The first Grey Cup, then called the Dominion Football Championship, was played at Rosedale Field, a pitch with temporary grand-stands in what is now Rosedale Park. Sharing the majority of its ancestry with rugby, the first major Canadian Football offshoot occurred in 1903 when the Ontario Rugby Football Union adopted the rules that U of T coach Thrift Burnside (legendary name) brought from the United States.
From there it was only a few short years before the Dominion Football Championship was established, fought between 'Rugby Football' clubs from Toronto, Ottawa, Hamilton, Montreal and Peterborough.
In 1909 Governor General Albert Grey (who by his British noble title is also called 'Earl Grey') announced the sponsorship of a trophy for the league. The first final played for this cup was an all-Toronto affair between U of T's Varsity Blues and the Parkdale Canoe Club 'Paddlers'. The final score was 26-6 for the Blues, and although both the cup and Earl Grey were missing from the field (the story goes that Grey forgot to order the cup in time), one hundred years of Canadian football history had been set in motion.
Since then, the Grey Cup has been held in Toronto a total of 45 more times. The next 26 finals in Toronto were held at Varsity Stadium. During the early years on Bloor, cross-town rivalries were tense, with the Blues and the Argos squaring off in 3 different finals from 1911-1920 (the Blues coming out on top 2-1).
It would still be a number of years before the Argos were the city's unequivocally supported football team. More competition came from the east end, where the Balmy Beach Beachers (say that five times fast) had a successful run in the late 1920's, going on to win the Cup in 1927 and 1930. In the mid 1930's the Argo's established a winning routine, with back to back victories in 1937 and 1938 (including such thrilling final scores as 4-3).