Gilzean began his career with local side Coupar Angus Juveniles, before signing provisional forms with Dundee in January 1956 as a 17-year-old amateur. He played once for their youth team Dundee Violet, but then played again for Coupar Angus while working as a despatch clerk for a carpet manufacturer in Perth. He signed professional forms with Dundee in February 1957, but then had a spell in Hampshire while he underwent National Service in the Royal Army Service Corps. Gilzean made his competitive debut for Dundee in August 1959, and then became a key part of a successful side. He scored more than 150 goals in the Scottish top flight as Dundee won the in and reached the semi-finals of the 1962–63 European Cup.
Gilzean joined Tottenham Hotspur in December 1964, moving for a transfer fee of £72,500. He made his first appearance for Tottenham a week later, in a home fixture against Everton.
Gilzean enjoyed a glittering career as a Tottenham player, while he also changed his style of play from being the main goalscorer to being an intelligent and creative forward. He formed a partnership alongside crowd favourite Jimmy Greaves. A member of the 1967 FA Cup Final winning team, Gilzean continued to be a regular first-team player after the arrival of Martin Chivers in early 1968 from Southampton.
After strike-partner Greaves moved to West Ham United in March 1970, Gilzean and Chivers formed a new and equally successful goalscoring partnership. This contributed greatly to Tottenham's cup triumphs in the first half of the decade, winning the in 1971, an all-English 1972 UEFA Cup Final against Wolverhampton Wanderers, and a second League Cup victory in 1973.
The season was Gilzean's last as a professional footballer as Spurs lost the UEFA Cup final to Dutch side Feyenoord Rotterdam. Tottenham awarded Gilzean with a testimonial match, played against Red Star Belgrade in November 1974, to recognise his ten years of service as a Tottenham player.
Following his retirement from Tottenham, Gilzean played in South Africa for three months with Highlands Park.
Gilzean made his debut for Scotland in November 1963, in a 6–1 win against . He had previously represented his country at Under-23 level and the Scottish League XI. He received four more international caps in the following twelve months while playing for Dundee. He also scored twice for a Scotland Select XI against Tottenham Hotspur in a November 1964 memorial match for Tottenham and Scotland player , who had died in tragic circumstances earlier that year.
Gilzean represented Scotland seventeen times during his Spurs career. In total he scored 12 goals in 22 full international appearances for Scotland, between November 1963 and April 1971.