Thompson was the son of Canon W. G. Thompson. He went to Marlborough College and took an MA at Sidney Sussex College, Cambridge. While at Cambridge, he joined the University Air Squadron and learnt to fly. He was commissioned into the Royal Air Force Reserve in 1936. In 1938, he joined the Malayan Civil Service as a cadet.
At the start of World War II, Thomson joined the RAF, and was serving in Macao when the Japanese attacked. He escaped the Japanese and with a suitcase full of money and a knowledge of Cantonese, he gambled his way across China to Burma.
He was a liaison officer with the Chindits in the Burma Campaign, being awarded the DSO and the MC, (the latter an unusual army decoration for an RAF officer).
 Later in the campaign, he flew Hurricanes and was promoted to the rank of Squadron Leader in 1945.
At the war's end, he returned to the Malayan civil service, becoming assistant commissioner of labour in the state of Perak in 1946. After attending the Joint Services Staff College at Latimer and holding the local rank of Lieutenant-Colonel, he was a member of the staff of the British director of operations during the Malayan Emergency. He would later say that much of what he had learned about counter-insurgency operations was learned while serving under Lieutenant-General Sir Harold Briggs and his replacement, General Sir Gerald Templer.
In 1959, (after Malayan independence), Thompson became permanent secretary for defence for Tun Abdul Razak (who later became Malayan prime minister). In response to a request from President Ngo Dinh Diem of South Vietnam, Tunku Abdul Rahman, the Malayan prime minister sent a team to South Vietnam to advise Diem on how to counter his insurgency problems. Thompson headed that team which so impressed Diem that he asked the British to second Thompson to the government South Vietnam as an advisor.
In September 1961, the British Prime Minister Harold Macmillan appointed Thompson head of the newly established
BRIAM (British Advisory Mission) to South Vietnam and - by extension - Washington. Thompson conceived of an initiative he called the Delta Plan, but when he saw the effects of the strategic hamlets initiative, begun in February 1962, he became an enthusiastic backer, telling President Kennedy in 1963 that he felt the war could be won. Under Thompson's leadership, BRIAM put economic pressure on the South Vietnamese government that Thompson described as a "straight invitation to a coup".
Kennedy was receptive to Thompson's ideas, but the American military establishment were extremely reluctant to implement them. His warning not to bomb villages went unheeded and his dismissal of American air supremacy was ignored. "The war [will] be won by brains and on foot", he told Kennedy, but competing interests in Washington and Saigon acted to marginalise Thompson and ultimately his strategies had no real effect on the conflict. He stepped down from BRIAM in 1965 and the organisation, deprived of the man who was essentially its raison d'être, folded up around him.
Despite his relatively acrimonious criticism of United States policy in Vietnam, Thompson returned to a post assisting the American government in 1969 when he became a special adviser on "pacification" to President Nixon.
In later life, Thompson wrote extensively about the use of commandos and counter-insurgency operations in asymmetric warfare.