Woo was born Wu Yu-seng (Ng Yu-sum in Cantonese) in Guangzhou, China, amidst the chaos of the Chinese Civil War at the end of October 1946. Due to school age restrictions, his mother changed his birth date to 22 September 1948, which is what remains on his passport. The Woo family, Christians faced with persecution during Mao Zedong's early anti-bourgeois purges after the communist revolution in China, fled to Hong Kong when he was five.:xv, 3
Impoverished, the Woo family lived in the slums at Shek Kip Mei. His father was a teacher, though rendered unable to work by tuberculosis, and his mother was a manual laborer on construction sites. The family was rendered homeless by the Shek Kip Mei fire of 1953. Charitable donations from disaster relief efforts enabled the family to relocate; however, violent crime had by then become commonplace in Hong Kong housing projects. At age three he was diagnosed with a serious medical condition. Following surgery on his spine, he was unable to walk correctly until eight years old, and as a result his right leg is shorter than his left leg.
His Christian upbringing shows influences in his films. As a young boy, Woo had wanted to be a Christian minister. He later found a passion for movies influenced by the French New Wave especially Jean-Pierre Melville. Woo has said he was shy and had difficulty speaking, but found making movies a way to explore his feelings and thinking and would "use movies as a language".
The local cinema would prove a haven of retreat. Woo found respite in musical films, such as The Wizard of Oz and in American Westerns. He has stated the final scene of Butch Cassidy and the Sundance Kid made a particular impression on him in his youth: the device of two comrades, each of whom fire pistols from each hand, is a recurrent spectacle later found in his own work.