Carly Simon was born in New York City. Her father was Richard L. Simon (co-founder of Simon & Schuster), a classical pianist who often played Chopin and Beethoven at home. Her mother was Andrea Heinemann Simon (née Heinemann), a civil rights activist and singer. Her father was from a German-Jewish family, while her maternal grandfather Friedrich was of German descent. Carly's maternal grandmother, Ofelia Oliete known as "Chibie", was a Catholic from Cuba, and was of Pardo heritage, a freed-slave descendant. A 2017 episode of PBS show Finding Your Roots tested Simon's DNA, which included 10% African and 2% Native American, likely via her maternal grandmother.
Her grandmother was sent to England and raised by nuns until the age of sixteen.
In 2015, Simon stated that when she was seven years old, a family friend in his teens did sexual things to her. She stated, "It was heinous", adding, "It changed my view about sex for a long time."
Simon began stuttering severely when she was eight years old. A psychiatrist tried unsuccessfully to cure her stuttering. Instead, Simon turned to singing and songwriting. "I felt so strangulated talking that I did the natural thing, which is to write songs, because I could sing without stammering, as all stammerers can."
Simon was raised in the Riverdale neighborhood of the Bronx, New York City and has two older sisters, Joanna (b. 1940) and Lucy (b. 1943), and a younger brother, Peter (b. 1947). They were raised as nominal Catholics, according to a book of photography Peter published in the late 1990s. She attended Riverdale Country School. She also briefly attended Sarah Lawrence College before dropping out to pursue music.