Southwold was mentioned in Domesday Book (1086) as a fishing port, and after the "capricious River Blyth withdrew from Dunwich in 1328, bringing trade to Southwold in the 15th century", it received its town charter from Henry VII in 1489. Over the following centuries, however, a shingle bar built up across the harbour mouth, preventing the town from becoming a major Early Modern port: "The shingle at Southwold Harbour, the mouth of the Blyth, is ever shifting," William Whittaker observed in 1887.
Southwold was the home of a number of Puritan emigrants to the Massachusetts Bay Colony in the 1630s, notably a party of 18 assembled under Rev. Young, which travelled in the Mary Ann in 1637. Richard Ibrook, born in Southwold and a former bailiff of the town, emigrated to Hingham, Massachusetts, along with Rev. Peter Hobart, son of Edmund Hobart of Hingham, Norfolk. Rev. Hobart had been an assistant vicar of St Edmund's Church, Southwold after graduating from Magdalene College, Cambridge. Hobart married in America Rebecca Ibrook, daughter of his fellow Puritan Richard Ibrook. The migrants to Hingham were led by Robert Peck, vicar of St Andrew's Church in Hingham and a native of Beccles.
fire in 1659 devastated most of the town and damaged St Edmund's Church, whose original structure dated from the 12th century. The fire created a number of open spaces within the town which were never rebuilt. Today this "series of varied and very delightful village greens" and the restriction of expansion because of the surrounding marshes, have preserved the town's genteel appearance.
On the green just above the beach, descriptively named Gun Hill, the six 18-pounder cannon commemorate the Battle of Sole Bay, fought in 1672 between English and French fleets on one side and the Dutch (under Michiel de Ruyter) on the other. The battle was bloody but indecisive and many bodies were washed ashore. Southwold Museum has a collection of mementos of the event. It has occasionally been said that these cannon were captured from the Scots at Culloden and given to the town by the Duke of Cumberland, who had landed at Southwold in October 1745 having been recalled from Europe to deal with the Jacobite threat, but they are much larger than those used by Charles Edward Stuart's army in that campaign. During World War I, it was widely thought that these cannon were one reason why this part of the coast was bombarded by the German Fleet as a "fortified coast". In World War II the cannon were prudently removed, reputedly buried for safety, and returned to their former position after hostilities.
On 15 May 1943 low-flying German fighter-bombers attacked the town and killed eleven people.