On August 16, 1856, the Texas Legislature enacted a measure providing for the establishment of a Texas Asylum for the Blind (now known as the Texas School for the Blind and Visually Impaired) in Austin. The state leased the Neill–Cochran House as a temporary site for the asylum while a permanent building was constructed. The asylum facility was built between 1856 and 1857 at a cost of $12,390 (equivalent to $333,000 in 2018). In 1857 the asylum moved into its building, where it operated until the end of the American Civil War in 1865, at which point the campus was commandeered by General George Armstrong Custer, who used the facility as his family's residence while contributing to the occupation and Reconstruction of Texas.
In 1866 the asylum was restored, and it occupied the campus from then until 1915, while the program was renamed the Texas Blind Institute in 1905 and then the Texas School for the Blind in 1915. During this period the complex was expanded with the construction of several additional buildings. During World War I the School for the Blind was displaced by a military pilot training program, and a barracks was added to the complex. After the war the School for the Blind relocated to a new and larger campus, and the original asylum facility spent several years housing the Texas State Hospital for the Senile.
In the mid 1920s the growing University of Texas at Austin purchased the campus from the State Hospital system, after which it came to be known as the "Little Campus" (by contrast with the main campus to the northwest). The university built a women's dormitory called the Little Residence Hall and renovated the existing buildings into a men's dormitory. The facility was again commandeered for military training during World War II, reverting to the university's use after the war's end.
On August 13, 1974, the Little Campus was declared a United States historic district and added to the National Register of Historic Places. In 1987 the area was officially renamed the "Heman Sweatt Campus," in honor of former UT law student and African-American civil rights plaintiff Heman Marion Sweatt.
Today, many of the Little Campus's historic structures have been demolished to make room for redevelopment, but two buildings remain: the asylum's original building, now known as the Arno Nowotny Building, holds the office of the director of the university's Dolph Briscoe Center for American History, and John W. Hargis Hall houses the university's Undergraduate Admissions Center.