Phillip Henry Pitts was born June 3, 1814 in Essex County, Virginia. He was the son of Thomas Daniel Pitts, a veteran of the War of 1812 who moved his family to Uniontown in 1833. Pitts married in 1841 to Margaret Davidson, the sister of Alexander C. Davidson.The Davidsons were descendants of William Lee Davidson, a general during the American Revolutionary War and founder of Davidson College in North Carolina. Pitts himself was a large contributor to the college within his lifetime and his plantation diaries were later donated to it for preservation.Phillip Pitts recorded in his diary that construction of the house began on February 27, 1852. He also recorded the house being completed in April 1853. His assets by 1860 were valued at $175,300. At this time he owned two additional plantations, "Rurill Hill" and "Kings" for a total of 2,200 acres (890 ha). The 1860 United States Census of Perry County indicates that Phillip Henry Pitts owned 75 slaves in that year, though his children are individually listed as owning an additional 68 slaves.Pitts also owned stock in the Alabama-Mississippi Railroad.The Pitts' had ten children, most of whom were raised in the house, with two dying in the American Civil War.Phillip Pitts remained a cotton planter until his death on April 22, 1884.The house continues to be occupied by the descendants of Phillip Pitts to the present day.
