Eliza Palmer was the daughter of James M. Palmer of Hillsborough, NC and Sophia Lutterloh, daughter of well-to-do Gen. Charles Lutterloh of Chatham County, NC. James M. Palmer was one of three sons of the widow Mary A. Palmer who died December 7, 1840. Interestingly, her tombstone in St. Matthew's Churchyard in Hillsborough, NC bears only one line, "Mother of J.M. Palmer."
James M. Palmer was popular and well-liked in the Hillsborough, NC community. He was chosen as a Town Commissioner in 1835 and held various positions of trust, but seemed to also have a talent for losing money in ill-started ventures. One of these, a newspaper scheme, started in 1843 when he announced plans to publish an area newspaper called the Orange Democrat. After 250 subscribers were secured, no record of the newspaper ever appeared.
In 1846 the Palmers purchased and moved into the Carleton Walker House on W. Margaret Lane in Hillsborough, NC. Today, the house is included on the National Register as the Walker-Palmer House in Hillsborough, NC. Sophia Lutterloh was a member of a large family who lived lavishly and easily, and it is clear the Palmers lived considerably beyond their income.
Eliza Palmer enrolled in the Burwell School as a day pupil c. late 1840s or early 1850s. Her name appears in the Burwell School Catalogue of 1848-51 . She undoubtedly took courses in music, since her pianoforte is listed in legal records. On January 12, 1856, Mrs. Burwell (Margaret Anna Robertson) wrote to her daughter Frances Armistead Burwell in New York, NY:
"Eliza Palmer has gone to St. Mary's, to be polished I suppose."
As early as 1853, Gen. Charles Lutterloh came to the Palmers' financial rescue. Deeds books in the Orange County Register of Deeds office preserve ample evidence of all the financial juggling necessary to secure the house and lot on Margaret Lane "to the sole and separate use of the said Sophia Palmer." Virtually all of James M. Palmer's property, including his daughter Eliza's "pianoforte," was conveyed in trust to his brother Nathaniel J. Palmer of Caswell County, NC.
James M. Palmer served as postmaster for four years from August 27, 1853 to September 16, 1857. He served as Hillsborough, NC's mayor from May 1856 to June 2, 1857. Financially, he was walking a tightrope during these years, for his brother Nathaniel J. Palmer had died in 1854, and his heirs and executors were settling his estate. By July 30, 1865, James M. Palmer and Sophia Lutterloh were living in Wayne County, NC and on that date conveyed their former home on Margaret Lane to James S. Watson .
Eliza Palmer was born in Hillsborough, NC.