Submitted by the Madison Genealogical Society

There were three Patterson pioneers in Madison County who arrived  before 1840 :  Robert Patterson,  David Patterson, and John Howell Patterson. We do not know if these Pattersons were brothers. David and John Howell Patterson arrived from South Carolina via Georgia. A fourth, Hector Paterson, arrived between 1850 and 1858. Hector probably was not a sibling to the Robert Patterson, David Patterson, or John Howell Patterson because he was born in Scotland and was similar in age to the other Pattersons.  Although there were variations in name spellings in the census and county records for all the branches of the Patterson/Paterson, generally the South Carolina Pattersons were a two "t" Patterson family, and the Patersons from Scotland were a one "t" Paterson family.

Robert Patterson was born around 1805; his wife was born around 1815, and the couple had one son, born around 1837. They arrived in Madison County some time during the 1830’s and were counted in the 1840 census.  By 1850, they had moved on, and we couldn’t find where they might have moved to.  There were three Robert Pattersons in Georgia in 1830, but we don’t know which may have been the one who arrived in Madison County in the 1840’s.

David Patterson, born around 1792, moved from Georgia with his wife Sarah, (possibly Sarah White from Richmond County, Georgia). 

John Howell Patterson, born around 1803 in South Carolina, arrived sometime in the 1830’s with his wife, Sarah Elkins Smith Patterson and two children.  The couple came from Decatur Georgia.  [Jeanie Blanton Trimble, a descendant of JOHN HOWELL, contributed much of the following information on JOHN HOWELL in an article in the Madison County Family History Book, and in an article in the July, 2007 edition of the Madison Genealogical Society Newsletter].

John and Sarah Elkins Smith were married on June 10, 1830 in Bainbridge, Decatur County, Georgia. John Howell received an original land grant for 120 acres in 1832.  He was elected to 1st Lieutenant in Company A, 8th Regiment of Florida Militia.  At some point he was a sheriff in Madison County; he voted in the 1845 election for Florida statehood.

John Howell and Sarah raised five children that we know of in Madison County:  
1. Andrew Elkins Patterson, 1839-1924; 
2. Joshua Joseph Howell Patterson, 1841-1865. 
3. Smith Parramore Patterson, 1845-1916; 
4. Sarah Eugenia Patterson, 1847-1929; and
5.  Lafayette Hunter Patterson, 1850-1929

Third Generation
Andrew Elkin Patterson first married Sarah Elizabeth Swift, 1844-1876.  The couple had nine children in Madison County: 
1. Sarah Eugenia., 1859-1950, married Louis Franklin Cameron, 1853-1885 and Lewis M. Sessions, 1859-1928; {Jeanie Blanton Trimble descends through Eugenia.} ; 
2. John Howell Patterson, born 1861, married Elizabeth Lowe Edgerton; 
3.  Ira Swift Patterson, born 1863, married Amanda Ellen Patterson, born 1858; 
4.  Mary Rebecca Patterson, 1866-1956, married Samuel Batie Cameron, 1858-1954;
5.  Joshua Joseph Patterson, born 1870, married Laura; 
6.  Andrew Elkins Patterson, born 1868; 
7. Ada Lucy Patterson, born 1871 married E. Johnson and then J.C. Harley; 
8. Agnes Buckner Patterson, born 1875, married Mr. Brooks and then Mr. Vickers. 

Sarah, first wife of Andrew Elkin, died in June, 1876, we believe, of complications of childbirth.

Andrew Elkin then married Lavinia Swift, sister of Sarah, 1854-1931; Andrew and Lavinia raised the following children: in Madison County.  
9. Russell A. Patterson, born 1879; 
10.  Finley Jesse Patterson, born 1883; 
11.  Charles L. Bertholdi Patterson, born 1885; 
12.  Gladys Lavinia Patterson, born 1888, married Thomas E. Chaires; 
14.  Grady Atticus Patterson, born 1890; 
15. Amanda Ellen Patterson, 1896-1932, married John E. Downing; and 
16.  Ruth Inez Patterson, born 1899, married Clifford A. Roach.  

In 1920, Andrew and Lavinia were living in Pinellas County, with their daughter, Ruth Inez.  Andrew died in 1924, and was brought back to Madison and buried in the Old Oakland Cemetery, next to his first wife, Sarah.  In 1930, Lavinia was living in Pinellas County with Ruth and her husband, Clifford Roach.  Lavinia died in 1931, and was also brought back to Madison County to be buried. 

Joshua Joseph Howell Patterson, born in 1861, died of disease on April 16, 1864 at Rock Island Union Prison, on the Mississippi River between Iowa and Illinois.  He was captured at Missionary Ridge, Tennessee.  He had joined the Confederate Army on October 5, 1862.  According to the internet site, "," prisoners began arriving in December 1863, temperatures were below 0, and sanitation was deplorable due to overcrowding and disease. A smallpox epidemic broke out, killing hundreds of prisoners.

Smith Parramore Patterson, born March 1, 1845 also served in the Civil War.  He and Rebecca Swift, born 1850, had one child in Madison County, 
1. Joseph Patterson, born in 1868. 

After 1870, Smith and Rebecca moved to Alachua County, and after 1910, they moved to Putnam County. They had several more children: 
2. Andrew J., born in 1868; 
3. Elvira, born in 1873; 
4. Ola, born 1876; and 
5. Lealia, born 1878 and 
6.  S P, born 1880; 
7.  Gussie, born 1884; and 
8. J.S., born 1890.    

In 1900, the family lived in Baldwin County, Bay Minette, Alabama. Smith and Rebecca are buried in Evergreen Cemetery in Jacksonville.

Eugenia Patterson, 1847-1929, married William George Washington Gornto, born in 1846.  The couple had the following children:  1. Charles Gornto, born in 1869; 
2. Arie Gornto; 
3. Allie Gornto, born in 1873; 
4.  Frank Gornto, born in 1875; 
5. Ida Gornto, born in 1877; and 
6.  William Dowman Gornto, born in 1879, married to Mattie Dehlia King, born in 1872.

Lafayette Hunter Patterson, 1850-1929 married Nancy DeLaughter, 1861-1925. 

Third Generation:  Andrew Elkin

We find a John Patterson in Marion County, Florida in 1870, who is possibly the son of Andrew Elkins and Sarah. 

Ira Swift is in Madison County in 1885, but by 1910, he and his wife Amanda have moved to Leon County; in 1920 he is in Hillsborough County, and 1930, in Duval County.  He was a Methodist minister.  We are not sure if Ira and Amanda had children.

Sarah Eugenia and Louis Cameron had two children:  Sarah E. born in 1876; and Andrew E., born  in 1879.  Sarah Eugenia and Lewis Sessions had the following children:  Bertha, born in 1891; Willie, born in 1892; and Ellis I. and Illa M., born in 1894. We think they raised their children in Madison County; however, Lewis died in St. Petersburg. 

We think Joshua Joseph wound up in Duval County, married to Laura; we are unsure if he had children.

Russell A. married Frances Sanders in 1900 and lived in Duval, Pasco, and Volusia Counties; the couple had several children.

Finley Jesse, also a minister, lived in Volusia and Polk Counties and was buried in Polk County.  He was married to Grace, was also a Methodist minister, and the couple had at least two girls.

Charles L. Bertholbi was married to Ruby B., was a manager at lumber yard, had at least two children, and lived in Pasco County.

Grady Atticus lived in Hillsborough County, was married to Grace, was a bookkeeper, and had at least one child.

 It appears that none of Andrew Elkins’ seven sons remained in Madison County.

Third Generation:  Lafayette Hunter

Lafayette Hunter and Nancy E.  “Nannie” DeLaughter married in 1880, and had two children:  
1) Lafayette Hunter Patterson, Jr. who lived from 1882-1935, and
2) Arthur D. Patterson, who lived from 1885-1891. 

Lafayette and Nannie moved from Madison County sometime between 1910 and 1920 to Duval County.  Lafayette died on July 19, 1929.  Nannie died August 20, 1925.  They are both buried in Oak Ridge Cemetery.  Lafayette  Hunter, Jr., first married Ruth and was employed as a druggist in Glynn, Georgia.  The couple had at least one child, Theodore, born 1916.  Next he married Carolyn Watkins and the couple lived for several years in Jacksonville.  He died in 1935, and is buried in Oak Ridge Cemetery, near his parents and brother.

As far as we know, none of John Howell Patterson's male Patterson descendants remained in Madison County.  If readers know better, we apologize for our errors would love to hear about corrections.  With three different Patterson lineages in Madison County at one time, errors could easily have been made in finding the correct descendants.

The Madison County Genealogy Society welcomes your input and invites you to join our organization.  We meet on the second Thursday monthly, except during summer months, in the Madison Public Library from 6 to 7 PM.  Annual dues are $25.  To add comments to our articles or to submit your own sketch of your ancestor, contact us at Madison County Genealogy Society, P.O. Box 136, Madison, FL 32341.  Or contact us by email at

Madison County


Genealogical Society