Green’s Dictionary of Slang

(US black) a poor white person, esp. a native of North Carolina.

1868		Putnam’s Mag. I 706/1: When [...] candidates refreshed their adherents by the barrelfull [sic] the low-downer enjoyed his periodical benders without expense [DA].
1872		Schele De Vere Americanisms 45: He [i.e. a very poor person] appears as Conch or Low Downer in North Carolina, and as Sandhiller or Poor White Trash in South Carolina and Georgia.
1883		R.L. Stevenson Silverado Squatters 151: They are at least known by a generic byword, as poor whites or low-downers [F&H].
1931		Sterling Brown ‘New St Louis Blues III’ in Botkin Folk-Say 121: Some low-down bummin’ cut plug from de passers-by.