Green’s Dictionary of Slang

brown bess adv.

yes.

1859		Hotten Dict. of Modern Sl. etc.
1864, 1867, 1870		Hotten Sl. Dict.
1873		Sl. Dict.
1938		Indiana Gaz. (PA) 16 Nov. 7/1: ‘Brown Bess’ means yes, and ‘Chevy Chase’ means the face.