Green’s Dictionary of Slang

cauliflower n.4

[? Fr. choux, cabbage, used as a term of affection]

[mid-19C] (US) a person, a fellow.