Green’s Dictionary of Slang

cauliflower n.⁵

[? its white ‘heart’; or rhy. sl. cauliflower ear = fear]

(US) cowardice, fear.

1978

W. Brown Tragic Magic 40: Don’t feel bad about showing some signs a weakness. Every man in here got some cauliflower in their heart.