Green’s Dictionary of Slang

duffy n.³

[? its popularity among Irish wearers, f. the common surname Duffy]

(US) a derby hat.

1930

‘Dean Stiff’ Milk and Honey Route 138: Neither does a hobo wear a hard hat or ‘duffy’.