Green’s Dictionary of Slang

an outcast; one who exists on the fringes of a given group.

1927		F.M. Thrasher Gang 76: In conventionalized gangs, a definite rule with regard to age is customary; this, however, usually is not enforced, or if it is, boys of the barred ages may still be hangers-on, ‘fringers,’ or an affiliated group.
1947		(con. 1944) J.H. Burns Gallery (1948) 55: Hal knew them all, even those fringers who didn’t belong to cliques.
1955		B. Schulberg On the Waterfront (1964) 25: Thank God he wasn’t a drifter, a fringer like his poor slob of a kid brother.