Green’s Dictionary of Slang

1. the keeper of an unlicensed pawnbroker’s.

1858		A. Mayhew Paved with Gold 71: A party as is kind to me on my rounds gave me some broken hits and an old jacket this morning, and I sold the jacket to the dolly-man for a bob.
1861		(con. 1840s–50s) H. Mayhew London Labour and London Poor II 110/2: A poor person driven to the necessity of raising a few pence, and unwilling to finally part with his lumber, goes to the dolly-man.

2. a Jew.

1937–84

Partridge DSUE (1984) 326/2: Anglo-Irish [...] late C.19–20.