Green’s Dictionary of Slang

clergyman n.

[the colour of both professions’ clothes]

a chimney sweep.

[[UK]Leeds intelligencer 11 Feb. 3/3: An honest Chimney-sweeper tricked a farmer out of twenty pounds [...] The too credulous country-man, supposing him to be a clergyman by his garb, took everything he said, for gospel].
[UK]Flash Minstrel! in Spedding & Watt (eds) Bawdy Songbooks (2011) I 108: [song title] Up the Flew, or, the Knowing Clergyman [...] Clergy he got burnt so bad, / To give the trade up he was glad.
[UK]Flash Dict. in Sinks of London Laid Open.
[UK]Farmer & Henley Sl. and Its Analogues.