Green’s Dictionary of Slang

a person who smokes, thus non-fagger, a non-smoker.

1911		F.J. Randall Bermondsey Twin 52: He opened the book. [...] His Great Fight against the Destroying Weed. From the Affiliated Confederation of Non-Smoking South Londoners: the Non-Faggers, the Empty Pouches, the Broken Pipes, the Anti-Cigs, the Clean-Throttles.’ A sigh escaped Jack Holdaway. ‘I shall never be up to his mark," he said regretfully. ‘ I’m too far gone. It’d take years to make a non-fagger of me’.
1915		Grey Brigade 26 June 2: What is to be done with the inveterate cigarette obtainer - the man who always has a box of 50 in his kit-bag but none in his case? He is to be found in all companies. ‘Got a fag, old man?’ is the favourite opening. The only way of escape, it seems to me, is to form an S.F.C.O.U.F. - Society For Choking Off Undesirable Faggers.
1947–53		W. Guthrie Seeds of Man (1995) 246: I’m an ole fagger from away back.