Green’s Dictionary of Slang

castle rag n.

[rhy. sl. = flag n.1 (1)]

(UK Und.) fourpence.

[UK]Hotten Dict. of Modern Sl. etc.
[UK]Sl. Dict.
[Aus]Morn. Bulletin (Rockhampton, Qld) 18 July 2/6: For fourpenny pieces we find the name of ‘bit,’ ‘castle-rag,’ ‘flag. [...] ‘joey,’ and ‘quarterer saltee’.