Green’s Dictionary of Slang

1. (US) a brothel.

1881		Trumble Sl. Dict. (1890).
1882		Breckenridge News (Cloveport, KY) 23 Aug. 3/3: My mother moved into a ‘castle’ and left me to shift for myself’.
1895		Crowe Aus. Sl. Dict. 15: Castle, a house of ill-fame.

2. (US black) one’s house, one’s home.

1940		Pittsburgh Courier (PA) 27 Apr. 7/6: If you dig one of those weird and unhep chics, don’t play the castle strong.
1953		L. Durst Jives of Dr. Hepcat (1989) 5: The castle gets groovy and the joint gently begans to rock.
1966		B. Jackson Get Your Ass in the Water (1974) 126: She drove me to her pad, she said, ‘Welcome home. / This is your castle, daddy, and inside is your throne.’.
2000		Indep. Rev. 21 Jan. 7: The face, he explains is donned every time a male leaves his castle.