Green’s Dictionary of Slang

fish-face n.

[early 17C; 1910s+] a stupid or ugly looking person; also used as a derog. term of address.