Green’s Dictionary of Slang

(US) a dismissive excl. either demanding that the subject leaves or suggesting that their statement is nonsense; also as go to grass and eat hay! (cf. go to grass v. (2)).

1844		‘Jonathan Slick’ High Life in N.Y. I 40: You go to grass!
1857		D.H. Strother Virginia Illus. 32: ‘Look here, gentlemen,’ said he triumphantly, ‘you may now go to grass with your shed’ [DA].
1859, 1860		Hotten Dict. of Modern Sl. etc.
1907		W.M. Raine Wyoming (1908) 55: Slim’s purple deepened again. ‘Y’u go to grass, Mac.’.
1910		B.L. Bowen ‘Word-List From Western New York’ in DN III:vi 442: go to grass, v.phr. imper. Get out! Stop talking! ‘Oh, you go to grass, I don’t believe a word you say.’.
1915		M.G. Hayden ‘Terms Of Disparagement’ in DN IV:iii 210: The little sass-box told me to go to grass.
1928		M. Bodenheim Georgie May 44: Oh, let him go to grass — ah’m not begging from no man.
1939		P.G. Brewster ‘Folk “Sayings” From Indiana’ in AS XIV:4 266: Expressions indicative of contempt are ‘cat’s foot,’ ‘Go to grass,’ and ‘Go to grass and eat hay’.
1943		C. McCullers Heart is a Lonely Hunter (1986) 101: She squelched him in a hurry: ‘You go eat grass!’.