Green’s Dictionary of Slang

(US) to go wrong, to fail.

1943		R. Chandler Lady in the Lake (1952) 113: Like a bad taste in the mouth [...] Like a romance that fouled out. Am I too blunt?
1959		E. De Roo Young Wolves 134: ‘You know, I didn’t foul out once.’ ‘You were good, Cliff.’.
1983		G. Tate ‘Cecil Taylor’ in Flyboy in the Buttermilk (1992) 26: They have to learn to play the game without fouling out.