Green’s Dictionary of Slang

(US black) an invitation to start fighting; esp. in phr. it’s gonna be me and you.

1964		R. Abrahams Deep Down In The Jungle 266: Me and you – Short for ‘there’s just me and you and we’re going to fight’.
1972		E. Folb Urban Black Argot 143: Me and You a challenge to a fight.