Green’s Dictionary of Slang

(Aus.) a dismissive excl. ‘don’t worry about me!’, ‘I don’t care!’.

1949		A. Marshall How Beautiful Are Thy Feet 50: ‘My worries,’ said Correll, contemptuously gesturing, ‘She’s got nothing on me.’ [GAW1].
1953		T.A.G. Hungerford Riverslake 174: ‘And you’re on the outer for sticking up for him?’ ‘My worries.’.
1963		A.W. Upfield Madman’s Bend 83: ‘It’s going to be another fine day, isn’t it?’ ‘My worries if it’s fine or wet.’ [GAW1].