I never promised you a rose garden - what you say when someone is giving you grief for something

what you say when someone is giving you grief for something that didn't go as expected and you're just like 'you had your chances.'

'I never promised you a rose garden' after my dog ate my homework again.

'I never promised you a rose garden' when my brother blamed me for the broken lamp.

'I never promised you a rose garden' after my friend forgot my birthday again.