is it not correct to say - whom did you see on the ball/game?
It depends on how you define 'correct'. According to many prescriptive grammarians
"Whom did you see at the ballgame?" is technically speaking correct.
But no native speaker actually says that. It sounds bizarre, maybe a little like pronouncing a heavy nasal vowel for every single -ę in Polish. Also technically correct but no one talks that way.
Probably 99.99 per cent of native speakers would say 'who' in that sentence.
'whom' is sometimes (not always) used directly after prepositions (especially when the who is a relative clause marker but if the who and preposition are separated most people will say 'who'.
Those are the people for whom I wrote the book.
vs
Those are the people who I wrote the book for.
who in the first sentence would sound wrong and whom in the second would sound wrong.
And English does not have a dative case anymore.