I really don't know why there are so many cars in GT4, possibly because it's been a stple of the GT series for so many years to have a large number of cars with the best possible physics.
Besides the licensing issue, which you're spot on with, there's really no need to show damage to these vehicles. Consider this, in a NASCAR game, they accurately detail damage and the physical downside of it because admage is a part of the race. You're always door handle to door handle and someone's going to bite the dust at some point. When you're racing in the GT series, you're not "supposed" to crash into things. Some of the things I've seen my friends do would total a car with a single blow. If the game did that, you'd have to buy a new car and sit out a week while you recover. Doesn't make for a very fun game when you have to keep buying cars left and right or keep spending money on repairs because you don't know how to drive.
I'm not saying you don't know how to play the game. I'm actually making a statement based on how all my friends and their friends play the game. I'm strange, as I actually like the tests and like "driving" the cars while everyone else I know will do everything possible to win and beat the game. They will, as we all like to put it, "game the game."
Certainly GT4 has an insane number of cars and a large number of tracks while TOCA is a good racer it doesn't come close to GT4 in my opinion. Comparing them side by side, they look about the same to me and GT4 seems to drive better for me. I'm a little biased because I still have the original three GTs in my room somewhere and I've played them forever. Not that TOCA isn't a good game, it's just not my cup of tea.
Besides, what other game lets you take pictures of a DeLorean DMC 12 stomping all over an RX-7.