I have noticed exactly zero improvement in FPS in beta 18. It's exactly the same for me as 17 and 16.
The only difference I've noticed is a reduction in texture loading freeze around new planes due to the caching. But that's at the cost of making it a pain to work on new skins, because I have to delete the cache folder for the plane before my changes show up in the game.
P4/1.7
512MB RAM
ATI Radeon 9600 Pro 128MB
WinME
DirectX 9.0