Maneuverability means a lot of things.
As the speed gets higher, the importance of turning ability increasingly falls down, while the importance elevator responsiveness and roll rates grow up. Since I started flying 109s, never a day goes by without envying how those P-51s handle at high speed. Elevator authority and roll ability, is the key to high-speed fights.
With the exception of the clip-wing Spitfires, to my knowledge all Spitfires were more or less average/mediocre in roll, and the tendency of heavy controls and reduction of roll rate were well noted.
A skilled opponent in a Spitfire is always a feared enemy, but if the pilot is more or less inexperienced, I can outmaneuver a Spitfire in a high speed fight. Even if he is skilled, there's high chance I can manage escape by a simple 'flick' of a roll, if, I can lure him into a chase of 350 mph+