Saxman, I understand that and in my first response to you.
I think you can do it if you are painting terrains.
If you are just using the colors in the water tab, no real idea on how to smooth it out.
However, after review I can see that my meaning might not come across clearly in my responses.
You are correct you can't achieve a nice transition just by using the water color brush.
You can achieve a nice anti-aliased water depth transition affect via the use of water terrain tiles. Its a work around basically.
You can set up multiple water tiles to simulate the colors of each water depth area you want and then paint them over water that has depth to achieve a very smooth affect. So if you want a smoother transition I am saying I have achieved good results with creating multiple wntt files, then create an area of depth that fades out via the waterd.bmp file, then paint a zone, then blend a zone between it and the next zone, and so on. It creates a nice transitional affect. That is the only way I have found so far to achieve what you want since painting the straight water color leaves much harder aliased edges.
If you don't use water tiles but just the straight water color brush as you stated the edges are much rougher and I have yet to discover anyway around it.