Check out soundfonts.it for the vb3 1.1. It's definitely worth checking out the free demo to see if you like it. Also there you will find a free organ, organized trio. Dirtbag is a cool organ sim for some over-the top distorted and delayed sounds.
As a former Hammond player, I'm pretty happy with B4II. The Leslie simulation is the weakest part, but that's usually true with simulations.
I agree with your assessment, and in fact, play B4II through a Motion Sound Pro 3 with a Roland KC500 bottom, the Leslie sim altogether off in my B4 performance presets. The other complaint I have with B4II is that the distortion is just too heavy at levels above 3 or 4. These too get reduced in my performance setup.
Otherwise, I like the sound of B4II very much, compared with either the latest XK product from Hammond-Suzuki, or an aging tonewheel.
The VB3 product from Italy is also quite realistic sounding when played through a true rotating horn amplifier like the Pro 3. I've read on KVR-audio that the author is again rewriting the engine, for reasons of his own, incidently.