Agreed, and I'm pretty good with the slow panning when required, but there are always those scenarios where a shark or codfish will dart across the frame.
If the difference between the two isn't significant then no big deal, but if it's pronounced then it will tip the scales.
I think that you are worrying about an immaterial thing. Rolling shutter is only obvious when panning across straight, distinct verticals. Underwater, about the only place you'll find straight, distinct verticals are on ship-wrecks. And then only obvious with a fast pan - which is likely to make your audience so dizzy that they won't notice the "rolling shutter" anyway.
Any shark darting across the scene might look slightly distorted if compared to a still photo but the audience will be oblivious.