Have photographed them only twice as i live in India. But the first time was a learning process. Ideally its best to take shots while they are coming towards you. Not when they are moving away either side. Panning your camera as they move is also a help.
Hopefully these two images may help get an idea of what i mean. The eyes are in focus, but the background and body of the seal has some blur. To avoid this completely, one has to shoot when the seal is com ing towards you as higher shutter speed is not possible.