That's what I thought at first, I had to swim through the gem to break it off, then I saw that literally any attack can break them off. Bommeranging them with the fins is too finicky for me, so instead i just used the fin shield and waited till he inhaled which i would bring up the magic shield. Another thing! I thought i had to consistently swim, I didn't know sinking down would be helful.