I don't think it's ever spelled out it's "actually true" in the game (the manga on the other hand...), but it's a solid theory, makes sense, and probably intended. Well, I don't think it makes sense, but that's because I don't think his tragic trauma makes sense in the first place; he should have been afraid of attachment (like Gerome), not women as a gender.