I think the worst bit about horses is how you can't pick up items, like Mushrooms and whatnot, while riding them, personally.. You either have to miss out, or jump off every time you see something you want to add to your inventory. Shame, really, because just messing about with them is a nice diversion.