Not only that, but you can't even interact with people while on horseback either. Horses are pretty much a dud in this game. After my last one went down due to a random bear trap in the road, I decided it to explore the rest of Skyrim on my own two feet.
My horse ran away the first time I got off of it and I bought him, didn't steal him. Then I checked all the cities and he wasn't there, so I just gave up. I turned it on today and the stupid fucker was just sitting there in one of the stables, so I have no idea what happened before.
I finally got ALL THE PS3 TROPHIES!!! I had to restart a new campaign to get the correct deadric artifacts (I mean, who didn't want to kill Barbas?). And then the theives guild glitch with Tonja (fuck yes that's a glitch!). It was worth it though.