Your horse's stamina gets spent faster if you run far out in a turn or go runaway. Otherwise it's spent bit by bit between each round of solitaire. If your horse has low stamina and is difficult to raise (I had a blue one whose stamina never did grow very much when it leveled) you may need to invest in a stamina carrot before long races.