What exactly are the greatest differences between ponies and horses?
Well, ponies are horses. A pony is a type of horse, so it’s not that there’s a difference between ponies and horses, but there are differences between types of horses.
As a rule, ponies are smaller than other horses, though of course that doesn’t apply to every single pony; Celestia is generally taller than any non-pony horse as well. But she is an exception.
Furthermore, ponies also have chubbier legs while other horses tend to have more slender legs. We also have larger heads in relation to the body.
I am attaching a picture from a diplomatic event years ago, when I first met Saddle Arabians, so you may see them in comparison to Celestia. Note that they are almost as tall as she is and look for the differences I mentioned.
Of course, another big difference is that ponies are divided into different types which in turn are very different and have their own specific traits. While that may seem like a purely taxonomic matter, it is relevant since every type of pony can produce offspring of another type of pony, allowing for quite diverse populations even in isolated locations.
Other horses don’t have the same kind of magic we do either. In fact most equines don’t have what we would consider magic, though our definition of magic is shaped by our culture, so there is some bias involved one should be aware of.
There are also cultural differences, but that might not be seen as a differences between horses per se, since it’s nothing inherent to one’s nature. Non-ponies who have emigrated to Equestria could easily end up adopting our culture. Biological differences are another matter however.