the gears are aligned in a way to always turn the little man in the opposite direction as the cart at the same rate of rotation. so if the man points at you, and you turn the cart clockwise 90 degrees, the man will be turned counterclockwise 90 degrees, and still be facing you. if you turn the cart counterclockwise 90 degrees, the man will be turned clockwise 90 degrees, and still be facing you
as for how they got the cart to point south to begin with, that goes into fengshui and cardinal direction geomancy. but long story short, the workshops that built these carts would have their front doors facing south to begin with (using the sun and the stars to figure out which way is south), so all they would have to do is build the cart facing that direction, and the little man will always point south