What caused the giant arthropods to shrink? It seems like, from an evolutionary standpoint at least, that large size would benefit them a whole lot.
You would think so! In this case, however, it’s less a case of evolutionary benefits and more that… all the giant bugs choked to death.
The Carboniferous Period, as the name might suggest, was chock full of plants. TONS of coal beds were laid down during this period, all from the weird, tree-like club moss and horsetail relatives that were dominant at the time.
A side effect of all these plants was that atmospheric oxygen levels were at a whopping 35% (compared to the 21% today). Turns out that atmospheric oxygen levels are one of the primary limiting factors in maximum arthropod size today, so: more oxygen meant more giant bugs.
Of course, all good things must die, and at the end of the Carboniferous an extinction/glaciation event known as the Carboniferous Rainforest Collapse occurred, taking the giant arthropods down with it as the oxygen levels plummeted. And that was the end of it.
OR WAS IT?
Larger-than-usual flying arthropods actually existed in more than just the Carboniferous, since arthropods got the “flying is cool” memo way earlier than the rest of us and dominated the skies, uncontested - and, more importantly, un-predated - for a significant portion of time. The peak size of the infamous griffinfly genus Meganeuropsis was actually reached in the Early Permian, and another jump was marked in the mid-Triassic. Until the Early Cretaceous, these jumps and dips in aerial body size paralleled the levels of atmospheric oxygen almost perfectly.
The decoupling of factors in the Cretaceous was abrupt, and suddenly it seemed as though flying arthropods couldn’t get tiny fast enough. Aerial insects were now prime dining for the new kids in town, and would remain so for the next ~150MY. Just in case you’re not quite picking up what I’m putting down, let me be frank.
In conclusion, the extremely tl;dr to take home from this is that giant bugs exist only when there are giant amounts of oxygen in the atmosphere. Unless there are birds, in which case it all goes to shit.