Financial forecasters almost universally agreed there would be no collapse before 2008. It happened anyway. Earthquakes? The Fukushima disaster and the tsunami that caused it reminded us how awful we are at predicting those, too.
So are weathermen, the most ridiculed of predictors, actually doing it right? Although they miss a day or two here and there, our National Weather Service has been consistently improving for over a century.
In the New York Times, Nate Silver discusses how their acceptance of chaos theory helps them get past simply relying on Big Data:
Why are weather forecasters succeeding when other predictors fail? It’s because long ago they came to accept the imperfections in their knowledge. That helped them understand that even the most sophisticated computers, combing through seemingly limitless data, are painfully ill equipped to predict something as dynamic as weather all by themselves. So as fields like economics began relying more on Big Data, meteorologists recognized that data on its own isn’t enough.