Survivorship Bias

I have posted about survivorship bias and how it affects your career choices: how a Hollywood actor giving the classic “follow your dreams and never give up” line is bad advice and is pure survivorship bias at work.

When I read up on the wikipedia page, I encountered an interesting story:

During WWII the US  Air Force wanted to minimize bomber losses to enemy fire. The Center for Naval Analyses ran a research on where bombers tend to get hit with the explicit aim of enforcing the parts of the airframe that is most likely to receive incoming fire. This is what they came up with:


So, they said: the red dots are where bombers are most likely to be hit, so put some more armor on those parts to make the bombers more resilient. That looked like a logical conclusion, until Abraham Wald - a mathematician - started asking questions: 

- how did you obtain that data?
- well, we looked at every bomber returning from a raid, marked the damages on the airframe on a sheet and collected the sheets from all allied air bases over months. What you see is the result of hundreds of those sheets.
- and your conclusion?
- well, the red dots are where the bombers were hit. So let’s enforce those parts because they are most exposed to enemy fire. 
- no. the red dots are where a bomber can take a hit and return. The bombers that took a hit to the ailerons, the engines or the cockpit never made it home. That’s why they are absent in your data. The blank spots are exactly where you have to enforce the airframe, so those bombers can return.

This is survivorship bias. You only see a subset of the outcomes. The ones that made it far enough to be visible. Look out for absence of data. Sometimes they tell a story of their own.

BTW: You can see the result of this research today. This is the exact reason the A-10 has the pilot sitting in a titanium armor bathtub and has it’s engines placed high and shielded.

10

Japanese destroyer Kikuzuki in Tokyo Bay off the coast of Japan, on 1 September 2016. He died 4 may 1942.

American amphibious LVT-1 remaining forever in the jungle of the Solomon Islands, on 7 September 2016

The wreckage of a Japanese fighter A6M, Palau, 29 Aug 2016.

Japanese light tank Ha-Go. Northern Mariana Islands, 28 August, 2016.

M3 Stuart, shot down by Japanese 37mm anti-tank gun. Arundel island, Solomon Islands.

Japanese light tank Ha-Go. Northern Mariana Islands, 28 August, 2016.

Floating tank LVT(A)-1, which is lined on Peleliu. 2012.

The skeleton of the Japanese tank Ha-Go on the island of GUAM.

M4 Sherman destroyed in 1944 during the battle for Peleliu

10

     Smithsonian’s National Air & Space Museum Udvar-Hazy Center in Chantilly, Virginia, offers the unique sight of a complete Mercury spacecraft. Many of these spacecraft are available for viewing all over the United States, but this one is special because it did not fly.

     During the course of a Mercury flight, several parts of the spacecraft are jettisoned and not recovered, including the retro package. This piece of equipment is visible here in my photos as the striped metal object strapped to the bottom of the heat shield. This small cluster of solid rocket motors was responsible for the safe return of the astronaut from space, making just enough thrust to change the shape of the orbit so that it would meet the atmosphere and use aerobraking for a ballistic reentry.

     If this package had not fired properly, the astronaut would be faced with the dire situation of being stuck in orbit. Fortunately, this never happened in real life, but it was captured in the fanciful novel “Marooned” by Martin Cardin, in which a NASA astronaut was stranded on orbit after his retro rockets failed. When the book was released in 1964, it was so influential that it actually changed procedures for Mercury’s follow on program Project Gemini, adding more redundancy to the spacecraft’s reentry flight profile.

     Alan Shepard, the first American in space and later Apollo 14 moonwalker, didn’t fail to notice that there was a leftover spacecraft at the end of the Mercury program. He lobbied for a second Mercury flight in this ship, speaking personally to both NASA Administrator James Webb and President John Kennedy about this flight. He told them his idea of an “open ended” mission in which they would keep him in orbit indefinitely until there was a malfunction or consumables began to run out. Webb stated (and Kennedy agreed) that it was more important to shelve the Mercury spacecraft in order to jump start the more capable Gemini Program. Thus, we now have this whole Mercury on display for future generations to appreciate.

Bessie Coleman by John de la Vega 

Bessie Coleman (January 26, 1892 – April 30, 1926)  was the first black woman to earn a pilot’s license. Because flying schools in the United States denied her entry, she taught herself French and moved to France, earning her license from France’s well-known Caudron Brother’s School of Aviation in just seven months. Coleman specialized in stunt flying and parachuting, earning a living barnstorming and performing aerial tricks. She remains a pioneer of women in the field of aviation.