SciTech Tuesday: Abraham Wald, Seeing the Unseen

Following up on last week’s election and polling post, let’s delve a little deeper into random sampling and the role of bias in statistical analyses.  Sampling bias occurs when data is collected in a way that decreases the chance that certain individuals are included in the sample.  The Literary Digest error in forecasting the 1936 presidential election was due to undercoverage bias: they polled Americans with home telephones service, excluding a large segment of the population (which just so happened to be more likely to vote for FDR).  For more information on election polling, check out last week’s blog.

During WWII, statistician Abraham Wald was asked to catalog the location of bullet holes on returning Allied aircraft and determine the best places to reinforce the planes with armor.  Wald developed the diagram below, shading the areas where bullets pierced the returning airplanes.

Wald’s diagram of bullet damage before (left) and after (right) the flight. Shaded areas indicate the locations of bullet damage to returning aircraft.

Naturally Wald’s supervisors concluded that the nose, wings, and fuselage were covered in damage and therefore needed more armor.  Instead, Wald suggested that the shaded areas did not need reinforcement and the Allies should armor the areas untouched by bullets.  Wald reasoned that aircraft returned home because neither the cockpit nor the tail sustained damage.  He recognized survivorship bias in the sample; the Allies were only sampling planes that completed their missions and returned home.  Abraham Wald saw what everyone else could not:  the aircraft that didn’t make it home.

Post by Annie Tête, STEM Education Coordinator at The National WWII Museum.