« Amazon and the Small Tech Company | Main | Fraud and Measurement »


Big Skills and Small Data

A common theme for us is the relationship between what we call "Small Data" and extremely smart and accomplished people in their work environments.  Small Data is a limited set of data that has been extensively tested and refined and then applied to a specific problem.  Our intuition might tell us that using small data-driven strategies to attack a problem in the environment of the most skilled practioners would not make sense.  To use David Halberstam's terminology: the Best and the Brightest

A current description of this paradox is Atul Gawande's book The Checklist Manifesto.  In his chapter "The Hero in the Age of Checklists," Gawande describes the concept of "expert audacity."  We sometimes have the conception that heroic action always produces the best outcomes on its own.  Somehow the idea of a great surgeon using a 19 item checklist doesn't fit.  It just wouldn't work on an episode of Grey's Anatomy.  But in some fundamental way, as Malcolm Gladwell points out, it's about humility.  Perhaps really, really great experts, e.g. Gawande, get small data and big skills together.

Gawande explains it much better than we ever could but we see the same phenomenon all the time.  In some conversations with prospective customers we feel compelled ask the question "You're the best and the brightest. Why would you need to do that?" Sometimes, as Gawande puts it, even in high skill processes there is a significant need for the "regimentation" of Small Data.

But perhaps the most interesting example is in the prediction of violent behavior.  The conclusion of the authors of Violent Offenders: Appraising and Managing Risk is that clinical judgment in predicting recidivism is not only of no use when combined with an actuarial assessment, but actually makes for poorer decisions.  Stated differently, the most highly trained pyschologist or psychiatrist cannot hope to match the predictive power of a 12-item checklist designed with historical data as its backing.  Assuming that the creational process of the checklist effectively covers the population it is applied to,  actuarial instruments that collect a relatively small number of data elements but have extensive data behind them are surprisingly effective.

Perhaps there are additional examples of this phenomenon.  Relatively simple statistics applied to baseball and basketball can make the difference in the outcome of a game.  Industrial Quality Control also can rely on very simple applications of data to make complex production create higher quality products.




Feed You can follow this conversation by subscribing to the comment feed for this post.

The comments to this entry are closed.

Contact Us