To process my grief over the imminent end of the season, I’ve been reflecting on baseball’s lessons for instructional designers. (Stick with me.)
Baseball—distinctively among major American sports—is constituted by sequences of discrete, observable events. Pitch, swing, ground ball to short, throw to first: that’s a common baseball play of four successive events involving four unique players, each performing a single act with the ball which, in a sense, he has no choice but to attempt. (Baseball is more like tennis or golf in this regard than like basketball, a game constituted by complex, fluid, and contingent actions.)
It’s partly because the events of baseball are so innocently measurable that the game is so rich in useless statistics. I’ll pick on the worst, the pitching win. (The winning pitcher in a baseball game is the pitcher who records the last out prior to the half-inning when the winning team takes the lead for the last time. There are two exceptions to that definition that only make it worse.) The problem with wins, as data, isn’t that they’re obscure or indeterminate. If anything, they’re too easy to see. They are readily observable facts about pitchers that tell you nothing useful about pitchers’ effectiveness. You can pitch brilliantly and lose, or pitch poorly and win. Both of these things happen all the time.
Lesson: not every measure is a measure of value. Not every fact about what someone is doing is a fact about how well they’re doing it. In honest moments no one contends that the quantity of time spent staring at a screen could measure someone’s level of engagement, or that the number of multiple-choice questions answered could measure much other than itself. Yet we’re tempted to think—and too often willing to pretend—that they must measure something, just because they’re there for the taking.
Data is easy. Meaning isn’t. Remember your baseball.
One thought on “Cheap seat time”
What you say is true. But what is also true is that you must measure and then decide which measurement to look at for different choices you might need to make. With the abundance of data collection devices available, there is no reason to measure a lot. And then to put our skill and experience in using the right measurements to make a choice. Moneyball was an interesting experiment to use measurements to make interesting choices.