March 11, 2013
Maybe I'm Wrong
Well, this year's Sloan Sports Analytics Conference has come and gone, and I wasn't able to attend. Worse, I couldn't go to the SABR Analytics Conference either. Of course, I've followed along as best as I could, but there's no substitute for actually being in the room.
With analytics having their big moment in the sun, and with the topic of how analytics fit into sports still something of an open cultural question, there have been a few writers who have considered that intersection and written something about it. Over at SB Nation, Andrew Sharp wrote a review of the Sloan Conference (seems that he was in the room), which contained this excerpt:
If there's genius on display at Sloan, it's this: When scouts or coaches or old school GMs get something wrong, it's an example of traditional scouting methods failing. When analytics get something wrong, it's "randomness" that you can't control. A small part of a much bigger process, and teams and fans should trust that process until they get a better outcome.
This might be the most damning critique of sabermetrics (and sports analytics in general) I've ever seen. Worse, it might be true.
Let's first name what we're talking about. It's a well-known phenomenon in psychology called the self-serving bias. When something good happens to you, you will tend to see it as the result of your own hard work and talent. When something bad happens, you will tend to blame bad luck. When something happens to someone else, especially to a rival, those attributions are generally switched. This has been proven in the psychological literature about 500 times. It's everywhere. You will do it today, I promise. I will too.
If you're reading Baseball Prospectus, you're probably tempted to reflexively say "Sharp is wrong." I was too. But, if we're going to be intellectually honest, this critique deserves a more thoughtful answer. One thing that sabermetrics can pride itself on is that we've made great strides in analyzing baseball while minimizing the biases that go along with "the human element." If we're going to be good scientists, we can't let this bias bring us down. Otherwise, we're just cheerleaders for spreadsheets.