September 22, 2011
Resident Fantasy Genius
What Sabermetrics Means to Me
On Sunday, I posted my review of “Behind the Seams: The Stat Story”—an MLB Network special on statistics and their place in the game. Despite being published on a Sunday, the article received a lot of attention, and I received a number of e-mails and phone calls from people with varying opinions on the piece, several of which wound up as discussions on sabermetrics in general. Given this and what’s sure to be widespread misunderstanding after Moneyball is wide-released tomorrow, I wanted to take today to talk about what sabermetrics means to me.
In my review of MLB’s documentary, I mentioned how there is an important distinction between sabermetrics and statistics that I wanted to expand upon today. For me, sabermetrics can be broken down into two components: statistical analysis and scouting.
I recently read a quote from Brewers’ GM Doug Melvin that will be useful in highlighting this difference:
You're trying to determine what they'll do this year based on what they did last year," he said. "Who is going to be the best pinch hitter? Last year, Joe Inglett had the most pinch hits in baseball. He's not even in the game now.
This is an example of statistics but not sabermetrics. You see, sabermetrics not only deals with statistics, but it deals with them within the proper context. It determines which statistics are worthwhile. While statistics tell us that Joe Inglett had the most pinch-hits in baseball last year, sabermetrics tells us that simply looking at a single-year leaderboard for a raw counting stat is a terrible way to judge a hitter’s pinch-hitting ability (and Melvin says as much). Sabermetrics would tell us that the proper way to predict a player’s pinch-hitting ability would be to create the best possible projection for the player’s overall talent level (including context adjustments, multiple years of data, regression, weighting, aging, scouting, etc.) and then applying the proper pinch-hitting adjustment to that (supplemented by whatever applicable scouting data you may have).
The same goes for hitting with runners in scoring position. Sure, Cantu had the most total hits, but that completely ignores how many chances he had to get those hits, not to mention the fact that even as a rate stat, a single year’s worth of data for a player with runners in scoring position is an incredibly small sample riddled with random variation—so much so that it essentially renders the data useless.
You see, sabermetrics is not simply looking at a statistic, taking it at face value, and drawing a conclusion. In the documentary, Lou Brock said, “We can make those things work anyway we want.” And he’s pretty much right. Using statistics, we could make a guy like Jorge Cantu look incredible if we really wanted to (hey, he had the most hits with runners in scoring position in all of baseball two years ago!). But we don’t do that—or, least, sabermetrics doesn’t. Sabermetrics is about finding the truth, not about spinning information.
Scouting is particularly important for players at the lower levels of the minor leagues. Because of the differences in competition levels, stats become less and less important the lower you go in an organization. For example, a pitcher could get by with a good fastball and no quality secondary offerings at Low-A and his numbers would look great, but that simply won’t play at the major-league level, which scouting would pick up on. Melvin echoes this sentiment as well:
The minor leagues are the development process. You have to know when a player's statistics are the result of him working on something.
The higher up you go, the closer the talent level approximates that of major-league players and the more use stats have. But scouting will always be important. While it’s the least important at the major-league level (relatively speaking), scouting is still an important component. Maybe a hitter is in a slump that a statistician would chalk up to a small sample size, but the scout may see a change in his swing that needs correcting. A scout may also notice a pitcher’s mechanics changing due to fatigue, which could lead to an injury. Scouting can even help inform our statistical models, creating, for example, more individualized means to regress to or more individualized aging curves based on the type of player in question. There are infinite uses for scouting, and it’s important for us to understand and appreciate this.
Additionally, it’s important to remember that the book (and the movie even more so, in all likelihood) took some liberties with what actually happened. It’s meant to entertain; it’s not meant to be a blow-by-blow recap of actual events. The A’s weren’t quite as anti-scouting as the movie will make it out to be, and regardless, scouting has (and deserves) an important place in the game of baseball. Similarly, statistics aren’t as all-encompassing as the movie will make them out to be and need to be viewed in the proper context. Theo Epstein once said that stats and scouting are two lenses of the same pair of glasses, and that pair of glasses is sabermetrics. He has it spot on.