September 19, 2011
Reviewing "Behind the Seams: The Stat Story"
When I volunteered to review “Behind the Seams: The Stat Story”—an MLB Network special on statistics and their place in the game that airs tonight at 10 p.m. EST—I wasn’t sure what to expect. When a big-name mainstream entity tries to talk about stats, it’s often overly simplistic, incorrect, or simply misses the point; ESPN’s stat segments on “Baseball Tonight” are often an example of this. And when I heard Bob Costas was hosting—remember, this is the same guy who, along with Buzz Bissinger, essentially ambushed Deadspin’s Will Leitch just a few years back on Costas Now—I really wasn’t sure what to expect.
Within the first five minutes—which included a couple minutes of introductory fluff—we hear Bob Costas utter the words “Numbers can only tell you so much. For instance, how do you rate a player’s standing from one era to another based solely on stats?” Uh-oh. I’m wondering if I’m in for a long hour, and I’m wondering if this will delve at all into the world of advanced statistics or even acknowledge the internet baseball community, since we actually have a pretty good solution to the “limitation” Costas cites in the form of stats like ERA+ and OPS+.
We’re then introduced to a parade of stat detractors who, like most stat detractors, just don’t get it. Steve Finley tells us how stats “[don’t] always tell you the story of how well you played as a player that year.” Maybe if we’re looking at seasonal totals we won’t be able to tell if you hit the game-winning home run to propel your team to the playoffs, but they’ll certainly give us a pretty darn good idea about how much value you had as a player.
Lou Brock—who, in the interest of fairness, makes sure to tell us that he was a math major—says, “We can make those things work any way we want.” Then the hammer is brought down in the form of opinionated old-school manager Tommy Lasorda, who flat out says, “Statistics are lies. Believe me; they’re lies.”
This misses the point entirely. Sure, I can find some random statistic or split that makes Jo-Jo Reyes look like a superstar, but it’ll only make him a superstar if you lack proper understanding, if you view it in the improper context. Sabermetrics, as Bill James famously said, is “the search for objective knowledge about baseball.” If you understand the limitations of numbers and how to properly treat them, you’ll be fine. Our job as baseball analysts is to find the truth in baseball. It’s not to warp statistics to fit whatever point we want to make. Sure, you can do that and fool the uninitiated, but to what end? Our job as baseball analysts is simply to find the truth and explain it, and statistics are a big help in this.
Bill James appears on screen at one point and says, “The statistics by themselves are never interesting. They are interesting only when you have a question that you’re trying to find the answer to.” And that’s exactly it. Stats will only tell you what they can tell you and nothing more. If you’re trying to get them to tell you something they can’t, then you’ll be fooled; then they’ll be lies.
Once we get past this little detour—which is much smaller than I’m sure I’ve made it out to be—“The Stat Story” actually levels off pretty well, attempting to shed a positive light on statistics and telling the story of how statistics have impacted the game of baseball. In fact, after it was all said and done, our Finley/Brock/Lasorda spectacle became much more of an afterthought, almost included to say, “OK, there are still people who don’t agree with all this, but it’s here, it’s undeniable, and here is the impact it’s having on front offices, Hall of Fame and awards voting, and the way fans view the game”. All in all, despite my wariness after the first few minutes, the documentary wound up being fair.
In fact, one of the first such things I noticed and really appreciated was that many of the interviewees were prominent internet writers. Of course there’s excellent work being done behind closed doors in front offices, but much of the modern day sabermetric movement is taking place on the internet at places like Baseball Prospectus, The Hardball Times, Beyond the Boxscore, etc. I wasn’t sure if this would be acknowledged in the documentary, and while specific mention was scarce, many of their authorities came from our little corner of the internet: former BPers Jonah Keri and Joe Sheehan, SBNation’s Rob Neyer, The Baseball Analysts’ Rich Lederer, FanGraphs’ David Appelman, and Retrosheet’s David Smith.
I was also very excited to see a segment—albeit a short one—on fantasy baseball. For a long time, fantasy baseball has had a stigma attached to it. While it still does to some extent, fantasy is becoming more and more acceptable, and this documentary aids in that. At one point, Major League Baseball’s vice president of stats Corey Schwartz says, “The average fantasy baseball fan knows more about what’s going on in baseball right now than a non-fantasy player.” The truth of the matter is, the game of fantasy baseball is good for the game of baseball. While I’m primarily a fantasy analyst, I also consider myself to be a real baseball analyst. Yes, there are differences between the two, but there’s also a lot of crossover. I thought it was great that several of the interviewees were fantasy players themselves, even if they have ties with non-fantasy pursuits in the world of baseball. Schwartz plays in Tout Wars, Sheehan participated up until this year, Keri has won multiple LABR championships, and Daniel Okrent was one of the inventors of the game.
Something I found interesting about the documentary was that there was very little talk of the stats themselves; it was more the story about the stats (which I guess I should have surmised from the title). Aside from Jonah Keri’s explanation of why batting average is flawed yet still manages to hold a prominent place in the game, there was little to no analysis of what makes a good stat and which stats are useful, with specific advanced stats receiving passing mentions at most.
If I were to criticize one thing about “The Stat Story,” it would be that they would often give an example of how stats were used throughout history, but it would be an incorrect use of stats and would be difficult to tell if the purpose was simply to show the evolution of the way stats have been used (which would be perfectly fine) or to illustrate ways in which stats can be helpful and have been helpful to teams who used them throughout history (which would not be fine, in the cases where the cited statistical analysis is flawed). For example, it talks about how Branch Rickey noticed that Jackie Robinson hit better with runners on base, so the next season he was moved to the cleanup spot, racked up RBI, and won the MVP. But a hitter’s single-season performance with runners on base is always too small of a sample to draw anything meaningful from, and of course a hitter is going to get more RBI in a better lineup position, regardless of whether he hits better with runners on. There was no clarification on points like these, the documentary rushing on and never again addressing the example.
One of the biggest things that casual observers of statistics—and, it seems to me, the makers of this documentary—fail to understand is the distinction between statistics and sabermetrics (words which the documentary used fairly interchangeably). Sabermetrics is the search for truth, which isn’t limited to numbers. I’ve long been a supporter of using qualitative information and was both honored and thrilled to have had the opportunity to attend MLB’s Scout School a couple years back. Scouting is a part of sabermetrics. On the quantitative side of the coin, sabermetrics isn’t just about using numbers; it’s about analyzing which numbers are useful and how to properly use them.
Statistics, on the other hand, are composed exclusively of numbers and, in the improper context, can be misconstrued. In this documentary, it mentions how Earl Weaver used batter/pitcher matchup data to set his lineups, with one of the interviewees chiming in, “And it worked!” But batter/pitcher matchup data is essentially useless. Yes, it’s statistics, it’s numbers, and it’s quantifiable… but it’s not sabermetrics.
The documentary’s greatest strength was in its storytelling. While it may have taken some liberties with declaring certain questionable statistics-based strategies successful, the story itself was engaging. The evolution of stats is something that interests me as a fan of baseball, and I found this documentary’s presentation of that story to hold my attention quite well. What I particularly appreciated was the way the story of statistics in front offices was presented. While many fans believe that the A’s were the pioneers of using advanced analytics in major-league front offices (and I’m sure many more will be of that belief come next week following the premiere of Moneyball), I applaud the documentary for noting and going into some detail about how the A’s were not the first. Yes, what they did was spectacular because of how small their budget was, but there were teams attempting to analyze stats as far back as the 1940s and some that were doing a pretty good job of it in the ‘80s and ‘90s.
All told, I believe the documentary is worth watching. There were some stories that Baseball Prospectus readers are likely to know—like how Bill James started the sabermetric movement or how Billy Beane became the poster boy for sabermetrics in front offices—but there were other anecdotes that you may not be entirely familiar with—such as how the White Sox were using a system that sounded remarkably similar to the Indians’ DiamondView as far back as 1981 or how the Red Sox used Tom Tippett’s Diamond Mind Baseball simulator to help them win the 2003 ALDS.
“Behind the Seams: The Stat Story” presents a fair case for the place of statistics in baseball, if one that is somewhat lacking in understanding. However, I do believe that it’s worth your time if for no other reason than for the story, which was plenty engaging and interesting.