CSS Button No Image Css3Menu.com

Baseball Prospectus home
Click here to log in Click here for forgotten password Click here to subscribe

<< Previous Article
Premium Article Kiss'Em Goodbye: Pitts... (09/19)
Next Article >>
Premium Article Divide and Conquer, NL... (09/19)

September 19, 2011

BP Feature

Reviewing "Behind the Seams: The Stat Story"

by Derek Carty

When I volunteered to review “Behind the Seams: The Stat Story”—an MLB Network special on statistics and their place in the game that airs tonight at 10 p.m. EST—I wasn’t sure what to expect. When a big-name mainstream entity tries to talk about stats, it’s often overly simplistic, incorrect, or simply misses the point; ESPN’s stat segments on “Baseball Tonight” are often an example of this. And when I heard Bob Costas was hosting—remember, this is the same guy who, along with Buzz Bissinger, essentially ambushed Deadspin’s Will Leitch just a few years back on Costas Now—I really wasn’t sure what to expect.

Within the first five minutes—which included a couple minutes of introductory fluff—we hear Bob Costas utter the words “Numbers can only tell you so much. For instance, how do you rate a player’s standing from one era to another based solely on stats?” Uh-oh. I’m wondering if I’m in for a long hour, and I’m wondering if this will delve at all into the world of advanced statistics or even acknowledge the internet baseball community, since we actually have a pretty good solution to the “limitation” Costas cites in the form of stats like ERA+ and OPS+.

We’re then introduced to a parade of stat detractors who, like most stat detractors, just don’t get it. Steve Finley tells us how stats “[don’t] always tell you the story of how well you played as a player that year.” Maybe if we’re looking at seasonal totals we won’t be able to tell if you hit the game-winning home run to propel your team to the playoffs, but they’ll certainly give us a pretty darn good idea about how much value you had as a player.

Lou Brock—who, in the interest of fairness, makes sure to tell us that he was a math major—says, “We can make those things work any way we want.” Then the hammer is brought down in the form of opinionated old-school manager Tommy Lasorda, who flat out says, “Statistics are lies. Believe me; they’re lies.”

This misses the point entirely. Sure, I can find some random statistic or split that makes Jo-Jo Reyes look like a superstar, but it’ll only make him a superstar if you lack proper understanding, if you view it in the improper context. Sabermetrics, as Bill James famously said, is “the search for objective knowledge about baseball.” If you understand the limitations of numbers and how to properly treat them, you’ll be fine. Our job as baseball analysts is to find the truth in baseball. It’s not to warp statistics to fit whatever point we want to make. Sure, you can do that and fool the uninitiated, but to what end? Our job as baseball analysts is simply to find the truth and explain it, and statistics are a big help in this.

Bill James appears on screen at one point and says, “The statistics by themselves are never interesting. They are interesting only when you have a question that you’re trying to find the answer to.” And that’s exactly it. Stats will only tell you what they can tell you and nothing more. If you’re trying to get them to tell you something they can’t, then you’ll be fooled; then they’ll be lies.

Once we get past this little detour—which is much smaller than I’m sure I’ve made it out to be—“The Stat Story” actually levels off pretty well, attempting to shed a positive light on statistics and telling the story of how statistics have impacted the game of baseball. In fact, after it was all said and done, our Finley/Brock/Lasorda spectacle became much more of an afterthought, almost included to say, “OK, there are still people who don’t agree with all this, but it’s here, it’s undeniable, and here is the impact it’s having on front offices, Hall of Fame and awards voting, and the way fans view the game”. All in all, despite my wariness after the first few minutes, the documentary wound up being fair.

In fact, one of the first such things I noticed and really appreciated was that many of the interviewees were prominent internet writers. Of course there’s excellent work being done behind closed doors in front offices, but much of the modern day sabermetric movement is taking place on the internet at places like Baseball Prospectus, The Hardball Times, Beyond the Boxscore, etc. I wasn’t sure if this would be acknowledged in the documentary, and while specific mention was scarce, many of their authorities came from our little corner of the internet: former BPers Jonah Keri and Joe Sheehan, SBNation’s Rob Neyer, The Baseball Analysts’ Rich Lederer, FanGraphs’ David Appelman, and Retrosheet’s David Smith.

I was also very excited to see a segment—albeit a short one—on fantasy baseball. For a long time, fantasy baseball has had a stigma attached to it. While it still does to some extent, fantasy is becoming more and more acceptable, and this documentary aids in that. At one point, Major League Baseball’s vice president of stats Corey Schwartz says, “The average fantasy baseball fan knows more about what’s going on in baseball right now than a non-fantasy player.” The truth of the matter is, the game of fantasy baseball is good for the game of baseball. While I’m primarily a fantasy analyst, I also consider myself to be a real baseball analyst. Yes, there are differences between the two, but there’s also a lot of crossover. I thought it was great that several of the interviewees were fantasy players themselves, even if they have ties with non-fantasy pursuits in the world of baseball. Schwartz plays in Tout Wars, Sheehan participated up until this year, Keri has won multiple LABR championships, and Daniel Okrent was one of the inventors of the game.

Something I found interesting about the documentary was that there was very little talk of the stats themselves; it was more the story about the stats (which I guess I should have surmised from the title). Aside from Jonah Keri’s explanation of why batting average is flawed yet still manages to hold a prominent place in the game, there was little to no analysis of what makes a good stat and which stats are useful, with specific advanced stats receiving passing mentions at most.

If I were to criticize one thing about “The Stat Story,” it would be that they would often give an example of how stats were used throughout history, but it would be an incorrect use of stats and would be difficult to tell if the purpose was simply to show the evolution of the way stats have been used (which would be perfectly fine) or to illustrate ways in which stats can be helpful and have been helpful to teams who used them throughout history (which would not be fine, in the cases where the cited statistical analysis is flawed). For example, it talks about how Branch Rickey noticed that Jackie Robinson hit better with runners on base, so the next season he was moved to the cleanup spot, racked up RBI, and won the MVP. But a hitter’s single-season performance with runners on base is always too small of a sample to draw anything meaningful from, and of course a hitter is going to get more RBI in a better lineup position, regardless of whether he hits better with runners on. There was no clarification on points like these, the documentary rushing on and never again addressing the example.

One of the biggest things that casual observers of statistics—and, it seems to me, the makers of this documentary—fail to understand is the distinction between statistics and sabermetrics (words which the documentary used fairly interchangeably). Sabermetrics is the search for truth, which isn’t limited to numbers. I’ve long been a supporter of using qualitative information and was both honored and thrilled to have had the opportunity to attend MLB’s Scout School a couple years back. Scouting is a part of sabermetrics. On the quantitative side of the coin, sabermetrics isn’t just about using numbers; it’s about analyzing which numbers are useful and how to properly use them.

Statistics, on the other hand, are composed exclusively of numbers and, in the improper context, can be misconstrued. In this documentary, it mentions how Earl Weaver used batter/pitcher matchup data to set his lineups, with one of the interviewees chiming in, “And it worked!” But batter/pitcher matchup data is essentially useless. Yes, it’s statistics, it’s numbers, and it’s quantifiable… but it’s not sabermetrics.

The documentary’s greatest strength was in its storytelling. While it may have taken some liberties with declaring certain questionable statistics-based strategies successful, the story itself was engaging. The evolution of stats is something that interests me as a fan of baseball, and I found this documentary’s presentation of that story to hold my attention quite well. What I particularly appreciated was the way the story of statistics in front offices was presented. While many fans believe that the A’s were the pioneers of using advanced analytics in major-league front offices (and I’m sure many more will be of that belief come next week following the premiere of Moneyball), I applaud the documentary for noting and going into some detail about how the A’s were not the first. Yes, what they did was spectacular because of how small their budget was, but there were teams attempting to analyze stats as far back as the 1940s and some that were doing a pretty good job of it in the ‘80s and ‘90s.

All told, I believe the documentary is worth watching. There were some stories that Baseball Prospectus readers are likely to know—like how Bill James started the sabermetric movement or how Billy Beane became the poster boy for sabermetrics in front offices—but there were other anecdotes that you may not be entirely familiar with—such as how the White Sox were using a system that sounded remarkably similar to the Indians’ DiamondView as far back as 1981 or how the Red Sox used Tom Tippett’s Diamond Mind Baseball simulator to help them win the 2003 ALDS.

 “Behind the Seams: The Stat Story” presents a fair case for the place of statistics in baseball, if one that is somewhat lacking in understanding. However, I do believe that it’s worth your time if for no other reason than for the story, which was plenty engaging and interesting.  

17 comments have been left for this article. (Click to hide comments)

BP Comment Quick Links

Ric Size

Bring sabermetrics into the mainstream is a long process. Any attempt by ESPN to give a fair look at the subject is welcome, in spite of its limitations. I look forward to watching. Thanks for the heads up!

Sep 18, 2011 06:02 AM
rating: 2
BP staff member Derek Carty
BP staff

Agreed. This one's MLB Network, though, not ESPN, if you're looking tonight :)

Sep 18, 2011 09:31 AM

That is an excellent point about the A's not being the first team to attempt to use these new statistics. I was shocked when I found this, but Branch Rickey, a manager even "older school" (and it isn't even close), wrote a piece in 1954 on new statistics. He advocated the use of on base; isolated power, which he called power quotient; and even the general uselessness of fielding percentage as a measure of defense. Not bad for a guy who was in the Hall of Fame ten years before Tommy Lasorda began managing.

Sep 18, 2011 07:09 AM
rating: 3
BP staff member Derek Carty
BP staff

Yeah, the program spends some time talking about Rickey.

Sep 18, 2011 09:32 AM

Small sample sizes may be useless for sabermetric purposes. As actual decisionmaking aids, they can still be very useful. In a situation of uncertainty, you give me some evidence that A is so rather than B, that's darn useful. Of course Weaver's individual matchup data helped him. Some, certainly.

Sep 18, 2011 07:43 AM
rating: 1

Exactly. Match-up stats are far from useless. If an average veteran hitter is 5-for-6 with 2 homers against a reliever, that is extremely meaningful. It's "only" 6 AB's, but in the context of the *hitter's career* it's a stand-out of many thousand AB's.

Sep 18, 2011 09:10 AM
rating: -2
BP staff member Derek Carty
BP staff

In actuality, a 6 AB sample -- even a 5-for-6 AB sample -- is probably going to do more harm than good, in the absence of other data that supports it (scouting reports, etc). It's far more likely to be random variation than anything else, and if you play that hitter over one who is superior on the whole (taking into account platoon splits and such), it's likely going to be a mistake. There's a whole chapter devoted to this in Tango, MGL, and Dolphin's "The Book," and I took a quick look at it myself in an article at FanDuel earlier this year: http://www.fanduel.com/insider/2011/07/06/how-important-are-batterpitcher-matchups-part-1/

Sep 18, 2011 09:39 AM

Lasorda's just angry he can't ruin the arms of pitchers any more by making them throw 150 pitches and 260 IP per season.

Sep 19, 2011 05:15 AM
rating: 0

I saw the show, and overall it wasn't bad at all. I do wish they gave more than a few seconds to discussing Strat-O-Matic, cause Arthur Richman's story is worth telling.

Sep 19, 2011 07:02 AM
rating: 0

I of course meant Hal Richman.

Sep 19, 2011 07:25 AM
rating: 0
Cory Schwartz

Arthur Richman was a fine gentleman with a great story worth telling, too: http://www.nytimes.com/1999/02/07/sports/sports-of-the-times-richman-s-love-in-with-the-game.html?src=pm

Sep 22, 2011 08:30 AM
rating: 0

As a Strat-O-Matic player for many years, I'll echo dianagram's comments. Trust me, playing Strat-O-Matic alone gives you the knowledge of in game management that is almost always consistent with what the real manager does. (Or doesn't, when the manager says "he had a hunch") Hal Richman's story is a great one, and the book is a must read for the Strat fan.

Sep 19, 2011 08:34 AM
rating: 1

So people who don't know how to use statistics think stats are useless. The problem comes from people like Steve Finley thinking that because they know how to hit a baseball and how to steal a base that they are equipped to answer a player value or base-running strategy question. I guess it's fair to say that athletic performance and humility don't go hand-in-hand.

It's like you have a chef and he assumes that because he's a great cook he should be able to run the restaurant. Sure, it's a great place to start, but there's more to it than making great food.

Sep 19, 2011 11:56 AM
rating: 3
BP staff member Derek Carty
BP staff

I think that's a great analogy.

Sep 19, 2011 12:19 PM
BP staff member Mike Fast
BP staff

In that analogy, sabermetricians also have to be careful about assuming they can run the restaurant without having more advanced cooking experience than being able to boil pasta. It's not impossible to do, but it does require extra effort to stay in tune with what the cooks are doing.

Sep 19, 2011 12:51 PM
Richard Bergstrom

That analogy is also why some people think (incorrectly at times) that baseball players make good TV analysts/broadcasters.

Sep 20, 2011 19:58 PM
rating: 0

Sorry, Derek, but your study buries the question underneath a broader but different one. I doubt Weaver ever figured, "lessee, Ralph's at career .260 versus Fernando, but Fred's at career .265; OK, in goes Fred!"

When an individual matchup shows extreme results in a limited sample, is that predictive? That's the action issue. Looking at non-extreme results detracts from that rather than shines light on it.

Sep 19, 2011 13:17 PM
rating: 1
You must be a Premium subscriber to post a comment.
Not a subscriber? Sign up today!
<< Previous Article
Premium Article Kiss'Em Goodbye: Pitts... (09/19)
Next Article >>
Premium Article Divide and Conquer, NL... (09/19)

Fantasy Article My Model Portfolio: Framing Decisions Around...
Fantasy Article Dynasty Dynamics: TINO Does Arizona, 2015
Pebble Hunting: The Case For Shaming the Cub...
Every Team's Moneyball: Texas Rangers: Short...
Every Team's Moneyball: Atlanta Braves: Shor...
Prospectus Feature: All Spin Is Not Alike
Premium Article Baseball Therapy: The Most Important Player ...

Premium Article Prospectus Hit and Run: Backing into the Pla...
Premium Article Divide and Conquer, NL East: The Importance ...
Premium Article Kiss'Em Goodbye: Pittsburgh Pirates
Premium Article Kiss'Em Goodbye: San Diego Padres
Premium Article Collateral Damage: Weakened Swings and Injur...
The Prospectus Hit List: Daily Hit List for ...
Premium Article Fantasy Beat: The Loney Island

2011-09-30 - Premium Article Playoff Prospectus: NLDS Preview: Brewers vs...
2011-09-26 - Fantasy Article Resident Fantasy Genius: Losing Speed
2011-09-22 - Fantasy Article Resident Fantasy Genius: What Sabermetrics M...
2011-09-19 - BP Feature: Reviewing "Behind the Seams: The...
2011-09-15 - Fantasy Article Resident Fantasy Genius: What's Wrong with L...
2011-09-12 - Fantasy Article Resident Fantasy Genius: Stockpiling Closers
2011-09-08 - Fantasy Article Resident Fantasy Genius: The Infield Fly Rul...

2011-09-19 - BP Feature: Reviewing "Behind the Seams: The...

2011-09-22 - Fantasy Article Resident Fantasy Genius: What Sabermetrics M...