May 23, 2017
How Long Can You Keep a Secret?
Baseball is a game of secrets and half-truths. All 30 teams employ a man whose entire job (well, most of it) is to stand there and dance around in code to relay instructions to the batter and runners. The pitcher and catcher have their own gestural language. After the game, players usually speak in a strange code in which they appear to be speaking English and answering questions, but they somehow don’t manage to say anything coherent at all. Then there’s the front office, where the secrets run so deep that depending on the day of the week, you might not be able to get the people there to admit that they are running a baseball team.
Secrets are important in the game. Teams have all sorts of things which they know that they prefer that no one else knows or—even moreso—that no one else even knows that they know. If that information is truly exclusive and if it’s important, then that secret can be worth a lot of money. If there’s a reason that the “analytics revolution” is now simply the “we all have an analytics department” statement of facts, it’s that an analytics department is a way for a team to potentially find a nugget of information that they can use to their advantage. It’s the same reason that teams have invested in all sorts of areas of knowledge.
The thing about secrets is that eventually, they get out. Ben Franklin famously observed that “three may keep a secret if two are dead.” As of this writing, there are 30 major-league teams.
How long does it take for a secret to make the rounds in baseball? One recent roster fad—the good-framing catcher—was once a secret super-power that teams sorta knew about, but few really realized how valuable (and worth chasing) it was. Some teams knew about it earlier than others, although knowing about it and doing something about it are two separate things.
Even once a secret gets out—the first public article quantifying pitch framing appeared in 2008—it took a while for the league to adjust, so there was a definite advantage to being the first to adopt the new technology. The now-famous example of the Rays signing Jose Molina (with his career .233/.282/.327 batting line), largely to take advantage of his otherworldly talents in framing the ball and provide them value well beyond what his contract actually paid, was an exercise in a team that hopped on the train early. They realized that a) this talent existed, and b) Molina was a good source of it, and no one else was paying attention to those facts.
Now, we hear about teams pricing that framing skill into their contract offers. Teams have begun training their catchers in the fine art of the frame and even moving players behind the plate if they feel they might have a good framing candidate. Now, framing is so passe that few bother to write about it. When Jason Castro got a three-year, $24.5 million deal from the Twins in the offseason, everyone nodded that while his batting line hasn’t been so great in recent years, he can still steal a few strikes. All the work that went into finding, then quantifying, then acting on that framing effect is now for naught. Everyone has it now.
But, let’s say that tomorrow, someone finds something interesting in one of the 30 MLB StatCaves out there. How long can they expect that advantage to last?
Warning! Gory Mathematical Details Ahead!
Let’s start with a strategic innovation that we know has completely saturated the league: the one-inning closer. The generally accepted theory is that the idea was “invented” in the late 1980s by A's manager Tony La Russa and his use of Dennis Eckersley (although as Rob Mains has pointed out, Pete Rose used John Franco in much the same way in 1987, before Eckersley really solidified his role in 1988).
How long did it take for the idea of having a pitcher pitch the ninth inning (and the ninth inning only) to record saves to overrun the league? First, here’s a graph of the percentage of saves from 1950-2016 that involved the pitcher getting three outs or fewer.
Saves were “invented” in the early 1960s and not formally adopted by MLB as a statistic until 1969, but any game that has an extant box score can be retroactively awarded a save. We can see that up until about the late 1980s, the majority of saves required more than three outs. In 1987, the number was 40.0 percent. Fourteen years later, in 2001, the number had more than doubled to 83.4 percent.
I looked to see, over that same period, what percentage of teams had 75 percent (or more) of their saves recorded with three or fewer outs.
From the time of Franco and Eck, it actually took more than a decade to get to near-universal use of the strategy, although within the first seven years or so, more than half the teams in MLB were using it. It is debatable whether the advent of the one-inning closer is a net positive or a net negative for teams, but clearly teams have voted on which side they prefer.
Let’s pull a number completely out of the air and say that using the strategy was worth 1.0 win above replacement in an environment where no one else was doing it. Once everyone was doing it, there was no more advantage to be gained. If it took roughly 10 years for the strategy to be fully adapted and implemented league wide, and the rate of implementation was mostly consistent (a few teams picked it up each year, and it was roughly the same number in each of the years), that insight would be worth about 5.5 wins over the course of those 10 years (1.0 WAR + 0.9 WARP + 0.8 WARP + ...). Those might not be pitch-perfect assumptions, but they give us the correct order of magnitude.
Now let’s look at a couple of other strategic trends in baseball that have appeared and that we can easily observe in the data. One is the aforementioned catcher framing, which helpfully, BP has statistics on back to 1988 (though the numbers from 1988 to 2007 are calculated in a different way—using play-by-play data—than those from 2008 to 2016.) I took data at the team level on how many framing runs each team had gotten from its catchers and then took the standard deviation of that number. Here’s the graph from 1988 to 2016.
If teams were all getting roughly the same performance from their catchers in terms of framing, the standard deviation would be low. It seems that from the late 1980s onward, the standard deviation actually drifted ever higher, suggesting that teams were getting wildly different amounts of value from their catchers (and apparently, no one really noticed how much).
Here’s the same graph, just zoomed in on 2008-2016.
It’s not entirely clear when the “framing revolution” began to be implemented, and when teams went from just trying to grab the catchers who were good at framing to actively trying to increase the supply of good framers in the league (and kicking out the Ryan Doumit types). The numbers show that the common wisdom that the catcher market is now being flooded with good framers, decreasing the relative advantage of having a good one, is correct.
We see that in 2011, the standard deviation between teams was over 20 runs, but in 2016, it had fallen to 14 runs. There’s still a lot to be gained from having a good framer, although at this point, not as much as there once was. We don’t know what the natural low for the standard deviation is. Even if we assumed that the only thing that teams cared about in their catchers was the ability to frame (not true), and that they were out to find the 60 best practitioners of that craft, one of them would still be better than the rest, and there would be some variance.
But we can see that certainly there’s been a move toward teams striving to find (or make) their own good-framing catcher. More importantly, the key nugget of wisdom is that teams should factor this skill into their evaluations of catchers. If we mark the time from the original Dan Turkenkopf catcher-framing article to the end of the 2016 season, we get a roughly nine-year period.
Now, let’s look at shifts. Again, it’s not entirely clear that The Shift is actually a net positive for the defense, but the league keeps shifting more and more. The fact that the shifting rate for the league overall has gone up by an order of magnitude over the past few years makes this analysis a little tougher, but we can still make a graph. This time, I’m using the coefficient of variation, which is just the ratio of the standard deviation of a distribution to its mean.
I pulled the number of batters that each team shifted on, from 2010 to 2016 (the only years for which we have public data). The publicly available data is only plate appearances that ended up with a ball in play, so strikeouts, walks, hit by pitch, and home runs are not included. I took the league-wide mean and standard deviation for each year, and created the coefficient of variation, which is nicely mapped below.
We see that variation among teams was highest in 2012, which is to say that teams were further apart in the number of shifts that they employed. By 2016, teams had gotten more similar. In 2012, the most shifty team was the Rays with 517 shifts, while the White Sox brought up the rear with 46, meaning that the first-place team shifted 11.23 more times than the last-place team. In 2016, the Astros led all shifters with 2,052 shifts, a mere 3.40 times more than the last-place Cubs, who shifted 603 times.
Teams are getting more similar, and that coefficient of variance has been halved over the course of a mere five seasons, but teams are not completely in line with each other at this point. Maybe this year, we will see a further tightening. If The Shift truly is spreading across the league, we will. Teams all face the same (basic) set of hitters and if it makes sense for one team to shift against a hitter, it probably makes sense for the other 29 to do so as well.
Unlike catcher framing, where to increase the supply of good framers in the ecosystem a team has to locate and groom a good framer, increasing the number of shifts is (in theory) as simple as pointing the third baseman to a new spot on the diamond. Once the strategy reaches full saturation, the variation will probably reach down near zero. Again, if we’ve deleted half of the variation within five years and the trend line holds, then we are looking at a situation where everyone shifts as much as everyone else (roughly) in about 10 years.
I’ll gladly admit that some of this is slap-dash math, but I’d argue that we have a decent case that it takes about a decade for an idea to fully engulf Major League Baseball and become the new normal.
What’s a New Idea Worth?
I’ve previously lamented the problem of the “second-move advantage” in baseball research. It’s easy to start a sentence with “if a win is worth $9 million dollars ...” and end it with “then the brilliant idea I have is worth a cool million at least!” The problem is that eventually your idea gets out into the baseball ecosystem and eventually every other team can copy it without paying for the R&D costs that it took to come up with it.
That line of thought might be a little too pessimistic. Yes, a brilliant idea can be copied after a while, and eventually everyone will adopt it, but there’s an intermezzo in there that lasts a few years where even if a team doesn’t have a monopoly on the idea, they still have a relative advantage over some of the other teams in the game. And that’s still worth something.
If it takes a decade (or so, and yes there are error bars and I’m fine with that) for the full depreciation to happen, then a new idea is going to produce value, even if it’s diminishing value, for a while. It’s possible that I’m over-estimating that time, and that these sorts of big-ticket items take longer to fully make their way into the fabric of the game, but the point is that the process is not instant. It’s gradual and incremental, and the spaces in between are where the money is made.
Earlier in this article, I completely pulled a number out of the air and suggested that for an idea that produced a strategic advantage worth one win in the very short term for the team that had initially adopted it, we could expect that idea to produce 5.5 wins (or so, and yes there are error bars and I’m fine with that) of value over 10 years. Maybe it’s a little less. Maybe it varies by the type of idea. But in that delay is value.
It’s hard to come up with an idea that’s worth one win, but we can start to mold a math equation around how much value an idea worth X short-term wins would produce, accounting for the depreciation that comes from the fact that in a few years, everyone will copy it. This also begins to show the value of competitive intelligence. Maybe a team isn’t the first to think of a new idea, but if they can identify what another team is doing and see the value in it, then they could be the second to jump on the bandwagon, and could gain value that way while everyone else catches up.
It probably also explains why general managers say so little about the fun things that are going on behind those office walls. After all, the longer they can hold their brilliant idea out from the light, the longer it will take until that “discover and adapt” process begins in the other 29 front offices. Silence is worth money. We need to accept a depressing conclusion that comes with this train of thought, too. In any business, sports included, you’re only as good as your last idea and your idea comes pre-installed with an expiration date on how useful it is.
But before we depart for the day, let’s look at this from one other angle. One baffling question that has never been quite fully answered is what a general manager (or more broadly, a front office) is worth. If we assume that a general manager’s job is to come up and implement strategies that make a team better, including coming up with cool ideas and not being the last one to take up other people’s good ideas once they inevitably leak out, we are now a little closer to a framework for figuring out the sum total of those contributions.