While looking toward the future with our comprehensive slate of current content, we'd also like to recognize our rich past by drawing upon our extensive online archive of work dating back to 1997. In an effort to highlight the best of what's gone before, we'll be bringing you a weekly blast from BP's past, introducing or re-introducing you to some of the most informative and entertaining authors who have passed through our virtual halls. If you have fond recollections of a BP piece that you'd like to nominate for re-exposure to a wider audience, send us your suggestion.
Baseball may seem best when you're 10 years old, but before you start building a time machine, take another look at Dan Fox's investigation of baseball's changing talent level over time, which originally ran as a "Schrodinger's Bat" column on January 18, 2007.
The golden age was first; when Man yet new,
No rule but uncorrupted reason knew:
And, with a native bent, did good pursue.
Unforc'd by punishment, un-aw'd by fear…
– Ovid, from his Metamorphoses (AD 8)
Throughout much of history humans have looked back on the remote past as a time of peace and plenty. As in the Roman poet Ovid's verse above, containing ideas that can be traced directly back to Hesiod writing in the late 8th and early 7th centuries BC, peace and harmony prevailed during this Golden Age, when the remote ancestors of the Greeks never aged and the earth brought forth food without effort.
That basic idea is repeated around the world. From the "First Time" of Osiris when abundance characterized the now-dry Egyptian landscape, to the "Reign of Saturn" when Jupiter's father ruled Italy, to the days when Krishna walked in India, to the rule of Quetzalcoatl and Viracocha in the Aztec, Mayan and Incan domains, and finally to our more familiar Garden of Eden.
It is only much more recently that humanity, aided in no small part by the Enlightenment and the insight of biological evolution, has adopted a more or less linear view of history that poses its own dangers. Under this view the future is generally bright and the lives of our ancestors are often viewed as "nasty, brutish, and short," a view C.S. Lewis labeled "chronological snobbery" when used to characterize more recent generations.
But be that as it may, the myth of the golden age in baseball has proven harder to shake. From Ty Cobb to Ted Williams to Joe Morgan (but notably not Casey Stengel), both the players and those who write about the game hearken back to the good old days when players were smarter, the level of play supposedly better with an emphasis on fundamentals, and when giants like Ruth, Johnson, and Lou Gehrig "roamed the earth," as the book of Genesis says. Williams himself summed up the view nicely in 1992:
Modern players are stronger, bigger, faster, and their bodies are a little better than those of thirty years ago. But there is one thing I'm sure of and that is the average hitter of today doesn't know the little game of the pitcher and hitter that you have to play. I don't think today there are as many smart hitters.
This week we'll review the arguments that show us, with all due respect to The Splendid Splinter, that there was, in fact, no golden age in baseball.
A Full House of Variation
Perhaps the most complete argument for the position that the general level of play in baseball has improved and not declined over time was articulated by the late paleontologist and baseball fan Stephen Jay Gould in his 1996 book Full House: The Spread of Excellence from Plato to Darwin.
In the book Gould devotes six short chapters to arguing that the disappearance of the 0.400 hitter, rather than documenting a decline in excellence, paradoxically records an ever-increasing level of play. Essentially his argument for why 0.400 hitters have become extinct can be broken down as follows:
One set of conventional explanations for the disappearance of the 0.400 employ the "tougher conditions" argument–that too much travel, too many night games, and too much publicity and media exposure have led to a lesser level of play. Those explanations don't hold water, as train travel and doubleheaders were likely every bit as exhausting as coast-to-coast flights, and night games offer an escape from the heat and, for some players, better visibility.
Another set of explanations (more promising in Gould's view) stems from "tougher competition" and includes better pitching (the development of the slider is often cited), better fielding (equipment and positioning), and better managing (defensive charting and the employment of relief specialists). However, if pitching and defense had gained the upper hand over hitting, then the fact would be evident from the fall of league batting averages as hitters struggled to keep pace. In fact league averages have remained relatively constant over time, and, as we all know, have actually jumped up in the last decade. Further, the powers that be have actively maintained some semblance of balance through subtle (changing the definition of the strike zone) and not so subtle (changing the height of the mound) rule alterations, the one primary exception being the prolific offensive environment ushered in by Babe Ruth that was not squashed due to the threat posed by the Black Sox scandal.
The idea that somehow today's baseball players, as opposed to athletes in all other sports, would be unable to keep up with players from a generation ago just doesn't make logical sense. The combined forces of the increasing population pool from which athletes are drawn (which in baseball includes African Americans since 1947, the influx of players from Latin America since the early 1960s and now the Pacific Rim) far outstrips the effects of expansion. Better training and medical care, a general increase in size and strength (the average player today is over two inches taller and 20 pounds heavier than the average player in the dead-ball era), and the improvement in absolute records in other sports such as running, swimming, and jumping, all point to the conclusion that baseball players as a group must be better athletically, and therefore able to outperform players from previous generations. Ty Cobb, Walter Johnson, Babe Ruth, and yes, even Ted Williams were the outliers in populations characterized by a lower general level of ability.
The relative stability of batting averages then allows for the possibility that 0.400 hitting has disappeared because the amount of variation in batting averages among players has shrunk. In other words, batting averages are distributed in a bell curve, and the spread of that curve has been restricted over time. To test this hypothesis Gould calculated the standard deviation of batting average over time for all regular players. What he produced was a graph like that shown below, which I updated to include data through the 2006 season for players with one or more at bat per scheduled game, and which uses the coefficient of variation (CV) rather than simply standard deviation in order to account for the small differences in league batting average across seasons.
As you can see from the graph, the CV shrinks over time. Gould, almost giddy with the regularity of the decreasing variation, concludes that his "general hypothesis is confirmed again: variation decreases steadily through time, leading to the disappearance of 0.400 hitting as a consequence of shrinkage at the right tail of the distribution." He also notes that this method reveals that while variation decreased rather rapidly and in lock step through the early years of the game, the rate of decline decreased in the twentieth century and began to stabilize after 1940. However, when viewed by decade, along with the CV for both slugging percentage and OPS, it's clear that the variation in all three seems to have declined over the past three decades as well (Gould's original data went to only 1983 or so), as shown below.
- Finally, Gould provides two arguments to support his contention that the shrinking variation documents a general improvement in play. First, he argues that complex systems improve when the best performers play by the same rules over extended periods of time. In his view, this permits the system to equilibrate and decrease variation. In essence the argument is that baseball has been around long enough that strategies for everything from defensive positioning and turning the double play, to fielding the bunt and pitching mechanics, to pitch selection and batting swings and stances have been refined to the point where they are increasingly becoming optimal, and therefore essentially standardized. In short, an elite player like Tony Gwynn "lacks the space for taking advantage of the suboptimality in others."
Although not discussed by Gould, Bill James echoes many of the arguments related to population and player size, while adding the increased ability to select good players within the population due to the development of the minor leagues, in his book The New Bill James Historical Abstract. In addition, James acknowledges the wider variation in past performance by applying a rudimentary "Time Line Adjustment" based on year of birth to his player ratings, since great players before 1950 "dominated their game to a greater extent than more recent players." Even that adjustment doesn't fully compensate. Only 34 of his top 100 players came to the majors since 1960.
This shrinking variation can also be seen at the team level in the decreasing variation in winning percentage, as shown in the graph below.
Secondly, as play improves the bell curve moves towards the "right wall" of human ability, causing the right tail to shrink and variation to decrease. There is a limit to what human bones and muscle can do (Sidd Finch excepted) and as play improves, the general population of players moves closer to that limit, or "right wall." The approach to that wall can more easily be seen in other sports, where incremental improvements in records such as the hundred meter sprint decelerate over time and eventually almost stop as we reach the limits of human performance.
In baseball, the human limits of what hitters can do in an environment where pitchers and fielders are also closer to the "right wall" will restrict the variation. This makes it ever more difficult for hitters to attain a 0.400 average as long as the mean stays in the .260-.270 range. Mathematically this can be illustrated by pointing out that in the 1920s players with two at bats per game were 2.7 standard deviations away from 0.400. By the 1990s, the analogous set of players were 3.98 standard deviations away, which explains why there were seven 0.400 hitters in the 1920s and of course none in the 1990s. Gould also supports this argument with an illustration of how fielding percentages (a more "absolute measure of changing excellence over time") have increased over time, particularly from 1870 through the 1960s.
Gould then summarizes by saying that hitting 0.400
is not a thing, but the right tail of a full house for variation in batting averages. As variation shrinks because general play improves, 0.400 hitting disappears as a consequence of increasing excellence in play.
Although this conclusion is generally accepted, and is supported by the data that shows decreasing variation, it doesn't tell us how much the level of play has improved.
Measuring the Rising Tide
Fans of Baseball Prospectus, especially those who have read Baseball Between the Numbers, will no doubt find many of Gould's arguments familiar. In the introduction to the book, titled "Batting Practice," Nate Silver discusses the idea that players of today are better for many of the same reasons discussed here. He then introduces a procedure called the "Baseball Time Machine" which uses EqA to compare players who remain in a league from one season to the next. This calculates a league difficulty factor normalized for the 1975 American League.
What is produced is a graph that shows a steady rise in difficulty over time, with a small blip for World War II. It also shows the NL stronger than the AL from the mid 1950s through the mid 1980s, thereby confirming Gould's thesis. For example, the 1927 AL is credited with a factor of 0.846 while the 2005 NL is at 1.167. These factors are then used to translate statistics and illustrate how a Honus Wagner (EqA 0.354 in 1908) might translate to the modern game (EqA 0.232, or roughly a Neifi Perez). Interestingly, the graph shows the same stabilization in the 1940s that was noted by Gould, as the slope of the trend noticeably decreases.
A more subtle approach then employed by Silver — termed simply the "Time Machine" — intended to level the playing field a bit in terms of equipment, training, and medical advances (and one would assume styles of play and strategy), lays a simple trend line over the league difficulty factors since World War II, and then extends that line backwards through time. This results in a much more gently sloping line, and so treats Wagner and Ruth more kindly in their comparison to Barry Bonds. The general conclusion remains, however, that there has been a steady improvement in the level of play over time.
This same methodology was previously used in a study by SABR's Dick Cramer in the 1980 Baseball Research Journal, where he compared batting averages for players who remained in the league and produced a similar graph with a similar trend line as the league difficulty factors.
Some have argued, however, that this general approach will always show such a trend regardless of whether one really exists because of the twin problems of skills deteriorating over time and regression to the mean. The first problem can be adjusted for by applying an aging curve, while the second must take into account the fact that players who remain in the league and are selected for inclusion in studies like these do so because they receive at bats, and those at bats are predicated on performance that is more likely to be better than their actual talent.
Applying both adjustments has led Tom Tango, co-author of The Book, to conclude that the level of improvement has decelerated over time, particularly in the last 30 to 40 years, but that "Babe Ruth would not be BABE RUTH, but more like a great hitter."
But is this approach the only way to approach the question of measuring the change in level of play over time?
I got to thinking about this question this week while reading a comment from SABR member Stew Thornley related to the hitting feats of pitchers of old on the SABR list server, which was in response to the contention from another member that pitching is so highly specialized in the modern game that pitchers don't have time to learn hitting. Rather than look at the issue as one of practice, Stew pointed out that it is more likely that if pitcher's hitting relative to the league has decreased over time (an assumption on his part but one that appears to have been researched by others), it is likely due to an increasing level of play.
Pitchers are increasingly selected from the amateur ranks based on their extreme right-hand-tail-of-the-distribution excellence in pitching. While there is certainly some athletic and experiential crossover that allows them to hit better than the general population (as evidenced by the best players at early ages being both the best hitters and pitchers), their hitting skill is not selected for in the evolutionary sense and so should remain relatively constant over time. In other words, pitchers simply don't hit as well in the modern game, not because they are not just as skilled (or slightly more so) with the bat as their predecessors, but because the selected skills of all players have increased over time.
Incidentally, this is why it has been increasingly difficult for pitchers to transition to position players over time, a fact I documented in a 2005 article titled "Rube Bressler Redux?" which discussed the career of Rick Ankiel. The rarity of crossover skills, in the words of Thornley, is "something to celebrate in that it is an indication of how much better the players are" today. And rather than posit that the designated hitter is responsible for the decline in pitcher's hitting, under this view the increase in level of play can be seen as a contributor to the introduction of the designated hitter.
It turns out Thornley's intuition is correct. Pitcher's offensive output relative to that of position players has declined over time. In the following graph I show the OPS of pitchers (defined as players who appeared in more than one game as pitcher) relative to non-pitchers in all major leagues since 1871 using the pink line and the y-axis on the left side of the graph, and the percentage of plate appearances that pitchers consumed during those years on the gray line tracked on the y-axis on the right side of the graph.
What is immediately obvious from this graph is that pitcher hitting has followed a workmanlike decline over the years as depicted with the pink line, with the increase during World War II consistent with the idea that the level of play was lower as minor leaguers took the place of major leaguers in the service. The large blip around 1920 is partly accounted for by the presence of Babe Ruth (1915-1919, 1921) and George Sisler (1915,1916,1918) — who count as pitchers since they appeared in multiple games as pitchers during that time.
The gray line illustrates the increasing specialization of the game, which was accelerated by the moving of the mound back to 60 feet 6 inches from 50 feet in 1893, causing pitchers to play other positions less frequently. The appearance of the designated hitter also is prominent.
What is more difficult to discern is that, as Gould found with decreasing variation in batting average over time and Silver with the "Baseball Time Machine," the slope of the line changes around 1946, with pitcher hitting decreasing at less than half the rate it did before World War II. The same graph with the two distinct trend lines is shown below.
While there may be other interpretations of the trend, the underlying regularity suggests a systemic cause, such as a generally increasing level of play. Those interested in additional discussion on the difference in hitting between pitchers and non-pitchers will want to check out this blog post by Tom Tango.
The End of an Age
There may be those who still argue that players of the past were every bit as good as players today. For my money, however, the combined evidence of demographics, other sports, technology, shrinking variation, and direct (albeit relative) measurement make for an enormously difficult circumstantial case to overcome. And while we may long for a golden age somewhere deep in our soul, we can be comforted by the fact that we get the privilege of watching the game played at an unparalleled level of excellence.