June 24, 2004
Testing the ''Mistake Hitter'' Hypothesis
Are Some Hitters Adept at Exploiting Bad Pitching?statistical evidence to the contrary, scouts, fans, and major league front offices continue to believe that some hitters are "clutch" and others are not. This is particularly evident in the playoffs, where the inability of a player with strong regular season statistics to hit in October is offered as evidence that the player is not "clutch," while other players are lauded for a few, well-timed base hits.
While there is no statistical evidence for systematic clutch hitting, however, it is still possible that some players do under (or over) perform in the playoffs, due to a tendency for "mistake hitting." Perhaps there are hitters who build their statistics up against bad pitching, but when faced with the quality pitching delivered in the playoffs, the holes in their game are exposed. Likewise, there may be players who do not have spectacular regular season numbers, but who have a solid batting approach that leaves them in an equally good position against low and high quality pitchers. The former type of player might be seen as "choking" in the playoffs, while the latter is seen as turning in a "clutch" performance.
"(Conine) can play first, play the outfield, he's not very athletic but he knows how to play the game. He can be pitched to, and he's a mistake hitter. Don't make many mistakes. He's a good clutch hitter, just not as athletic as others on club. He fits in good."One thing that is apparent in these quotes is the concept of "mistake hitting." According to the scouts, Conine is a mistake hitter. The implication is that Conine--by a greater degree than other hitters--benefits from getting a lower quality pitch to hit. On the other hand, Lowell's scouting report suggests that he is affected by good pitching to a lesser degree than Conine is.
The hypothesis makes sense. It is certainly conceivable that some hitters have holes in their swings that are easily exploited by a well-spotted fastball or a nasty hook. Since good pitchers throw good pitches more frequently than bad pitchers, it would follow that Lowell's production is affected less by facing a good pitcher than Conine's production, if the scouts are right.
So if this is the case, mistake hitters would be less valuable to contending teams. They would tend to wilt in the playoffs, when the quality of opposing pitching is generally high. General managers for contending teams would be best served to identify hitters who can hit good pitchers and stock his lineup with them to increase his odds in the postseason roll of the dice.
While the existence of these two different types of hitters makes sense, previous studies suggest that this is not likely to be the case. In one such article, David Grabiner showed that a hitter who can hit Triple-A pitching should be able to hit big league pitching. That is, the transition from facing minor league pitchers to facing major league pitchers affects all hitters similarly. If this is the case, it is possible that facing even higher quality pitching would also affect all hitters the same. But this proposition has never been publicly tested, and we propose to do so in this article.
Testing the Hypothesis
It's not that we don't trust the scouts on this one, but armed with the information we learned from Grabiner's article, we wanted to verify this "mistake hitter" hypothesis ourselves using cold, hard data. To do this, we gathered 1983-1992 data from Retrosheet.org on each at-bat for all major league players. We measured the "mistake differential" as the difference between batter performance against "good pitchers" and batter performance against "bad pitchers." We used OPS as a crude measure of output and OPS Against as a crude measure of pitching ability. If the pitcher's OPS Against was in the upper half of his league, he was--for the purpose of our study--a good pitcher. If he was in the bottom half, we considered him a bad pitcher. For these purposes, we measured starters against other starters and relievers against other relievers. Median OPS Against was actually a little bit better for relievers than starters, so a reliever had to be slightly more effective than a starter to be classified as "good." We also threw out appearances against starters who threw fewer than 100 IP and relievers with fewer than 35 IP.
Here is a list of the top and bottom five hitters in 1992 in terms of difference in OPS.
(OPS vs. good Ps) - (OPS vs. bad Ps) Kent Hrbek -.4479 Thomas Howard .2461 Mike McFarlane -.4164 Kevin Mitchell .2289 Reggie Sanders -.3910 Fred McGriff .2114 Jay Bell -.3848 Damon Berryhill .1991 Ricky Henderson -.3589 Willie Wilson .1779So in 1992, the players in the left column were the biggest mistake hitters. As for the guys on the right, we'll call them the professional hitters, since they hit good pitching better than bad pitching in 1992.
It is clear that in any one year there are large differences in this mistake differential. But the real test of this hypothesis is whether these differences persist over time. Are there players who systematically overperform against bad pitchers and underperform against good pitchers? We performed this test by comparing the consistency over time for players in the extent to which they are mistake hitters.
We included a hitter in our analysis only if he had more than 100 plate appearances against good pitchers (that is, pitchers with an OPS Against below the median) and 100 plate appearances against bad pitchers in consecutive seasons. We calculated for each player their (OPS vs. good pitchers) - (OPS vs. bad pitchers) for as many years as Retrosheet had data for the 1983-1992 span. We then compared pairs of odd and even years in this span (e.g., 1983 vs. 1984), and asked: Are players who are mistake or professional hitters in one year likely to be mistake or professional hitters in the next? That is, is there a statistical correlation between mistake differentials across years? If so, then it suggests there are systematic differences in the ability of players to hit good vs. bad pitchers that could be exploited by major league front offices.
The answer to this question is a resounding no. There is absolutely no correlation in mistake differential across years. Those who have a high mistake differential in one year are no more likely than average to have a high mistake differential in another year. Remember our top/bottom five mistake hitters from 1992? Here's how they did in the previous year according to mistake differential:
(OPS vs. good Ps) - (OPS vs. bad Ps) Kent Hrbek .1329 Thomas Howard n/a Mike McFarlane n/a Kevin Mitchell -.0243 Reggie Sanders -.0044 Fred McGriff -.1490 Jay Bell -.3285 Damon Berryhill n/a Ricky Henderson -.0692 Willie Wilson -.0235Remember that in 1992 the hitters on the left were the mistake hitters and the hitters on the right were professional hitters in 1992. In 1991, the mistake differential of all these hitters looks to be pretty randomly distributed. According to mistake differential, Kent Hrbek went from being a player with one of the highest mistake differentials in 1991 to the lowest in 1992. On first glance, it looks like the mistake hitter hypothesis doesn't have much of a backbone.
More generally, the following data plot shows clearly the lack of relationship between mistake differential over time:
Each of these data points represents one of the players used in our project. The data is centered around (-.100, -.100), meaning that for the average player, facing a good pitcher instead of a bad one costs them about 100 points of OPS.
The best fit line for this graph produces a zero slope and a zero R^2 value, which means that the players who look like mistake hitters by the numbers (like Hrbek circa 1992) only look that way because of random chance. If mistake hitting were a repeatable skill, we would see clusters of data points in the upper right and lower left quadrants. This is clearly not the case: there is no such thing as a systematic mistake hitter.
This conclusion is not driven by the particular structure of our study. No matter how we slice the data, we consistently find evidence against the notion of a systematic mistake hitter. If we use OPS ratio instead of OPS differential, or if we slice the sample of pitchers into the top quarter and bottom quarter by OPS Against, we continue to get the same finding.
In spite of the many approaches we took to looking at this data, it looks like the scouts are wrong about Conine and Lowell. We weren't able to find even a little bit of evidence that supports the idea of mistake hitting. Like clutch hitting, mistake hitting in professional baseball appears to be a myth. A good hitter is a good hitter no matter who his competition is.
A special thanks goes out to Jonathan Gruber and Evan Taylor, from the Massachusetts Institute of Technology, for their contribution to this article.
Cliff Roscow is an intern for Baseball Prospectus and a starting pitcher for the M.I.T. varsity baseball team. You can contact Cliff at firstname.lastname@example.org.