“On the other hand, if Rod Carew asked me to check the ball, I didn’t even glance at it. That ball was gone. If Rodney didn’t like it, I didn’t like it. That’s one of the reasons we were such a good hitter.”

–The late Ron Luciano, former umpire, in his 1982 book The Umpire Strikes Back.

Bias in sports officiating isn’t a topic to be taken lightly, and one need only recall the recent furor over a New York Times article written by Alan Schwarz, where he reported a study on racial bias in the officiating of NBA Games. But as discussed in this space a month ago, PITCHf/x data does give us a limited window into asking questions about how players are treated by umpires; today, we’ll continue trekking through this new world and see what we can learn about pitchers, hitters, and the umpires who like them, like Ron Luciano with Rod Carew.

The Catfish Effect?

It has never been seriously suggested that racial bias plays any role in the calling of balls and strikes. But the notion certainly exists that the performance or stature of the player may influence the outcome. The better the player, the larger the benefit of the doubt. I first recall reading about this idea when Luciano humorously discussed it in The Umpire Strikes Back where he had the following to say about pitchers:

During a game an umpire gets into a groove with a pitcher. People like Catfish Hunter and Ron Guidry are always going to be around the plate, so an umpire gets into the habit of calling strikes. Even when they miss the plate, it’s usually a situation pitch intended to setup the batter for the next pitch or entice him to swing at a pitch outside the strike zone that he can’t hit solidly. The umpire becomes so used to calling strikes that it’s difficult to call a ball. Strike one, strike two, foul ball, it’s close to the plate, strike three.

Then there are pitchers like Ed Figueroa. He was all over the place. One pitch would be high, the next pitch would be in the dirt, the third pitch would be in the concession stand. He would throw three pitches outside the strike zone, then nip the corner of the plate by a quarter inch and expect the umpire to be ready to call a strike.

As we discussed a month ago, we can use the pitch location data provided by PITCHf/x to calculate how frequently a pitch that is called a strike probably was a strike, and how often a called ball was really out of the strike zone. With over 43,000 pitches to look at–twice as many as reported last month–we find the following totals:

Type          Pitches   Agree     Pct
Called Strike   14555   11820   81.2%
Ball            29377   27728   94.4%
Total           43932   39548   90.0%

So umpires are essentially getting it right nine times out of 10. It should be noted, however, that in making this calculation, we’re giving an inch benefit of the doubt to the pitcher, since that’s the reported granularity of the measurements. And although they should be few and far between, it’s also the case that the final position reported is taken as the ball crosses the front of home plate; there may be occasional pitches that catch the side or back of the plate and are therefore technically strikes and called that way. Finally, generally speaking, umpires call a strike zone that is slightly different in shape than the rule book interpretation used by the software in making these calculations. The actual shape of that strike zone is a topic for another day.

Before we look at individual pitchers, we need to determine if there is any difference between left and right-handed pitchers overall that would skew the results for individuals. The following table breaks it down; from it we can conclude that pitches from both sides are called in the same fashion by umpires:

Throws  Type          Pitches   Agree     Pct
Left    Called Strike    3344    2786   83.3%
Left    Ball             6963    6570   94.4%
Total                   10307    9356   90.8%
Right   Called Strike   10417    8379   80.4%
Right   Ball            20803   19634   94.4%
Total                   31220   28013   89.7%

Finally, we can now list the top 20 pitchers who have benefited the most and least at the hands of the men in blue. To do this, we’ll not only calculate the percentage of agreement on called balls and called strikes, but also a plus/minus number termed Adv (Advantage), which shows the aggregate number of pitches on which the pitcher benefited (actual balls called strikes minus actual strikes called balls, and so a negative number represents a pitcher who has not gotten an overall benefit) along with what the number is as a percentage of the total number of pitches thrown (%Adv). We’re also looking only at pitchers who’ve thrown 100 or more pitches captured by the system–since the system is still operative in fewer than half of the ballparks, that means we’re missing a substantial number of pitchers.

Pitcher           Throws  Pitches     Adv  %CSAgree  %CBAgree    %Agree      %Adv
Jason Davis       R           131      -7     0.968     0.920     0.931    -0.053
C.J. Wilson       L           233     -12     0.889     0.888     0.888    -0.052
Frank Francisco   R           147      -6     0.818     0.864     0.850    -0.041
Brandon Mccarthy  R           246     -10     0.847     0.879     0.870    -0.041
Alan Embree       L           136      -5     0.974     0.939     0.949    -0.037
Matt Desalvo      R           139      -5     0.914     0.923     0.921    -0.036
Scott Feldman     R           172      -6     0.805     0.893     0.872    -0.035
Joe Beimel        L           132      -4     0.956     0.931     0.939    -0.030
Jay Marshall      L           139      -4     0.961     0.932     0.942    -0.029
Cla Meredith      R           117      -3     0.971     0.952     0.957    -0.026
Javier Vazquez    R           382      29     0.688     0.961     0.872     0.076
Buddy Carlyle     R           167      14     0.758     0.990     0.904     0.084
Odalis Perez      L           107       9     0.756     0.985     0.897     0.084
Eric Gagne        R           103       9     0.636     0.957     0.854     0.087
Heath Bell        R           203      18     0.750     0.944     0.852     0.089
Jake Peavy        R           548      49     0.682     0.957     0.856     0.089
Tyler Yates       R           109      10     0.542     0.988     0.890     0.092
Justin Germano    R           247      23     0.735     0.972     0.874     0.093
Kyle Davies       R           279      30     0.634     0.978     0.864     0.108
Takashi Saito     R           149      18     0.610     0.944     0.812     0.121

By this measure, Jason Davis of the Mariners has benefited the least and Takashi Saito the most. And as you can probably tell from the percentages, in the aggregate pitchers (perhaps surprisingly) get more calls than hitters and in fact hold a 1,100-pitch advantage over those 43,000 pitches. From the table you can also probably guess that the variability in %CSAgree is higher than that for %CBAgree. In fact, it’s three times higher because the sheer amount of area outside the strike zone is much larger than the area within it, and so most balls are easier to call correctly than most strikes.

The crucial issue pertaining to Luciano’s comments is whether or not there is any pattern to the table above. Do pitchers who are around the strike zone get more calls than we’d expect?

Saito has walked just three men in 32 innings and Justin Germano eight in 54, while Davis has given up 23 free passes in 30 2/3 innings–all of which lend support to the idea of the “Catfish Effect.” Depending on how you look at it though, it could be that these pitchers’ good or bad fortune has resulted in their walk totals and is not to be credited to their command. On the other hand, Kyle Davies has walked 37 men in 78 innings and Tyler Yates 15 in 33 1/3. Further, perhaps the ultimate control pitcher, Greg Maddux (15 walks in 91 1/3 IP), ranks 64th out of 149 pitchers; it’s not obvious from the table that control pitchers are always or even primarily getting an advantage.

A more systematic approach is to calculate the percentage of pitches that were either swung at or were called strikes for each pitcher as an indication of a pitcher who is around the strike zone, and then run a regression against %Adv to see whether there’s any correlation. When this procedure was done for these 149 pitchers, the result was a very small positive correlation (r of .19) that was not statistically significant at the 95% level. In other words, at this point we can not conclude that as a general rule umpires immediately favor pitchers who keep the ball around the strike zone. To check whether it still might be the case that over the long haul umpires, like Luciano, do favor certain pitchers, %Adv was also correlated with career walk rate (to an r of .03) but, once again, no evidence was found of a statistically significant effect.

The Carew Effect?

Just as the common wisdom dictates that some pitchers gain an advantage from umpires, so too is it said about hitters, as Luciano again explains:

The situation is exactly the same with hitters. Rod Carew is a perfect example. He’ll have a three-ball, two-strike count on him and foul off six consecutive pitches. Then he’ll finally let one go. Well, if he swung at six and didn’t swing at that one, it’s got to be ball four. The great hitters get treated that way because they’ve proven they know the strike zone. If Ted Williams didn’t swing at a pitch, it was a ball, wherever it was…. Over a period of seasons the good hitters have proven they know a strike when they see one. So when Carew doesn’t swing at a pitch, I have to believe it’s off the plate. If a future Hall of Famer like Yastrzemski doesn’t think it’s a strike, is Ron Luciano going to argue with him? So if it’s close and the good hitters don’t swing, it’s going to be a ball.

Surely Luciano is using a bit of hyperbole for comedic effect, but you can’t argue that his reasoning doesn’t follow. But before we can test this question for hitters we need to account for the differences between right-handed and left-handed batters. Last month we showed that lefties seem to get the shaft to some extent, and that conclusion hasn’t changed with more data, as the following table shows:

Bats    Type          Pitches   Agree     Pct
Left    Called Strike    6277    4740   75.5%
Left    Ball            12672   12012   94.8%
Total                   18949   16752   88.4%
Right   Called Strike    8283    7084   85.5%
Right   Ball            16718   15729   94.1%
Total                   25001   22813   91.2%

Particularly where strikes are concerned, lefty hitters incur a 10% penalty over their right-handed brethren, while on called balls, they’re running neck and neck. As a result of this disparity, we’ll list the leaders and trailers in separate tables using the same definitions as those in the table for pitchers; the only exception is that Adv is inverted and calculated as actual strikes called balls minus actual balls called strikes.

Left-Handed Hitters
Hitters           Bats    Pitches     Adv  %CSAgree  %CBAgree    %Agree      %Adv
Jose Vidro        L           219     -26     0.608     0.964     0.836    -0.119
Willie Harris     L           164     -19     0.621     0.972     0.848    -0.116
Garret Anderson   L           121     -13     0.706     0.971     0.860    -0.107
Chipper Jones     L           160     -17     0.614     1.000     0.894    -0.106
Matt Stairs       L           187     -19     0.635     0.968     0.856    -0.102
Curtis Granderson L           112     -11     0.692     0.986     0.884    -0.098
Brian Mccann      L           201     -18     0.661     0.978     0.881    -0.090
Nick Markakis     L           116     -10     0.778     1.000     0.914    -0.086
Andre Ethier      L           249     -21     0.671     0.971     0.876    -0.084
Dan Johnson       L           334     -27     0.724     0.961     0.871    -0.081
Adam Dunn         L           105      -1     0.733     0.907     0.857    -0.010
David Dejesus     L           119      -1     0.851     0.917     0.891    -0.008
Terrmel Sledge    L           187      -1     0.868     0.955     0.930    -0.005
Russ Branyan      L           121       0     0.773     0.949     0.917     0.000
Jim Thome         L           277       1     0.821     0.938     0.910     0.004
Brad Wilkerson    L           222       2     0.868     0.918     0.901     0.009
Hideki Matsui     L           109       1     0.854     0.897     0.881     0.009
Hank Blalock      L           154       2     0.811     0.923     0.896     0.013
Johnny Damon      L           109       4     0.894     0.855     0.872     0.037
Carl Crawford     L           107       4     0.917     0.928     0.925     0.037

For left-handed hitters, poor Jose Vidro has seen 26 out of 219 pitches, or 12%, go against him. As is the case for pitchers, the magnitude of the balls called strikes is much larger than the reverse. For Vidro there were 79 pitches called strikes, only 48 of which we calculated as actually having been strikes. Carl Crawford, on the other hand, saw only two strikes out of the 24 called go against him. The inclusion of some shorter players in Vidro, Willie Harris, Matt Stairs, and Brian McCann at the top of the table and Adam Dunn, Russ Branyan, and Jim Thome in the bottom might at first lead one to believe that the size of the hitter’s strike zone makes a difference. Unfortunately for that theory, an analysis shows there is no correlation between height and %Adv for either left- or right-handed batters.

Right-Handed Hitters
Hitters           Bats    Pitches     Adv  %CSAgree  %CBAgree    %Agree      %Adv
Albert Pujols     R           103      -9     0.686     0.971     0.874    -0.087
Matt Diaz         R           121      -8     0.667     0.966     0.884    -0.066
Ty Wigginton      R           106      -6     0.767     0.987     0.925    -0.057
Elijah Dukes      R           126      -7     0.694     0.956     0.881    -0.056
B.J. Upton        R           142      -7     0.804     0.979     0.923    -0.049
Derek Jeter       R           107      -5     0.536     0.899     0.804    -0.047
Dustin Pedroia    R           130      -6     0.836     0.960     0.908    -0.046
Mike Piazza       R           100      -4     0.829     0.969     0.920    -0.040
Magglio Ordonez   R           103      -4     0.839     0.986     0.942    -0.039
Marlon Byrd       R           145      -5     0.786     0.921     0.869    -0.034
Jason Phillips    R           124       1     0.865     0.931     0.911     0.008
Aaron Hill        R           336       3     0.879     0.923     0.908     0.009
Vernon Wells      R           297       3     0.837     0.919     0.896     0.010
Michael Napoli    R           250       4     0.855     0.904     0.888     0.016
Richie Sexson     R           349       6     0.839     0.918     0.897     0.017
Kenji Jojima      R           180       4     0.772     0.862     0.833     0.022
Shea Hillenbrand  R           207       5     0.887     0.904     0.899     0.024
John McDonald     R           152       5     0.912     0.895     0.901     0.033
Gary Sheffield    R           125       5     0.844     0.892     0.880     0.040
Reggie Willits    R           135      10     0.980     0.872     0.911     0.074

While the list of leaders and trailers for left-handed batters seems agnostic on the question of whether better hitters are getting better calls, the list of right-handed hitters seems to go against the idea. Albert Pujols, Derek Jeter, Mike Piazza, and Magglio Ordonez all have good reputations and, as a whole, are doing well this season. Yet they find themselves at the top of the list for those most justified in criticizing the umpires. One would think that at least the great Pujols might be seeing some advantage if one existed.

Both lists, however, seem to provide some promise that perhaps more disciplined hitters (Dunn, Thome, Brad Wilkerson, Hideki Matsui, Richie Sexson, versus Matt Diaz, Elijah Dukes, B.J. Upton, and Garrett Anderson) are in fact picking up a few extra calls. As we did with pitchers, we can take a more systematic approach and run a regression on %Adv with the Fish metric on plate discipline we calculated in a previous column. What that analysis shows is that %Adv has no correlation with Fish (r of .05 for lefties and -.025 for righties) for either side. For now then, it would appear that the current season’s plate discipline is having no effect on how umpires make calls.

But what if the “Carew Effect” kicks in as a result of an entire career’s performance and is not correlated with a single season? To check for this we can correlate %Adv with career walk rate; but once again, we find that there is no correlation between the two, and hence no evidence that umpires favor hitters in any systematic or statistically significant way.

Strike Three?

We’re left with an idea that pitchers with better command and hitters with better plate discipline will tend to get the benefit of the doubt from the home plate umpire. And with this limited set of data, that’s all we have at the moment. At this point we can find no evidence that umpires favor either.

Thank you for reading

This is a free article. If you enjoyed it, consider subscribing to Baseball Prospectus. Subscriptions support ongoing public baseball research and analysis in an increasingly proprietary environment.

Subscribe now

Dan Fox


You need to be logged in to comment. Login or Subscribe