“I shouldn’t get so mad when I think there’s been a scoring mistake, but I do. You just want it to be fair to everybody involved, and I understand that’s hard. But you can’t forget that players in the major leagues should be expected to make major-league-caliber plays.”

–Hall of Famer turned announcer Tom Seaver

On some days official scorers have a tough job. I witnessed one of them from the press box in May of 2004, at a Royals/Twins game at Kauffman Stadium. In the bottom of the fifth, the Royals had loaded the bases with one out. Desi Relaford was perched on third, Angel Berroa on second and Carlos Beltran on first. What came next was a series of events that you’ll likely never see again.

The batter is Mike Sweeney, and he lofts a short pop-up behind first base. First baseman Doug Mientkiewicz, second baseman Michael Cuddyer and right fielder Jacque Jones all converge on the ball while first-base umpire Jeff Kellogg signals an infield fly. With Mientkiewicz’s back turned toward the infield, the ball hits his glove and falls to the ground as he and Cuddyer brush against one another. Kellogg signals that the batter is out. Meanwhile, Beltran has left first and gone halfway to second, and seeing the ball drop, continues to second. However, Berroa hasn’t moved from second, so both Beltran and Berroa are now standing within a couple feet of the base. Relaford speeds toward home from third as soon as the ball hits the ground.

Back in the outfield Mientkiewicz picks up the ball, whirls and throws it in the general direction of home plate, apparently in an effort to nab Relaford. However, the throw hits Sweeney square in the back as he stands near first base watching the action. Sweeney hits the ground in pain (“I thought I got shot” he said later) as the ball slowly rolls in the direction of the Royals dugout. By this time, Berroa is madly waving Beltran to get back to first, and convinced by Berroa’s theatrics, Beltran indeed starts heading that way. Catcher Henry Blanco retrieves the ball by the Royals dugout and rifles it to first, where right fielder Jones catches the ball and just beats a sliding Beltran, tagging him and touching the base for the second out of the play, and the third of the inning. The home-plate umpire signals the press box to indicate that Relaford’s run counts. The Twins, slightly confused, trot off the field.

Needless to say all of this caused a considerable amount of consternation in the press box for both the scorer and the media, as both the former and the latter scrambled to consult their rule books, and then offered several conflicting opinions as to whether Sweeney should be credited with a sacrifice fly and/or a run batted in, whether Mientkiewicz should be charged with an error, whether the Twins get credit for a double play, whether the run is unearned, and whether Beltran had to retreat to first.

In the end, after multiple changed rulings and mediation by the Elias Sports Bureau, the answer to the above questions were no and no on Sweeney’s sac fly or RBI (under rule 10.09), yes on Minky’s error (the error allowed the run to score), no on the double play (there was a “misplay” involved), yes as to the run being earned (since the run scored on the play with the third out), and no on Beltran’s having to retreat to first (he was at liberty to advance “the same as on any fly ball”). While there wasn’t a tremendous amount of room for interpretation here, few would argue that the play wasn’t difficult to assess and render judgment on.

I got to thinking about this play in the wake of an interesting question in last week’s chat that went as follows:

Dills (Chicago): How can we account for the differences in local “official” scorers in regards to defensive rankings? Too often, an error in, say, St Louis on a ball hit by a Padre would be a HIT in San Diego or vice versa. How can this possibly be accounted for? The E vs. HIT scoring will always be a subjective decision made in many cases by fairly biased scorers.

I’ll admit that at first I took issue with the assumption the part of the questioner that scorers were inherently biased. In my experience, the scorers I’ve worked with have acted professionally and strive to do the best job they possibly can. But of course the general perception among fans and the media alike remains, and although scorers don’t have as much influence as they do in at least one hilarious spoof, both players and coaches often get worked up when decisions don’t go their way.

This question, then, spurred me to thinking how one could measure scorer bias, if indeed it exists. So this week, we’ll ruminate a bit on that topic, and hopefully dispel any concerns that all I would write about this year was baserunning.

Bias, or Just Home Cookin’?

Over the past few years the performance-analysis community has made great strides in evaluating the contribution that fielders make. Augmenting simple counting statistics like putouts, assists and errors with considerations for opportunities has revolutionized the way in which we can evaluate the differences between a Derek Jeter and an Adam Everett.

Be that as it may, controversies over errors persist. Too often, the mainstream media uses errors as the sole basis by which fielders are judged. Since that is still the unhappy case (barring the completion of the Kuhnian revolution in baseball) the topic of bias and the assignment of errors by official scorers also remains problematic.

As many readers know, the official scorer for each game is a local person, often a retired sportswriter or sports information director from a nearby college, whose duties are defined in the last section (10.00) of the official rule book. What you may not know, and what was shared in a very informative article by Nationals scorer and SABR member David Vincent, is that up until 1980 the position was held by a newspaper writer. In a move to make the position less susceptible to conflicts of interest MLB began hiring independent contractors.

That move has seemingly done little to quash the oft-heard complaints of scorer bias. These fall into basically two categories revolving around the notion that, being from the home team’s metropolitan area, the scorer will inevitably make rulings and interpretations that benefit the home team. As a first pass we can define those categories as follows:

  • A scorer might be prone to crediting a hit to a player on the home team in an effort to boost his average, resulting in fewer errors for the visiting team than there otherwise would have been. For our purposes in this article we’ll call this “Home Team Hit” (HTH) Bias.

  • A scorer might be prone to withholding an error from a player on the local nine in an effort to spare him embarrassment or contribute to a reputation for poor fielding, resulting in fewer errors for the home team than there otherwise would have been. We’ll call this “Visiting Team Hit” (VTH) Bias.

It should be noted that in the case of VTH, one might speculate that there is also a mitigating bias in play where official scorers may shy away from crediting home-team pitchers with earned runs, and therefore in some cases award the home-team fielder with an error. One might assume, however, that giving errors to the home team would have the greater resonance because of its immediate consequence–after all, earned runs aren’t always counted immediately, so the potential exists to get out of the inning without giving up any earned runs in cases where a close play went in favor of giving the opponent a hit. Keep in mind though that it also complicates things when you consider that the home team probably should field better in the park with which they are the most familiar rather than on the road, and so would naturally commit fewer errors, thereby blurring any evidence of VTH, even if it exists.

You’ll notice that these two categories put us in the unfortunate position of not being able to easily quantify bias. In the case of HTH, we’ll see more hits for the home team, and in the case of VTH we’ll see more for the visitors. In other words, ceteris paribus, the two may cancel each other out by suppressing errors across the board, making it difficult to detect bias.

The consequence is that if both HTH and VTH biases exist, then in effect what we actually have is a bias against pitchers, and in favor of fielders and hitters. In my opinion, that is often the situation. Many are the times when I’ll assume a play will be ruled an error (invoking the “ordinary effort” clause of rule 10.13 in my mind’s eye while channeling Mr. Seaver), only to see the hit designation flash on the scoreboard. Announcers and team personnel also invoke a form of this argument when, as I witnessed recently, a member of a team’s public relations department pleaded his case to the official scorer by repeatedly citing the phrase “but this is supposed to be the Major Leagues!”

But still, we’d like to see what–if anything–the actual data tell us. To look at this question I examined the number of errors credited to the home and visiting team for each team and season from 2000 through 2006 (through games through September 15th of this season). In an effort to detect HTH and VTH I calculated the number of errors per game credited to each team at home and on the road and converted them into percentages. (It’s an imperfect method, I know, because the number of innings is not the same for both teams.) For example, to try to detect VTH, I found that the Los Angeles Angels of Anaheim at home were charged with 9% more errors than on the road, while visitors in Angels games were charged with 14% fewer errors than when in their own home parks–possibly reflecting HTH. Overall, either because VTH is stronger than HTH or the fact that home teams simply play better defense in their own parks, or perhaps a combination of both, teams were charged with 3% fewer errors at home over the seven season sample. This difference would be larger if we had used innings rather than games to calculate our rate, since visiting teams play fewer innings in the field.

From a team perspective then, here are the 31 teams (Montreal is included for 2000-2004) ordered by our VTH column.

2000-2006 Error Percentages
Team         VTH     HTH
CHA         -22%      0%
FLO         -20%     -4%
WAS         -16%     13%
MIN         -15%      0%
PHI         -13%    -15%
CIN         -12%     -5%
SEA         -10%     -5%
ARI         -10%     -7%
OAK          -9%      7%
KCA          -9%      1%
DET          -5%     10%
CLE          -5%     22%
SFN          -5%      4%
MON          -5%      1%
TOR          -5%     12%
TBA          -4%     10%
NYA          -2%      8%
BAL          -2%     -3%
CHN           1%      1%
TEX           2%      1%
COL           2%     27%
ATL           3%     12%
SLN           3%     -3%
MIL           3%     11%
PIT           4%      9%
HOU           5%     -4%
SDN           6%    -17%
LAN           7%     -3%
ANA           9%    -14%
BOS           9%     14%
NYN          18%     -2%

To reiterate, what this table records is that the White Sox were charged with 22% fewer errors at home than on the road, while their opponents were charged with errors at the same rate both in Chicago and in their home parks when playing the Sox. This could be interpreted as indicating that there is some VTH bias in play. In fact, the Sox were the only team where the percentage of errors at home was negative for all seven seasons.

On the other hand, the Mets were charged with 18% more errors at home, while their opponents were charged with 2% fewer when playing in New York. In terms of detecting VTH, had we reordered the above list we’d find that Padres opponents were charged with 17% fewer errors in San Diego than when playing in their own parks.

Obviously, we’re not dealing with a huge sample size, and complicating factors such as scorer turnover, scorers sharing responsibilities and home park influences all conspire to skew the numbers, making this effort somewhat dubious. To give you a feel for the number of scorers and an inkling as to their differences (I’m making no value judgments here about individual scorers because of the variability in the data and team defensive quality which this analysis doesn’t account for), the following table lists the scorers who have worked games in 2005 and 2006 for each team, along with a simple calculation of errors per game for both home and visitors:

Scorer             Team               G      HE      VE      H E/G   V E/G
Ed Munson           ANA             148     102     101       0.69    0.68
Mel Franks          ANA               7       3       1       0.43    0.14
Rodney Johnson      ARI              72      40      40       0.56    0.56
Gary Rausch         ARI              69      30      46       0.43    0.67
Tyler Barnes        ARI              15      15       6       1.00    0.40
Mark Frederickson   ATL              77      34      52       0.44    0.68
Mike Stamus         ATL              75      48      49       0.64    0.65
Tony Schiavone      ATL               1       0       0       0.00    0.00
Paul Newberry       ATL               1       1       0       1.00    0.00
Jim Henneman        BAL             118      71      72       0.60    0.61
Mark Jacobson       BAL              40      20      26       0.50    0.65
Charlie Scoggins    BOS              60      27      43       0.45    0.72
Mike Shalin         BOS              41      22      24       0.54    0.59
Joe Giuliotti       BOS              35      15      28       0.43    0.80
Ed Carpenter        BOS              13       6       8       0.46    0.62
Bob Ellis           BOS               4       4       5       1.00    1.25
Tony Massarotti     BOS               1       1       2       1.00    2.00
Bob Rosenberg       CHA             100      40      71       0.40    0.71
Don Friske          CHA              39      19      18       0.49    0.46
Scott Reed          CHA              16      12       8       0.75    0.50
Don Friske          CHN              71      56      41       0.79    0.58
Bob Rosenberg       CHN              57      42      41       0.74    0.72
Allan Spear         CHN              27      12      17       0.44    0.63
Glenn Sample        CIN             123      75      61       0.61    0.50
Ronald Roth         CIN              36      23      20       0.64    0.56
Chad Broski         CLE              47      27      35       0.57    0.74
Chuck Murr          CLE              47      46      29       0.98    0.62
Hank Kozloski       CLE              45      25      23       0.56    0.51
Bob Maver           CLE               8       7       8       0.88    1.00
Bob Price           CLE               6       3       2       0.50    0.33
Dave Einspahr       COL             108      63      86       0.58    0.80
Dave Plati          COL              42      30      35       0.71    0.83
Dave Moore          COL               2       2       2       1.00    1.00
Chuck Klonke        DET              75      46      56       0.61    0.75
Steve Lysogorski    DET              72      49      48       0.68    0.67
Ron Kleinfelter     DET               7       3       4       0.43    0.57
Ron Jernick         FLO             135      89      84       0.66    0.62
Doug Pett           FLO              21       9      18       0.43    0.86
Trey Wilkinson      HOU              43      31      18       0.72    0.42
Dave Matheson       HOU              39      18      24       0.46    0.62
Ivy McLemore        HOU              36      19      22       0.53    0.61
Rick Blount         HOU              35      22      20       0.63    0.57
Del Black           KCA              92      65      66       0.71    0.72
Will Rudd           KCA              52      34      37       0.65    0.71
Alan Eskew          KCA              11       8       6       0.73    0.55
Don Hartack         LAN              81      51      52       0.63    0.64
Ed Munson           LAN              72      50      49       0.69    0.68
Tim O'Driscoll      MIL             125      96      78       0.77    0.62
Wayne Franke        MIL              30      22      17       0.73    0.57
Tom Mee             MIN             147      81     105       0.55    0.71
Barry Fritz         MIN               4       1       3       0.25    0.75
Gregg Wong          MIN               4       5       0       1.25    0.00
Howie Karpin        NYA              68      49      52       0.72    0.76
Bill Shannon        NYA              58      36      36       0.62    0.62
Jordan Sprechman    NYA              22      11      15       0.50    0.68
Billy Altman        NYA               2       1       0       0.50    0.00
Dave Freeman        NYA               2       1       0       0.50    0.00
Howie Karpin        NYN              52      34      36       0.65    0.69
Bill Shannon        NYN              52      40      32       0.77    0.62
Joe Donnelly        NYN              25      13      19       0.52    0.76
Jordan Sprechman    NYN              21      19      19       0.90    0.90
Billy Altman        NYN               2       2       3       1.00    1.50
Dave Freeman        NYN               2       1       1       0.50    0.50
David Feldman       OAK              75      32      43       0.43    0.57
Chuck Dybdal        OAK              43      26      25       0.60    0.58
Al Talboy           OAK              34      15      19       0.44    0.56
Art Santo Domingo   OAK               1       0       0       0.00    0.00
Bob Kenney          PHI              88      48      66       0.55    0.75
Jay Dunn            PHI              34      23      18       0.68    0.53
Mike Maconi         PHI              28      16      16       0.57    0.57
John McAdams        PHI               5       2       3       0.40    0.60
Bob Hertzel         PIT              40      36      20       0.90    0.50
Bob Webb            PIT              39      26      34       0.67    0.87
Tony Krizmanich     PIT              38      23      25       0.61    0.66
Evan Pattak         PIT              37      19      21       0.51    0.57
Bill Zavestoski     SDN              94      64      54       0.68    0.57
Dennis Smythe       SDN              62      30      26       0.48    0.42
Eric Radovich       SEA              73      30      48       0.41    0.66
Dan Peterson        SEA              45      24      33       0.53    0.73
Darin Padur         SEA              36      22      21       0.61    0.58
Vinnie Richichi     SEA               2       1       2       0.50    1.00
Art Santo Domingo   SFN              77      40      51       0.52    0.66
Chuck Dybdal        SFN              32      22       9       0.69    0.28
Al Talboy           SFN              27      15      25       0.56    0.93
Michael Duca        SFN              20      10      10       0.50    0.50
Mike Smith          SLN              67      47      50       0.70    0.75
Gary Mueller        SLN              66      43      51       0.65    0.77
Jeff Durbin         SLN              20      18      13       0.90    0.65
Rick Martin         TBA              81      63      40       0.78    0.49
Jim Ferguson        TBA              72      53      51       0.74    0.71
Bill Mathews        TBA               2       1       1       0.50    0.50
Steve Weller        TEX              65      45      43       0.69    0.66
John Mocek          TEX              46      25      16       0.54    0.35
Dan Schimek         TEX              43      29      21       0.67    0.49
Doug Hobbs          TOR              55      39      27       0.71    0.49
Louis Cauz          TOR              53      31      35       0.58    0.66
Joe Sawchuck        TOR              25      13      22       0.52    0.88
Stephen Utter       TOR              19       9       7       0.47    0.37
Howard Starkman     TOR               1       1       0       1.00    0.00
David Vincent       WAS              90      52      51       0.58    0.57
Benjamin Trittipoe  WAS              61      44      40       0.72    0.66

As you can see, in the case of the White Sox, Bob Rosenberg has credited the team with fewer errors per game than either of his co-workers, and at a rate that is low by comparison to other scorers who have worked a similar number of games. Interestingly, he’s also worked a significant number of Cubs games.

Other Options?

I’m sure all readers will remember the controversy surrounding last year’s postseason with regards to umpiring. What stood out in that context appears to be the same issue that ruffles feathers with regards to scoring decisions–essentially a lack of uniformity and standardization. Just as fans have no desire to endure umpires using different mechanics for strike, ball and out calls, there is no place in scoring for divergent interpretations of the rule book or the introduction of bias.

Two of the proposed solutions, which both seem reasonable, are to increase the standardization through better or additional training for scorers, or to complete the move started in 1980 and scrap the idea of local scorers altogether by increasing the umpiring crews to five men, with one serving as the scorer.

The more radical solution among the performance-analysis community would like to get rid of errors altogether. Despite my respect for those who do the job today, I’ll admit some sympathy for either of the former solutions. You won’t find in me in the more radical camp; while there may be problems in some quarters, the information we get from designation of errors is more valuable than not having that information at all.