March 1, 2004
Baseball Prospectus Basics
Evaluating DefenseIt is one of the most suspenseful moments in a baseball game. There's a smash to the second baseman, he slides, knocks it down, picks up the ball, throws from his knees, and the first baseman can't dig it out. The crowds waits, and then the message appears on the scoreboard "On the last play, the official scorer has ruled: HIT."
Many of the problems inherent in evaluating defense are evident in the situation above. The first, and most crucial, is the fact that one of the most basic statistics involved in defense, the error, is assigned by one of baseball's loosest rules, left to the interpretation of the various official scorers. While the league has struggled for the past few seasons to remove the subjectivity inherent in calling the strike zone, it has done nothing to remove the same from the assignment of errors. Rules 10.05.a-e discuss in detail what is to be considered a "base hit"--essentially any ball that could not be fielded with "ordinary effort," a phrase that is never defined or clarified. In any field, statistics are only valuable if they are consistent and accurately reflect the action on the field. Errors, especially recently, have become assigned in such an ad hoc fashion as to relegate the statistic to nearly unusable status.
Second, the conclusions about defense performance we draw with our eyes are often based on insufficient or severely flawed evidence. Rey Ordonez made more than his fair share of exciting plays at shortstop over the past few seasons, but while they certainly look impressive, you must ask yourself if the same plays are made by other fielders more consistently without the all the spins and ninja rolls, either because of greater range or better positioning before the play. Distinguishing between the economically boring and the inefficiently flashy is simultaneously essential and difficult.
Finally, defense is intrinsically a team activity, thus metrics assigning individual applaud or fault can be as deceptive as RBIs and other team dependent hitting measurements. Statistics like Range Factor are largely based on the number of balls fielded by a particular player. However, the fact that a right fielder, for example, doesn't field as many balls as other right fielders in the league may be the result of many other factors besides actual defensive ability; he may be playing next to an extraordinary center fielder, in a park with a small or irregular rightfield area, or playing behind a pitching staff that gives up significantly fewer flyballs than a league average staff. As with virtually all metrics in baseball, the context is the key, not the actual statistic.
When analyzing defensive performance, the most important mental adjustment to make is to hold the defense accountable for every hit on balls that could have been fielded. Initially, this does not seem fair--defense and pitching are almost inseparably entangled. Determining if the causality lies with the pitcher or the fielders is even more perplexing than drawing conclusions about the play itself. This approach certainly requires some refinement, but it removes the decisions of the official scorer from the equation. Bill James suggested this approach in one of his Abstracts in the 1980s, calling the new metric "Defensive Efficiency (DE)." It is, quite simply, the percentage of balls in play fielded by the defense. The best teams are usually around .7300 with the worst around .6900.
While DE works fine for entire teams, it doesn't yield any information about individual defensive accomplishment. To that end, BP's Clay Davenport has developed several metrics that allow us to better evaluate each fielder. First and foremost, these metrics are based on range, complimenting players more for the plays they make than the plays they screw up. The reason for this is quite simple: there is no difference between reaching a ball and dropping it and letting it drop in for a hit. It's like the old axiom in golf: 97% of the putts that don't make it to the hole don't go in; in baseball, if you can't get to a ball, you can't field it. To determine the number of opportunities the player had to make a play, we start with the league average defensive performance at that position and adjust it for the player's particular situation. There are five key adjustments:
Using these, it's possible to estimate better the performances of individual fielders compared to the league average. From there, the number of runs allowed above or below average can be determined for each fielder. In Baseball Prospectus 2004, the "Defense" column for fielders contains values like "154-1B 3"; this indicates that the player played 154 games at first and was 3 runs above average (FRAA). You can also see these numbers on BP's PECOTA Cards.
Davenport Fielding Runs are a major step up from other widely available defensive statistics such as Fielding Percentage, Range Factor, and the enigmatic Zone Rating, but statistical analysis of baseball defense is still in its infancy. Front offices have access to much more advanced metrics than the public, some specifically charting where each batted ball is hit, how hard, and how high. As that wealth of data expands and reaches the public, it will be possible to better evaluate individual defensive performances. For the time being, however, just try to use an objective eye the next time ESPN's Web Gems comes on.