CSS Button No Image Css3Menu.com

Baseball Prospectus home
  
  
Click here to log in Click here for forgotten password Click here to subscribe

Futures Guide 2014 is Now Available in Paperback and Three E-book Formats.

Premium and Super Premium Subscribers Get a 20% Discount at MLB.tv!

<< Previous Article
Premium Article What You Need to Know:... (05/14)
<< Previous Column
Baseball Therapy: Shou... (05/09)
Next Column >>
Premium Article Baseball Therapy: Woul... (05/20)
Next Article >>
Premium Article Skewed Left: Replaceme... (05/14)

May 14, 2013

Baseball Therapy

How Reliable Are Our Fielding Metrics?

by Russell A. Carleton


A little more than a week ago, Jon Heyman of CBS sent out a tweet wondering why it was that Starling Marte and Bryce Harper had the same WAR. Heyman was quoting Baseball-Reference's version of WAR, which at that moment in time showed Marte and Harper tied at 1.7 wins. Harper had clearly been the superior hitter, but drilling down, it turned out that the fielding metric used by Baseball-Reference loved Marte's defense enough (and thought Harper's was average enough) to call them equals.

The problem with any sort of number this early in the season is that on many measurements, we're still at a time when players haven't logged enough playing time for the measure to be considered reliable. But of course, some measures are more reliable than others. The more reliable a measure, the sooner we can be more confident that it actually reflects what the player's talent level was during that time. The less reliable it is, the more likely it is that there will be fluky spikes and valleys over short (and sometimes long) periods of time. Fielding metrics are an estimate of how many outs a player saved from Opening Day onward, and what that was worth. However, in the same way that a player who went 3-for-4 on Opening Day is technically a .750 hitter for the moment, it’s not real. A fielding metric might need some time to stabilize as well before we get a good read on what’s going on.

There's been research on how quickly various batting and pitching statistics stabilize, but in general, few people have asked the question of how reliable our fielding metrics are. One reason is that several of the most commonly cited fielding metrics (UZR, DRS) rely on proprietary data not available to the general public. We just don't have the ability to peek under the hood. And so, we're going to have to get a little creative.

Warning! Gory Mathematical Details Ahead!
There is a publicly available data set that has batted ball type and hit location data for Major Legue Baseball. Retrosheet (put them in the Hall of Fame!) data files from 1993-1999 have the type of ball hit (grounder, fly ball, line drive), as well as zone data on where the ball was hit. This isn't the ideal data set for a few reasons. First, the zones aren't very granular, and they were input by stringers scoring the game from the press box, so the difference between a line drive and a fly ball might be in the eye of the beholder. Also, the youngest of these data are old enough to be enrolling in high school this fall. However, if anyone would like to show me a publicly available data set that is better...

I started by looking at ground balls for infielders. First, I calculated what zones "belonged" to an infielder. For each zone, I looked at which infielder(s) made the play at least 25 percent of the time (when the ball did not scoot through) over all seven years in the data set. When a zone had more than one fielder assigned to it, for example, a ball in the 56 zone (between short and third) might belong to the shortstop or the third baseman, I did not penalize the third baseman for not fielding the ball if the shortstop got there first. It simply went as a "no play" for the third baseman. Conversely, I did not reward the shortstop for somehow making a play in short right field. (What the heck was he doing out there anyway?)

My criterion for success was whether or not an out was recorded on the ground ball (either by force out, or just good ol’ throwing the ball to first). I played around with whether or not he got to the ball (regardless of whether he finished the play) or whether he fielded and threw cleanly. (If the first baseman dropped the throw, whose fault is that?) It didn't change the results all that much. All events were coded 0/1 (not out/out).

The rest of this article is restricted to Baseball Prospectus Subscribers.

Not a subscriber?

Click here for more information on Baseball Prospectus subscriptions or use the buttons to the right to subscribe and get access to the best baseball content on the web.


Cancel anytime.


That's a 33% savings over the monthly price!


That's a 33% savings over the monthly price!

Already a subscriber? Click here and use the blue login bar to log in.

Related Content:  Fielding,  Stabilize,  Defensive Metrics

17 comments have been left for this article.

<< Previous Article
Premium Article What You Need to Know:... (05/14)
<< Previous Column
Baseball Therapy: Shou... (05/09)
Next Column >>
Premium Article Baseball Therapy: Woul... (05/20)
Next Article >>
Premium Article Skewed Left: Replaceme... (05/14)

RECENTLY AT BASEBALL PROSPECTUS
Premium Article Painting the Black: Super Twoing
Premium Article What You Need to Know: Meltdown in Minnesota
Premium Article The Prospectus Hit List: Friday, April 18
Premium Article Notes from the Field: A-Ball Notes
Premium Article Raising Aces: Masahiro Tanaka, the Debut Ant...
Baseball Prospectus News: New Stat Reports a...
Premium Article Skewed Left: How Service Time Dictates Top P...

MORE FROM MAY 14, 2013
Premium Article Punk Hits: Javier Lopez Learns BABIP
Premium Article The Prospectus Hit List: Tuesday, May 14
Premium Article Scouting the Draft: Middle Infielders to Kno...
Premium Article Transaction Analysis: Barton Parcel
Fantasy Article Sporer Report: Evaluating Early-Season Outli...
Premium Article Daily Roundup: Around the League: May 14th, ...
Fantasy Article Free Agent Watch: American League, Week Seve...

MORE BY RUSSELL A. CARLETON
2013-06-03 - Premium Article Baseball Therapy: Why Sabermetricians Should...
2013-05-23 - Premium Article Baseball Therapy: Are Starters Motivated by ...
2013-05-20 - Premium Article Baseball Therapy: Would the Astros' Piggybac...
2013-05-14 - Premium Article Baseball Therapy: How Reliable Are Our Field...
2013-05-09 - Baseball Therapy: Should I Worry About My Fa...
2013-05-06 - Premium Article Baseball Therapy: What is a Good Hitting Coa...
2013-04-29 - Premium Article Baseball Therapy: On the Evolution of the Pa...
More...

MORE BASEBALL THERAPY
2013-06-03 - Premium Article Baseball Therapy: Why Sabermetricians Should...
2013-05-23 - Premium Article Baseball Therapy: Are Starters Motivated by ...
2013-05-20 - Premium Article Baseball Therapy: Would the Astros' Piggybac...
2013-05-14 - Premium Article Baseball Therapy: How Reliable Are Our Field...
2013-05-09 - Baseball Therapy: Should I Worry About My Fa...
2013-05-06 - Premium Article Baseball Therapy: What is a Good Hitting Coa...
2013-04-29 - Premium Article Baseball Therapy: On the Evolution of the Pa...
More...

INCOMING ARTICLE LINKS
2014-03-27 - Premium Article Prospectus Preview: NL West 2014 Preseason P...
2013-05-14 - Manufactured Runs: Listen to What the Heyman...