keyboard_arrow_uptop

Let’s get defensive. Probably the simplest defensive metric available that’s of any practical utility is Defensive Efficiency Rating, a Bill James creation. Simply put, DER is:

If you want to calculate it from official offensive statistics, you can figure it as:

In other words, it is one minus Batting Average on Balls In Play (BABIP). It’s actually quite a clever concept, and it does a fairly good job of measuring defense. The big problem is simply that it tells us a lot about a team, but very little about the players on the team.

Now, let’s think about this for a minute. By all rights, individual player defense should add up to team defense. Let us, for the moment, tackle a portion of the issue for the sake of clarity: For right now, we’re only looking at ground-ball defense. And let’s (for the time being) ignore balls fielded by the pitcher and catcher. Throw them out on both sides of the equation. And pretend for just a moment that all four of our infielders are newly minted Cal Ripken Jr. clones, and they play every game. Now, let’s take our DER and try and break it up into components:

“PM” stands for “plays made,” and “CH” stands for chances. In short, what we’re trying to do is split up balls in play and “credit” them to the individual fielders to produce each player’s individual DER. Can we do this?

The simple part (assuming one has play-by-play data) is figuring out each fielder’s plays made. In the infield, a play is made when the player fielding the ball is credited with either a putout or an assist. (In some rare circumstances-mostly when a missed catch error is committed-this occurs even when there is no out recorded on the play.) For the outfield, a play is made when the player fielding the ball is credited with a putout. An outfield assist is generally what we refer to as a “baserunner kill,” an important part of the puzzle, but something that’s better measured by an arm rating rather than including it in a range-based fielding metric. We’ll swing back around to looking at outfielder arms eventually, though.

So we can apportion out fielding plays made between fielders pretty readily. What can we do about splitting up balls in plays into fielding chances? For that, we need to look at where the ball is hit on the field.

The Danger Zone

Typically, when looking at balls in play, dividing the field into zones indicates the hit location, like so:

This particular zone diagram is adapted from the Project Scoresheet scorekeeping system; different data providers adopt different zone systems, and the specifics of each zone aren’t really important right now. The location of where a ball is hit on the field is recorded based upon whatever zone it’s hit in. For ground balls, it’s typically recorded as the zone where it passed through the infield, regardless of where the ball eventually ends up.

So when we want to determine something about a ball in play, using its zone location and batted-ball type, we can compare it to its peer group. But let’s consider a specific example:

Suppose we’re interested in the ball indicated by the blue dot in that diagram. In a zone-based system, that particular fielding play would end up being compared to the play indicated by the red dot on the far left, but not the play represented by the red dot right next to it on the field.

Of course we can always divide into smaller and smaller zones to address this issue. But you end up slicing your sample smaller and smaller, making yourself more susceptible to random variation. And you’re always going to end up with an arbitrary distinction between what batted balls are peers and what aren’t.

So, rather than dividing the field into zones, what can we do? Instead, let’s compare every batted ball to all other batted balls. Let’s simply weigh the closest ones more heavily.

Say we describe every batted ball, not by a zone, but by an angle, where home plate is the origin, a straight line out to second base is zero degrees, first base is 45 degrees, and third base is -45 degrees. If we have a ball hit at 10 degrees (or just a bit to the right of second base), we can compare it to a ball hit at zero degrees and 20 degrees equally. A ball at -5 degrees can also be compared, but we put less emphasis on it in determining what it should be.

To achieve this, we can use a tool known as local regression, also called a Loess or Lowess regression.

Finding Chances

Since we’re dispensing with the idea of zones, we can also dispense with the notion that some fielding chances are “in zone” and fieldable and some are “out of zone” and unfieldable. All batted balls in play are fieldable; it’s simply a matter of responsibility.

So, to figure out who is responsible for each portion of the field, we are going to throw out hits and simply look at plays made for the time being. Essentially, we are trying to give out one chance per batted ball. We can certainly use fractional chances, though-some balls might be 50 percent the responsibility of the shortstop and 50 percent the responsibility of the second baseman, for instance. Using some local regressions, we can apportion out chances like so:

Blue represents first base, purple second base, orange shortstop, and red third base.

That’s still a little messy, especially along the foul lines-not everything adds up to one exactly as we would like. That’s not entirely unexpected-the local fit is going to come with a standard error for each fitting point as well, and since we know the actual value, we can “push” everything in the right direction and get everything to add up correctly.

So we’ve got what we came here for, right? I mean, we can figure out a DER for each fielder that aggregates to team DER, at least for ground balls.

What we’re missing here is that not all ground-ball chances are equal. So, before we can compare two players, we need to know not just the number of chances they had, but also the difficulty of those chances. What we want to look at is the expected outs made on each ball in play as well:

This time, not everything does-or should-come close to adding up to one. Particularly, you can see that a ball hit straight up the middle is rarely, if ever, converted into an out.

Doing the Split

All of the above works for figuring out responsibility on hits. But there’s a caution when applying it to outs (or errors, for that matter). Once a ball is fielded, it is no longer a chance for any other fielder, and it shouldn’t be counted against them. So, for any ball a player fields, they are credited with one chance, regardless of location.

Since we are crediting the fielder with the entire chance, we also need to credit that fielder for the entire expected out as well, not just the expected out for his position:

This is key for two reasons-it makes sure everything properly reconciles at the team level, and it avoids over-crediting a “ball hog” for stealing plays from his teammates.

Putting it Together

So, for every ball in play, we figure both a player’s chances and their expected outs. A player’s individual DER is:

And their expected DER is:

From here, we can compute a “normalized” individual DER for each player that controls for the difficulty of chances each player received. We can also subtract expected outs from plays made to figure a fielder’s plays above (or below) average.

This gives us a framework for evaluating the contributions of individual fielders. We’re not quite ready to apply it to individual fielders, though. First, we need to account for a few other factors, such as baserunners, the number of outs, number of strikes, and park factors. We’ll attend to those next week.

Notes and Asides

All regressions and graphs for the article were produced using GNU R with the locfit package, with a smoothing value of 0.2.

Several existing fielding metrics served as an inspiration for this system, primarily Mitchel Lichtman’s Ultimate Zone Rating and Shane Jensen’s SAFE.

### Latest Articles

3/22
3
3/22
1
• ##### Prospectus Feature: Won’t Get Fooled Again \$
3/22
0
You need to be logged in to comment. Login or Subscribe
Guancous
1/29
Interesting approach and it feels intuitive. I'm curious to see where this goes with more variables thrown in.
jtrichey
1/29
I think you may be onto something exciting here. Can't wait for more.
Berselius
1/29
Colin, What kind of batted ball data do you have to generate these angles? I don't know a ton about the UZR underpinnings either. Does it all come from the project scoresheet data, and you're converting the zones to angles?
Zaxell
1/29
Good stuff; looking forward to part II. Any plans on working velocity into the model, in addition to angle? One would expect some teams to cough up more hard-hit balls than others, and it would be nice not to penalize their fielders for it. But maybe I'm getting too greedy.
sunpar
1/29
This looks like great stuff, Colin. I'm excited. Just out of curiosity, where does angle of batted ball data come from?
Schere
1/29
Once this is perfected, we'll need to somehow distinguish positioning vs. range. Note to MLB: put RFID chips in the uniforms and the baseballs, so we can solve this problem ASAP!
Wereslug
1/29
Great work, this is the direction fielding metrics should be moving in. I like Jensen's approach and this seems to improve upon it particularly because individual DER is designed to add up to team DER and because I don't know if there is accurate/precise enough data to account for everything they tried to include (positioning of fielders, spread/location of hit balls, run values of locations) without compounding any systematic errors. I could be wrong about this because I don't know enough about statistics to look at the data myself. One thing I like about SAFE is that I think it was important, for infielders at least, that ground balls down the line went for more bases on average. I was never as sure about using SLGBIP vs BABIP for outfielders because I was told that differing parks effects made the sample sizes too small to work with and too difficult to sort out. How is the outfield DER going to be calculated? Is there any chance you are going to add expected run value of various hit locations/angles to the metric?
cwyers
1/29
For the outfield, the process should be very similar, it just requires more input variables. The local regression technique used here "scales up" quite nicely to accommodate multiple independent variables, just like any other least-squares regression technique. As for expected run values - yes, certainly, that's something that we're going to progress toward.
Wereslug
1/30
It was pointed out to me that I made some factually inaccurate claims in my post above. I guess what I am trying to say is that I like the idea of having a fielding metric that is based on smooth functions and not discrete bins. That seems to mirror reality better even if, as mgl says, for large enough sample sizes, there won't be a significant difference in the results. The reason I like your attempt at DER better than Jensen's SAFE (Here is where I'm not completely sure about how SAFE is calculated. I could have misinterpreted Jensen's explanation and therefore this paragraph would be wrong. Let me know if that's the case) is that DER gives fielders credit for exactly the number of chances they had. So that if by random chance, in a particular season, a fielder has many fewer chances than one would expect, he's not penalized/rewarded for more that what he really did. On the other hand, in SAFE, each player is measured using a density estimate rather than actual chances. If I'm correct here, SAFE is telling us anticipated value if the player received an average number and distribution of chances. That feels more like a projection than actual value of past performance.
3n2sports
1/29
It might behoove BP to allow everyone (not just subscribers) to read about the novel defensive metric about to be rolled out. First, it will let more people see the idea and possibly offer insightful ways to improve upon it. And second, this is the kind of work that gets readers excited and ready to open their wallets for a subscription. Just a thought.
3n2sports
1/29
PS, This is a great first step and I'm glad to see that you guys are prepared to ditch the outdated FRAA that's been in use (still need a name? How about Fielding Runs Indicator (FRI), simple, familiar, yet distinguishable from the older FRAA). I can't wait until you guys are ready to integrate hit f/x data a few years down the line...now that will be a damned good fielding stat.
Oaktoon
1/29
Outs for balls up the middle obviously are quite dependent on the presence, or absence, of a shift. What is the percentage of times that ML infields now employ an exaggerated shift where the SS is standing almost directly behind 2nd base-- partic. for left hand hitters? I would think the drop to 0 that the writer assumes for a ball hit directly up the middle is not correct. Plus there is the added chance that one of the fielders is favoring the middle because of the possibility of a steal. And there is the added chance of a high chopper hit by a relatively slow runner-- enough time for the middle infielder to catch the ball hit up the middle and still throw the batter out. So i am having a bit of a problem with that particular metric.
Oaktoon
1/29
and to clarify-- obviously all the shifts with a LHB involve the SS standing behind 2B-- does anyone even use such an exaggerated shift against a RHB??
cwyers
1/29
We can certainly take these things into account. With the hit location data it's pretty simple to identify, for instance, what hitters are frequently "shifted" against and adjust the data accordingly. We can also look at different infield alignments for various things - for instance, when fielders might be holding runners on. We can also look at when fielders might be playing up for the bunt, and how that affects their ratings. We're also going to do a LOT of work on park factors here, and do so in a way that takes into account different types of scoring bias in the batted ball data. But before we can do any of that, we have to establish the sort of baselines that I mentioned above. So this is just sort of the introduction, to get everyone familiar with the tools that we're going to be using. There is still a lot of work to be done here.
Peter7899
1/29
Phew, thanks Colin, I understand the concept. I have a question though. Say a ball is hit between the first baseman and second baseman in the overlapping angle of the two, but neither makes the play. They both had a chance to make the play, though, so do you credit them both with a chance, or with a partial chance? How is that handled?
cwyers
1/29
Right, let's say that a ground ball is hit at about 30 degrees, or right where the first base and second base lines meet up. Each fielder would receive credit for a half chance. As you shift over more toward the line, the first baseman receives proportionally more credit for a chance, until finally you reach the point where the first baseman gets full credit and the second baseman gets no credit for that fielding chance.
sgturner65
1/29
Are you making allowances for how hard a ball is hit? Albert Pujols, for instance, smokes a lot of balls past infielders that, say, Jack Wilson won't.
cwyers
1/29
Sure. At the same time, if Michael Bourne and Jim Thome hit the exact same ball at the same vector and same speed, the odds of each reaching base is different. And I don't know that we have good information on how hard a ball is hit. So instead maybe we control for the identity of the batter instead. It's something to look at.
nateetan
1/30
I thought you were using the hitFX data. Doesn't that have batted ball velocity?
cwyers
1/31
This is ball location data from Gameday, which is fed out in x,y coordinates. What I do is transform those coordinates - technically those are Cartesian coordinates - into polar coordinates, which are listed using angle and distance. (Distance is going to come into play more when we start talking about the outfield.)
nateetan
1/31
blcartwright
1/31
'real' data sets? Gameday comes from mlb.com. The only hit f/x data released to the public was for April 2009. We are hoping for more. Peter Jensen has studied Gameday's ball locations, compared them to BIS and Stats, and although Gameday is slightly less accurate than those, it's close, and all have an error of around 6 feet in x or y in the outfield. A year ago I published park factors studies that showed recorded line drive rates varied +/- 20% from park to park in the majors. The 'hard' or 'soft' designation had an even larger variance, and I have chosen not to use those. So yes, it's difficult to tell how hard the ball was hit.
nateetan
2/01
Who has access to the full HitFX then? Nobody or only people who pay for it? If it's pay, why doesn't BP pay for it, or is something that costs a few million and so only the teams buy it? For sure if they are just using Gameday, then they don't have great speed data then.
sgturner65
1/31
You first answered "sure" to my question and then went on to stat you don't know if you have good information on how hard a ball is hit. To me, defensive metrics are sorta like the origins of our galaxy. With what we presently know, there is no answer to the question that will satisfy a majority of the people. Every at bat and every play in baseball is unique. Some things just may not be quantifiable.
nosybrian
1/31
I think a better way to put your last sentence is that "while it's possible to quantify defense there's a certain irreducible amount of noise or uncertainty in the data."
Oleoay
1/31
No one really knows what protons, electrons and neutrons actually look like and how they interact in an atom, but quantum mechanics have done a very good job at coming up with models that are useful.
mbodell
1/29
It seems like two things are going to be left out of this model (at least directly, you may capture some part of them indirectly). One part is fielder positioning. This includes the shift, but also includes things like holding runners, individual players tendencies, etc. One part is the speed of the ball. A sharply hit ball might be more (or less!) likely to be a hit than a slow ball. The type of field/turf might effect that as well. Not all balls hit at -10 are equal.
benharris
1/29
This is awesome. I'm really excited to see where this goes. Just looking at the graphs is pretty informative. I'd be interested to see how the graphs differ on a team-by-team basis. Also, you can kind of see how (right) handedness might play into fielding, as all expected outs directly up the middle were fielded by the SS (on his glove side), while the 2B had virtually no chance of making the play. Cool stuff.
vetrini
1/29
Thanks for allowing the reader a look into the process of building a new statistic. Fascinating.
blcartwright
1/30
Positioning is a key part of fielding. If a fielder is better than others in selecting an optimal starting positon, then he should get a better rating.
alaime
1/30
Excellent, I like this direction. Agreed that positioning is quite important. Not just playing closer to the line or further, but up to cut off the run or back at double play depth. So it's not just up to the fielder as positioning is adjusted by game situation. I imagine in the future as data gathering gets more mature, we'd also be able to correlate the fielders position at the time the ball is hit. Not sure if we have that now. So for example if the SS is playing closer to 2B and the 3B is closer to the line, both the GB chances and the GB expected outs lines would shift for each fielder. However IMO this would be in the future and I think baselining and generating some raw data for evaluation using this concept is fine for now. Can't wait to see it.
benharris
1/30
Won't that just show up in the data anyway? If a player's better at positioning then he should get to more balls, which is what you're measuring. I don't know that any adjustments need to be made for it.
rawagman
1/30
it isn't about fielding to get to more balls, but to get to different sorts of balls in play
Peter7899
1/30
Positioning is a part of fielding batted balls. The better defenders will be better positioned, therefore getting to more balls. I don't think the stats differentiate between a smart fielder and a athletically gifted fielder. A good fielder is a good fielder period.
cwyers
1/31
Well the question is if the fielder is allowed to optimally position himself, or if there are strategic concerns (say, if he has to position himself to hold a runner on the bag, or if he's playing up for the bump) that require him to be positioned suboptimally. And we don't have the positioning data to figure that out, no. But we do have plenty of baseball sense that tells us in what situations this typically occurs - really, there are a few things (what bases have runners on them, how many outs there are, number of strikes, etc.) that give us a pretty good gauge of where the runners should be positioned. And we can take the data we have and test some of these and measure how much of an effect there is. As a general rule of thumb - corner infielders tend to shift up and down the line, middle infielders tend to shift laterally - this is illustrated here and here. (That said, certainly there are times when the corner men shift over to their left or right, or the middle infielders play closer in or further back.) So you need a slightly different process for adjusting corner men than middle infielders. We'll look at all that.
Oleoay
1/31
If we are able to generate a baseline for a given fielder in all situations, we can also learn more about positioning. For example, we can also tell if a higher-than-normal percentage of hits, as opposed to outs, happen at a certain angle of hit in a given runner/out situation, but in a normal situation that fielder would've had the range to get to that ball. Thus, positioning is most likely at fault.
sroney
1/30
Defensive Efficiency is not QUITE "1 - BABIP", because 1-BABIP gives the defense credit for making the plays which are scored as errors, when it should not.
cwyers
1/31
Yes. However, if you want to calculate DER from official batting stats, that's the closest you're going to come. If you want to do it from official pitching stats, it looks something like: (IP*3-SO)/(IP*3-SO+H-HR) If you have extra data like sacrifices and reaching on errors and so on you can do a better job. But especially with historical data, sometimes you really don't have that. Not that this is especially relevant here - we have full play by play data for this.
DrDave
1/30
Interesting; it will be fun to see how this plays out. For presentation purposes, though, please choose a different word than "Chances" to describe opportunities to field a ball. "Chances" already has an official definition, and it ain't that. The opportunities for confusion are legion. If "opportunities" is too long, how about just "opps"?
Oleoay
1/31
How do you factor for batted balls that tip off one fielder's glove (like the pitchers) and are then handled by another position player for an out?
devine
2/01
Wow. Colin, thanks. I'm sure I need to read it again (and maybe a third time) but it really sounds promising.
ncimon
2/03
This is a really great idea, the first one I've seen that seeks to realistically distribute fielding responsibilities. And it does it without discretizing the playing field which is so un-natural. I'm really looking forward to your development of these empirical distributions and the resulting analysis: how different fielders do with respect to these distributions.