Our objective at BP Stats is to provide sophisticated and useful metrics. Our mission is to do this while working transparently. Part of this involves reviewing what we have. As much—if not more—is about building new things.

Today we arrive at an update to FRAA—Fielding Runs Above Average—that represents both lanes of the process. FRAA is the metric we use at BP to measure the defensive contributions made by players in the field. In reviewing what we had, we found something we really needed to fix—outfield assists were not being counted in FRAA. We’re also adding some new metrics for catcher throwing skills (SRAA and TRAA) along with the framing and blocking metrics (CSAA and EPAA) we already were adding. This will allow FRAA to reflect player defense across a variety of important spectrums.

All of these improvements are happening even while we work on the next generation of FRAA using some of the techniques that have produced things like DRA and CSAA. This new version of FRAA will be the next “old” version if our offseason projects go well.

Zoom Out: WARP

A quick review of how we make our Wins Above Replacement Player (WARP) metric. Here’s how we generate WARP, in two parts.


BRAA = Batting Runs Above Average

BRR = Baserunning Runs

BRR_ARM = Fielder arm ratings

FRAA = Fielding Runs Above Average

POS_ADJ = Positional adjustment

REP_LEVEL = Replacement level

RPW = Runs per win

PWARP = DRA converted into WARP scale.


Notice something? Fielder arm ratings are not part of FRAA, only WARP.

Arms and Fielding
BRR_ARM is supposed to take into account "fielder arm ratings"—everything from outfielder throws to catcher SB/CS-against to infielders allowing base advancements. But a) it has some quirks that make it such that it's not adequately representing what happened historically, and b) it's not currently included in FRAA, which makes FRAA an incomplete metric for defense.

The basic concept of BRR_ARM is to take the RE_DIFF before and after for plays where a fielder is involved. In this, it's probably best measuring infielder arms, so we're first replacing catcher throwing and outfielder assists. This will result in:

* Infielders will see their FRAA change, based on how good their arms are. Their WARP will be unaffected.

* Outfielders will see their FRAA change, and their WARP change to a lesser extent (BRR_ARM was only capturing a small portion of the runs saved by outfield assists, so high-assist players will see big changes).

* Catchers will see their FRAA change, and their WARP change by a different amount.

Outfielder assists are evaluated using a new method. Each event has a different run impact depending on the base-out state (RE24). This value is compared to the actual change in run expectancy (RE, and including actual runs scored) that takes place on each “assist opportunity” play to determine the number of runs which were either a) prevented by that assist (often a sizeable run impact), or b) allowed by the failure to record an assist (a very small run value). These runs disrupted a league-season neutral FRAA adjustment for outfield assists.

Catchers' throwing contributions are now being measured by SRAA (Swipe Rate Above Average) and TRAA (Takeoff Rate Above Average). The former measures base-stealing success, the latter measures base-stealing attempts, both of which affect the game. You may be familiar with these measures for baserunners and pitchers, but now we’re including SRAA and TRAA as add-ons to FRAA for catchers. TRAA is a small component, providing less than a run plus-or-minus, while SRAA stretches out toward five or more runs plus-or-minus. No surprise there, as throwing runners out is obviously more important (and detectable) than intimidating them back to the bag.

SRAA is, like FRAA, a work in progress. As you peruse the catcher data, you may notice 2011 is strangely close to zero for all catchers. This is something we have tentatively addressed using a provisional model which doesn’t “degenerate” for 2011. (For the uninitiated, an explanation of degenerate models.) It also provides conservative estimates and is too computationally intensive to be a production process. As we work on improving the model and tuning its performance, we plan on updating the current method.

Blocking and Framing Updates

We’ve recently discussed our revisions to Errant Pitches Above Average, or EPAA, our second biggest contributor to catcher FRAA in our new method.

Framing provides the biggest potential impact for a catcher’s defensive contribution. When merging the CSAA data into FRAA we found an overcount of “extra strikes.” However, after harmonizing the two (we had 99.5 percent accounted for), we found little or no change at the top end of the rankings. The bottom end dropped back down…apparently this is where our extra strikes were going. Once viewed as a possible feature (conservative estimates of negative framing impacts) it was truly a bug and is now corrected. Apologies to Ryan Doumit.

Zoom Out: WARP Revised

The end result is a top-line BWARP ‘formula’ that looks mostly the same, but actually includes a handful of new and improved items.


*includes OF assists, catcher CSAA, EPAA, SRAA and TRAA

This will also show up in our projections, which have been missing OF assists—potentially skewing our assessment of a team's defense. The new additions, particularly around catcher defense, are bound to impact our looks ahead.

The first batch of PECOTA projections for these new metrics can be found in the 2016 Annual. So go preorder one.

See it Now
You can review the updated FRAA estimates for individual player seasons here. The components for catchers are available here and here.

Thank you for reading

This is a free article. If you enjoyed it, consider subscribing to Baseball Prospectus. Subscriptions support ongoing public baseball research and analysis in an increasingly proprietary environment.

Subscribe now
You need to be logged in to comment. Login or Subscribe
Excellent explanation, thanks. One question that springs to mind about controlling the running game: how do you apportion blame/credit between the pitcher and the catcher? I'm not clear on it.
catcher and pitcher, along with lead runner and batter, are 'random effects' in the model. So ... it's magic! Or Judge can explain better.
The model keeps track of success / attempt rates by runners with all combinations of pitchers, catchers, and lead_runners. It then shrinks everyone toward the mean except to the extent any of those participants shows a demonstrated ability to excel / not excel in their role. The estimates that are left assign credit (or blame) for the most likely role of each participant in base-stealing success, multiplied times their number of opportunities.
or magic.
The other explanation is good, and thanks for it, but I like magic even better. Now I have a mental image of Jonathan standing in front of a mirror with a wand in his hand, conjuring statistics for the sole purpose of impressing Emma Watson.
To be fair, he's probably not the only one doing this.
I'm a little confused. I went to look up a player, Wilmer Flores, to check how he fares relative to other SS. I looked him up at SS and found his numbers, 1.2 FRAA - but this isn't broken out into SS and 2B, it's just his total FRAA. Will this be broken out at some point?
Also, on his player card, under the Standard heading, his FRAA is 0.2, compared to 1.2 under Advanced, yet his WARP is 2.2 in both cases.
Yeah, things aren't quite matching up yet. I think all the player cards are wrong in the standard heading right now unless there were no changes to their data. The advanced heading has the correct information.
Does you commitment to transparency extend to PECOTA, or will it be a black box mystery forever?
Sidenote: thanks for explaining degenerate models. At least they are funny, so they don't add nothing.
Thanks and keep up the good work!
One of FanGraphs' best features is its publication of the components of fielding metrics — on a player's page, you can see his arm runs, range runs, etc. for both DRS and UZR. Will you make the FRAA components available on player cards?
(Also, this is awesome — any adjustment that adds six runs to Kevin Kiermaier's defense checks out with me.)
yes, we do plan on making the component breakdowns available on Player Cards in the near future.
I'm a little confused on catchers' WAR timeline. When did framing skills start being incorporated? If I look at previous years' data (or back to a certain year) is it now recalculated with framing?
This data is still being updated on the sortables and player cards. I'll leave a note here when everything is complete (hopefully tomorrow).
2008-2015 are up. There are two links in the "Advanced" section of sortables:

And on player cards, the "Catching" section is available for catchers, while FRAA has been updated on both sortables and player cards to include the stats discussed in this article.
It goes back to 2008, but we will likely go back to 1988 as well (with a different non PITCHf/x based model)