As you know, different pitching estimators tend to agree on which pitchers are good and which ones are not. The interesting cases are when they disagree—strongly. In those situations, the proper response is not to decide which one is “correct” (to the extent there is such a thing), but rather to look at why they disagree.

On a related note, people have recently asked for us to do more explaining of how Deserved Run Average (DRA) works. Often, it’s easiest to do that with an example.

Today, our example is Twins right-hander Ervin Santana. Santana has a 1.80 ERA, a 1.80 RA9, a 4.00 FIP, and a cFIP of 102, but a DRA of 2.74. He is striking out 6.4 batters per nine innings, walking 3.5 batters per nine innings, and giving up just under one single home run per nine innings.

ERA and RA9 suggest an extraordinary pitcher; FIP and cFIP see an average pitcher, and DRA sees him somewhere in between, as a very good, but not-as-good-as-his-RA9 pitcher. Why the difference in opinion?

FIP and cFIP, as you know, look only at home runs, strikeouts, hit batsmen, and walks. Santana gives up a below-average number of home runs, generates fewer strikeouts than average, and gives up a tad more walks than average. Home runs count for more than the other aspects, so he comes out as an average-ish pitcher overall.

DRA sees a more interesting profile. Santana has a left-on-base percentage of 91 percent and a batting average on balls in play of .136. If you look at Santana’s player card, you’ll see that he has played in hitter-friendly stadiums (pitcher park factor, or PPF, of 107), faced roughly average opponents (oppTAv of .258), and most importantly of all, has held batters to a True Average of .173. Since the league TAv is .260, this is an incredible amount of damage control on contact.

DRA applies linear weights after adjusting each event, so strand rate isn’t a factor; ignoring it is certainly consistent with Santana’s DRA being almost a full run higher per nine innings than his charged RA9. But the interesting things here are the BABIP and the True Average allowed. FIP expresses no opinion on balls in play at all. This often is just as well. But the disadvantage is that when a pitcher does exert meaningful control over balls in play, FIP will not pick it up outside of home run prevention.

Although luck has plenty to do with it, Santana’s results on balls in play are simply too low to completely deprive him of credit. Twins pitchers overall have a BABIP of .263 and a TAv allowed of .253. Although the BABIP and True Average figures need to be discounted to some extent, it is most likely that some of it is justifiably credited to Santana. That is what DRA is doing here: shrinking the credit, but still giving credit where each model sees it as being due.

The DRA runs leaderboard is your friend here. In addition to each player’s DRA, we tabulate each player’s total NIP runs (not-in-play events like strikeouts, HBP, and walks), Hit runs (value of hits allowed), and Out runs (essentially, an adjusted BABIP) above average. The simplest thing to do is to look at the sign next to each value: if the value is negative, the pitcher is performing better than average in that category (runs saved, if you will). If the value is positive, the pitcher is performing worse than average in that category (extra runs allowed).

As you can see, Santana is below average in NIP runs (consistent with a pitcher who generates fewer strikeouts and greater walks on average), but the best in baseball at controlling Hit runs (restricting damage on contact); his Out runs are also above average.

None of this means that Santana will necessarily keep his RA9 (or DRA) that low all year. It does mean that a pitcher’s ability to restrict damage done on balls in play, and to keep some balls from being hits at all, is worthy of at least partial recognition. DRA’s ability to do that is one reason to keep an eye on its assessments, particularly when they diverge from other estimators.

Thank you for reading

This is a free article. If you enjoyed it, consider subscribing to Baseball Prospectus. Subscriptions support ongoing public baseball research and analysis in an increasingly proprietary environment.

Subscribe now
You need to be logged in to comment. Login or Subscribe
very helpful, thanks!
The thing about Santana is he knows what he's doing. He's been around forever. I was at Game 5 in Anaheim in 2005 when he bailed out Bartolo Colon and shut the Yankees down. He was credited with the win that night. He's had some great years and he's had some decidedly mediocre years, but he's at the point in his career where he knows what can hurt him and what can't. It'd be different if we were talking about some 23 year old kid sustaining what he's doing now, but Santana has been nails for nearly two years now. The Twins are relevant again and that is great for both Santana and baseball. Last year he got nothing to work with, but pitched very well most times out. This year he might have one or two bad outings. He's steady as a rock. The rush to quantify all things baseball is fun and revealing at times, but I'm filing this guy's success under plain old guile. He's not elite, but at this point he's certainly better than most. Wish I had eight guys just like him on my fantasy team. I owned Santana all last year, but at $16 I had to make a decision between him and $15 Kevin Gausman on protected roster day. I went with Baltimore's offense and Gausman's alleged upside. So much for the best laid plans. Veterans don't get to be veteran's by accident. If the Twins keep hitting this guy will win 15+ games..Just my two cents...
Jonathan, can you work through 2016's out_runs for Jose Fernandez (high BABIP) and Verlander+Scherzer (low BABIP). They were among the league leaders in worst out_runs. And as a point of contrast, Porcello, low BABIP and among league leaders in best out_runs.

Fernandez and Porcello would be obvious. It would be interesting to see how the other two guys got hurt in out_runs, but wasn't obviously reflected in BABIP.

We calculate out probability over average for generating putouts at each fielder position, with some positions obviously being more important than others due to the large n they generate. Essentially, the material numbers come from P3, P7, P8, and P9 --- so P3 and OF essentially.

We calculate Fernandez, Verlander, and Scherzer as being below average at generating P3 outs (not surprising given their profiles) while Porcello was a leader. We then have Fernandez as also being notably below average at generating OF outs [so his hole gets deeper], Verlander and Scherzer as about average [which doesn't help them overcome the hole dug at P3, but stops it from getting worse], and Porcello as above average in generating OF outs, which only pours it on to his existing P3 advantage.

Porcello managed to capture the best of both worlds, which, while I haven't looked at him specifically, probably is one reason why he was so successful last year. It certainly was a big driver of his 3.09 DRA.
If I am following you, because Verlander and Scherzer are flyball pitchers, the number of putouts by 1B (meaning number of groundball outs to their infielders), will be low.

But given that they are flyball pitchers, you would expect them to be above average in getting outs from the outfielders, just by sheer volume. It sounds like you are saying that they are average compared to all pitchers, meaning below average compared to flyball pitchers.

Am I following all that?

I guess I am having a hard time reconciling these guys being among league-leaders in having a low BABIP, and yet being among league worst in out_runs.

I was hoping to see how the individual components, including luck, would give these guys a .255 BABIP.
With regards to Ervin Santana, I can tell you that among the 150 pitches with the most batted balls, he's 3rd best in terms of best combination of speed+angle.

I am glad to see that and it certainly makes sense.
The best, or worst, example of relying on these numbers is the Nationals handling of Tanner Roark in 2015. Despite an excellent record in 2014, with a low ERA, he was replaced in the rotation because his dreaded "peripherals"(not enough K's) didn't fit the template. The result was that Nationals Park was empty in October and the players enjoyed the post-season on the golf course. This year the poster boy is Antonio Senzatela. To suggest that he has been -0.7 WAR is beyond absurd and is based on one number, his low K rate. Fewer hits than innings pitched, excellent WHIP, good ERA, runs still count the last I looked, all the while pitching in Denver. Come on guys, this type of projection does nothing to enhance belief in your projections.
Tanner Roark's 2014 DRA suggested he's a mid-rotation starter, as has his 2016 DRA, his 2017 DRA-to-date, and, I would suggest, the general consensus answer to the question "what is Tanner Roark?" He was not bounced from the Nationals rotation before the 2015 season due to poor peripherals, he was bounced because ownership signed Max Scherzer and Roark was the least established of the five incumbents. Four of the five 2015 Nats starters pitched roughly to expectations, and Doug Fister isn't the guy you would've suggested replacing coming off his 2014 2.41 ERA and eighth-place Cy Young finish. Why the 2015 Nationals missed the playoffs is somewhere between a book and an article and definitely not a comment, but it sure didn't have much to do with starting pitching.

In contrast, Antonio Senzatela has 62 innings of a good ERA and red flag peripherals; yes, his strikeout rate is low, but he's also giving up over a homer per nine and 80 percent of his runners are being stranded. I liked him some as a prospect, but coming into the season he had made all of seven starts above A-ball, so there's no track record even in the high-minors here to point at for sustainability. He's bordering on a two-pitch pitcher (throwing his change less than five percent), which is another red flag. He just seems like a weird place to start if we're critiquing an advanced pitching stat.
DRA is not a projection.
I really gotta figure out how to embed gifs.

Thanks for this, and I think the explanation of how DRA is utilizing balls-in-play/BABIP information. In a case like Santana's, you can easily understand why DRA, RA9, and FIP would come to different conclusions. Since his success is centered on exactly the info FIP ignores, FIP may "underrate" him.

My question though is: what about the pitchers for whom the difference is much larger? Let's take Ivan Nova. Nova's ERA is 2.92 and his FIP is a bit higher at 3.15. His DRA is 5.21. If I check his DRA run values, he's giving up runs in not-in-play and hits, but making up more than any other pitcher in out runs. The net of these is a seemingly inconsequential ~2 runs. So: if the big disagreement isn't in the DRA run values, where is it? His TAV against is low, he's played in roughly neutral parks (PPF=97), and he goes from a guy who's nearing 2 WAR by fWAR to a replacement-level arm.

The same is true for his teammate Wade LeBlanc. The DRA run values net to ~2, and yet his DRA-RA9 is *higher than his RA9*. What's the source of these anomalous (that is, not immediately apparent in the run values) discrepancies?

PS - Nova's DRA tanked after being traded to Pittsburgh in 2016, even as his TAV-against got much better (as did his RA9). What could that be about?