CSS Button No Image Css3Menu.com

Baseball Prospectus home
  
  
Click here to log in Click here for forgotten password Click here to subscribe

<< Previous Article
Premium Article What You Need to Know:... (08/12)
<< Previous Column
Premium Article Baseball Therapy: Big ... (08/05)
Next Column >>
Premium Article Baseball Therapy: Beco... (08/19)
Next Article >>
Premium Article The Prospectus Hit Lis... (08/13)

August 12, 2014

Baseball Therapy

I Believe In Clutch Hitting

by Russell A. Carleton


I know, I’m not supposed to, but I believe in clutch hitting.

By clutch hitting, I mean that certain players have some sort of ability to perform better in higher leverage situations. Leverage, for the uninitiated, is a concept formalized by sabermetrician Tom Tango. We know that some situations in a game are more important than others. When it’s 15-1, no one cares what happens in a plate appearance. When it’s the bottom of the ninth with runners on second and third, two outs, and the home team is down by one, pretty much the entire game rides on this at-bat. Leverage index is a mathematical model of how much more important that late game situation is.

Leverage is based on the idea of win probability. We can look at each game situation (let’s say, bottom of the third, one out, runner on first, and the home team down by two) and figure out over some past time frame how often the home and visiting team won. More to the point, we can figure out how much that win probability can change based on whatever is about to happen next. In the 15-1 situation, whatever the batter does is going to move the needle very little. In the bottom of the ninth, the win probability could go from roughly 50–50 to 100–0 in a hurry. When a batter does something positive that increases his team’s chances of winning, we give him credit for adding win probability (even if giving him all the credit is silly). In a high-leverage situation, a batter can accumulate a lot of win probability in a single at-bat.

The standard test for whether there is such a thing as clutch hitting has been to look at the win probability that a player records over the course of a season and compare it to what his win probability would have been in situations where the leverage index was 1. (This is the basis of how our friends at FanGraphs calculate clutch.) From season to season, players show very little correlation on this measure of clutch. In general, the interpretation has been “clutch doesn’t exist” rather than “we had a poor measure of clutch to begin with.” Indeed, I have found that this measure of clutch eventually does become reliable. It just takes a while. Maybe there is signal in all that noise; maybe we need a better antenna.

Warning! Gory Mathematical Details Ahead!
In the 2014 Baseball Prospectus Annual, I introduced the idea of looking a little more closely at individual players to see how they reacted to pressure situations. I examined how, for each player, the leverage of a situation affected his tendencies to swing at the first pitch. There’s a separate regression equation for Daniel Murphy, David Murphy, and Donnie Murphy. Since every plate appearance has a first pitch and the count is always 0-0 when it happens, I’m able to hold a few things constant. But my program runs a logistic regression only looking at Daniel’s at-bats and what he did in them, creates an equation describing his behavior, and then does it again for David, and again for Donnie.

I then took each equation and calculated the chances that each player would swing at a first pitch when the leverage index was 1 (average) and 2 (a situation twice as important as the average situation). Then, I subtracted the two and got a rough indicator of how high leverage began to affect a player (at least on this one behavior). I used a minimum of 250 plate appearances in a season and looked at players from 2009 to 2013. In the past, I’d found that clutch, as described above, had a year-to-year correlation of .074. (I used a method known as auto-regressive intra-class correlation.) For this group, across the five years, the ICC was .30. That’s not huge, but we call home runs a true outcome for pitchers with year-to-year correlations in the same neighborhood. I termed this difference between predicted first-pitch swing rate “swing difference.” Some players swing a lot more when the leverage goes up. Some barely notice. A few start to freeze.

Next, I wanted to see if swing difference predicted changes in outcomes. For the years 2009 to 2013, I used the log-odds ratio method (which I have used multiple times before) to create a predicted percentage that each plate appearance would end in a strikeout based on the batter and pitcher’s usual rates in that area. I did the same for walks and singles and home runs and the rest of it. Next, I looked at all plate appearances in which a batter with 250 PA in that season faced a pitcher with 250 batters faced in that season. I created a binary logit regression in which I had my predicted percentage of a strikeout (for the initiated, expressed in a log of the odds ratio), and then entered in the leverage index for each plate appearance, the swing difference stat for the batter and the multiplicative interaction of swing difference and leverage.

This type of analysis, called a moderator analysis, is well-suited to answering the “clutch question.” If certain players have some sort of clutch factor (and here, we’re using swing difference as a rough measure of clutch) then as leverage increases, we would expect to see those who are higher on this clutch factor to show greater increases (or sharper decreases). That’s what the interaction term between swing difference and leverage does. If it’s significant, it means that as leverage goes up (or down), the effect it has will depend, at least in part, on that clutch factor.

What I found is that for hitters who show more of an effect on swing difference (leverage makes them swing at the first pitch more), they were less likely than expected to walk and less likely to strike out as leverage went up. Instead, they showed higher rates of both extra base hits and outs in play. To show some sense of how much of an effect this could have, here are the numbers for strikeout rate.

Let’s say that our pitcher-batter matchup stats alone would suggest that the chances of a strikeout are 20 percent. Now, let’s take a look at what would happen in a situation that has a leverage value of 1, and compare a batter who has a swing difference of .10 (he swings at first pitches ten percent more often in higher leverage situations than he does in medium-leverage situations) and a batter who has a swing difference of 0 (he swings equally in both situations). The values are the likelihood of a strikeout happening.

High Swing Difference No Swing Difference
Leverage = 1 19.3% 19.3%
Leverage = 2 17.7% 18.3%

In an average-leverage situation, the two hitters are about the same (they differ at the fourth decimal place), but once the leverage is turned up a bit, they get different results. Not by a lot, but it’s there. You get the same basic effect sizes for the other outcomes.

Before we go further, the careful observer will note that there’s a certain tautology that goes along with these analyses. I think it doubles as both a feature and a bug. A batter who is more likely to swing at the first pitch in high-leverage situations is probably just more likely to swing in high-leverage situations. It’s no wonder he sees a drop in his expected walk rate (and in some sense his expected strikeout rate). And if we’re saying that his swing rate drops because of leverage (or at least in accordance with leverage), then it’s not surprising that the effect appears. We’ll talk about this more in a bit.

Clutch. Heart. Grit. Myocardial Infarction.
Let’s clear a few things up. Clutch is not a result of having superior moral character, notwithstanding the plot of every sports movie. It is also not a guarantee that a hitter will always come through. My contention is a much more reserved one. Clutch is likely some combination of ability to deal with pressure combined with some particular change in approach, whether conscious or unconscious, that results in slight variations from what we might otherwise expect. For some, that change makes a hitter better and in some it makes him worse.

These analyses may not completely prove that clutch ability exists, but they do lay what I hope is a foundation for how we might continue the search. “Clutch” is a way of saying that the situation matters because players are human. What we have here is an indicator that has reasonable (if not great) consistency across years, and it explains differences between players in how leverage affects them. More searching might find something with more consistency. Even then, year-to-year consistency is not the only way to establish that a measure is reflective of a player’s true talent level. Using a more tracking-based approach might help. Players can and do change, even within a season. There’s no reason clutch needs to be an enduring trait, rather than a state we can detect with some reliability. The rest is simply showing that the factor, whatever it is, can explain some of the differences between players’ performances in different leverage situations.

As to these specific analyses, it might very well be that what’s driving things is that some players are looking at the sorts of relievers they face in high-leverage situations and saying “Well, he usually comes right at me, so no point in messing around. I might as well swing when I see something interesting.” It might not be a mystical force at work, but a very reasonable reaction to the circumstances. In that case, clutch isn’t even something psychological, but a mental skill. Still, there could be problems with multi-colinearity. What this might be showing is that some players swing more in high-leverage situations, and so we would expect them to take fewer walks, somewhat by definition. Then again, even knowing that information could have strategic value. Maybe when we have other data sets to work with, we might be able to look at measures of how leverage affects a player that aren’t based on game results.

The other piece of this, and it’s one that I tried to drive home in the piece in the Annual that started everything, is that knowing that a player swings more (or less) often in high-leverage situations might be good within the context of one skill set and bad within another. These analyses fall into the large-N trap that assumes that more swinging is better (or seems to be) for everyone. But if nothing else, I’d present these analyses as a way of re-opening what had been assumed to be a closed debate. Clutch hitting might just exist.

Russell A. Carleton is an author of Baseball Prospectus. 
Click here to see Russell's other articles. You can contact Russell by clicking here

28 comments have been left for this article. (Click to hide comments)

BP Comment Quick Links

GreggB

Question to the community from a relative neophyte: have the studies that indicate little indication of clutch performance taken into account that, in the highest-leverage at bats, the batter is generally facing the freshest, most skilled pitcher the other team has to offer? If a batter has an OPS of .850 over the course of the season (against league-average pitchers), and an .850 OPS in high-leverage situations when the game is on the line, he will usually be performing against far more skilled pitchers. I would interpret that as clutch performance.

Aug 12, 2014 03:28 AM
rating: 3
 
NYCRuss

How much of "clutch hitting" is contextual in that rather than a player performing better in such situations, he's really performing worse in non-clutch situations?

For example, could it be that some players react to stress so that they're more focused, but without that stress they're actually under-performing?

Aug 12, 2014 05:45 AM
rating: 2
 
mertes79

I also wonder if a player always knows when he is hitting in a high leverage situation? Some are obvious, but I assume based on some of the relief pitcher literature I've read that there are perhaps less obvious high leverage situations in the 5th or 6th innings included in the analysis. Seems an important factor might be determining what situations a hitter perceives to be high leverage in terms of ferreting out whether he demonstrates "clutch hitting" ability that is somehow different from other players.

Aug 12, 2014 07:46 AM
rating: 3
 
BrewersTT

If I follow you, this seems like a distinction without a difference, if the question is "do they perform better in clutch situations?".

Aug 12, 2014 10:17 AM
rating: 1
 
gweedoh565

This is a good point, but mertes79's comment does address that other clutch-related question of whether players can "turn it up" when called upon, or when they sense that they need to, or something to that effect.

Aug 12, 2014 10:22 AM
rating: 0
 
BrewersTT

True, the reason behind any difference would be important to understand.

Aug 12, 2014 10:32 AM
rating: 0
 
BP staff member Russell A. Carleton
BP staff

The Yerkes-Dodson curve suggests that people are actually at their peak at a moderate amount of stress, and that too little or too much have the same effect of driving down performance, but we don't know where the high point of the curve is.

Aug 12, 2014 10:32 AM
 
godfather
Other readers have rated this comment below the viewing threshold. Click here to view anyway.

there are, and always have been, performers who shine in the moment, those who know they are right where they belong; those who deny that there are such players are swallowing sabermetric bilge; these guys believe the pitchers, not them, are in trouble here

Aug 12, 2014 10:19 AM
rating: -13
 
BrewersTT

You may be right about how the players view themselves, but wouldn't we expect to easily find a difference in statistical performance in different game situations if this were a repeatable, significant effect? How could it not show up in the statistics? Russell certainly could be right that we haven't looked at it in the right way yet, but if we have to look at it with careful focus, then it has to be a limited effect: limited to a small group of players, limited to certain conditions, very limited in scope, or some such constraint.

Aug 12, 2014 10:27 AM
rating: 2
 
BP staff member Russell A. Carleton
BP staff

I can live with that. I think clutch is majorly over-sold as a narrative, but a limited view of it might stand up to more scrutiny.

Aug 12, 2014 10:35 AM
 
Johnston

Bah. Romantic nonsense.

Aug 12, 2014 12:15 PM
rating: 3
 
Dodger300

The corollary to what you are saying is that the players who "shine in the moment" must be giving up lots of at bats when the game isn't on the line.

That is hardly praise for them. I think they should be trying hard all the time.

Aug 13, 2014 04:55 AM
rating: 2
 
BrewersTT

I follow the statistical approach and your caveats about it, but I'm not sure I see why swing tendencies are a proxy for clutch hitting. I would agree that a significant swing difference would tend to result in different performance, but not necessarily better or meaningful clutch production.

You state positive swing differences correspond to striking out less. I find this one of the more surprising discoveries here. Taken to its extreme, it would suggest that hitters with no discipline would seldom strike out, but that isn't what happens. More swinging would only guarantee more contact if the mix of pitches faced remained the same.

Aug 12, 2014 10:22 AM
rating: 1
 
BP staff member Russell A. Carleton
BP staff

I used swing tendencies as a proxy for "Is affected by high leverage."

Aug 12, 2014 10:43 AM
 
rrvwmr

Swing tendency can be a proxy for "is affected by high leverage," but shouldn't you be looking for a proxy for "performs better/worse in high leverage?"

Aug 12, 2014 14:45 PM
rating: 0
 
StefanAFrisch

I'd be curious to know the relation between the leverage measure and perception of leverage. Some of the noise might be from the relation not being linear, or the leverage measure over or under valuing leverage relative to perception.

Aug 12, 2014 10:33 AM
rating: 0
 
BP staff member Russell A. Carleton
BP staff

Oh if only I could find that dataset...

Aug 12, 2014 10:35 AM
 
gweedoh565

You could at least look at the REALLY obvious "clutch" situations, right? Like >8th inning, tied or down by 1 or 2 with men in scoring position. Surely those situations are universally recognized as critical even by young players of the game.

And in lieu of treating that as a binary clutch/not-clutch variable you could use LI in those "obvious" situations and some baseline for any non-clutch situation.

Also would be interesting if those clutch situations could be weighted by the importance of the game itself (i.e. a game for a team in a pennant race means more (has higher "game leverage"(?)) than a Rangers game).

Also, lasers! You could include lasers!

Aug 12, 2014 17:45 PM
rating: 3
 
Randy Brown
(189)

This reminds me of the concept of pitching to the score, with "swing more or less often" taking the place of "throw more strikes even at the expense of giving up more hits". And as I recall the last research I saw on that subject showed no observable difference in ERA/FIP based on the score.

Is it possible, or even likely, that the batter outcomes are similar even if the player changes approach?

Aug 12, 2014 11:26 AM
rating: 0
 
BP staff member Russell A. Carleton
BP staff

He could end up with the same results, just through different means (grounding to second, rather than striking out). Even here, I found that Ks and BBs went down, but outs in play and doubles/triples went up. It's possible that in terms of run production, it all washes out.

Aug 12, 2014 11:44 AM
 
bmrelyea

Did your regression analysis control for factors such as number of outs, number of runners on base, whether those runners were in scoring position, run differential, scoring opportunities remaining, etc.? Depending on the "high-leverage" situation, a "clutch" hitter might be more or less inclined to swing at the first pitch.

Aug 12, 2014 13:02 PM
rating: 0
 
ravenight

One analysis I haven't seen, though I would be surprised if it hasn't been done, is to base the measure of clutch hitting on the probability and potential of the result, not just the effect of it. So, for example, if you are up in a 1-run game with runners on 1st and 3rd and 1 out, hitting a deep fly ball is clutch, even if the outfielder throws out the runner or the runner stumbles or whatever. Likewise, hitting a walk-off homer in the bottom of the 9th is just as clutch as hitting a game-tying homer, even though it increased the odds of victory by more. Basically, if BABIP is noisy, and a batter's ability to improve in the clutch is noisy, it would be good to try to remove as much BABIP noise as possible when measuring clutch ability.

You could argue that a better analysis would be to categorize the PA into more controlled outcomes (BB, SO, GB, FB, LD), and see how the predicted results of an outcome of that type would affect Win Probability, with respect to the maximum (positive & negative) effect they could have. So if the worst you can do is drop the probability from .4 to .35, but the best takes it from .4 to .7, then you get a little credit for leaving it at .4; if the worst takes it from .4 to .25 and the best from .4 to .5, you get more credit for leaving it at .4. Or, in other words, some of clutch hitting is avoiding negative results at important moments.

The other aspect that I'm a little less clear on is the probability of a result. WPA should take into account the typical distribution of results in that situation, but it has less to say about the rarity of a particular outcome. Even normalizing for leverage doesn't exactly remove that as a factor - a rare outcome probably changes the WP by more, but not necessarily in proportion. This could actually cut both ways - perhaps HRs are a powerful enough outcome that they are actually disproportionately represented in WP.

I think if we are talking about fans watching certain players and saying that guy is clutch, we have to assume that for that to really be true in a way that is meaningful, the effect has to be fairly large (at least, on the order of perceptible difference in BA). But when a fan watches a situation, they don't see "well, a walk only adds .01 of a win, a double adds .05, and a homer adds .2" and then judge the double as a mediocre outcome - they see the possibilities of making an out, driving in a run, or getting on base. They are more likely to call a guy who drives in runs consistently a clutch hitter, obviously, but a guy who strikes out, grounds into a double play or hits a lazy fly ball is going to seem a lot less clutch than a guy who hits is hard somewhere. So maybe the question we've been asking is "do hitters accomplish anything useful with clutch hitting, and can they repeat it?" and we should really be asking only one of those questions at once.

Aug 12, 2014 13:34 PM
rating: 4
 
apfeffer

I wonder if the relative value of a walk is lower in high leverage situations, with the tying or go ahead run already on base. If so, increasing your swing tendency could be a sign of baseball intelligence.

Aug 12, 2014 16:56 PM
rating: 0
 
sbnirish77
Other readers have rated this comment below the viewing threshold. Click here to view anyway.

What next?

BP to acknowledge that PEDs actually help performance?

Shocking.

Aug 12, 2014 17:58 PM
rating: -15
 
MGL

"It's possible that in terms of run production, it all washes out."

I am confused. You found somewhat of a "talent" for swinging more or less at the first pitch depending on leverage, and you also found that swinging more or less at the first pitch affects walks, K, outs, and extra base hits, right? But you don't know if this actually makes the batter better or worse - it could be that just their approach changes, but not necessarily the win impact, right?

So, why, in your first sentence, do you say:

"By clutch hitting, I mean that certain players have some sort of ability to perform better in higher leverage situations."

Did you find that some players do perform "better" by virtue of this swing change, or not? "Better" has to be that their win impact goes up. It can't just be a change in behavior without knowing how that affects their win impact, right?

Aug 12, 2014 19:52 PM
rating: 3
 
BP staff member Russell A. Carleton
BP staff

Actually, that's just because of the realities of the publication schedule. I didn't have time to look into whether it washed out or whether it was a net positive. Your critique is valid. My only defense is that I had a busy week.

Aug 12, 2014 19:59 PM
 
bigpete123
Other readers have rated this comment below the viewing threshold. Click here to view anyway.

David Ortiz has something in him, be it ability to deal with pressure or mental fortitude whatever you want to call it, but the best way i know how to describe it is clutch. There may not be statistical evidence to back this up, but at the plate with the game on the line he elevates his game.

Aug 12, 2014 22:15 PM
rating: -7
 
BrewersTT

The statistics record what actually happened. If there's no sign of it in there, it didn't happen. If a guy like Ortiz gets more loud hits in key situations, it could be because he gets more loud hits than other guys in all situations.

It must be said though that even a rigidly data-driven person on this subject will remember Ortiz's 2013 World Series for a long time.

Aug 13, 2014 08:25 AM
rating: 3
 
You must be a Premium subscriber to post a comment.
Not a subscriber? Sign up today!
<< Previous Article
Premium Article What You Need to Know:... (08/12)
<< Previous Column
Premium Article Baseball Therapy: Big ... (08/05)
Next Column >>
Premium Article Baseball Therapy: Beco... (08/19)
Next Article >>
Premium Article The Prospectus Hit Lis... (08/13)

RECENTLY AT BASEBALL PROSPECTUS
Fantasy Article Player Profile: David Robertson
Fantasy Article The -Only League Landscape: National League ...
Fantasy Article Tale of the Tape, Dynasty Edition: Nick Burd...
Fantasy Article Dynasty League Positional Rankings: Top 50 R...
Premium Article Rumor Roundup: Huston Street Might Have Reli...
Pitching Backward: In Search of the Winninge...
Rubbing Mud: A Solution Does Not Exist

MORE FROM AUGUST 12, 2014
Premium Article What You Need to Know: August 12, 2014
Premium Article Pebble Hunting: The Unlikeliest Part of the ...
Notes About Baseball: Hit By Pitches: On Acc...
Fantasy Article The Stash List: 17th Edition
Fantasy Article Deep Impact: Week 19
Premium Article Daily League Strategy: Up Latos
Premium Article Minor League Update: Games of Monday, August...

MORE BY RUSSELL A. CARLETON
2014-08-27 - BP Unfiltered: Playing Favorites
2014-08-26 - Premium Article Baseball Therapy: How Billy Beane Built the ...
2014-08-19 - Premium Article Baseball Therapy: Becoming An Adult f/x
2014-08-12 - Baseball Therapy: I Believe In Clutch Hittin...
2014-08-05 - Premium Article Baseball Therapy: Big Extension, Big Mistake...
2014-07-30 - BP Unfiltered: The Ultimate Showdown of Ulti...
2014-07-29 - Premium Article Baseball Therapy: Trading Ryan Howard For No...
More...

MORE BASEBALL THERAPY
2014-09-04 - Premium Article Baseball Therapy: I Guess You Just Throw The...
2014-08-26 - Premium Article Baseball Therapy: How Billy Beane Built the ...
2014-08-19 - Premium Article Baseball Therapy: Becoming An Adult f/x
2014-08-12 - Baseball Therapy: I Believe In Clutch Hittin...
2014-08-05 - Premium Article Baseball Therapy: Big Extension, Big Mistake...
2014-07-29 - Premium Article Baseball Therapy: Trading Ryan Howard For No...
2014-07-15 - Premium Article Baseball Therapy: Why Are We Playing Hunger ...
More...

INCOMING ARTICLE LINKS
2014-10-08 - BP Unfiltered: The Cardinals' Flickering Mag...
2014-09-05 - Premium Article Painting the Black: The Short Lifespan of a ...
2014-08-15 - Premium Article Pebble Hunting: Arizona's Pitching Problems