BP Comment Quick Links


August 12, 2014 Baseball TherapyI Believe In Clutch HittingI know, I’m not supposed to, but I believe in clutch hitting. By clutch hitting, I mean that certain players have some sort of ability to perform better in higher leverage situations. Leverage, for the uninitiated, is a concept formalized by sabermetrician Tom Tango. We know that some situations in a game are more important than others. When it’s 151, no one cares what happens in a plate appearance. When it’s the bottom of the ninth with runners on second and third, two outs, and the home team is down by one, pretty much the entire game rides on this atbat. Leverage index is a mathematical model of how much more important that late game situation is. Leverage is based on the idea of win probability. We can look at each game situation (let’s say, bottom of the third, one out, runner on first, and the home team down by two) and figure out over some past time frame how often the home and visiting team won. More to the point, we can figure out how much that win probability can change based on whatever is about to happen next. In the 151 situation, whatever the batter does is going to move the needle very little. In the bottom of the ninth, the win probability could go from roughly 50–50 to 100–0 in a hurry. When a batter does something positive that increases his team’s chances of winning, we give him credit for adding win probability (even if giving him all the credit is silly). In a highleverage situation, a batter can accumulate a lot of win probability in a single atbat. The standard test for whether there is such a thing as clutch hitting has been to look at the win probability that a player records over the course of a season and compare it to what his win probability would have been in situations where the leverage index was 1. (This is the basis of how our friends at FanGraphs calculate clutch.) From season to season, players show very little correlation on this measure of clutch. In general, the interpretation has been “clutch doesn’t exist” rather than “we had a poor measure of clutch to begin with.” Indeed, I have found that this measure of clutch eventually does become reliable. It just takes a while. Maybe there is signal in all that noise; maybe we need a better antenna. Warning! Gory Mathematical Details Ahead! I then took each equation and calculated the chances that each player would swing at a first pitch when the leverage index was 1 (average) and 2 (a situation twice as important as the average situation). Then, I subtracted the two and got a rough indicator of how high leverage began to affect a player (at least on this one behavior). I used a minimum of 250 plate appearances in a season and looked at players from 2009 to 2013. In the past, I’d found that clutch, as described above, had a yeartoyear correlation of .074. (I used a method known as autoregressive intraclass correlation.) For this group, across the five years, the ICC was .30. That’s not huge, but we call home runs a true outcome for pitchers with yeartoyear correlations in the same neighborhood. I termed this difference between predicted firstpitch swing rate “swing difference.” Some players swing a lot more when the leverage goes up. Some barely notice. A few start to freeze. Next, I wanted to see if swing difference predicted changes in outcomes. For the years 2009 to 2013, I used the logodds ratio method (which I have used multiple times before) to create a predicted percentage that each plate appearance would end in a strikeout based on the batter and pitcher’s usual rates in that area. I did the same for walks and singles and home runs and the rest of it. Next, I looked at all plate appearances in which a batter with 250 PA in that season faced a pitcher with 250 batters faced in that season. I created a binary logit regression in which I had my predicted percentage of a strikeout (for the initiated, expressed in a log of the odds ratio), and then entered in the leverage index for each plate appearance, the swing difference stat for the batter and the multiplicative interaction of swing difference and leverage. This type of analysis, called a moderator analysis, is wellsuited to answering the “clutch question.” If certain players have some sort of clutch factor (and here, we’re using swing difference as a rough measure of clutch) then as leverage increases, we would expect to see those who are higher on this clutch factor to show greater increases (or sharper decreases). That’s what the interaction term between swing difference and leverage does. If it’s significant, it means that as leverage goes up (or down), the effect it has will depend, at least in part, on that clutch factor. What I found is that for hitters who show more of an effect on swing difference (leverage makes them swing at the first pitch more), they were less likely than expected to walk and less likely to strike out as leverage went up. Instead, they showed higher rates of both extra base hits and outs in play. To show some sense of how much of an effect this could have, here are the numbers for strikeout rate. Let’s say that our pitcherbatter matchup stats alone would suggest that the chances of a strikeout are 20 percent. Now, let’s take a look at what would happen in a situation that has a leverage value of 1, and compare a batter who has a swing difference of .10 (he swings at first pitches ten percent more often in higher leverage situations than he does in mediumleverage situations) and a batter who has a swing difference of 0 (he swings equally in both situations). The values are the likelihood of a strikeout happening.
In an averageleverage situation, the two hitters are about the same (they differ at the fourth decimal place), but once the leverage is turned up a bit, they get different results. Not by a lot, but it’s there. You get the same basic effect sizes for the other outcomes. Before we go further, the careful observer will note that there’s a certain tautology that goes along with these analyses. I think it doubles as both a feature and a bug. A batter who is more likely to swing at the first pitch in highleverage situations is probably just more likely to swing in highleverage situations. It’s no wonder he sees a drop in his expected walk rate (and in some sense his expected strikeout rate). And if we’re saying that his swing rate drops because of leverage (or at least in accordance with leverage), then it’s not surprising that the effect appears. We’ll talk about this more in a bit. Clutch. Heart. Grit. Myocardial Infarction. These analyses may not completely prove that clutch ability exists, but they do lay what I hope is a foundation for how we might continue the search. “Clutch” is a way of saying that the situation matters because players are human. What we have here is an indicator that has reasonable (if not great) consistency across years, and it explains differences between players in how leverage affects them. More searching might find something with more consistency. Even then, yeartoyear consistency is not the only way to establish that a measure is reflective of a player’s true talent level. Using a more trackingbased approach might help. Players can and do change, even within a season. There’s no reason clutch needs to be an enduring trait, rather than a state we can detect with some reliability. The rest is simply showing that the factor, whatever it is, can explain some of the differences between players’ performances in different leverage situations. As to these specific analyses, it might very well be that what’s driving things is that some players are looking at the sorts of relievers they face in highleverage situations and saying “Well, he usually comes right at me, so no point in messing around. I might as well swing when I see something interesting.” It might not be a mystical force at work, but a very reasonable reaction to the circumstances. In that case, clutch isn’t even something psychological, but a mental skill. Still, there could be problems with multicolinearity. What this might be showing is that some players swing more in highleverage situations, and so we would expect them to take fewer walks, somewhat by definition. Then again, even knowing that information could have strategic value. Maybe when we have other data sets to work with, we might be able to look at measures of how leverage affects a player that aren’t based on game results. The other piece of this, and it’s one that I tried to drive home in the piece in the Annual that started everything, is that knowing that a player swings more (or less) often in highleverage situations might be good within the context of one skill set and bad within another. These analyses fall into the largeN trap that assumes that more swinging is better (or seems to be) for everyone. But if nothing else, I’d present these analyses as a way of reopening what had been assumed to be a closed debate. Clutch hitting might just exist.
Russell A. Carleton is an author of Baseball Prospectus. Follow @pizzacutter4
28 comments have been left for this article. (Click to hide comments) BP Comment Quick Links NYCRuss (74983) How much of "clutch hitting" is contextual in that rather than a player performing better in such situations, he's really performing worse in nonclutch situations? Aug 12, 2014 05:45 AM mertes79 (32946) I also wonder if a player always knows when he is hitting in a high leverage situation? Some are obvious, but I assume based on some of the relief pitcher literature I've read that there are perhaps less obvious high leverage situations in the 5th or 6th innings included in the analysis. Seems an important factor might be determining what situations a hitter perceives to be high leverage in terms of ferreting out whether he demonstrates "clutch hitting" ability that is somehow different from other players. Aug 12, 2014 07:46 AM BrewersTT (1952) If I follow you, this seems like a distinction without a difference, if the question is "do they perform better in clutch situations?". Aug 12, 2014 10:17 AM gweedoh565 (48688) This is a good point, but mertes79's comment does address that other clutchrelated question of whether players can "turn it up" when called upon, or when they sense that they need to, or something to that effect. Aug 12, 2014 10:22 AM The YerkesDodson curve suggests that people are actually at their peak at a moderate amount of stress, and that too little or too much have the same effect of driving down performance, but we don't know where the high point of the curve is. Aug 12, 2014 10:32 AM godfather (13975) Other readers have rated this comment below the viewing threshold. Click here to view anyway. there are, and always have been, performers who shine in the moment, those who know they are right where they belong; those who deny that there are such players are swallowing sabermetric bilge; these guys believe the pitchers, not them, are in trouble here Aug 12, 2014 10:19 AM BrewersTT (1952) You may be right about how the players view themselves, but wouldn't we expect to easily find a difference in statistical performance in different game situations if this were a repeatable, significant effect? How could it not show up in the statistics? Russell certainly could be right that we haven't looked at it in the right way yet, but if we have to look at it with careful focus, then it has to be a limited effect: limited to a small group of players, limited to certain conditions, very limited in scope, or some such constraint. Aug 12, 2014 10:27 AM BrewersTT (1952) I follow the statistical approach and your caveats about it, but I'm not sure I see why swing tendencies are a proxy for clutch hitting. I would agree that a significant swing difference would tend to result in different performance, but not necessarily better or meaningful clutch production. Aug 12, 2014 10:22 AM StefanAFrisch (72555) I'd be curious to know the relation between the leverage measure and perception of leverage. Some of the noise might be from the relation not being linear, or the leverage measure over or under valuing leverage relative to perception. Aug 12, 2014 10:33 AM Oh if only I could find that dataset... Aug 12, 2014 10:35 AM gweedoh565 (48688) You could at least look at the REALLY obvious "clutch" situations, right? Like >8th inning, tied or down by 1 or 2 with men in scoring position. Surely those situations are universally recognized as critical even by young players of the game. Aug 12, 2014 17:45 PM Randy Brown (189) This reminds me of the concept of pitching to the score, with "swing more or less often" taking the place of "throw more strikes even at the expense of giving up more hits". And as I recall the last research I saw on that subject showed no observable difference in ERA/FIP based on the score. Aug 12, 2014 11:26 AM bmrelyea (61233) Did your regression analysis control for factors such as number of outs, number of runners on base, whether those runners were in scoring position, run differential, scoring opportunities remaining, etc.? Depending on the "highleverage" situation, a "clutch" hitter might be more or less inclined to swing at the first pitch. Aug 12, 2014 13:02 PM ravenight (45272) One analysis I haven't seen, though I would be surprised if it hasn't been done, is to base the measure of clutch hitting on the probability and potential of the result, not just the effect of it. So, for example, if you are up in a 1run game with runners on 1st and 3rd and 1 out, hitting a deep fly ball is clutch, even if the outfielder throws out the runner or the runner stumbles or whatever. Likewise, hitting a walkoff homer in the bottom of the 9th is just as clutch as hitting a gametying homer, even though it increased the odds of victory by more. Basically, if BABIP is noisy, and a batter's ability to improve in the clutch is noisy, it would be good to try to remove as much BABIP noise as possible when measuring clutch ability. Aug 12, 2014 13:34 PM apfeffer (28628) I wonder if the relative value of a walk is lower in high leverage situations, with the tying or go ahead run already on base. If so, increasing your swing tendency could be a sign of baseball intelligence. Aug 12, 2014 16:56 PM sbnirish77 (17711) Other readers have rated this comment below the viewing threshold. Click here to view anyway. What next? Aug 12, 2014 17:58 PM MGL (2121) "It's possible that in terms of run production, it all washes out." Aug 12, 2014 19:52 PM bigpete123 (68101) Other readers have rated this comment below the viewing threshold. Click here to view anyway. David Ortiz has something in him, be it ability to deal with pressure or mental fortitude whatever you want to call it, but the best way i know how to describe it is clutch. There may not be statistical evidence to back this up, but at the plate with the game on the line he elevates his game. Aug 12, 2014 22:15 PM BrewersTT (1952) The statistics record what actually happened. If there's no sign of it in there, it didn't happen. If a guy like Ortiz gets more loud hits in key situations, it could be because he gets more loud hits than other guys in all situations. Aug 13, 2014 08:25 AM Not a subscriber? Sign up today!

Question to the community from a relative neophyte: have the studies that indicate little indication of clutch performance taken into account that, in the highestleverage at bats, the batter is generally facing the freshest, most skilled pitcher the other team has to offer? If a batter has an OPS of .850 over the course of the season (against leagueaverage pitchers), and an .850 OPS in highleverage situations when the game is on the line, he will usually be performing against far more skilled pitchers. I would interpret that as clutch performance.