keyboard_arrow_uptop
The BP Merch Store is now open! BP360 is back, plus tons of apparel options!

There’s a meme that I occasionally see floating around about how having a “cheat day” when on a diet is actually a good thing for those trying to lose weight. There’s probably something to that. Dieting requires a level of commitment and discipline that can be hard to sustain, and if knowing that you have a “cheat day” keeps you from giving up entirely, it might end up being a net-positive to have one. But the meme goes further to say that if one cheat day is positive, imagine what seven cheat days would do.

Lately, I’ve been writing a lot about the infield shift. I’m not a fan. I’m not one of those people who screams “play the game the right way!” If it worked, I would gladly encourage teams to do it. I’m just not fully convinced that it works as well as thought. The problem seems to be that while the shift does a good job at converting balls in play into outs, it comes with side effects. It changes how pitchers pitch. With the left side of the infield short-staffed (against a left-handed hitter), they tend to shy away from the outer-third of the plate where the batter might be tempted to poke it to the left side, and as a result, they throw more balls than they otherwise would. This seems to lead to the obvious side effect of more walks, but also puts the batter in a more advantageous count more often. It’s not a total effect. Pitchers still do go toward the outer-third, but the effect is big enough to have some real consequences for the game, and—importantly—it seems to allow more walks than it prevents in singles.

Last week in this space, I found that the “walk penalty” isn’t a static effect. There appear to be certain types of hitters and pitchers who appear to be more immune to it. In particular, pitchers who get a lot of swings-and-misses seem to do better (from their perspective, meaning more outs). On the flip side, batters who make a lot of contact seem to get better results (meaning fewer outs). But knowing that there were things that might change the magnitude of the “walk penalty” meant that there was a chance that some cases were OK to shift in. Even if the walk penalty was brought just a tiny bit below the benefit from the extra infielder on the right side, it’s still net-positive, and so a team should do it.

Well … let’s see if we can find those shifts that are worth saving and … maybe a few surprises in the math as well.

Warning! Gory Mathematical Details Ahead!

Once again, I’ll be working with data downloaded from Baseball Savant detailing events from 2015-2019. The data have an indicator for each pitch of whether the defense was lined up in a shifted formation. They note two different types of shift (in addition to a standard formation) of a partial shift and a full shift. Previous research that I (and others) have done shows that these partial shifts aren’t really all that effective at suppressing singles, and as such, I exclude them from the data set.

I used a regression to create a model of what the expected outcome would be for a given set of circumstances, with and without the shift on. Teams have to make the decision about whether or not to shift before they know the outcome, and the batter has the added benefit of knowing that decision before stepping into the batter’s box. Teams should make the best decision that they can. I used wOBA coefficients (located conveniently in the data set) as my outcome measure. I created a regression which included both the batter’s average wOBA contribution, and the pitcher’s, per PA over the course of that year, along with some key situational variables (is the batter left-handed, is the platoon advantage in effect?) and some batter and pitcher statistics, including swinging strike rate, ground ball rate, pulled ground ball rate, and called strike rate for batter and pitcher. 

(As a check, I ran mostly the same analysis using whether the plate appearance ended in an on-base event or not, and got the same basic model and answers to the subsequent questions below.)

The regression I used was stepwise, so that the model could pick the most influential variables. Not surprisingly, batter and pitcher wOBA per PA rates were the most strongly correlated (with … wOBA itself). I first used all plate appearances which did not feature a shift and entered the rest of the variables to generate an equation that would predict the expected outcome in wOBA terms. Platoon advantage, and whether the batter was left-handed entered in. I did the same for all PA involving a shift. On that one, in addition to the same variables as above, how often both batters pulled their ground balls and how often pitchers had their grounders pulled (for obvious reason) and how often pitchers got swinging strikes, which we saw last week was an important piece of the puzzle.

Once I had the two equations, I could estimate—independent of what actually happened—the expected value of shifting and not shifting. If a team could do likewise, the correct answer is to pick the one with the lower value. 

First, I ran a simple frequency of how often shifting made sense and found that … 40 percent of the time it made sense? (I told you that there’d be surprises…) Huh?

Here’s a different question: Among the shifts that actually happened, how many of them were supported by the model? The answer: 61.8 percent. You can be a cynic and say that 38.2 percent were not good ideas. Or in statistical parlance, teams reacted to a non-present signal, which is a Type I Error.

But wait a minute, I thought that the data said that shifting overall was a bad idea. It is, even though the majority of shifts were actually supported. When you look at how big a penalty those “bad idea” shifts had versus the average gain in expected value from the “good idea” shifts, the bad ones had a magnitude that was about 1.5 times greater than the good ones. The shift is still a net-loss for the defense, but … the data suggest that the best cure isn’t deleting the shift altogether, but cutting back the bad ones. 

And now a question even I had never thought to ask. I’ve looked at cases where the shift was employed and found overall that the effects were actually bad for the defense. What about the cases when a team didn’t shift? What happened there? The answer was that 33.4 percent of cases where a team should, have shifted, according to the model, they didn’t. (For the initiated, they made a Type II error.)

In a sentence that makes no sense but is perfectly correct, the shift is both over-used and under-used. Put a little more succinctly, teams aren’t necessarily shifting the wrong amount, but they are shifting in the wrong situations.

There is one problem:

2015: 29.1%
2016: 36.0%
2017: 41.2%
2018: 38.3%
2019: 45.9%

These are the percentages of shifts that teams “got wrong” according to the model. As the use of the shift as a strategy has been increased (with a blip dip in in its use between 2017 and 2018), teams have actually been loading up on more “bad” shifts (Type I errors). Over that same time period, they did cut down on the percentage of times when they didn’t shift, but should have (Type II errors), but the Type I errors outpaced the reduction in Type II errors.

What gives? I isolated all shifting situations and then split them by whether the model thought kindly of them or not. I compared the overall statistical characteristics of both batter and pitcher in the situations that the model liked and the ones that it didn’t like, and found … nothing that interesting. And then, in a throw-away analysis, I looked at the two variables that I’d included in the regression that were categorical: whether the batter was left-handed and whether the platoon advantage was in effect.

Among shifts that did happen, if the batter was left-handed, the model thought it was actually a good idea 84.1 percent of the time. Among shifts that happened against right-handed batters, the model had a favorable opinion 1.6 percent of the time. That’s not a mis-print. One point six percent. The model clearly thought that the secret to the shift was simply noting where the batter was standing. Could it be that simple? I looked at the situations where the shift didn’t happen and found the model telling me the same things. Shift the lefties. Don’t shift the righties.

(Here I’m going to pause and send you to read (two) something(s). There’s an amazing Sabermetrician who found this a year and a half ago whose name is Russell. It just isn’t me. It’s Russell Eassom at the blog “Bat Flips and Nerds.” Go read those.)

Once I had Eassom’s findings in hand, I pulled out the analysis code that I have used in the past concerning the effectiveness of the shift, and quickly paneled the data by batter handedness. For a quick recap, this takes hitters who had more than 50 non-shifted PA and sets their results from those PA as their expectation for when they are shifted. For example, if a hitter gets 20 singles in 100 non-shifted PA, if the shift is a better defense, we would expect to see singles in fewer than 20 percent of the same batter’s shifted PA. Small sample sizes will overwhelm the analysis of an individual hitter, but if you pile everything together, this can give us an idea of what’s going on in the aggregate. Data are once again from 2015-2019.

Change from baseline for outcome Left-handed batters Right-handed batters
Strikeouts + 2.2% – 4.7%
Walks + 1.1% + 0.9%
HBP – 0.0% + 0.0%
Single – 2.0% + 0.2%
Double/Triple – 0.0% + 0.3%
Homerun + 0.0% + 0.4%
Out in Play – 1.3% + 2.8%
On Base Event – 0.9% + 2.0%
BABIP – .016 – .006
Linear Weight Runs (per PA) – .007 + .018

This is where I sheepishly say, “Yeah, why didn’t I think of that?”

The shift does seem to “work” for reducing BABIP for everyone the way that it’s supposed to, a little better for lefties than righties, but profit is profit. The walk penalty is still a real thing. We see that walk rates do go up in front of the shift, on both sides of the plate. What differs is that the shift also seems to increase the number of strikeouts among left-handed batters as well, while having the completely opposite effect for righties. Righties also see an increase in their extra-base hits and their home run output, while for lefties, it’s mostly flat. Handedness is clearly the key variable. (I broke the numbers down further to see if the handedness of the pitcher made a difference, and it did, but the result was still the same and the handedness effects for the pitcher are confounded with the platoon effect.)

With lefties, the shift does steal back more outs than it gives away in walks, though surprisingly, in the form of strikeouts. It seems that pitchers go for a style of pitching which favors a lack of contact, which does produce more walks, but also more K’s. Considering everything, the shift is a net positive against lefty swingers. Not so much for righties, where the ball tends to go into play more with destructive results.

Which brings me to this chart which makes me feel a little better about myself, even though I totally got things wrong:

Year Percentage of LHB Shifted Percentage of RHB Shifted
2015 21% 4%
2016 27% 7%
2017 25% 5%
2018 34% 9%
2019 48% 15%

Major-league teams have been increasing the rate at which they have been shifting everyone, left and right. While they’ve been increasing their proportion of shifts on left-handed batters, on which they accrue value, they’ve also been increasing their percentage of shifts against right-handed batters, which are toxic and cancel out much of the benefit that they would have gotten by just sticking to shifting the lefties. 

Major League Baseball didn’t see it either.

It means that much of my own writing on the topic has been conflating two different effects. The walk penalty is a real thing, and I stand by it. I assumed that this was what was driving the finding that the shift was a net negative. It certainly doesn’t help, but what seems to have been driving the bus is that somehow, shifting against right-handed batters somehow supercharges them and reduces their interest in strikeouts. Even though the majority of shifts are done against lefties, the negative effect of shifting against righties is much greater than the effect of the benefit of shifting against lefties, so the net effect of these “missing” righty strikeouts (and the fact that some portion of them became extra base hits and home runs) was turning the shift into an overall negative. Teams were doing something right and something wrong at the same time.

I just sort of assumed that if the shift worked (or didn’t) against lefties, it would work (or not work) the same way against righties. I assumed that it was all a mirror. I feel a little foolish making that assumption now. There’s the simple geography that if you shift a lefty, the one infielder on the left side can play in the shortstop hole and probably at least sorta defend a broader area of fair territory than the lonely first baseman who has to stick close to first base to receive a throw on a potential ground ball out. There’s the reality that most pitchers (and batters) are right-handed, so it was more likely that a right-handed batter was facing a same-handed pitcher, while a lefty was facing an opposite-handed pitcher. Same vs. opposite-handed pitching involves two different sets of strategies and the ball breaks differently from the batter’s perspective.

I looked in that mirror and didn’t realize that I was the one who was backwards.

***

Before I go, I want to tease one possible implication of this finding. I don’t feel fully confident to recommend this yet, but the data are at least suggestive. I admit it. I’m hedging.

The data that we saw above say that left-handed hitters should almost always be shifted. Somehow—and until I really can understand the “how” on this, I want to be careful—a shift changes either the pitcher or batter in such a way that more walks happen, but also more strikeouts. Maybe it’s as simple as the pitcher still feels a little antsy about letting the ball go into play and there being a silly single to the left side, and so the pitcher goes to the strikeout bag of strategies. Whether that’s part of the shift or something that could be altered by just realizing that it was happening and changing behavior, I don’t know. But according to these preliminary data, left-handed batters should almost always be played in a 3-1 formation. Righties should be played 2-2.

I’m someone who gets worked up over words. I find that when there’s a concept that doesn’t have a word, there is often opportunity. (I call them groffles or “unspoken words.”) I (and others) have written about baseball evolving toward what might be called “positional anarchy.” When the shift first came to prominence, there were chuckles when a ground ball to the 34 hole found the third baseman in short right field, and the scorecard said G5-3 as a result. But we all understood that it was just part of a special strategy that was used against special hitters. Now it seems that the special lefty might be the one that doesn’t get shifted. What do we call it when the third baseman playing in short right field is now standard practice, at least when a lefty is up? What happens when the third baseman spends a third of the game nowhere near third base and that’s just an accepted part of the game. Nothing weird about it.

We might end up needing to create a new word for that player.

Thank you for reading

This is a free article. If you enjoyed it, consider subscribing to Baseball Prospectus. Subscriptions support ongoing public baseball research and analysis in an increasingly proprietary environment.

Subscribe now
You need to be logged in to comment. Login or Subscribe
Shaun P.
9/02
Wow! So walk penalty is having an impact, but it's not the driving force.

Ok, now I am super curious about something, Russell. Mark Teixeira. The shift seemed to utterly defeat him in his last years with the Yankees. But he was a switch hitter - could he have simply just hit righty anytime he got - or was likely to get - shifted? Asked more broadly, given the difference in better handedness when facing a shift, are switch hitters (theoretically) shift proof? Should they be? Would looking at the few hitters who can hit from both sides of the plate reveal anything about this? I know there aren't a lot of switch hitters...
brewerstt
9/02
Wouldn't the defense shift the shift to the other side in such a case, to negate the strategy? Or maybe what you are saying is that switch-hitters are more used to using the whole field.
Mike Juntunen
9/02
I mean, if you read the article what Russell demonstrates here is that shifted outcomes are substantially different for left handed and right handed hitters, and shifting right handed hitters is not equally effective because of it; particularly, that left handed hitters strike out more when they're being shifted, and right handed hitters don't. It isn't clear exactly what causes that and whether those tendencies would be true for the same player on opposite sides of the plate, but it's certainly possible and the answer isn't as simple as "just use a right handed shift". The whole point is that right-handed shifts are less effective than left handed ones for reasons that we can see but perhaps not entirely explain just yet.
brewerstt
9/04
Right, my question was very poorly worded. The assumptions in switching to bat RHed are that the general failure of the shift strategy against RHers would continue no matter the hitter, no matter the pitcher, and no matter the frequency with which it is employed. If I were a manager, I would want to see that play out before giving up on shifting against RHers.
Shaun P.
9/02
Maybe, but I think Mike answered that below. And yes, I think switch hitters in general do tend to be more "all field" hitters, though as I recall Tex - especially on his later years - was more a pull hitter (I could be mis-remembering). But it seems to me that looking at switch hitters, because they can hit from both sides of the plate, might reveal something. At least it can't hurt!
Shaun P.
9/02
Sorry, above, not below. I forget that even if I reply to a comment, the reply goes to the bottom of those comments instead of nesting.
Mike Juntunen
9/02
I've enjoyed this series even though I've had skepticism about some of the conclusions, particularly because some teams employ several dozen people as brilliant as Russell, and it's hard for me to imagine none of them have considered or investigated this, yet they still shift.

That LHH shifting is quite profitable aligns with MLB team behavior overall and makes sense. But there's a thing that I would really like to ask, though I understand that it may be possible, and this is in part inspired by Russell's recent Effectively Wild apperance where he discusses this with a pro-shift proponent from SIS (the creators of Defensive Runs Saved):

DRS tracks team shift runs as a team-level statistic, and that statistic tells us that certain teams shift really effectively and some teams do not. Since Team Shift Runs was introduced, the same couple teams have occupied the top of the list, most notably the Dodgers. The Dodgers were also famous for being an early adoptor of coach/card-oriented defensive positioning micro-adjustments, and using that sort of information to take people who were considered very bad defenders and get largely average results from them (2018 Matt Kemp) or guys who were not viewed as long-term possibilities at a position and keeping them there (Corey Seager). I think we have evidence that some teams, including the Dodgers, are much better at this sort of micro-positioning in order to optimize their player performance.

This leads to my question: Is it possible that some teams also position within their shifts more optimally? Choose their base/count/hitter and other situations better? Coach their pitchers to perform better with the shift behind them, or even have recognized the walk penalty and tried to coach to it directly by easing pitcher confidence?

I recognize this is hard to research because a single team's defensive performance over even a whole season is a pretty small sample, and different teams may get their results in different ways, so considering them as a group may mask their individual strategies. But it seems to me that it is worth considering, and I would really be interested into a deep-dive of the teams that DRS consistently rates as the most positive-value shifting teams in MLB (The Dodgers and Diamondbacks and Rays topped the list in 2019. Searching this stat is limited by what appears to be a bug: if you try to do a multi-season team defensive search on FanGraphs, where this data is searchable, you get the multi-season totals, but the 2019 Team Shift Runs data no matter how big your range. )

The research and analysis that Russell's done on this here at BP is really good, but it's really hard for me to believe that teams with 30+ person analytics departments that shift so heavily haven't gone through some of this same material, and still concluded they should shift. I have to suspect they've either methods they believe optimizes their shift performance, believe they can coach their players to not change their approach, or have found some other piece of information that we haven't yet arrived at that makes the choice clearer.

When I think about the penalty for walks I think a lot about something I hear on Dodger broadcasts whenever Orel Hershiser takes a day off and Nomar Garciaparra fills in. Nomar is not a big fan of the shift, but he doesn't criticize it directly. Instead, when he sees a player do something that they would only do in a shifted environment, he comments about how weird and awkward that play would be: a 3rd baseman turning a double play, the shifted outfielder having to charge the ground ball in order to be able to feed it to second for the force, etc. He constanty bangs the drum that these plays would be so strange because you have not practiced them. He seems to miss out on the reality that modern baseball players have practiced them, for years, and, as Russell mentions, some of these alignments are as common or more common than the standard 2-2 infield alignment on some teams. The idea that a 3rd baseman being called on to turn the double play like a shortstop is inexperienced with it is silly: he probably practices that play nearly as much as the shortstop, and he's been doing it for years. And as time moves on and players have been running shifts since they were in the low minors, their familiarity with all these will only continue to increase, and it won't be something they had to learn during their MLB careers like it is for some older infielders today. I feel like you must see the same thing with pitchers. Older MLB pitchers have been resistant or critical of the shift at times and vent their frustration after or even during the game about it. I cannot think of a single incident of a 24 year old pitcher doing that sort of thing, because it's totally normal to them: it's the way they've always experienced professional baseball. The psychology of the players themselves is going to change over time, even if the individual players are not able or willing to change, due to the turnover and introduction of young players accustomed to the shift. I would think it might be reasonable to expect to see the walk penalty decrease over time, or that it might be possible to research it and see if there is a difference in the profile of which pitchers do and do not have larger walk penalties by age range and/or handedness. In particular, if we find that younger pitchers do not suffer the same penalties, we might be able to conclude that MLB teams have decided that the shift will be worthwhile in the long run and they're going to continue it, even if individual resistant players are indeed penalized by it, because complete normalization of it over years is the only way to realize that advantage.
Shaun P.
9/02
I too wonder about the tons of team analysts who either haven't found this or think they can counteract it. Part of me wonders if we are giving them too much credit, which I think Ben alluded to on that episode of Effectively Wild. If they are focused on current projects and not reviewing something that their team already "knows" works, they might indeed miss it. Which is not a knock on them!
There are only so many hours in a day and plenty of problems to solve. But just because teams are a lot smarter than they used to be, doesn't mean they are omniscient. We give a lot more deference to teams than we did 15 years ago, and maybe we should be just a little more skeptical sometimes.
Mike Juntunen
9/03
The only problem I have with this argument is that one of the things BP promotes about itself on the book jacket are quotes from prominent front office types who say they read the site every day. Russell's research is out there in public and I think it's kind of unfathomable to believe that no one inside team orgs is reading it; in fact, that Russell's so good at this is why he was hired on with a team for a while in the first place. Even if teams didn't revisit this on their own, this has been researched in public for a while now, and I think the idea that no one is seeing these articles and continuing on blithely is implausible. I would bet that most FOs with an analytics staff of any size have some intern or lower-level staffer whose job it is to filter new public research from well-regarded names and send it out to the team. Research published on BP has rocked the industry: most teams didn't have private framing data when BP published FRAA. Even as teams have gotten well ahead of public research due to the combination of superior, private data feeds and huge investments in staff, they're not in a bubble, and they're still aware of what's getting published on the outside.
Craig Goldstein
9/03
Mike, the certitude with which you're addressing this is uncalled for. When BP published framing stats and emphasized the value of it, the industry didn't turn on a dime. Just because the research exists doesn't mean it won't take time to verify it, or that they might see things that complicate it, or just that they spent X amount of hours training and communicating a certain strategy and it doesn't benefit a team to undo all of that at once. Just because they're continuing on doesn't mean they do so blithely. Maybe chill out a touch.
Craig Goldstein
9/03
Also your estimation of the size of analytics staffs is off by about about 10 and those are likely split by manner of assignment, so it's not like they have 20 people crunching numbers on shift data.
Mike Juntunen
9/03
I mean, there are some arguments I'd like to point out: response to catcher framing wasn't instantaneous, but it didn't take that long (a few years), which is remarkable given how few teams had analytically-driven front offices at the time, and I'd certainly presume the response to a present-day development of similar magnitude would be quite a bit faster (of course, they've done their best to make sure that'll never happen by omitting critical data fragments from publically searchable statcast data, like batted ball spin and spray vector, in order to ensure that public research can't spoil internal development the way some teams lost their cutting edge framing research), or inquire what exactly you mean in terms of analytics staffs, when I've seen acccounts that the Dodgers and Yankees R&D departments have numbers approaching 100 (not all researchers, and not all those of Russell's caliber, obviously), to name a few, but since the tone of my arguments seems to so frequently be taken as not-chill, I think the better course is to stop writing them out in the first place.
Mike Juntunen
9/03
Reply threading is such a wonky beast on BP, you can figure out where this goes, ah well.
Craig Goldstein
9/03
I don't think it needs to either be language that leaves little room for discussion or no arguments at all, but that's your call, ultimately. I am not aware of any R&D departments approaching close to that figure, especially in the sense that you're talking about and when I inquired the answer I got was ~20 for the larger groups.
Craig Goldstein
9/03
I guess, ultimately I don't see the value repeatedly proclaiming that these guys know all this and more and haven't reacted. Certainly it's possible, but if we're going to appeal to authority to that extent instead of research ourselves and wonder aloud, why are we here?
brewerstt
9/02
These studies are highly interesting. Somewhere along the series, it would be interesting to see how/whether shifts have affected base-running decisions and success.
morillos
9/02
Cricket-based suggestion for the 3rd base-right field guy: silly point. Just 'cause I love that name. (In cricket it IS silly, as you're about 2 feet from the batter, ready to catch a "foul tip", but also ready to be creamed my a drive!)