Notice: Trying to get property 'display_name' of non-object in /var/www/html/wp-content/plugins/wordpress-seo/src/generators/schema/article.php on line 52
keyboard_arrow_uptop

Over the last few weeks, a press release has been making the rounds. It’s a persuasive press release that reports some interesting research, and wherever it goes, it produces a post. There’s just one problem: the research it reports is a little misleading.

The press release, which you can view here, was put out by the American Academy of Sleep Medicine. It summarizes the results of a recent study on the effect of fatigue on strike-zone judgment. The source of the study is a research abstract published in an online supplement of the journal SLEEP—you can access the abstract (PDF) on page A408 here—and the principal investigator behind it is Scott Kutscher, MD, an assistant professor of sleep and neurology at Vanderbilt University Medical Center.

The study’s conclusions are simple. Here’s one of the quotes from Kutscher in the press release:

Plate discipline—as measured by a hitter’s tendency to swing at pitches outside of the strike zone—got progressively worse over the course of a Major League Baseball season, and this decline followed a linear pattern that could be predicted by data from the six previous seasons. We theorize that this decline is tied to fatigue that develops over the course of the season due to a combination of frequency of travel and paucity of days off.

To make things even more interesting, the study promises a PED connection. As the abstract notes, “This effect became more pronounced following a ban on stimulants in 2005.” Or as one of the articles about the study says:

But whatever players put into their bodies today to fight fatigue, it no longer includes amphetamines—or at least it doesn't unless those players want to risk getting slapped with a stiff suspension. It's that distinction that Kutscher cites as proof that fatigue, and not other factors, is driving the impaired strike-zone judgment that hitters experience as the season wears on.

So, to summarize the study’s claims (and strong suggestions): Plate discipline declines as the season goes on. This decline stems from increased fatigue. And the effect has become more pronounced since baseball banned amphetamines, which players used to take to counteract fatigue.

There’s a reason—actually, there are a few reasons—why this study has been spreading so fast. It’s about baseball, which means that it appeals to a wider audience than your average research abstract. It reports an intriguing discovery with real implications: if fatigue is making batters worse, then perhaps teams could improve their performance by taking steps to minimize that strain. It’s tied to PEDs, and it appears to isolate a performance-enhancing effect, which is notoriously difficult to do. It comes from a reputable source—a doctor/professor who specializes in sleep and neurology at a university you’ve heard of. And it feels like a fairly intuitive finding—we’ve all experienced the fatigue associated with travel and a busy schedule, and we know how lack of sleep can impair our performance. Why shouldn’t it impair baseball players’ performance, too?

Thanks to that potent confluence of factors, there’s a pretty good chance that you’ve seen something about this if you read about scientific research or baseball on the internet. On the science side, we have ScienceDaily, Medical Daily, PopSci , Medline Plus, and Medpage Today. On the baseball side, we have Grantland, Hardball Talk, Baseball Musings, Reddit, and The Boston Herald. And then there are the general news sources, like the Nashville Business Journal and US News & World Report. (And so on—I’ve been told that the New York Times will soon publish its own report on the research. *Update* Here it is.) Some of those posts merely regurgitate or link to the press release, while others interpret and editorialize. But none of them raises any serious doubts about the study’s methodology or conclusions.

So, is this study misleading? There are three aspects of the research that deserve some scrutiny. First, there’s the claim that plate discipline erodes throughout the season. Second, there’s the claim that the cause of that erosion is fatigue. And third, there’s the claim that baseball’s amphetamine ban has exacerbated the effect of this fatigue.

Let’s tackle each of those in turn.

Does plate discipline erode throughout the season?
Here’s what the abstract says about the data behind the study:

We have two publicly available sources of plate discipline data: PITCHf/x, which powers plate discipline stats at both BP and FanGraphs, and Baseball Info Solutions, which licenses information to FanGraphs. Kutscher’s study draws on the BIS data, which is collected by stringers from video and extends a few years into the pre-PITCHf/x era. The abstract reports a straight April-to-September comparison, but the researchers also looked at data from each intervening month.

We were able to verify that the April and September O-swing rates reported in the abstract are accurate, according to the BIS information at FanGraphs. And the same trend appears in BP’s PITCHf/x-based plate discipline data, although the numbers aren’t the same: from 2008 to 12, league-wide O-swing rates were 27.6 percent in April and 29.0 percent in September.

But that “linear increase” breaks down when you look at the monthly splits:

Month

Pitches

O-swing Rate

March

5002

29.48

April

512712

27.56

May

610150

28.19

June

574238

28.64

July

555887

29.23

August

608948

28.99

September

579149

29.04

October

42628

29.50

O-swing rates were at their lowest in April. But (disregarding the smaller sample size of October) they peaked in July, then declined slightly and held steady in August and September. If fatigue were impairing plate discipline, we’d expect the rate to be higher in August than July, and higher in September than July and August. But that’s not what we see.

Moreover, reporting O-swing rates alone offers an incomplete picture of plate discipline. In an email exchange with Russell Carleton, Kutscher said that the researchers focused on O-swing because swinging at a ball is always a bad idea. (Pablo Sandoval begs to differ.) Judgment and reaction time are known to be affected by sleep deprivation, so O-swing rate would seem to be the plate discipline stat most susceptible to fatigue.

Except, well…what do we see when we look at not just O-swing rate, but also Z-swing rate (the percentage of pitches inside the strike zone that drew swings) and overall swing rate? Again from 2008-12:

Month

Pitches

O-swing Rate

Z-swing Rate

Swing Rate

March

5002

29.48

63.05

45.92

April

512712

27.56

61.56

44.72

May

610150

28.19

61.87

45.36

June

574238

28.64

62.24

45.81

July

555887

29.23

62.48

46.13

August

608948

28.99

62.41

45.98

September

579149

29.04

62.37

45.87

October

42628

29.50

62.95

46.43

It’s not just O-swing rate that’s higher in September than April. Z-swing rate follows the same trajectory. So hitters don’t just swing at pitches outside the zone less often in April—they swing at all pitches less often in April. And swinging at strikes is generally a good decision, or at least not a bad one. Why would fatigue cause batters to swing at more pitches inside the strike zone? When you examine the other swing rate stats, the narrative that batters make worse decisions about when to swing later in the season doesn’t hold up.

The correlation between monthly O-swing and Z-swing rates is 0.96. In other words, the two move in tandem. There’s no evidence that one could reduce fatigue-related swings at pitches outside the strike zone without also reducing swings at pitches inside the strike zone.

One last thing before we move on to the study’s second claim. Colin Wyers pulled the monthly BIS-based plate discipline split stats for April and September—the same data set that the researchers drew on—from FanGraphs, then took the difference in overall swing rate and O-swing rate for each season. Here’s what that looks like:

As Colin put it, “the reported effect essentially evaporates if you look at Swing% instead of O-Swing%.”

Is fatigue the cause of the erosion in plate discipline over the course of the season?
Well, first things first: as we’ve just shown, there is no erosion in plate discipline over the course of the season, so it doesn’t make much sense to ask this question. But while there’s no erosion in plate discipline, there is a difference in plate discipline, at least between the beginning and the middle-to-end of the season: batters swing less often (at all pitches) in April than they do in subsequent months. Does that have something to do with fatigue?

We can’t prove that it doesn’t, though you’d have to come up with a convoluted argument to suggest that it does. But there are factors other than fatigue that could influence swing rates. For one thing, the player pool in April isn’t the same as the player pool in September. Rosters expand in September, and teams give a higher percentage of their plate appearances to rookies who weren’t around earlier in the season. Those rookies might tend to be free swingers, which could skew the stats. The abstract doesn’t offer any evidence that the researchers made an attempt to control for this.

Colin Wyers did make an attempt to control for this, and as it happens, it doesn’t make much of a difference. Even after controlling for changes in the player pool, swing rates are still at their lowest in April, higher in September, and highest in July. (You can see the controlled results and a brief explanation of the methodology here.) Teams give a higher percentage of their innings in September to rookies, too, and perhaps those rookie pitchers are less capable of inducing swings outside the zone. So maybe it all evens out. (Which brings up another question: If hitters are fatigued later in the season, mightn't pitchers be fatigued, too?)

However, Colin’s attempt to control for changes in the player pool doesn’t control for changes in the players. And there’s one important change in player performance that could influence plate discipline: pitchers don’t throw as hard in April as they do in subsequent months. The average four-seam fastball reaches peak velocity in July, as temperatures rise and pitchers stretch out their arms. A graph of average fastball speed by month looks a lot like a graph of swing rate by month. When pitchers are throwing harder, hitters have to decide whether to swing earlier. Maybe they’re more likely to swing when they have less time to weigh their options.

Another possible cause: as Max Marchi’s research earlier this year suggested, hitters may well be “ahead” of pitchers earlier in the season, maybe because pitchers need more time to build up their arms and return to peak playing shape after a winter off. That could be a factor here, though it wouldn’t necessarily explain why hitters swing at fewer pitches inside the zone in April. Or maybe there’s something about the weather that makes hitters more or less likely to swing, aside from its effect on pitch speed. When the weather is warm, the ball travels farther, which gives hitters more incentive to swing.

Even if there were a clear, progressive decline in plate discipline over the course of a season, it would still require quite a leap to conclude that fatigue was the cause. To do that, you'd have to assert that something that did change (plate discipline) was linked to something that was only speculated to change (level of fatigue), and was also very difficult to verify. There might be dozens of other factors that one could pinpoint as the cause just as plausibly. Fatigue might be a sensible explanation, but to single it out as by far the most likely smacks of just-so story.

Has the amphetamine ban made players more fatigued?
It certainly stands to reason that bidding goodbye to greenies might make players more sleepy (although some of them may have simply switched to Adderall.) But the evidence—“researchers found a much sharper uptick in O-Swing rate from 2006 to 2012 than in 2002 to 2005”—isn’t conclusive (even aside from the fact that that, as we’ve shown, the uptick in O-swing is accompanied by an uptick in Z-swing, which seems to call the significance of the O-swing rise into question).

Unfortunately, we can’t check to see whether the PITCHf/x data from 2002 to 2005 shows the same trend as the BIS data, because PITCHf/x data from 2002 to 2005 doesn’t exist. But as Colin Wyers has documented (here and here), the BIS plate discipline data from the pre-PITCHf/x era is somewhat inconsistent, and subject to certain biases. For instance, over the 2002-10 period that Colin studied, the BIS-recorded overall swing rates and take rates inside the strike zone remained pretty stable from season to season, but the rate of recorded swings at pitches inside the zone declined dramatically:

As Colin wrote,

If a batter takes a pitch (that is to say, if the umpire has to make a decision on whether the pitch was in the zone), the percentage of pitches scored as in or out of the zone stays pretty consistent season to season. If the batter swings, and the umpire doesn’t have to make a determination of whether or not the batter swung on the pitch, we see a pretty constant decline in the willingness of BIS’ scorers to mark it as in the zone.

There doesn’t seem to be an explanation for this that comes from the actual pitched ball themselves—it really isn’t plausible that pitchers could change their approach that much without it showing up in a batter’s swing rate or the called strike rate.

There’s a likely explanation for why the data on pitches taken is more stable than the data on pitches swung at—in the case of a pitch taken, the video scout is getting immediate feedback in the form of the umpire’s calls. Now, there will be times when the scorer disagrees with the assessment of the umpire, but they always have the umpire’s assessment available to them when it comes time to make the decision. On pitches where the batter swung, the scorer must make the decision unaided—his only recourse is another scorer, who shares the same observational problems he has. (Since 2008, all parks have had PITCHf/x data, which could serve as an additional check; 2007 had partial coverage. But from 2002-06, no such objective data was available publicly.)

Or, in chart form, with “delta” representing the difference between April and September:

Period

SwingDelta

OSwingDelta

2002-2005

1.18%

1.50%

2006-2012

1.29%

2.90%

In other words, the difference attributed to the amphetamine ban might simply be a difference in the way the data was recorded.

I sent a draft of this article to Dr. Kutscher, and he kindly responded via email to some of my criticisms. He acknowledged that he had considered the possibility of scoring bias in the BIS data, and he noted that our explanations aren't necessarily mutually exclusive. For instance, while increased pitch velocity and the resultant reduced time to make decisions could explain the rise in swing rate, fatigue could still be a factor, since it's known to cause reaction time to suffer. While this is true, it's also part of the problem: the "fatigue effect" explanation isn't incompatible with any interpretation of the data, because we don't have any of the information that could prove or disprove a tie to plate discipline. In the absence of hard data on player fatigue, all we have is a hypothesis. And if all we have is a hypothesis—and some data that might be biased—why put out a press release?

Lessons
One lesson you shouldn’t take from this article is that fatigue can’t impair players’ performance, or that teams shouldn’t try to ensure that their players get sufficient sleep. It can, and they should. Russell Carleton wrote about the effects of fatigue at BP a few years ago, and a new study—mentioned in the same press release as the one about Kutscher, and also presented as an abstract (PDF) in an online supplement of SLEEP on page A326 here—finds that players who self-report elevated levels of sleepiness have shorter careers. (Of course, correlation doesn’t imply causation—maybe the sleepy players lose sleep because they’re playing poorly, not the other way around.)

The principle investigator for that study, W. Christopher Winter, MD, of the Martha Jefferson Hospital Sleep Medicine Center, has consulted for teams, so maybe there will be a trend toward paying more attention to players’ sleep habits. That wouldn’t be a bad thing. As Kutscher put it to me, "Fatigue has been shown to have a profound impact on performance in a wide variety of fields, and sleep represents a way for athletes to potentially improve their performance. In what ways, and to what extent, remains in question, but in a game of inches (or, perhaps more accurately, tenths of seconds) even a small improvement could make a big difference." On that we can agree.

But while a sleepy player might suffer from impaired performance, it’s a stretch to say that the league as a whole has worse plate discipline due to fatigue later in the season, and an even greater stretch to suggest that the amphetamine ban has produced a marked uptick in player fatigue. Those things might be true, but this study hasn’t shown them to be true. (It’s important to note that since this study has only been published as an abstract in an online supplement to SLEEP, not as a full paper in the journal itself, it hasn’t been through a formal peer review process.)

It may be that flawed studies are more likely to produce interesting results, and we know that studies with interesting results are more likely to be published and posted about. So maybe the lesson is: approach research and studies with a skeptical eye (and that includes studies published at Baseball Prospectus). Not everyone has the time and access to data necessary to evaluate every researcher’s claims, and when those claims come from a reputable source, cite statistics, and don’t sound implausible, it’s easy to accept them (especially when you really need something to write about). But caution is always in order.

Thanks to Dan Brooks, Russell Carleton, and Colin Wyers for research assistance.

Thank you for reading

This is a free article. If you enjoyed it, consider subscribing to Baseball Prospectus. Subscriptions support ongoing public baseball research and analysis in an increasingly proprietary environment.

Subscribe now
You need to be logged in to comment. Login or Subscribe
rcbussbp
6/21
Here's an article on baseball players and sleep:

http://www.popsci.com/science/article/2013-05/want-mlb-players-improve-and-stay-league-longer-give-them-naps

Want MLB Players To Suck Less And Stay In The League Longer? Give Them Naps
bornyank1
6/21
I linked to that one above.
jfranco77
6/21
I'm kind of stunned that reputable researchers used team-based stats. The composition of the team isn't the same in April as it is in September.

If you could show me that specific players swing more wildly in September I might believe that. Find a sample of 200 players who had at least 60 PAs in every month and test them. Something like that.
WoodyS
6/21
Well done.
walrus0909
6/21
I agree that there is a huge leap from "O-Swing % is up in September" to "the amphetamine ban is having a huge impact on performance". But I had a couple of additional observations:

- The only month where the O-Swing % doesn't increase is August -- i.e., right after 90% of baseball gets a 3-4 day vacation. I wonder what the July numbers look like pre- and post-ASG. It's possible that O-Swing % in the first half of July is greater than June, and then less than August in the second half of July.
- You mention that Z-Swing % also goes up month-to-month. Swinging at all pitches is still considered bad plate discipline, and doesn't seem inconsistent with hitters becoming less selective or discerning.
- You wrote, "Maybe there’s something about the weather that makes hitters more or less likely to swing, aside from its effect on pitch speed." I haven't played baseball in awhile, but I remember it *hurts* when you hit a ball poorly in the cold. I imagine this effect is amplified with faster pitches and wooden bats. Maybe players are more picky because they're less likely to square up pitches outside the zone. This is a complete wild guess.
- You also mentioned the fact that team composition changes from April to September thanks to rookie callups. Could this also be a function of injuries? As the season wears on and players get placed on the DL, they are replaced by less skilled hitters, who may be less selective. And, since a lot of players use the All-Star break to get healthy, it makes sense that the O-Swing % decreases shortly thereafter until rosters expand.

Lastly, I can't believe baseball players don't take naps. How early do players get to the field for a 7:00 game? 3:00? The road team typically has BP last, starting around 4:30 or 5:00, right? Doesn't there seem like there's a lot of downtime? And I've heard NBA players frequently nap, especially on the road, but I forget if that includes game days.
rmunter
6/21
One thing that you always hear from ex-players is that the great test of the baseball season is maintaining focus through the fatigue of the long, hot summer. That tracks with a lot of this to me. Fatigue leads to less focus, less focus leads to decline of discipline. Heck my golf swing always falls apart on the back 9 because I get tired, so it's no surprise to me that that would be the case.
chabels
6/23
And as Sam points out, that's why this study got so much press: it's conclusions are intuitive and relate to our own experiences.

That does not make it true, and given the questionable statistic work, it may appear that this concept is in fact incorrect as the data do not show less discipline.
jdeich
6/21
Another data point would emerge from looking at swing rate as a function of a proxy for rest:

1) Game played in the same city as the day before.
2) Rest day (travel required).
3) Rest day (no travel required; e.g. homestand).

Maybe break down category #2 by the distance of the travel.

To exclude September callups, etc., maybe only look at "everyday" players. Presumably that fourth outfielder is more rested than the starters.
HPJoker
6/21
Ben Lindbergh: Investigative Journalist. He's on the case! Nothing gets past Ben Lindbergh! He has a NOSE for finding the truth and will stop at nothing to get there!

Plus-Plus article Ben. Sharing with all my friends.
bornyank1
6/21
Thanks! I hope you're a popular person.
HPJoker
6/22
I'm a stathead, so I live in my mother's basement, therefore not popular. Sorry.
lichtman
6/21
Nice job guys!

Ditto what everyone else said and what they were thinking!

The first thing that I find confusing is the idea that any fatigue is caused by sleep deprivation. Sleep deprivation? Where did that even come from? I have never heard of that in relation to fatigue in sports.

I've always assume that you may get fatigued because you are playing almost every day. I have never heard of players being sleep deprived during the season. Most of your games are at 7:00 and you don't arrive at the park until 3 or 4. Even for day games, you arrive at what, 9 or 10?

Why would a baseball player be sleep deprived as the season goes on? If he stays out all night during the baseball season, I imagine that he would do he same in the off-season.

So, tired from playing every day, fine. But sleep deprived? Huh?

Anyway, as far as the actual study, if you are not, at the very least, controlling for the pool of players, which apparently they did not, then I don't want to even hear about the results! This needs to be done using some kind of "delta method". Of course. Any study that looks at a player effect, which is most studies, needs to control for the pools of players. That is especially true when your effect is over the course of the season when the player pools change considerably. To not control for the players (batters and pitchers) is just ridiculous.

Ditto for the weather. Can't the weather be controlled for using the fixed effects methods that all the statisticians/sabermetricians seem to be using these days?

And finally, batter and pitcher approaches change as the season progresses, for various reasons. Whether they change for the better or worse (or the same), and why, is often impossible to ascertain. Some of these changes may have something to do with fatigue (but not sleep deprivation!) or maybe not.

In order to implicate fatigue as a culprit, first you want to show that the changes are a detriment to the hitters, which, as I said, is not easy, and second, you pretty much have to rule a lot of other things out before you conclude that the effect is a result of fatigue, and even then, it is just a guess.

Finally, if hitters are experiencing fatigue as the season progresses, and pitchers don't, we would see a decline in offensive stats, after controlling for weather? Do we? I actually think we do, at least for April versus the rest of the season, but I have always thought that it was because "hitters were ahead of pitchers" early int he season. If it were due to fatigue, I think we would see a gradual but continuous degradation from April to September. I don't think we see that.
lyuchi
7/09
Great debunking.