CSS Button No Image Css3Menu.com

Baseball Prospectus home
  
  
Click here to log in Click here for forgotten password Click here to subscribe

Premium and Super Premium Subscribers Get a 20% Discount at MLB.tv!

<< Previous Article
Premium Article On the Beat: Unbearabl... (05/30)
<< Previous Column
Premium Article Overthinking It: The M... (05/28)
Next Column >>
Premium Article Overthinking It: The L... (05/31)
Next Article >>
The Lineup Card: Makin... (05/30)

May 30, 2013

Overthinking It

Bill James, Base Umpires, and the Sabermetric Significance of Checked Swings

by Ben Lindbergh

When I got home from the SABR Analytics Conference in mid-March, I spent a week or so writing and talking about some of the most interesting things I’d heard there. But there was one particularly intriguing topic that I wasn’t yet ready to write about.

That topic was brought up by Bill James, who's given us more than his fair share of interesting insights. Here’s all the backstory you need to know: James was on an “Analytics Super Panel” with Brian Kenny and Joe Posnanski, and he was asked by an audience member whether there’s any utility for teams in gathering information on home plate umpires. If I’ve embedded it right, this video should play only a short clip from the relevant portion of the panel (59:40-1:00:29). If I’ve embedded it wrong, you can either skip to that section yourself or read the transcript below.

One thing that, to answer your question, the different first base, third base umpires are wildly inconsistent in how they call the checked swing strike. Major league teams are aware of that, and under certain conditions, the starting pitcher will be aware that that first base umpire likes to call, will call the strike, because if you throw a slider down and away, he’ll start to swing at it, he’ll check, there’s a swing, and it’s really important that that’s an umpire that tends to call that a strike or tends not to call it a strike. So, yeah, we’re aware of that stuff, and we watch it, and I would bet almost every team is on top of that issue.

We all know that home plate umpires have different strike zones, and we’re aware that teams pay attention to and try to exploit that information. (Here’s a recent picture of some of Inside Edge’s umpire charts taped up in the Dodgers’ dugout.) But before I heard James say it, I’d never considered that checked-swing strike rates might vary by base ump, or that pitchers might alter their approach accordingly. Of course, there’s no precise definition of what constitutes a swing in the official rulebook, so it makes sense that certain umpires would see swings differently. Some might rule based on whether the batter’s wrists “break,” or whether the barrel of the bat passes in front of the plate. Others might try to assess the batter’s “intent to swing.”

So I went in search of stats that could support what James said. They’re pretty tough to come by. MLBAM, Baseball Info Solutions, and STATS LLC don’t track checked swings. The only major data provider that does is Inside Edge, and even Inside Edge doesn’t track them quite the way you’d want. The company doesn’t record whether an appeal to a base umpire was made, except in rare cases where video replay revealed that an ump had made an obvious wrong call (in the opinion of the Inside Edge scout). It simply records whether there was a checked swing, regardless of outcome. Of course, that’s as subjective a call as the umpire’s is. Here’s how Inside Edge instructs its video scouts to consider checked swings, according to Product, Marketing, and Sales Director Kenny Kendrena:

The checked swing is subjective, obviously, but the basic rule of thumb we train our guys on is to log it for the type of check that ‘the umpire needs to think about.’ In other words, they won’t log it if the action of the hitter was very subtle. It’s more for the borderline check, or at least one that puts the thought of check swing in the umpire’s head.

Insidge Edge tracked check swings on a trial basis in 2008, but began tracking them for all games in 2009. The company recorded 11,807 checked swings in 2012 alone.

Baseball Prospectus acquired Inside Edge’s checked-swing data from 2009-12 and merged it with our pre-existing PITCHf/x database, which made all sorts of analysis possible. Max Marchi ran the numbers through a multi-level logistic regression with random effects—essentially, a “with or without you” model that works with more than two actors. (This is the same technique Max uses to isolate the impact of catcher receiving by correcting for all of the other factors that can influence a ball/strike call.) Max looked at checked swings that resulted in either a swinging strike or a ball, accounting for pitcher, batter, pitch type, count, home plate umpire, and relevant umpire on base (third base ump with a left-handed hitter at the plate, and first base ump with a righty up). I’ll report all of his findings below, but first I want to discuss whether the stats back up Bill James, whose comments inspired the study.

Max split the dataset into even and odd rows and ran a correlation on the extra strike calls by base umpire, weighting for the number of check swings that occurred with each umpire on base. The year-to-year correlation was 0.34—not strong, but tough to dismiss, given the number of data points and the fact that we aren’t even considering whether the home plate umpire appealed. (There’s a stronger correlation between a base ump’s rates at first and third.)

Here’s a scatterplot of checked-swing strike rate on odd-row checked swings vs. checked-swing strike rate on even-row checked swings. (“Swing rate” in the axis labels refers to how often the checked swing was ruled a real swing.) The darker the dot, the larger the sample.

So it does seem that there’s a persistent difference in the rates at which base umpires call checked swing strikes, as James suggested. However, Max found no difference in the percentage of sliders thrown when umpires who call more checked-swing strikes are on base. That means that we can’t confirm James’ contention that pitchers are altering their approaches to take advantage of the rates at which base umps call checked swings strikes. Nor can we disprove it. As Max put it, “I believe his claim might be true, but pitchers try to exploit the knowledge of the base ump tendency so rarely that it would not come out from analysis of the data.” If we tried to look for evidence of particular pitchers or teams tailoring their approaches to these tendencies, or studied only certain situations, we’d be slicing the base-ump samples so small that we’d lose statistical significance, regardless of what the results revealed.

That said, I’m a little skeptical that “almost every team is on top of” umpire checked-swing strike rates. James’ comments certainly suggest that the Red Sox have studied the issue, maybe with more granular information than we obtained from Inside Edge. But the difficulty of obtaining the necessary data, and the modest potential payoff, makes it unlikely that most of the other 29 teams have done the same studies. I asked statistical analysts from a few teams whether they have data on checked swings, and whether they’ve studied checked-swing strike rates. All of them said that they didn’t, and hadn’t. “I don’t know what every team in baseball is doing,” one of them told me. “I do know what a handful of them are doing, and they aren’t doing that.”

Front-office types are under no obligation to tell me the truth, but it makes sense that checked swings wouldn’t be anywhere close to a team’s top priority. Last season, 1.6 percent of pitches led to checked swings. Teams saw, on average, 145 pitches per game (290 for both clubs combined). That means there were an average of 2.3 checked swings per team, per game (4.6 for both clubs combined). A team could steal an extra strike from time to time if it knew the probabilities of checked-swing strikes by base umpire, but since the third parties that teams typically get data from aren’t providing that data, they’d have to collect it themselves or get Inside Edge’s info and then watch every checked swing to see whether there was an appeal. (There's no technology currently installed around the majors that tracks the path of the bat.) That would take some serious time, and few teams have a large enough army of interns to spare one for that sort of project.

***

Let’s take a look at the rest of what the checked-swing information has to offer:

The umpires
Here are the increases and declines in the probability that a checked swing will be called a strike, broken down by base umpire:

Base Ump

Rise in Strike Probability

Base Ump

Decline in Strike Probability

Jerry Meals

4.6%

Phil Cuzzi

-5.2%

Brian Runge

3.9%

Joe West

-4.0%

Angel Hernandez

3.1%

Adrian Johnson

-3.8%

Scott Barry

3.0%

Tim Welke

-3.7%

Chris Guccione

3.0%

Laz Diaz

-3.4%

Here's Meals on the left, calling a strike on Alex Liddi last May, and the more lenient Cuzzi on the right, letting Matt Holliday off easy last April. (Liddi did appear to go around farther, so this isn't the best example of the disparity between the two.)

According to Max’s model, the difference between the highest and lowest is only about 10 percent, but the true difference is likely more dramatic. These results are somewhat diluted, since we can’t focus only on checked swings that were appealed.

And here are the increases and declines broken down by home plate umpire:

Home Plate Ump

Rise in Strike Probability

Home Plate Ump

Decline in Strike Probability

Doug Eddings

2.1%

Gary Cederstrom

-2.1%

Bill Miller

2.0%

Paul Nauert

-2.1%

Joe West

1.9%

Dana DeMuth

-1.8%

Alfonso Marquez

1.5%

Chad Fairchild

-1.4%

Brian Runge

1.4%

Wally Bell

-1.3%

Brian Runge is unlikely to call a checked-swing a strike wherever he’s stationed, while Joe West is much more likely to do so when he’s behind the plate than when he’s on the corners.

These differences are smaller, but it makes sense that there would be some distinction, since home plate umpires can decide to call checked-swing strikes themselves. Their ball calls can be appealed by the catcher, but their strike calls can’t by appealed by the batter.

Where checked swings happen
Here are two plots of pitches that led to checked swings over the past four seasons, both from the catcher’s perspective. Non-checked-strike balls are green, checked-swing balls are blue, and checked-swing strikes are red. First for right-handed hitters:

Then lefties:

And here’s a GIF with both combined:

The highest concentration of checked-swing strikes on those plots is in the same region to hitters of both handedness: down and in to lefties and down and away to righties. As it turns out, pitcher handedness is a bigger factor than batter handedness. Here are the checked swings to righties, first against left-handed pitchers and then against right-handed pitchers:

And here are the lefty batters against pitchers of both handednesses:

Now we see the difference.

Finally, a breakdown of checked-swing rates by location inside or outside of the strike zone:

Pitch in Zone?

Pitches

Checked Swings

Checked Swing Percentage

No

1,396,581

32,559

2.33%

Yes

1,422,235

6,554

0.46 %

Pitches outside the typical called strike zone are about five times more likely to be checked swings than pitches inside the typical called strike zone. That makes sense: hitters hesitate less when deciding whether to swing at strikes.

The count
After accounting for the players, umpires, and pitch types involved, Max found that the count has an impact on the probability that a checked swing will be called a strike. Here’s the impact on probability by count, in ascending order. (In this case, 0-0 pitches are the baseline relative to which all other counts are measured.)

Count

Change in Strike Probability

0-2

-19.0%

1-2

-17.9%

2-2

-15.3%

3-2

-10.8%

3-1

-5.5%

2-1

-4.4%

1-1

-2.5%

0-1

-2.4%

0-0

0.0%

2-0

0.5%

1-0

2.7%

3-0

4.1%

This is pretty consistent with the “compassionate umpire” effect John Walsh discovered when he looked at the size of the called strike zone by count. Umpires are much less likely to call strikes on 0-2 than they are on 3-0; they’re also less likely to call checked swings swinging strikes.

Pitch type
This table tells us the rates at which each pitch type induces check swings, in descending order of check swings divided by pitches:

Pitch Type

Checked Swing Rate

Knuckle-curve

6.8%

Slider

2.7%

Curveball

2.5%

Splitter

1.6%

Cutter

1.5%

Changeup

1.2%

Four-seam

0.9%

Knuckleball

0.9%

Two-seamer

0.9%

Sinker

0.7%

And here’s a related table that tells us the change in the probability that a checked swing will be called a strike, broken down by pitch type instead of count, and with four-seam fastballs as our baseline:

Pitch Type

Change in Strike Probability

Cutter

7.7%

Slider

7.2%

Splitter

5.3%

Changeup

3.6%

Knuckle-curve

3.4%

Sinker

2.9%

Curveball

0.4%

Two-seamer

0.4%

Knuckleball

-3.5%

These results also make sense. A checked swing at a cutter or slider is more likely to be called a strike; a checked swing at a fastball is less likely to be called a strike.

Batters
The batters with the highest checked-swing rates—checked swings as a percentage of pitches—are, unsurprisingly, batters who swing and strike out a lot:

Batter

Checked Swing Rate

Carlos Pena

3.9%

Cody Ransom

3.7%

Chris Carter

3.7%

Matt LaPorta

3.7%

Brandon Inge

3.4%

Dan Johnson

3.4%

Justin Maxwell

3.3%

Juan Uribe

3.2%

Jim Thome

3.1%

Nolan Reimold

3.0%

Also unsurprisingly, the batters with the lowest checked-swing rates are guys who don’t swing a lot and tend to make contact. The only surprise is that Jeff Keppinger couldn’t quite crack the top 10 list (he finished 14th).

Batter

Checked Swing Rate

Wes Helms

0.0%

Chris Coghlan

0.1%

Trevor Crowe

0.2%

Emilio Bonifacio

0.2%

Jorge Cantu

0.2%

Ross Gload

0.3%

Gaby Sanchez

0.4%

Alexi Amarista

0.4%

Tony Abreu

0.4%

Julio Borbon

0.4%

Wes Helms was the only hitter to see at least 1000 pitches from 2009-2012—he saw 2658—without a single checked swing. Say what you will about Helms’ offensive abilities, but the man has no doubts about whether he wants to swing. (You might notice that that list is lousy with Marlins, which could potentially point to some sort of scorer bias.)

And here are the batters whose checked swings were called strikes more and less often than expected, after controlling for other factors:

Batter

Rise in Strike Probability

Batter

Decline in Strike Probability

Marlon Byrd

6.9%

Denard Span

-7.6%

Adam Jones

6.6%

John Mayberry

-7.4%

Chris Getz

6.1%

Robinson Cano

-7.3%

Austin Kearns

5.8%

Johnny Damon

-7.3%

Ian Desmond

5.8%

Delmon Young

-7.2%

Daniel Descalso

5.7%

Albert Pujols

-7.1%

Magglio Ordonez

5.5%

Alexei Ramirez

-6.7%

Roger Bernadina

5.4%

Peter Bourjos

-6.7%

Ivan Rodriguez

5.4%

Justin Upton

-6.4%

Torii Hunter

4.9%

Everth Cabrera

-6.2%

Pitchers
The story is similar for pitchers. High-swing/chase/whiff guys get check swings often:

Pitcher

Checked Swing Rate

Al Alburquerque

3.7%

Daniel Schlereth

3.4%

Jason Jennings

2.9%

Boone Logan

2.9%

Garrett Richards

2.8%

A.J. Burnett

2.8%

Craig Kimbrel

2.7%

Corey Kluber

2.7%

Jason Bulger

2.6%

Billy Wagner

2.6%

And low-swing/chase/whiff guys don’t:

Pitcher

Checked Swing Rate

Dan Meyer

0.0%

Brad Hand

0.0%

Alex Sanabia

0.0%

Juan Oviedo

0.0%

Sean West

0.0%

Renyel Pinto

0.0%

Brian Sanches

0.03%

Burke Badenhop

0.2%

Chris Volstad

0.3%

Tim Wakefield

0.3%

Oviedo threw 3187 pitches without seeing a single checked swing. And here are the pitchers whose induced swings were called strikes more and less often than expected, after controlling for other factors:

Pitcher

Rise in Strike Probability

Pitcher

Decline in Strike Probability

Ryan Dempster

4.5%

Kevin Slowey

-3.9%

Daniel Hudson

4.4%

Johnny Cueto

-3.9%

Mark Buehrle

3.9%

Bruce Chen

-3.9%

Yovani Gallardo

3.9%

Francisco Rodriguez

-3.8%

Brett Cecil

3.7%

Livan Hernandez

-3.4%

Mat Latos

3.6%

Clayton Kershaw

-3.1%

Sean Marshall

3.5%

Josh Tomlin

-3.1%

Felix Hernandez

3.3%

Max Scherzer

-3.1%

Brandon Lyon

3.3%

Justin Verlander

-3.1%

Anibal Sanchez

3.0%

Franklin Morales

-3.0%

I’ll leave it to you to decide whether there are any perceptible patterns among the arms on those lists.

Outcomes
There were no home runs or triples marked as check swings, but there were 146 singles and 13 doubles. Here’s the most recent double, a leadoff blooper hit by Norichika Aoki off of Travis Wood last August:

There was also one checked swing on a pitchout, courtesy of Nick Punto last June:

The hit-and-run was on, but Ryan Kalish stayed put, having either anticipated the pitchout or missed the sign. Punto looked silly, and home plate umpire Sam Holbrook pointed at him to make sure the whole ballpark saw.

The pitchout to Punto was also the farthest outside of any pitch that induced a checked swing. We’ll wrap up with some other extremes. Here’s the highest pitch that induced a checked swing, a 3-2 offering from Brandon Beachy to Brett Lawrie last June:

Intensity!

And here’s the lowest (well, the lowest viewable on video), an October 2010 swing by Ryan Howard that was ruled a strike by third-base umpire Jim Wolf:

As for the inside offerings: there were 96 checked swings on pitches that hit batters.

Thanks to Max Marchi and Ryan Lind for research assistance, and to Inside Edge for checked-swing data.

Ben Lindbergh is an author of Baseball Prospectus. 
Click here to see Ben's other articles. You can contact Ben by clicking here

Related Content:  Bill James,  Checked Swings

9 comments have been left for this article.

<< Previous Article
Premium Article On the Beat: Unbearabl... (05/30)
<< Previous Column
Premium Article Overthinking It: The M... (05/28)
Next Column >>
Premium Article Overthinking It: The L... (05/31)
Next Article >>
The Lineup Card: Makin... (05/30)

RECENTLY AT BASEBALL PROSPECTUS
Fantasy Article Fantasy Starting Pitcher Planner: Week 19
Barbecue State of Mind
Premium Article Transaction Analysis: Give the Red Sox Your ...
Premium Article Transaction Analysis: Astros, Marlins, Swap ...
Premium Article Transaction Analysis: The Great Big David Pr...
Premium Article The Call-Up: Mike Foltynewicz
Trade Deadline

MORE FROM MAY 30, 2013
Premium Article Dissecting the Draft: Examining Kris Bryant
Fantasy Article Bullpen Report: Movin' on Up
Premium Article The Call-Up: Alex Wood
Premium Article The Call-Up: Alex Colome
Premium Article The Call-Up: Michael Wacha
Premium Article Daily Roundup: Around the League: May 30, 20...
Fantasy Article Free Agent Watch: American League, Week Nine

MORE BY BEN LINDBERGH
2013-05-31 - Premium Article Overthinking It: The Longest Plate Appearanc...
2013-05-31 - BP Daily Podcast: Effectively Wild Episode 2...
2013-05-31 - BP Unfiltered: The Royals and Two Weeks of W...
2013-05-30 - Overthinking It: Bill James, Base Umpires, a...
2013-05-30 - BP Daily Podcast: Effectively Wild Episode 2...
2013-05-29 - BP Daily Podcast: Effectively Wild Episode 2...
2013-05-28 - Premium Article Overthinking It: The Montero-Pineda Trade in...
More...

MORE OVERTHINKING IT
2013-06-06 - Premium Article Overthinking It: The Plate Discipline-Only P...
2013-06-01 - Overthinking It: This Week in Catcher Framin...
2013-05-31 - Premium Article Overthinking It: The Longest Plate Appearanc...
2013-05-30 - Overthinking It: Bill James, Base Umpires, a...
2013-05-28 - Premium Article Overthinking It: The Montero-Pineda Trade in...
2013-05-25 - Overthinking It: This Week in Catcher Framin...
2013-05-24 - Overthinking It: The Longest Plate Appearanc...
More...