CSS Button No Image Css3Menu.com

Baseball Prospectus home
  
  
Click here to log in Click here for forgotten password Click here to subscribe

<< Previous Article
Premium Article Painting the Black: Th... (05/20)
<< Previous Column
Baseball Therapy: Anal... (05/13)
Next Column >>
Premium Article Baseball Therapy: The ... (05/27)
Next Article >>
Premium Article What You Need to Know:... (05/20)

May 20, 2014

Baseball Therapy

Beware the Genius Tag for Coaches

by Russell A. Carleton


How do you know whether your team’s hitting coach or pitching coach is doing a good job? Generally, the answer is “Well, how are his hitters/pitchers doing? Are they getting better?” That seems to be the justification given when he gets fired, after all.

Is that fair? Can a pitching coach really be blamed if his pitchers aren’t performing well? Like a manager, he’s not the one out there throwing the pitches, and the guys who are out there throwing the pitches may not be good. You can’t make a chicken salad out of a sow’s ear. It’s entirely possible that our hitting (or pitching) coach is actively tinkering with the swings of each hitter on his team, that he’s a mad genius, and that he could turn anyone into Babe Ruth. Or he might turn Babe Ruth into Mario Mendoza. Or maybe he’s more of a hands-off guy and just happens to be around when Mario Mendoza turns into Babe Ruth. If it does happen, can we safely proclaim him a coaching genius?

Warning! Gory Mathematical Details Ahead!
This is a tricky question to answer (and the math is going to get very gory this week). First, we ask the question of what a hitting (or pitching) coach’s job is. In theory, it’s to make the hitters (or pitchers) on his team better than they had previously been. Even if he’s not working with the most talented bunch, they should be showing some improvement.

Last year, I tried tackling this question, both with hitters and pitchers, using a very different method (mixed linear modeling, for the initiated). I tried to control, as best as I could, for the talent level that a coach had on hand, and then charged any variation from that talent level to the coach. I figured that some of it was random variance, but the randomness would largely cancel itself out across multiple hitters (or pitchers). At the end of the articles, I put together a “best and worst” section, but found myself asking whether I could actually trust the sample sizes that I had. For example, in the hitting coach article, I sang the praises of Kevin Seitzer, because his Royals hitters had been doing better than my model expected, and because he had what I guessed was a long enough track record. Was it really long enough?

I borrowed some methodology that I used in a previous article on figuring out whether we can trust changes in performance from year to year. The method, known as the reliable change indicator, looks at the difference between this year and last year in some stat (say, Smith picked up 30 points of OBP by going from .290 to .320) but also controls for what sample sizes produced those two numbers and the fact that those numbers are more reliable when they are produced at bigger sample sizes (for the initiated, I created a pooled standard error of the difference, based on the reliability of the stat at X PA and a good estimate of the population variance that we might expect at X PA). The result is something akin to a z-score. This gives us an indicator of whether a player has actually improved or declined.

From 1993-2013, I looked at the (raw, unadjusted) strikeout rate posted by all hitters, minimum 100 PA. In the following year (again, minimum 100 PA), I looked to see how much their strikeout rate had changed, by the reliable change index (RCI) method I just referenced. RCI can tell us whether the change in some measure has been positive or negative and how much faith we should put in those changes, in much the same way that a t-test can tell us whether the difference between two means is significant enough.

The rest of this article is restricted to Baseball Prospectus Subscribers.

Not a subscriber?

Click here for more information on Baseball Prospectus subscriptions or use the buttons to the right to subscribe and get access to the best baseball content on the web.


Cancel anytime.


That's a 33% savings over the monthly price!


That's a 33% savings over the monthly price!

Already a subscriber? Click here and use the blue login bar to log in.

Related Content:  Pitching Coaches,  Coaching,  Hitting Coaches

7 comments have been left for this article. (Click to hide comments)

BP Comment Quick Links

godfather

this conclusion can be drawn about a lot of sabre matter...isolated incidences such as kevin long with Curtis granderson work for me, and chili davis seems well represented in Oakland's approach, but conclusions rest largely on slippery slopes

May 20, 2014 07:09 AM
rating: 0
 
Tony B

As always, Russell, nice article that made me think!

You're hinting at this in the final few paragraphs, but did you consider adding more years to the sample? Unfamiliar with RCI (though I read the "clinical significance" wiki!), but something as crude as a CAGR over, say, 3-4 years with a more lenient z-score threshold could reveal something.

May 20, 2014 10:16 AM
rating: 1
 
BP staff member Russell A. Carleton
BP staff

Thought about that... maybe we need 2-3 years to see the process unfold fully. My decision to keep it to one year though was based more on the fact that people a) freak out when hitters/pitchers are doing so much better/worse than last year and b) people call for firings as a result.

May 20, 2014 11:04 AM
 
Geopipp

Good article, I quite enjoyed it and it gave me pause to think. A couple of thoughts though. Why the disclaimer about gory math ahead? People either should learn it or they will skip it to the end looking for the conclusion. Secondly we always seek to quantify any impact a variable will have on the game. I suspect this is one of variables you won't be able to quantify. I believe that coaching at this level is more to what you have alluded to, a comment here or there that might have an impact. Firings are a byproduct of perceived underachievement. Whether this underachievement is attributable to the coach or the player, is something the Manager or the GM will be required to determine.

May 20, 2014 12:33 PM
rating: 0
 
BP staff member Russell A. Carleton
BP staff

The gory math warning was just something that I've turned into a personal trademark/gimmick. It was actually based on Dante's Inferno (Abandon all hope ye who here enter!) I started doing it a few years ago actually to specifically allow people to skip the details if they just wanted the conclusion. Some people like hearing about covariance matrices. Some don't.

May 21, 2014 09:14 AM
 
Pat Folz

A bit late to this party, but one thing to check w/r/t "maintenance" might be whether certain coaches' hitters (or pitchers?) are streakier than others'.

May 25, 2014 22:26 PM
rating: 0
 
Truthteller

God's gift to statisticians. Random error. The body's resistance for a repeated behavior to be consistent. Once learned, one RE can enhance or take away another. The statistician is bewildered. Aha, it must be regression. Not the Coach, I say. Bah, Humbug. But it does provide fodder for another article. Enjoyed.

Jun 10, 2014 12:11 PM
rating: 0
 
You must be a Premium subscriber to post a comment.
Not a subscriber? Sign up today!
<< Previous Article
Premium Article Painting the Black: Th... (05/20)
<< Previous Column
Baseball Therapy: Anal... (05/13)
Next Column >>
Premium Article Baseball Therapy: The ... (05/27)
Next Article >>
Premium Article What You Need to Know:... (05/20)

RECENTLY AT BASEBALL PROSPECTUS
Fantasy Article My Model Portfolio: Framing Decisions Around...
Fantasy Article Dynasty Dynamics: TINO Does Arizona, 2015
Pebble Hunting: The Case For Shaming the Cub...
Every Team's Moneyball: Texas Rangers: Short...
Every Team's Moneyball: Atlanta Braves: Shor...
Prospectus Feature: All Spin Is Not Alike
Premium Article Baseball Therapy: The Most Important Player ...

MORE FROM MAY 20, 2014
Premium Article Going Yard: The Swing Problems Troubling Tro...
Premium Article What You Need to Know: Cincinnati Stays Aliv...
Premium Article Painting the Black: The Werth-Case Outcome
Premium Article Minor League Update: Games of Monday, May 19
Fantasy Article The Stash List: Sixth Edition
Fantasy Article Dynasty Dynamics: The U25 Top 150, Part Two
Daily League Strategy: Two Stud Pitchers

MORE BY RUSSELL A. CARLETON
2014-06-03 - Premium Article Baseball Therapy: Introducing My Daughter to...
2014-05-30 - Premium Article Baseball Therapy: The Hard Part About Preven...
2014-05-27 - Premium Article Baseball Therapy: The Annual Amateur Draft G...
2014-05-20 - Premium Article Baseball Therapy: Beware the Genius Tag for ...
2014-05-13 - Baseball Therapy: Analytical Master or Leade...
2014-05-07 - BP Unfiltered: Baseball Prospectus at Fox Sp...
2014-05-06 - Baseball Therapy: Is Oakland's Run Different...
More...

MORE BASEBALL THERAPY
2014-06-03 - Premium Article Baseball Therapy: Introducing My Daughter to...
2014-05-30 - Premium Article Baseball Therapy: The Hard Part About Preven...
2014-05-27 - Premium Article Baseball Therapy: The Annual Amateur Draft G...
2014-05-20 - Premium Article Baseball Therapy: Beware the Genius Tag for ...
2014-05-13 - Baseball Therapy: Analytical Master or Leade...
2014-05-06 - Baseball Therapy: Is Oakland's Run Different...
2014-04-29 - Premium Article Baseball Therapy: Do Innings Limits Work?
More...

INCOMING ARTICLE LINKS
2015-02-24 - Baseball Therapy: The 10th Man in the Lineup
2014-11-04 - Premium Article Baseball Therapy: Why Joe Maddon Matters
2014-06-10 - Premium Article Baseball Therapy: Can Draft Lightning Be Bot...