CSS Button No Image Css3Menu.com

Baseball Prospectus home
  
  
Click here to log in Click here for forgotten password Click here to subscribe

<< Previous Article
Premium Article Pebble Hunting: Martin... (04/28)
<< Previous Column
Moonshot: What PITCHf/... (04/23)
Next Column >>
Moonshot: The Winding ... (05/05)
Next Article >>
Premium Article What You Need to Know:... (04/28)

April 28, 2014

Moonshot

How Quickly Do Team Results Stabilize?

by Robert Arthur


With the end of April looming, we can begin to shed some of our fears regarding small sample size. Statistics like strikeout and walk rates have passed critical thresholds on their march toward stabilization, and so we are beginning to get a first look at how well individual players will perform. The requisite early-season loss of ~20% of each team’s starting rotation to the failure of a certain crucial ligament has taken its toll, resulting in a clearer picture of who will make each team’s starts.

All of which is to say, we can begin to turn our attention to matters larger than individual players. Since the ultimate goal of every team is to win a championship—and the best way to win a championship is simply to field a very good team—the question of utmost importance is simply: How good is my team?

In light of this question, I examine here how quickly team quality stabilizes over the course of a season. At a fundamental level, good teams are defined by 1) scoring lots of runs, and 2) not allowing the other team to score many runs. Therefore, I take as my measurements of quality runs scored per game and runs allowed per game.

While there is a simple relationship between the number of runs scored/allowed and wins (via the Pythagorean expectation), that relationship is quite noisy. First and foremost, the noise results from sequencing, or the luck a team has in apportioning its runs to individual games. A bad team may thus end the season with an excellent record and a playoff berth, despite an underlying lack of quality. Nevertheless, all else being equal, good teams (those that score many runs and don’t allow many runs) are more likely to make the playoffs and win championships than bad teams.

Estimating Quality
A simple estimate of the quality of your team’s offense is simply the number of runs it has scored so far in the season divided by the number of games it has played (RS/games). Naturally, the accuracy of that estimate will improve as the season progresses and more data is available, but how just quickly does it become accurate? I used Retrosheet game logs from 2000-2013 to examine this question.

The rest of this article is restricted to Baseball Prospectus Subscribers.

Not a subscriber?

Click here for more information on Baseball Prospectus subscriptions or use the buttons to the right to subscribe and get access to the best baseball content on the web.


Cancel anytime.


That's a 33% savings over the monthly price!


That's a 33% savings over the monthly price!

Already a subscriber? Click here and use the blue login bar to log in.

19 comments have been left for this article. (Click to hide comments)

BP Comment Quick Links

PeterCollery

You say: "In general, a given team’s projected RS number is going to be somewhere between PECOTA’s projection and the RS number the team has accrued so far."

Why would it ever be anything else? How, for instance, do we account for the Orioles who are outperforming their PECOTA so far, but are projected to finish below that level?

Apr 28, 2014 07:20 AM
rating: 1
 
BP staff member Robert Arthur
BP staff

Mathematically, a linear model has the variables in it (in this case, RS so far and PECOTA projected RS), and also an intercept. So if the intercept for a particular model is say -.1, and the RS and PECOTA numbers are each pointing towards 4.2 RS, the model might spit out 4.1, because of that intercept.

Generally, I would take the above numbers with a grain of salt. They are provided for illustrative purposes, rather than as definitive predictions. The point of this article was to show how quickly RS/RA stabilized, and to demonstrate that preseason predictions still carry some weight (and will until ~ game 100). If people are interested in maximally accurate predictions, maybe I can do a follow-up with some more sophisticated models.

(With that said, it's also possible I just made a mistake in entering the numbers in the table. I'll go back and check to make sure.)

Apr 28, 2014 07:48 AM
 
R.A.Wagman

Robert - Is it safe in looking at the final numbers that the model assumes no player movement between teams?

Apr 28, 2014 08:47 AM
rating: 0
 
BP staff member Robert Arthur
BP staff

Yes. All of this is based on PECOTA's depth chart projections, which assume that player X will get N plate appearances with his current team. There's no accounting for what would happen if that player gets traded. That's probably a source of some inaccuracy, given trading deadline dynamics (good teams tend to buy, bad teams tend to sell).

Apr 28, 2014 08:53 AM
 
jrcolwell

Before running your projection model, did you update PECOTA depth chart projections to account for new playing time projections (like in the case of injuries)? Or are you still using the same preseason playing time projections?

Apr 28, 2014 13:43 PM
rating: 0
 
BP staff member Robert Arthur
BP staff

Yes, I ran it with the updated predictions.

Apr 28, 2014 15:50 PM
 
dianagram

Very nice work Robert!

Apr 28, 2014 08:58 AM
rating: 1
 
evo34

What exactly is the model you came up with?

Apr 28, 2014 10:05 AM
rating: 0
 
BP staff member Robert Arthur
BP staff

There's 162 slightly different models, one per game number. As you might imagine, the weight on PECOTA decreases as the season goes on, and the weight on RS/G increases. As I said below, I'm going to look into this again and try to get a simple formulation for how much to weight PECOTA per game number. The intention here wasn't to maximize accuracy so much as to illustrate the overall trends.

Apr 28, 2014 18:54 PM
 
Peter Benedict

MN scoring the third most runs in all of baseball? Unpossible!

If only we had some pitching to go along with our projection-annihilating offense...

Apr 28, 2014 10:20 AM
rating: 0
 
Greg Ioannou

For about half the teams (BAL, DET, CIN, CLE, MIL, NYA, NYN, PHI, SFN, TBA, TOR, HOU, and WAS), the projected RS is below both the actual RS and Pecota. I think there's a problem with the methodology. (Or perhaps you just made a mistake.)

Apr 28, 2014 10:28 AM
rating: 1
 
BP staff member Robert Arthur
BP staff

I think that this is a feature, not a bug. Why? Because teams generally scored fewer runs per game later in the season than earlier in the season, reducing the final RS/G number (or at least they did in the years I looked at [2012/2013]). So the model is systematically underpredicting the runs/game to account for that.

Apr 28, 2014 12:23 PM
 
Michael Bodell
(89)

If that were the case for nearly half of the teams then wouldn't a better prediction for PECOTA just be even fewer runs per teams? Might not there be some small sample size issue (maybe a shift of offense from 2012 to 2013 or from 2013 PECOTA expectations to 2013 actual) that causes this effect.

If PECOTA says 4.6 and actual to date is 4.4 I'm suspicious of a prediction that is 4.2, and when it happens to nearly half the teams (13 out of 30) it suggests a bug.

Apr 29, 2014 18:50 PM
rating: 0
 
Michael Bodell
(89)

Also, in general the teams that you have projected outside this band are the teams who have actual runs scored most similar to projected runs score. In some sense that is expected since the band of possible values between the two is smallest. But in other senses this is surprising: The teams PECOTA has most accurately projected what they are doing are ones that your model doesn't trust! You'd think the evidence to date should make you more trust PECOTA more, not less.

BTW the issue happens in both directions (actual lower than PECOTA and PECOTA lower than actual). For instance:
Baltimore: 4.64 actual, 4.34 PECOTA, 4.23 projected.
Toronto: 4.35 actual, 4.48 PECOTA, 4.24 projected.

Somewhat dubious.

Apr 29, 2014 18:55 PM
rating: 0
 
newsense

I think I see an error: The Nationals' projection is below both its current runs scored and PECOTA.

There has been some prior work that suggests that if you pro-rate the PECOTA winning percentage to 69 games and add a team's current record to the PECTA 69 game record, that it will be close to the best predictor of the team's record going forward. That might serve as a good check on what you have done.

Apr 28, 2014 11:02 AM
rating: 0
 
BP staff member Robert Arthur
BP staff

See above. But I agree; the tendency to underpredict to RS/G is a little weird. I'm fairly certain that the numbers are correct and consistent with my method, but perhaps I need to re-examine the method with some external checks as you suggest.

Apr 28, 2014 12:27 PM
 
ravenight

Would be interesting to see the full version (perhaps trained on 3 years of data), incorporating RA to make a wins prediction.

Since the model's strength is its simplicity, I'm curious what the weights it's using are. Are you using one set of weights derived to optimize the predictive value at all points along the curve? Would it be better / close to just use [PECOTA * 162 + RS * (GP / 162)] / 2?

Very interesting that teams scored more of their runs early in the season, so improving on PECOTA's projection requires even more RS. If that's really a trend across all seasons, it has a lot of implications for fantasy...

Apr 28, 2014 14:37 PM
rating: 0
 
BP staff member Robert Arthur
BP staff

I use a different set of weights for each game number.
A lot of good questions and suggestions here, and in the comments above (as usual). I will look into some of these for the next article.

Apr 28, 2014 18:50 PM
 
myshkin

I assume the sort order for your table was games played and then alphabetical by team abbreviation? That was a bit mystifying.

Apr 29, 2014 00:13 AM
rating: 0
 
You must be a Premium subscriber to post a comment.
Not a subscriber? Sign up today!
<< Previous Article
Premium Article Pebble Hunting: Martin... (04/28)
<< Previous Column
Moonshot: What PITCHf/... (04/23)
Next Column >>
Moonshot: The Winding ... (05/05)
Next Article >>
Premium Article What You Need to Know:... (04/28)

RECENTLY AT BASEBALL PROSPECTUS
Premium Article The Prospectus Hit List: Friday, April 24
Premium Article Daisy Cutter: Carlos Rodon and the White Sox...
Eyewitness Accounts: April 24, 2015
Premium Article Rubbing Mud: Of Signals and Sausages
Raising Aces: Debut Ante: Raisel Iglesias
Painting the Black: No D In Desmond?
BP Bronx

MORE FROM APRIL 28, 2014
Premium Article Pebble Hunting: Martin Perez Turns Two, 12 T...
Premium Article Prospectus Hit List: Monday, April 28
Premium Article Minor League Update: Games of April 25-27
Baseball Prospectus News: Lindbergh Receives...
The Week in Quotes: April 21-27
Fantasy Article Interleague Report: Week Five
Fantasy Article Fantasy Freestyle: Transaction Paralysis

MORE BY ROBERT ARTHUR
2014-05-21 - Moonshot: PITCHf/x and Batter Breakdowns
2014-05-12 - Moonshot: Predicting Batter Breakouts with P...
2014-05-05 - Moonshot: The Winding Path of Albert Pujols,...
2014-04-28 - Premium Article Moonshot: How Quickly Do Team Results Stabil...
2014-04-23 - Moonshot: What PITCHf/x Can Tell Us About Ba...
2014-04-14 - Premium Article Moonshot: Does April Velocity Last?
2014-04-08 - Premium Article Moonshot: Miguel Cabrera and the Bearable He...
More...

MORE MOONSHOT
2014-05-21 - Moonshot: PITCHf/x and Batter Breakdowns
2014-05-12 - Moonshot: Predicting Batter Breakouts with P...
2014-05-05 - Moonshot: The Winding Path of Albert Pujols,...
2014-04-28 - Premium Article Moonshot: How Quickly Do Team Results Stabil...
2014-04-23 - Moonshot: What PITCHf/x Can Tell Us About Ba...
2014-04-14 - Premium Article Moonshot: Does April Velocity Last?
2014-04-08 - Premium Article Moonshot: Miguel Cabrera and the Bearable He...
More...