Notice: Trying to get property 'display_name' of non-object in /var/www/html/wp-content/plugins/wordpress-seo/src/generators/schema/article.php on line 52
keyboard_arrow_uptop

In 2007, the Angels won 94 games despite a third order win total of 86. In 2008, the Angels won 100 games despite a third order win total of just 84. In 2009, the Angels won 97 games with just a third order win total of 87. Going into 2010, we were left wondering whether the Angels were the luckiest team in the world or whether they were doing something that made them appear to be a slightly above average team but actually among the best in the league. A clue was thrown our way in 2010 when the Angels came back to earth, dipping to a record of 80-82. This might have resolved the issue if the Angels third order win total had been more than 72, because even a mediocre team beat the stuffing out of its third order record!  The Adjusted Standings that we publish under our Statistics tab here at Baseball Prospectus are designed to give fans a clue about how lucky teams have been, but the Angels’ performances have thrown some major question marks our way in recent years about the methodology of those standings.

So what is going on here?  To understand this, let’s first walk through what the third order standings are trying to tell us.  There are three orders of luck that are gradually stripped away, and while the math is tough, the intuition is simple.  In order of presentation in the Adjusted Standings:

 

  • W-L: Actual Record, how many games did teams win?

 

  • W1-L1: First Order Standings, how many games would a normal team win given their runs scored and runs allowed?

 

  • W2-L2: Second Order Standings, how many runs would a normal team score and allow given their hits, hits allowed, walks, walks allowed, etc., and how many games would a normal team win given that run differential?

 

  • W3-L3: Third Order Standings, what if the Second Order Standings were adjusted for difficulty of opponents?

 

Knowing how to calculate them would be a challenge, but understanding the intent is simple. The goal is to infer which teams were good, and which teams just won a lot of close games, and also which teams just got a lot of timely hits. Looking at third order records as more valuable implies that teams that win a lot of close games are getting lucky, and that teams that get five runs on five hits and a walk are getting lucky too. Is that really true?

I figured that one of the best ways to answer that question would be to look at which “standings” actually correlated the most with the following year’s actual standings. Keep in mind that although players switch teams and some teams actually get better or worse from year to year, that will all wash out. For example, losing John Lackey to the Red Sox costs the Angels in both real record and adjusted record because his effect on runs allowed and wins depart. 

So, while the Angels in 2009 clearly were much closer to their 2008 actual record than their 2008 third order record, is that true for most teams?  And if so, which standings correlate the best with actual record the following year?  I used the standings for 2005-10, giving me a solid 150 pairs of consecutive years to consider.

 

Standings version

Correlation with next year’s standings

Actual Standings

.487

First Order Standings

.493

Second Order Standings

.504

Third Order Standings

.485

 

Do not be disappointed in the third order standings.  The difficulty of opponents is something that is persistent, so the constant bonus added to the Orioles third order win total due to playing the Yankees, Red Sox, and Rays is not supposed to predict an Orioles resurgence the following year, because those teams are just going to keep on beating the Orioles next year. However, we see solid evidence that each adjustment adds a small something to our estimate of team skill, so even though the Angels are beating their first, second, and third order records every year, on average we are still better off looking at a team’s second order record if we want to guess how well it will do the following year.

That does not mean the Angels do not have a knack for beating their first and second order records. To prove that, we would need to look at how well teams that beat their first and second order records repeat that feat the following year. If there is no year-to-year correlation of teams’ abilities to beat their adjusted records, then we are left with a conclusion that the Angels are simply very lucky.  That might sound like a copout, but it is not at all.  There has to be one team that gets the title of “the luckiest team ever” just as someone out there needs to be flipping a coin and calling it in the air correctly 10 times in a row.  One of every 1,000 people will accidentally call a coin correctly 10 times in a row without any skill (and one out of every 1,000 will guess wrong 10 times in a row).  Statistically, some team needs to be the luckiest.

However, we see some pretty clear evidence that the Angels might have some skill.  There is a small but real skill level in beating one’s first order record.  The difference between first order wins and actual wins for teams had a .079 correlation from year to year.  So while almost all of the fluctuation around one’s first order record is luck, about 8 percent of that fluctuation is actual skill level. 

Not only that, there is a .193 correlation in the difference between second order wins and real wins, though only a .103 correlation in the difference between second order wins and first order wins. 

The .193 difference comes from simply aggregating the two effects: that some teams are good at winning close games, and that some teams are good at generating a bigger run differential than their total base, hit, and walk differentials suggest.

The ability to generate that bigger run differential than hit and walk differentials suggest is actually made up of two parts which should be looked at separately. Firstly, do some teams have the ability to sequence their total bases, hits, walks, and outs in such a way that they score more runs than other teams? Secondly, do some pitching staffs have the ability to sequence their total bases, hits, walks, and outs in such a way that they allow fewer runs? The answer to the first question is more likely to be yes than the second.

The correlation from year to year of the difference between runs and EQR is .104, while the correlation from year to year of the difference between runs allowed and EQR allowed is just .055. Both suggest some evidence of a skill, while also highlighting that the majority of this comes from luck.

So, can we do any better than looking at second order standings if we want to predict next year’s standings?

 

Standings version averaged

Correlation with next year’s standings

Actual & 1st Order

.501

Actual & 2nd Order

.511

Actual & 3rd Order

.504

Actual, 1st, & 2nd Order

.510

Actual, 1st, 2nd, & 3rd Order

.509

1st & 2nd Order

.507

1st, 2nd, & 3rd Order

.503

2nd & 3rd Order

.496

 

It appears that the most information comes from averaging the actual standings with the second order standings.  There are obviously more things that could be done such as weighted averages, but these will lead to only small gains.  The lesson to be learned is that although you are better off looking only at the second order standings if asked to pick just one column, there is something added by looking at the real standings, too.  Adding in which teams are likely to win close games might do something. too, but this information is probably already contained when adjusting for second order standings.

For readers’ information, and also to fuel discussion about the natural follow-up question of “which teams win more games than their run differentials or batting lines suggest?”, I leave you with a few lists: one that answers the question of which teams have won more games than their run differentials suggest over the last six years, one that tells you how much each team has outscored their expected runs over the last six years, and another tells you how much each team has beaten their expected runs allowed over the last six years.

 

Team

WW1

Angels

28.2

Astros

22.5

White Sox

15.7

Diamondbacks

10.4

Padres

9.5

Marlins

9.1

Mariners

8.6

Yankees

8.2

Brewers

7.1

Red Sox

5.7

Reds

4.6

Cardinals

4.2

Rays

2.3

Giants

2.0

Phillies

1.5

Twins

0.9

Royals

-1.4

Nationals

-2.7

Mets

-4.1

Orioles

-4.4

Tigers

-5.0

Dodgers

-7.1

Cubs

-7.1

Rangers

-8.0

Pirates

-8.8

Rockies

-9.4

Athletics

-10.6

Indians

-22.6

Blue Jays

-24.5

Braves

-25.1

 

Team

Runs – EQR (2005 to 2010)

Angels

111

Twins

83

Cardinals

79

Rangers

77

Braves

61

Pirates

51

Royals

46

Giants

42

Athletics

38

White Sox

25

Rockies

15

Blue Jays

14

Tigers

11

Astros

10

Phillies

9

Marlins

6

Dodgers

-1

Padres

-2

Indians

-19

Mariners

-27

Brewers

-28

Nationals

-39

Yankees

-39

Reds

-41

Cubs

-43

Mets

-46

Diamondbacks

-51

Rays

-90

Orioles

-113

Red Sox

-138

  

Team

Runs Allowed – EQR Allowed (2005 to 2010)

Phillies

-170

Angels

-100

Pirates

-95

Twins

-86

Reds

-72

Braves

-67

Red Sox

-67

Blue Jays

-46

Astros

-36

Mets

-18

Indians

-11

Giants

-8

White Sox

-8

Orioles

-5

Cardinals

-3

Athletics

3

Padres

15

Tigers

15

Mariners

20

Rockies

37

Rays

38

Nationals

40

Brewers

45

Yankees

51

Cubs

52

Marlins

60

Rangers

79

Diamondbacks

89

Royals

111

Dodgers

145

Thank you for reading

This is a free article. If you enjoyed it, consider subscribing to Baseball Prospectus. Subscriptions support ongoing public baseball research and analysis in an increasingly proprietary environment.

Subscribe now
You need to be logged in to comment. Login or Subscribe
bozarowski
11/11
As there seems to be some degree of skill to 'overachieving' the predicted wins, I'm curious what the results would look like if we applied this sort of analysis to managers' seasons instead of team seasons. This sort of methodology seems like it could be a jumping off point to start to come to some sort of quantitative analysis of the impact of a good or bad manager. Considering that over that 6 year sample every one of those teams had massive personnel turnover (just looking at the Angels alone, Ervin Santana and Scot Shields seems to be the only relevant contributors to both teams), it seems like the cause might be linked more in the manager or organizational philosophy than the actual players.
crperry13
11/11
Got any ideas for how to "fix" this? Those charts at the end have a huge variation. Looking at the absolute values of those charts shows that on average, the projections are off by 9.4 Wins, 52.1 runs allowed, and 45.2 runs scored.

That's a pretty significant average difference between projected and actual.
crperry13
11/11
Course, that is over six years, which isn't so bad after all.
tkniker
11/11
Just to throw out something. This piggybacks on something Matt you looked at about 15 months ago.

Evidence suggests that the variability of runs scored (and runs allowed) can have a significant impact on a team outperforming their Pythag expected Win Pct.

A team who has a higher than normal variability on Runs Allowed perform better, (and the corrollary that a team who has a lower than normal variability on Runs Scored) seem to consistently outperform their Pythag.

With that said, are the Angels more consistent in Runs Scored and "less consistent" in Runs Allowed than most teams?
swartzm
11/11
Wow, I didn't even think of relating that article to this. That is definitely worth checking- thanks!
jjaffe
11/11
As the creator of the Prospectus Hit List, which averages the actual, first, second and third order standings to craft the Hit List Factor, I'm pleased to find further backing that its predictive value holds up just about as well as any Matt checked (.509), and incrementally better than using just the third-order standings.
swartzm
11/11
This is definitely vindication of that method, and even more so because the Hit List tries to adjust for strength of schedule. The Orioles do have to play in the AL East every year, so even if 3rd Order Record overestimates their future performance, it more correctly states their value.
jjaffe
11/11
Furthermore, having studied the matter in the past relating to the Angels, I can report that some of the persistent difference owes to bullpen management. As I reported here (http://www.baseballprospectus.com/article.php?articleid=9529), the correlation between a team's cumulative WXRL and its D3 (the difference between actual and third-order record) is .42, whereas it's just .20 for SNLVAR.
joelefkowitz
11/11
I understand that players switching teams should wash out, but if there existed certain roster management tendencies (not sure what they'd be off the top of my head) of teams that outperform or under perform couldn't that create a bias?
jjaffe
11/11
In the specific case of the Angels, their track record appears to suggest that Mike Scioscia does have some knack for bullpen management beyond most other skippers. I can't recall what the correlations were if I removed the Scioscia Angels from the study, nor did I try to identify any other skippers with similarly persistent tendencies - it's an area of study I've always hoped somebody with more time and better numerical chops would pick up and expand upon, either to confirm or explain away what I'd found.
cwyers
11/11
I've puttered around with the Angel's Pythag before, and one of the things I've found is that there's a certain amount of over-performance built into simply having a good record for a sustained period of time. That's because the spread of Pythagorean win percentages is smaller than what we see in real life.
ckahrl
11/11
The 2010 Astros are a particularly interesting ]case: they were dramatically awful in the first two months, particularly on offense (17-34, 155 RS, 256 RA), and performed below their Pythag. They were then significantly better afterward (59-52, 456 RS, 473 RA), both offensively and in outperforming their Pythag.

I don't think there's any particular genius or "luck" involved, beyond noting they were initially trying to play guys like Towles and Feliz and Manzella, and coming to the belated recognition that these weren't just bad answers, they were horrific. Because they were *so* bad early, that contributed to a huge deficit in their eventual RS that helps makes them seem like massive overachievers.

Or to put it another way, Brad Mills seems like a decent skipper, but I wouldn't fire up the "certified genius" bandwagon just yet.
crperry13
11/12
I think you just illustrated the point that player/personnel changes shouldn't just be swept under the rug. Any self-respecting Astros fan knew that Feliz and Manzella were experiments doomed to failure, and all untested prospects have huge uncertainty. There should be some way to use a player's liklihood of actually finishing where he started to adjust the team projections.

For example: Feliz started 2010 as the 3B starter projected for 500+ PA, but most pundits would agree that if the Astros were to make a change at that base, the most likely outcome would be a small improvement in Runs Scored. Let's say a 60% chance that any change at 3B would mean more Runs Scored, based on his own projected VORP.

His own history and projections should also imply that a change at that base during the course of 2010 would be more likely than not - let's say a 70% chance, whereas a player like Hunter Pence would have a 5% chance.

Quantifying or projecting the likelihood of player and personnel changes would be an interesting experiment to see how it would affect the projections. 4th Order?

P.S. You forgot that Kazuo Matsui started at 2nd base. That surely didn't help.
momansf
11/12
The only problem with the Prospectus Hit List is that in the end, there are 5 AL teams ahead of an NL team because they are considered to have stronger competition. But, when a team like the Phillies, who had the top record in baseball this year and had a dominant team overall finishes lower than a team like the Red Sox due to schedule strength, it seems that matchups are overvalued. It seems a bit too disproportionate to have the AL be so much better than the NL.
joelefkowitz
11/12
There's a separate league adjustment which basically just penalizes NL teams. The Phillies probably finished where they did because this league adjustment is too big (for some people's tastes) and not because of matchup adjustments.