CSS Button No Image Css3Menu.com

Baseball Prospectus home
  
  
Click here to log in Click here for forgotten password Click here to subscribe

World Series time! Enjoy Premium-level access to most features through the end of the Series!

<< Previous Article
Under The Knife: The D... (11/08)
Next Article >>
Premium Article Schrodinger's Bat: Fir... (11/09)

November 9, 2006

Predictatron Recap

We Have a Winner!

by Ben Murphy

Just as I did last year, I'm here to follow up the HACKING MASS Wrap with a look at this year's Predictatron results. This is the second year we've done the Predictatron contest, and it continues to be popular, for obvious reasons--trying to predict the order of finish and teams' eventual records is one of the oldest hobbies of baseball fans.

For those that haven't had the pleasure to compete, Predictatron is the annual contest at Baseball Prospectus where entrants can win $500 by predicting the total wins for each of the 30 major league teams, and the results of the playoffs. Basic scoring is set up so that everyone starts with 1000 points, and you lose points for every win you are off for each team; you can win points back with the playoffs. There are also a few wrinkles, like the Mortal Lock, so I'd encourage everyone to read the full rules.

This year, we didn't have time for the detailed statistics investigation, but we will take a look at the entire pool of Predictatron guesses, the winning ballot, and a comparison to some other preseason predictions. We'll start by going through each division and seeing how the Predictatron guesses picked the playoff teams. To avoid any allegations of falling victim to media bias, we'll traverse the divisions randomly...well, pseudo-randomly, anyway.

AL West

TeamActual
Wins
Average
Wins
Mortal
Lock%
Playoff%DIV%WC%
OAK9393.2518.4%89.7%87.4%2.2%
ANA8986.631.0%17.6%11.2%6.3%
TEX8079.120.4%1.6%1.0%0.6%
SEA7874.471.1%0.4%0.3%0.1%

By a pretty healthy margin, readers pegged the A's to take the AL West. In fact, the A's had the highest average win guess amongst all MLB teams, just barely higher than the Yankees. Curiously, the A's managed to outperform their third-order wins by a little over 10 wins to reach their total of 93 wins on the year. In instances where ballots didn't favor the A's to win the West, they almost all picked the Angels. The division shook out as expected, with the average guesses for the A's and Rangers basically dead-on, and the Angels and Mariners about three wins off.

NL Central

TeamActual
Wins
Average
Wins
Mortal
Lock%
Playoff%DIV%WC%
SLN8392.3227.3%92.9%90.0%2.9%
MIL7584.140.8%15.0%5.8%9.2%
CHN6681.870.6%7.6%3.2%4.3%
HOU8281.380.3%5.7%1.0%4.7%
CIN8073.344.1%0.2%0.0%0.2%
PIT6772.455.1%0.2%0.0%0.2%

Over 90% of ballots correctly picked the Cardinals to take the division, but like most preseason guesses, overestimated their actual wins. Predictatron ballots were optimistic about the Brewers, but were likely disappointed when they only won 75 games--injuries to their pitchers, especially to their frontline starter, Ben Sheets, didn't help. The Cubs vastly underperformed expectations, and didn't come close to the average prediction of 82 wins; they underperformed the average ballot prediction more than any other team. The average ballot pegged the Astros' win total, and were pretty close on the Pirates. The Reds almost made it to .500 this year, overperforming the average ballot by a few wins, even after a late season swoon following their trading two key hitters to Washington.

AL Central

TeamActual
Wins
Average
Wins
Mortal
Lock%
Playoff%DIV%WC%
CLE7889.805.2%65.1%59.2%5.9%
CHA9088.042.6%35.8%28.4%7.5%
MIN9684.310.9%13.9%11.3%2.6%
DET9578.260.1%1.3%1.1%0.2%
KCA6260.6435.4%0.0%0.0%0.0%

Most readers liked Cleveland to win the division, despite a tight bunch of three teams at the top. Of course, the division did play out with a relatively tight three-team race, but it was the Tigers in the thick of things, not the Indians. In fact, no team in the AL underperformed expectations as much as the Indians did. Readers picked the Royals for mortal lock 35.4% of the time, the highest for any team, and contestants were rewarded accordingly, as the average ballot was within a win and a half of their actual wins, which is critical for the mortal lock pick. Ballots were also close on the White Sox, but the other three teams' average win guesses were all off by at least 11 wins.

AL East

TeamActual
Wins
Average
Wins
Mortal
Lock%
Playoff%DIV%WC%
NYA9793.2118.6%90.4%61.1%29.4%
BOS8691.9211.3%80.9%38.4%42.5%
TOR8782.880.2%3.1%0.4%2.7%
BAL7074.721.0%0.2%0.1%0.1%
TBA6170.983.8%0.0%0.0%0.0%

The Yankees and Red Sox tied for the lowest standard deviation of win predictions for all teams, meaning that readers tended to agree about how many games they would win more than any other team. Both teams were picked to make the playoffs the large majority of the time, usually with the Yankees taking the division and the Red Sox the wild card. Of course, the Red Sox disappointed everyone and underperformed by roughly six wins, and also finished third in the division and well out of the wild card race. Usurping second place in the East were the Blue Jays, who finished about four wins ahead of the average prediction. The Orioles and Devil Rays were both content to finish out the set in more typical fashion, although both underperformed expectations, by five wins and 10, respectively.

NL West

TeamActual
Wins
Average
Wins
Mortal
Lock%
Playoff%DIV%WC%
LAN8885.621.4%63.8%63.0%0.9%
SDN8881.790.3%17.6%16.6%1.0%
SFN7681.430.7%17.2%16.7%0.6%
ARI7677.001.1%3.9%3.7%0.2%
COL7669.2014.3%0.1%0.1%0.0%

The NL West seems like it is consistently a little bit zany, and last year was no exception. The Giants and Diamondbacks looked like they might make a run at the division or the wild card, but faded down the stretch, leaving the Dodgers and Padres to duke it out. Predictatron ballots picked the Dodgers by a few wins, but anticipated a tight three-team rumble, similar to the AL Central. The majority of ballots had the Dodgers winning, analogous to the Indians, only this time the ballots got it right, while the Dodgers and Padres both made the playoffs.

NL East

TeamActual
Wins
Average
Wins
Mortal
Lock%
Playoff%DIV%WC%
NYN9788.685.8%72.7%40.6%32.1%
ATL7988.586.0%68.7%45.5%23.2%
PHI8586.092.0%34.0%13.8%20.2%
WAS7170.6911.9%0.0%0.0%0.0%
FLO7867.1818.2%0.2%0.1%0.1%

This was the most hotly contested division in terms of picking the division winner. We have seen other divisions where the average win guesses for the top teams were very close, especially the AL Central, but in all other cases the leader for division winner was favored by at least 59% of ballots. Here, though, we saw the Mets and Braves basically splitting the bill, with the Phillies picking up some votes as well. We all know now that the Mets cruised for most of the year, overperforming the average predicted win total by almost 10 wins. The Braves stumbled, falling to third behind the Phillies (whose win total was basically pegged by the Predictatron ballots). Nobody saw the Marlins doing as well as they did, and they over shot the projection by about 11 wins. The Nationals' average win guess was basically dead on.

The Winning Ballot

The winning ballot was completed by Vince Femenella, who correctly picked the Cardinals to go all the way, and also got 6 of the 8 playoff teams correct.

When asked about his ballot, Vince expressed relief that his faith in the Twins paid off, and was pleased that his Mortal Lock picks had done so well. Vince was able to predict all but a few teams within single digits, and got an impressive 40 points back in the playoffs to finish with a 13-point lead.

When asked what he might do with the prize, Vince was unsure what he would do with the money, but vowed that "as a once contraction-threatened Twins fan, you can be sure the Bud Selig photo will find a place of honor." Congrats, Vince!

Now that we know how the divisions looked, we can break down how the Predictatron ballots envisioned the playoffs working out. We've already seen that the ballots favored the A's, Cardinals, Indians, Yankees, Dodgers, and Braves to win their divisions. That leaves the wild cards, which shook out as follows:

American League Wild Card

TeamAvg. WinsWC%
BOS91.942.5%
CHA88.07.5%
ANA86.66.3%
TOR82.92.7%
MIN84.32.6%
TEX79.10.6%
DET78.30.2%
SEA74.50.1%
BAL74.70.1%
KCA60.60.0%
TBA71.00.0%

This is basically as expected, as the Red Sox and Yankees have been so strong in recent years that most people expected the wild card to come out of the East. That thinking gave the Red Sox over 40% of the wild card votes, while the Yankees also got almost 30% of the wild card votes, likely from those voters that gave the Red Sox the edge in the division.

National League Wild Card

TeamAvg. WinsWC%
NYN88.732.1%
PHI86.120.2%
MIL84.19.2%
HOU81.44.7%
CHN81.94.3%
SDN81.81.0%
SFN81.40.6%
ARI77.00.2%
CIN73.30.2%
PIT72.50.2%
FLO67.20.1%
COL69.20.0%
WAS70.70.0%

Similar to the American League, the majority of ballots saw the wild card winner coming out the East. The top two teams on this list came out of the East, obviously, and the Braves also had 23% of the wild card votes, mimicking the Yankees wild-card voting in the American League. To summarize the wild card, 74.6% of wild card votes in the American League came from the East and 75.7% of wild card votes in the National League came from the East, which is pretty astonishing.

Once the entrants completed their predictions for the standings, each balloter was required to predict how the playoffs would proceed. We'll only consider analyzing the playoff ballots for teams who were picked to make the playoffs over 10% of the time. This excludes the random ballot picking the Mariners to win the World Series and its brethren.

Below is a chart describing how the playoff ballots shape up. Before you get too engrossed in the details, allow me to explain what the numbers are showing. The first column is simply what percentage of ballots picked the given team to make the playoffs. Each of the next six columns describe the team's performance in all ballots where it was picked to make it to the given round of the playoffs. Each pair of columns is the team's record and winning percentage in the Division Series (DS), Championship Series (CS), and World Series (WS).

For example, that means that the Padres were picked to make it to the playoffs 158 times; the were picked to go 27-131 in the division series, so they were on the ballot for the Championship Series 27 times. In those 27 chances, ballots picked them to win and advance to the World Series six times. In those trips to the World Series, balloters had them winning zero times.

So, without further ado, let's dig in...

TeamPlyoff%DSDS%CSCS%WSWS%
SLN92.9%612-2230.733349-2630.57094-2550.269
NYA90.4%358-4550.440150-2080.419105-450.700
OAK89.7%656-1500.814444-2120.677348-960.784
BOS80.9%279-4480.384115-1640.41284-310.730
NYN72.7%417-2370.638242-1750.58059-1830.244
ATL68.7%293-3250.474111-1820.37933-780.297
CLE65.1%276-3090.472101-1750.36661-400.604
LAN63.8%136-4380.23739-970.2875-340.128
CHA35.8%105-2170.32628-770.26716-120.571
PHI34.0%152-1540.49778-740.51320-580.256
ANA17.6%50-1080.31623-270.46015-80.652
SDN17.6%27-1310.1716-210.2220-60.000
SFN17.2%43-1120.27713-300.3028-50.615
MIL15.0%48-870.35624-240.5009-150.375
MIN13.9%53-720.42424-290.45316-80.667

As you peruse the table, here are somethings you might notice and find fascinating:

  • The entrants loved the A's; they have the highest average predicted regular season wins, the highest projected record in all three rounds of the playoffs, both in terms of number of times they were picked to win each series and aggregate winning percentage in each series (amongst these teams), and they were picked to win the World Series on 38.7% of ballots
  • 94 contestants correctly picked the Cardinals to win the World Series
  • The Cardinals are picked to make the playoffs more than any other team, but their predicted record declines rapidly as you go deeper into the playoffs, .733 to .570 to .269
  • The Dodgers were picked to make the playoffs over 63% of the time but were picked to advance less than 24% of the time
  • The Red Sox, similarly, were picked to make the playoffs over 80% of the time, but only picked to advance 38.4% of the time

There are also a few other tidbits and curios which are not shown on the chart:

  • The American League Champion Detroit Tigers were picked to make the World Series by one entrant, David Goldman; congrats to David;
  • The Royals, Devil Rays, and Nationals were the only teams that were never picked to make the playoffs;
  • The Orioles, Reds, and Rockies were picked to go to the playoffs a few times, but not to win any postseason series;
  • A few teams were picked to make the playoffs a handful of times and basically win out; for example, the Mariners were picked to make the playoffs four times and win the ALDS and ALCS all four times, and the World Series on three out of those four ballots. Now that's what I call fandom, except for that one slacker;
  • Similarly, all four of the people that picked the Rangers to win in the ALDS also picked them to win the ALCS and World Series.

Like Nate Silver did recently, we'll compare the Predictatron average guesses to the PECOTA projections, the ESPN experts' average predictions, and the Diamond Mind simulation predictions. Below is a listing for all teams, similar to the one that Nate ran in his piece but not broken out by division.

TeamActualPredictatronPECOTAVegasESPNDMB
ANA8986.68188.59079
ARI7677.07774.07476
ATL7988.68589.09185
BAL7074.77776.57274
BOS8691.99390.59186
CHA9088.08291.59286
CHN6681.98585.08185
CIN8073.37874.07077
CLE7889.88890.09088
COL7669.27468.57067
DET9578.38377.07779
FLO7867.27167.06169
HOU8281.48183.58478
KCA6260.66164.06162
LAN8885.68785.58786
MIL7584.18480.58379
MIN9684.38480.08490
NYA9793.294100.09493
NYN9788.78891.09087
OAK9393.39388.59296
PHI8586.18681.58486
PIT6772.57975.57675
SDN8881.87877.58077
SEA7874.57775.57180
SFN7681.48081.58486
SLN8392.38694.09395
TBA6171.06967.57070
TEX8079.18080.58081
TOR8782.97986.08583
WAS7170.77077.07375

Below is a chart showing how each set of guesses correlates with the actual wins:

SystemCorrelation
Predictatron0.679
PECOTA0.669
Vegas0.618
ESPN0.636
DMB0.644

As you can see, Predictatron has the closest correlation with the actual wins, narrowly edging out PECOTA. This lends credence to the theories that support the wisdom of crowds. Compared to PECOTA, the Predictatron ballots did especially well with the Angels, White Sox, and Pirates. Since both systems did so well, we can't really say that Predictatron is a superior method to forecast team wins, but it is pretty fascinating that a large group of relatively well-informed guesses could match wits with such a finely-tuned system.

Both Predictatron and HACKING MASS continue to bring entrants back, and we enjoy reviewing and recapping things every year. Thanks to everyone who competed, and we hope you're excited about taking another crack at it next season.

0 comments have been left for this article.

<< Previous Article
Under The Knife: The D... (11/08)
Next Article >>
Premium Article Schrodinger's Bat: Fir... (11/09)

RECENTLY AT BASEBALL PROSPECTUS
Pebble Hunting: An Illustrated Guide to the ...
Baseball Therapy: The Truth About Butterflie...
Pitching Backward: How To Get A Hit Off Madi...
Minor League Update: Games of Tuesday, Octob...
Playoff Prospectus: PECOTA Odds and Game Two...
Moonshot: The Royals, the Strike Zone, and a...
Playoff Prospectus: The Other Royals: World ...

MORE FROM NOVEMBER 9, 2006
Premium Article Schrodinger's Bat: First Contact

MORE BY BEN MURPHY
2011-03-26 - Fantasy Beat: Improved SS/SIM for 2011
2008-04-21 - Fantasy Article BP Kings Preview
2007-04-11 - Fantasy Article BP Kings Update
2006-11-09 - Predictatron Recap
2006-11-02 - The AL-Kings
2006-11-01 - Hacking Mass
2006-07-26 - AL-Kings July Update
More...