March 21, 2003
BP Does Tout Wars
How PECOTA Fared
So when Jason Grey and the crew of mastersball.com asked me to join Tout Wars this season, I was torn. Having met Jason at Ron Shandler's Arizona Fall League Symposium in November and being a veteran of countless home roto leagues, I loved the idea.
But that still left one question: What was in it for BP? Sure, the draft offered a chance for some publicity. But holding Pizza Feeds, hosting Scoresheet drafts, and having Jeff Bower run out naked to the 50-yard line during halftime of the Super Bowl already offered great marketing opportunities.
To truly leverage the draft (we chose National League), we decided to get into the spirit of Tout Wars. Known as a venue for trying out new drafting and team-management strategies, it became obvious BP could use the draft as a way to showcase Nate Silver's PECOTA system. A short chat with Nate and BP injury guru Will Carroll later, we had our BP dream team--or at least two guys with perfect skill sets for the occasion, and one who could breeze through the draft still cracking jokes and bouncing in his chair after four hours.
Now all we needed was a strategy. PECOTA was originally designed to project statistics that are team-independent, or at least reasonably close to it. Sure, we can be snobbish at times in our relegation of RBI and its brethren to the dustbin, but the conceptual advantage in projecting team-independent statistics is that it's possible to do so with a minimum of user intervention. Without that, the first virtue of any projection system--that it be objective--is easily compromised.
Six of the 10 categories used in Tout Wars fall into this bundle: batting average, home runs, stolen bases, ERA, strikeouts, and WHIP. The projections, of course, do include an adjustment for park effects, as well as a correction for team defense that affects a pitcher's ERA and WHIP. But those things are reasonably easy to evaluate objectively. For the remaining categories, we needed to get our hands a little dirty.
Runs and RBI were estimated based on constructing each team's probable batting order, and running each player's weighted mean PECOTA projection through a simulator program thousands of times. Constructing batting orders is a dangerous game: We don't know, for example, whether Larry Bowa is going to settle on Jimmy Rollins or Marlon Byrd or Placido Polanco as a leadoff hitter. We do know that Jim Thome should add 40 or 50 runs to that lineup, and that whoever fills the top couple of spots in the batting order stands to cross home plate more often as a result. Information is the name of the game, and to fail to account for batting order effects is, at best, a little lazy.
Projections for pitcher wins were constructed based on a variation of the Pythagorean formula. If you know a pitcher's runs allowed, his run support, and his innings pitched, you can do a reasonably good job of estimating what his wins total should be. We don't know any of those things, of course, but we think we do, and we ran with it. Roy Oswalt, projected for a 3.29 ERA in 200 innings, was forecast at 15 wins; Odalis Perez, projected for a 3.25 ERA in 200 innings but with a weaker offense behind him, was forecast at just 13. In addition, we tweaked the projections by making sure that the individual win totals summed up to the team's predicted W-L record, which is an indirect way to account for bullpen support.
You'll notice that a pitcher's own won-loss record from previous seasons isn't used at all; if there is such a beast as the Pitcher Who Just Knows How To Win (a.k.a. Jack Morris), sightings of him are so unusual that we're not going to bank on being able to capture him for the purpose of creating projections. Forecasting any pitcher to win 20 games is a risky proposition, as it requires a perfect storm of great pitching, run support, durability, and luck. Randy Johnson and Curt Schilling were tied for the projected league lead at a pedestrian 17, a figure driven in part by an expected drop-off in their support from the offense and the bullpen. Somebody will win 20, but a dartboard is as well equipped as a spreadsheet to determine who it is.
There's no elegant sabermetric way to come up with proper estimates for saves. We did discover that there are some predictable relationships between a team's W-L record, its runs scored and allowed totals, and the number of saves that it accumulates. Naturally, teams that win more games wind up with more saves; teams that win with pitching and defense instead of hitting usually finish with a few more saves on top of that. Although the variance occurs within a fairly narrow range--Arizona topped the charts at 45 saves, while Milwaukee trailed with 38--it's just enough to be worth worrying about (the range would have been quite a bit wider within the American League, where there's more differentiation between the top and the bottom teams).
Thus, we were able to come up with an estimate of the total number of saves for each team, and proceeded to allocate them among individual pitchers accordingly. In order to do so, we considered the historical usage patterns favored by each team's manager--some managers will give all but a handful of save opportunities to their main guy, while others aren't as afraid to go with the hot hand--as well as injury reports, and the PECOTA forecasts themselves. For example, we're projecting Antonio Alfonseca to be only the fourth or fifth best pitcher in the Cubs' bullpen this season, so it seemed wise to spread his save opportunities around a little.
Underlying all of the projections were estimates of playing time. We won't go into too much detail about the process for generating those here, as it's identical to that used in the PECOTA season preview articles that we're running this week and next. Suffice it to say that it's based on a combination of objective and subjective factors, including Will Carroll's input. As a result, appropriate hedges against injury risk were built into the projections, which seems like a neater way to do things than to run with a value that you know is too high, and chop off an arbitrary amount in the heat of the auction.
Creating your forecasts, of course, is only half the battle; you also need to translate them into appropriate dollar values. Because we felt that our comparative advantage lies in the projections themselves, we tried to apply a neutral approach to this problem, relying on the standard process of allocating marginal auction dollars based on projected marginal gains in the standings.
We did invest some time in building and applying an adjustment based on position, but it turned out not to make much difference except in the case of catchers; the middle infield positions, which are often the hardest to fill for a flesh-and-blood baseball team, have plenty of value in roto since those guys tend to steal a lot of bags. We went with a $180/$80 allocation between hitting and pitching in accordance with the going rates at previous Tout Wars drafts. (That turned out to be prophetic, as the split in the auction came out to an identical amount this time around, right down to the dollar).
A complete set of projections in hand, we proceeded to compare our dollar values with some of the published alternatives, and discovered almost immediately that ours were more top-heavy. That is, the best players in the league came out at a few dollars higher for us than in the consensus view, taking some budget away from mid-level talent. We took that for a good thing; one of PECOTA's strengths is in its ability to differentiate truly elite talent from flukish performances. We didn't realize until draft day, however, how much that would motivate a stars-and-scrubs approach.
In fact, we came into the draft without a traditional drafting strategy. One popular tack, known as the LIMA Plan, encourages owners to load up on hitting, leaving only a small fraction of the budget for drafting a closer or two, plus a pitching staff full of pitchers with strong peripherals who might lack defined roles and almost always lack name value. Though loading up on strong middle relievers and pitchers lacking set roles in the starting rotation tends to work better in a 4x4 league, where strikeouts don't count, the idea of hoarding offense and going light on pitching has achieved near-gospel status among roto players.
In an effort to zig where others might zag, we figured we might try to go against the grain a bit and bid a little more than might be expected on pitching. But that wasn't a set plan or anything. We didn't come in hell-bent on drafting hitters or pitchers. We weren't targeting power or speed. We didn't focus on counting stats or rate stats.
We had just one overriding goal in this draft: get value, no matter where and when it may turn up. If we ended up with 200 steals, five aces, three closers or six Thomes, so be it. We'd use the PECOTA dollar figures as our guide, and try to draft a team full of players going for less than PECOTA's dollar projections. We'd mix in a little draft awareness to monitor runs on positions, position scarcity and other trends. But value was king.
As with most drafts (or auctions), the big names came out early. As we mentioned, most roto players are scared to death to bid aggressively on elite players. The theory goes that by spreading your dollars around, you lower your risk. Maybe so, but PECOTA told us that Vlad Guerrero was worth $54. So when the NL's most valuable roto player came up, we jumped in with both feet, eventually landing in a two-team bidding war. We bid Vlad all the way up to $52, some $10 after most teams had sighed and given up. When the bid went to $53, we passed. Vlad wasn't a bargain anymore at $54, and given the other 11 teams' reticence, we felt confident we'd nab a bunch of other marquee players for less than full price.
We were dead right. We took care of saves by grabbing Billy Wagner and Scott Williamson early, at $25 and $19, vs. PECOTA's $26 and $24 figures. Though we didn't know it at the time, we were far from done with our pitching spending.
Meanwhile, the focus shifted to hitters. When Barry Bonds came up, the air seemed to get sucked out of the room. Yet despite the universal interest in the world's best player, teams refused to put their money where their drool was. When the bidding stopped at $38, we'd nabbed Bonds at $5 less than PECOTA value, securing our projected second-best hitter (tied with Bobby Abreu).
Todd Helton's name came up not long afterwards. Since roto couldn't care less about ballpark effects, positional value or other similar real-life value adjustments, Helton's a perennial must-have player. This one was trickier though--we were both worried about Helton's bad back, and Will echoed our concerns. We'd grab Helton if he came at a significant discount. At $33 (vs. PECOTA's $41), we were happy.
The same thing kept happening throughout the entire first half of the auction. While our esteemed and seasoned opponents sat on the sidelines waiting for what they perceived as value, we snared our bargains early and often. It can be a little scary seeing your roster with 11 spots still to fill and all of about $50 left to spend, while three other teams have only bought a player or two and have over $200 each at their disposal. But we stuck closely to our strategy. Brad Wilkerson at $16 when PECOTA said $22? Definitely. BP 2002 Cover Boy Adam Dunn at $24 when PECOTA said $30? The three of us came this close to jumping out of our chairs and doing a celebratory St. Patrick's Day weekend jig.
Because everyone saved their money for hitting, we kept finding pitching bargains. Byung Kim for $14 when PECOTA said $20 we loved--that even elicited a few "nice pick" calls from the room. PECOTA thinks Ryan Dempster can have a mini-turnaround season, so $4 vs. an expected $10 made sense. And so on.
But Scott Stewart, when we already had two closers? Unorthodox, sure. But even with PECOTA accounting for injury concerns and Frank Robinson's occasional tendency to spread saves around, it had Stewart at $14. We got him for $9. Sure, we'll probably want to trade a closer later on. But by getting a saves surplus at a discount, we set ourselves up to acquire whatever we might need later in the year, while not blowing money we'd use to grab values like Jose Vidro ($23 vs. PECOTA's $26) or Felipe Lopez (a pre-emptive opening bid of $6 that got him for $10 less than PECOTA's "Barry Larkin is toast" estimate).
By the end, we'd landed a top-heavy offense and deep pitching staff. Of course, spending $109 of a $260 budget on nine pitching spots (vs. 14 hitting spots) will do that, especially when, as noted earlier, the average split worked out to $180 for hitting and $80 for pitching.
So here's our roster (click on NL Rost. (XLS) or (PDF) at the Tout Wars site to view the BP roster under "Keri," as well as our 12 opponents' teams):
C Y. Torrealba $1, Estalella $1 1B Helton $33 2B Vidro $23 SS F. Lopez $6 3B K. Ginter $1 MI J. Wilson $1 CI Houston $3 OF Bonds $38, Dunn $24, Wilkerson $16, K. Robinson $2, Ledee $1 UT Dellucci $1 SP Wood $22, Kim $14, Lawrence $13, Dempster $4, Marquis $2 RP Wagner $25, Williamson $19, S. Stewart $9, Shuey $1 Bench Neal, D. Minor, Eischen, Linden, R. Martinez, C. MillerHow does PECOTA think we'll do? Not surprisingly, it has us winning by a landslide. That's a natural bias of any predictions system: Follow it better than anyone else, and the system will tell you you're a runaway winner. What was most interesting wasn't so much the final projected standings, but rather some of the individual categories. While we'd have expected to fare well in saves, ERA and strikeouts (PECOTA says 1st in all three categories), it's the hitting side of the ledger that stands out. Despite carrying such luminaries as Ricky Ledee, Keith Ginter, and Yorvit Torrealba on the roster, PECOTA thinks we'll finish second in batting average, home runs, RBI and runs scored, thanks to the big five hitters we snatched up at a discount. In fact, our only obvious weakness appears to be stolen bases, a category we can address as the season goes on if necessary.
Of course this is all theoretical. Injuries will happen, off years will happen, our opponents will find a rookie-year Albert Pujols here, an out-of-nowhere Vicente Padilla there. It'll be a long season, and we'll be scouring the waiver wire and the trade hotline often, looking to upgrade whenever possible.
As the season goes on, we'll provide periodic updates on how our team is doing. We'll also work with PECOTA to develop optimal ways to add talent at optimal prices. (check out PECOTA's complete AL and NL roto projections, also running today, at BP Premium--if you're not a BP Premium subscriber, click here to sign up)
We hope you'll come along for the ride. E-mail Nate with PECOTA-specific roto questions. E-mail either one of us with ideas for improving the team. Marketing is nice, and we're sure to have fun along the way. But we're also in it to win it. PECOTA's a very sore loser.
Jonah Keri is an author of Baseball Prospectus.