Suppose Player A plays at league average level for all 162 games, and Player B plays at a league average level for half a season and then gets hurt and misses the rest of the season. Both put up the same number of wins above average (0 WAA), but Player A helped out his team far more than Player B. Average players have significant value, and teams pay far more than the league minimum for their services on the open market. Contrast this with the scenario where a replacement-level player gets injured. Typically he can be replaced by another player who would be willing to play for the league minimum. So, in a sense, a full season of 0 WAR and a half season of 0 WAR have the same value, whereas the same is not said of a full season of WAA vs. half a season of WAA. There can be legitimate reasons to favor other baselines of course though.

This seems a bit backwards. Each implementation of the WAR framework makes slightly different assumptions and thus answers slightly different questions. One should start with figuring what question(s) one wants to answer and what assumptions one is comfortable with, and that can dictate which implementation to use. Starting with the answer and then searching for what question produces it works better in Jeopardy than sabermetrics.

There *is* something special about these guys. Taillon might be the best prep pitching prospect in a decade, and Allie and Heredia have tons of potential as well. Also, their #1 pick in 2009 comes in at #4, and their #1 pick in 2008 is already starting for their MLB team. Also I suppose it's worth mentioning that players drafted before 2008 were drafted under a different front office.

Regarding #2 and the 2003 Tigers, even though you acknowledge the potential effects of random variation in performance, I'm not sure if you give it enough credit. You've selected them because they were historically bad, but a small number of replacement-level teams might perform that poorly by random variation alone. Furthermore, I'm not sure about your assumption that they were making an effort to win every game. Yes the players put on the field probably were, but I imagine there might have been young sub-replacement level players who were played to gain experience in the hopes of being above replacement-level in future years (off the top of my head without looking at numbers, Jeremy Bonderman was probably such a player). Finally, there may have been a lot of replacement-level or better players that should have been given playing time but were not due to mistakes by management. So the fact that on occasion a team can have an *observed* performance a little below replacement-level is not too troubling.

What Tango said. BillJohnson, you do recognize that PECOTA itself comes with a list of percentiles, and that is what we are evaluating, right? I want to make sure there's no confusion on that. So PECOTA is telling us that it thinks something will happen 10% of the time, so we want to check to see if that does happen 10% of the time. Now, if we were just using the mean PECOTA forecast, then yes that could very well be Gaussian and would be tighter if PECOTA were better. But that is not what we are talking about here. We are not evaluating how well PECOTA projects mean performance, we are evaluating the very percentiles (really deciles) that PECOTA is giving us.

It's not about "who is better than who." Forgive me for putting words in his mouth, but I'd bet that Tango, for instance, would LOVE for PECOTA to absolutely destroy Marcel and for the percentiles to work as we expect them to. He's not worried about some chest-thumping competition to prove the superiority of "his system." He pretty much tells you that "his system" is the worst possible acceptable system and he pleads for you to do better if possible. If there are flaws in existing techniques, it is a good thing for the entire baseball analysis community for those flaws to be corrected. We all want to see the best possible analysis whether it comes from BP or somebody else. I think the main problems arise when a lack of transparency (perhaps coupled with some marketing hype) gives something the illusion of being the best it could be when it in fact is not. I applaud Colin for doing the dirty work to make the process much more transparent and therefore much more open to improvement.

Have you ever done "retrojections" with PECOTA? That is, use the latest version of PECOTA but only using data available at the time (so we could have a forecast for 2009 using only data available through March 2009, etc). That would give us much larger samples to at least judge PECOTA against itself (this could also be done with Marcel since it's open-source). Also if the algorithm for PECOTA is tweaked we could examine the new retrojections to see how much they improved (although much care must be taken to avoid over-fitting). I recognize that this would be a huge undertaking, you might not have access to all the right data required, and you are surely busy with other stuff, but it would be cool to have.

Thanks Colin for the much-appreciated transparency. While I don't think any projection system is "deadly accurate," this restores my lost belief that PECOTA is in the same group at the top as the other major players in projections. Is it fair to normalize every projection to the league offensive environment? My instinct says no; the projections are the projections. However, I'd like to hear some arguments from both sides on this one as I'm not too sure what is most fair.

I would think Anthony Rendon would mean that Pedro Alvarez probably won't be a 3rd baseman after 2013 or so. I don't know that his defense would have held up there for long anyway, but Rendon would give them more incentive to have Alvarez switch corners.

I look at the "Moneyball approach" and sabermetrics as two different, but related, ideas. From my understanding, the Moneyball approach is mostly about identifying skills (or classes of skills) that are undervalued by the market and making them a target of your organization. The A's used sabermetrics to identify that OBP was such a skill. The Rays' sudden rise in 2008 was largely a result of their improvement on defense, something thought to be undervalued by the market. However, I'm not sure that sabermetrics was how they accomplished this; it might have involved a large amount of scouting instead (I don't know for sure).

Exactly...Matt Morris pretty much epitomized "spending money" on a "veteran, proven Major League starting pitcher." Incidentally, that move may have been the straw that broke the camel's back for GM Dave Littlefield, as he was fired not 2 months after that.

I don't know if this counts as "recently," but the Pirates traded former closer Mike Williams to Houston near the deadline in 2001, and he proceeded to re-sign with the Pirates that offseason.

What I saw in this article was a handful of cherry-picked examples...where is the analysis? Is this not a website claiming to be a leader in the objective analysis of baseball? A star player who might get traded has a certain projected performance going forward and a certain cost. The prospects have the same thing, albeit with higher variance. A nice article might have compared the two historically to see if the star players actually had more value, on average, than the prospects received in return. Victor Wang already did much of the legwork for such a study in his research over prospect values. No offense, but I expect better from Baseball Prospectus.

I can't speak for Matt and others, but when I use the word "unsustainable" that does not mean I think it's completely impossible for the phenomenon to sustain for some finite period of time. If you go to the casino and double your money in a night of blackjack, I will call that unsustainable. If you do the same thing the next night it does not prove me wrong. Note that you might have done this while skillfully applying blackjack strategy and card-counting skills, but you still performed way above your long-term true talent rate...nobody is *that* good at blackjack. It's possible Jimenez will continue to post these unsustainable numbers, but calling them unsustainable was warranted according to how I use the term.

Do you have any educated guesses about what kind of money it would take to get some of these guys? Are we talking about $8 million for Taillon vs. $5-6 million for Machado?

"The best organizations use both." I think even the geekiest of stat geeks will agree that evaluations should be made with statistical analysis AND scouting. Personally I know that I cannot do the latter, so my analyses will always be incomplete. I think there is a difference though between recognizing one's own ignorance of a complementary technique and offering complete disdain for the technique (and I think both sides have been guilty of the latter).

I don't really disagree with most of what you are saying, but I don't really see your point. Yes we try to separate outcomes from processes and yes we do an imperfect job at it. Until we have a perfect "universe simulator" that includes ALL the variables (including those regarding the minds of the players) then we will always fall short of being able to completely separate the processes from the outcomes. That doesn't mean we can't do a pretty good job of estimating. Our estimations (even estimations of true talent) have uncertainties, and there are even uncertainties around those uncertainties ad infinitum. We should definitely be aware of that, but that shouldn't stop us from trying to improve and it doesn't give sabermetrics' detractors a free pass.

Yes, he did normalize things that way: "I’ve "normalized" those values so that a single is equal to one." Great work Colin. I often encounter people who criticize my use of "made-up" stats while defending their love of OPS. I love how you explain how it's not really even that simple when you look at it.

This is an interesting idea, but I think you do need to account for playing time in some reasonable way. Some of the teams may have an artificially high "Original W-L" because their roster now includes more than 25 players. If say a farm system produces 4 1st basemen who each go on to accrue 3 WARP on other teams, I'm not sure it's fair to credit the original team with 4*3=12 WARP since obviously not all of them would be playing. Obviously there is conservation of the 162*30 games started per position across the league, but some teams may be at 200-300 in some positions while others may even be at 0 for certain positions. I'm not saying I have a better way to account for this, but it strikes me as a bit incomplete without it.

Currently the "Leaders" on FanGraphs will only show wOBA back to 1974, but you can go back more than that for individual players. For instance, Babe Ruth had a wOBA of .600 in 1920 (and keep in mind that wOBA is purposefully set to be in the OBP scale). Perhaps it's true of TAv, but I don't know that it's accurate to say wOBA is "tuned" to a given time period. Instead I believe it is tuned to the run environment of the league for that season. In other words, we don't (or at least shouldn't) apply the correct linear weights from 2009 to figure out Babe Ruth's wOBA in 1920...we apply the correct linear weights from the AL in 1920 to figure Babe's wOBA in 1920. There are also different machinations you can go through to adjust for team-specific run-environments if you like (and I believe Rally at Baseball Projection does this for his linear weights if I'm not mistaken). I look forward to reading the internal studies that have been floating around BP, but I think it would take quite a substantial result to trump wOBA. In my opinion the tie should go to the system that has better logical foundations, and I think wOBA fits that bill. It finds the "correct" value for each type of batting event and adds them together. If TAv has a tiny edge in RMSE that wouldn't make it better than wOBA because wOBA has a significant edge in principled foundations (also note that wOBA was not engineered for the purpose of minimizing RMSE of predicted team runs scored).

It does not *have* to be either/or, but for much of the last year it seems to me that it has been that way. I think the geeky minority (of which I am a member) was starting to become a bit marginalized here in favor of appealing to the masses more. I signed up for BP because I was willing to pay money for the absolute best statistical analysis out there. I'm part of the small minority that wants the gory details outlined and that much prefers "10.8 WARP3" to "2 wins better." That said, I found that in 2009 BP was *not* providing the absolute best statistical analysis out there. Many of the stat heads were not contributing as much and their metrics were not keeping up with the likes of FanGraphs. I was willing to pay a premium for the best statistical analysis, but I was actually getting the best elsewhere for free. As a result I let my account expire. However, in recent months I think BP has made several good steps in the right direction in adding the likes of Colin Wyers and Pizza Cutter. This showed me enough to renew my account, but I hope you don't let it stop there. Appeal to the masses if you must in order to make some money, but please make sure you don't alienate the rest of us in doing so. Even if this means charging more for a more stat-friendly BP experience, that's OK with me. I'm eager to see if BP can continue to make more steps in the right direction.

This is interesting as it relates to batters. As for pitchers, we know that pitchers only have a small amount of control over whether or not a ball in play is converted to a hit, but I believe they do have more control over the batted ball type (sinkers tend to produce more ground balls, etc.). I'm a little confused as to what a pitcher should strive for as it relates to batted ball type. I seem to recall an article where Brian Bannister mentioned that for 2008 he focused striking more batters out, but in doing so he gave up more line drives and thus had worse results. He changed his approach to try to give up a greater percentage of ground balls, and he has had some success at that. I would have thought ground balls would be what a pitcher would want to get when he's not striking them out. Line drives are converted to hits a lot and fly balls turn into home runs a lot. That leaves grounders, but if I'm reading this right then hitters should strive for groundballs as well. What am I missing here? Could it be that grounders might yield base hits more than flyballs but produce worse results overall (since there will be fewer extrabase hits)?

I actually have tickets to the game itself from another group (at the low price of "free"). Is there any way I'd be able to get into the event without also buying the game ticket? I wouldn't even need the burgers and booze; I just want to talk baseball.

I have read a lot about the expectations for various BP Idol contestants. Well Brian helped to show how expectations for a top prospect may not mesh with his performance on the field. As a Pirate fan I've followed McCutchen's career for several years and I was never particularly impressed with what he did on the field, but I found few that saw the same thing. It's nice to see somebody agree that he might not even reach what McLouth did offensively...though I hope we are both wrong at that front. Thumbs up for helping us to discern reality from expectations in this instance.

If true, that tells me two things. First, the Pirates had better make sure to sign Sano. They also better use the money saved on Sanchez to get some top-flight guys in later rounds with signability issues.

As a fellow Bucco fan (my are we well-represented here), I agree. I don't see why money should be a factor, especially with the couple million saved in the McLouth deal. Personally if they draft someone like Crow and are able to sign Sano I would consider that a great message that things are different in this front office. I know some have been saying that a cheaper pick at #4 will free up money for Sano, but I don't see why we can't have it both ways...

First of all, I'll defend Brittany regarding the "one of the best outfields in the Majors last season" bit. I think for this purpose what needs to be looked it is how well they actually performed while they were together rather than how well they could be projected to perform in the future. While he was with the Pirates in 2008, Xavier Nady had an EQA of .318, while Jason Bay had an EQA of .312 for the Pirates that year before getting traded. Nate McLouth had an EQA of .300 for the season (I don't know where to find his EQA at the trade deadline, but it was likely slightly higher). So all three outfielders, while in Pittsburgh, were near the top of the offensive leaderboards for their respective positions in 2008. Dragging in sustainability or defense into the argument belabors the point a bit, but at the time the trades started they were playing like one of the best outfields in baseball. As a Pirate fan I wanted to vote for this article simply because it was about a topic that interested me. However, I didn't really find anything new in the article. For instance, I would have loved to read why Morton's minor league numbers improved so much in 2008 and 2009 (and perhaps why he hasn't yet translated that to MLB success). Perhaps this is an unfair standard since I've read a lot more on this topic than most have, but regardless I decided not to vote for this article. Good luck to Brittany and everyone else nonetheless.

It seems to me that the Pirates are placing a much higher premium on defense than they had in the past. They aren't fooled by McLouth's gold glove and they recognize that he is poor defensively. With their type of pitching staff that doesn't strike a ton of guys out, defense may be more important to them than average. They may in fact now have the best defensive outfield in the league; I'm sure Zack Duke et al appreciate that.

Funny you should bring up McGwire. As a teenager in the mid to late 90's, Mark McGwire was my favorite baseball player. I also felt that Stairs was like the poor man's McGwire, so I always kind of liked him. When my favorite team, the Pirates, acquired Stairs in the 02-03 offseason, I was pretty excited about the acquisition. My college roommate at the time made fun of me for thinking that Stairs would have any positive impact at all, and it turns out that he had quite the season once he was finally allowed to play. Since then Stairs has been involved in a lot of hyperbole in the back and forth between my friend and me; I of course insist that Matt Stairs is the greatest hitter ever and my friend insists that he has no value at all as a player. I especially love to point out that Stairs owns more career home runs and a higher career OPS than the legendary Roberto Clemente...I bet even Matt himself would be surprised to find that out...

I loved using a variant of this pitching strategy several years ago in leagues that capped the number of games *started* for pitchers but did not enforce any type of innings pitched limit. Those leagues also used daily changes so that I could get the most value out of the relievers. Instead of giving an active roster spot to a starter not starting that day, I could have a chance at an extra few K's while lowering my rate stats and maybe getting a vulture win. Using some starters though was necessary. This league structure is pretty rare though, and I haven't found it to be particularly useful in other structures.

Everybody remembers Kendall's ankle injury. However, what gets lost is an injury a couple years later. I don't recall the exact details, but I believe he injured his thumb either prior to or during the 2001 season. He played through it though, but the thumb in injury seemed to take away his ability to drive the ball, thus destroying the little power he had. He successfully bounced back from the ankle injury in 2000, but many Pirate fans (myself included) think the subsequent thumb injury is what destroyed his career.

The last year of Three Rivers Stadium was 2000. PNC Park opened in April of 2001, and Kendall turned 27 a couple months later. When PNC Park opened, Kendall was on track for Cooperstown (having just signed a new contract to pay him accordingly), but oh how he has fallen...

Add one more to the tally for Bucco fans. When you are waiting 'til next year before the season even starts, you need someplace like BP to take your mind off the current team. That said, I'm optimistic about what Neal Huntington and BP alum Dan Fox can do with this club.