CSS Button No Image Css3Menu.com

Baseball Prospectus home
  
  
Click here to log in Click here for forgotten password Click here to subscribe

Premium and Super Premium Subscribers Get a 20% Discount at MLB.tv!

<< Previous Article
Premium Article On the Beat: Mid-Week ... (01/27)
<< Previous Column
Premium Article Checking the Numbers: ... (01/20)
Next Column >>
Premium Article Checking the Numbers: ... (02/01)
Next Article >>
Premium Article Transaction Action: Se... (01/27)

January 27, 2010

Checking the Numbers

Preceding the Value of Volatility

by Eric Seidman

the archives are now free.

All Baseball Prospectus Premium and Fantasy articles more than a year old are now free as a thank you to the entire Internet for making our work possible.

Not a subscriber? Get exclusive content like this delivered hot to your inbox every weekday. Click here for more information on Baseball Prospectus subscriptions or use the buttons to the right to subscribe and get instant access to the best baseball content on the web.

Subscribe for $4.95 per month
Recurring subscription - cancel anytime.


a 33% savings over the monthly price!

Purchase a $39.95 gift subscription
a 33% savings over the monthly price!

Already a subscriber? Click here and use the blue login bar to log in.

In last week's column, we discussed the free agent market through a pair of bifocals, with one lens focused on volatile pitchers and the other concentrating on consistent moundsmen. It stood to reason that the more volatile pitchers would be of much greater utility to teams with nothing to lose, while the steady and reliable ones were better suited to add stability instead of putting a team over the top. Throughout the discussion, which personified the aforementioned archetypes with Joel Pineiro and Jon Garland, I operated under a major assumption: past volatility correlates strongly to wider performance distributions in the future, and more future volatility than consistency club card carriers. The popular saying that assumptions can make asses out of "u" and "mption" (or something like that-I'm not really up on what the kids say these days) used in conjunction to the axiom that past performance is not always a clear-cut indicator of what to expect moving forward suggests that a prequel needs to be written to better understand what was intuited last week. With that in mind, today's efforts will be spent investigating what I simply figured to be true: Do volatile pitchers project to be more volatile than those considered consistent in a predetermined span?

For starters, why is this important? Recall our previous conclusion,that the Pineiros of the world should be sought after by teams like the Mets, with a wide range of potential outcomes, since the low end of his distribution would not realistically cause a hindrance, while the high end could conceivably push them into post-season contention. Well, if we find out that Pineiro and his brethren project similarly in the variance department to Garlandites, that takeaway is essentially rendered moot from an individualistic standpoint. We would be advocating decisions to be based on unreliable past information.

Methodology

The research steps involved here are fairly simple, starting with the culling together of every four-year stretch starting between 1974 and 2006. For the sake of transparency, and since I get questions every now and then about why I avoid using 1954-1973 data, 1974 was set as the minimum first year since Retrosheet is missing some games prior to that season; 2006 was the maximum first year because of the length of the desired spans as a four-year span starting in 2006 ends last season. In each span was the number of games and starts as well as park-adjusted ERA. Sure, an estimator could be used in its place, but this is much more back of the envelope in scope and the adjustments for park definitely add more validity to the numbers. From there, the standard deviation of the adjusted metrics in the first three seasons was computed and the pitcher-spans were classified based on their relationship to the median deviation: The top 50 percent fell into the volatile bin with the rest categorized as consistent.

As an aside, pitcher-spans does not refer to unique pitchers, as many hurlers will appear multiple times in the sample-Roy Halladay 2001-04, 2002-05, 2003-06, you get the point-and further, some may vacillate between classifications. The next step involved the computation of the fourth-year park-adjusted ERA based on a hint of regression, a dash of aging and a simple, Marcel-esque weighting scheme of the first three seasons. With the data assembled, three main questions are raised:

  1. Which group proved easier to project?
  2. How did each group perform in the fourth year, on average?
  3. What were the standard deviations in actual performance in that fourth year?

These questions will inform us whether or not volatility yields more volatility in the future than consistency; in other words, whether or not Pineiro can actually be expected to have a wider performance distribution in 2010 than Garland, even though they would fall into different bins over the last few seasons. Each of the three questions above could be answered differently based on restrictions in playing time, but the first point can be discussed through root mean square error testing, with the second calculated through a weighted average and the third through a general standard deviation function. Enough stalling, I know, get to the results!

Results

The table below shows the following year 4 data for each group, in different runs based on playing time parameters: Projected park-adjusted ERA, actual park-adjusted ERA, the root mean square error between actual and projected, the standard deviation of actual park-adjusted ERA, and the percentage of those whose actual mark fell two or more standard deviations from the mean. It should be stated, too, that playing time parameters refer to a minimum number of games started in each of the four seasons, so for the group of 20 each member made 20 or more starts in every season.


Type       GS Min    N   Projected Actual   RMSE    St.Dev   2+ SDS
Volatile     20     710    3.93     3.88    0.79     0.88     4.93%
Consistent   20     711    3.73     3.76    0.78     0.87     4.64%

Type       GS Min    N   Projected Actual   RMSE    St.Dev   2+ SDS
Volatile     10    1196    4.08     4.06    0.96     1.04     5.10%
Consistent   10    1186    3.82     3.91    0.98     0.99     4.38%

Type       GS Min    N   Projected Actual   RMSE    St.Dev   2+ SDS
Volatile      5    1560    4.19     4.17    1.18    1.26      4.29%
Consistent    5    1557    3.86     3.98    1.10    1.19      4.50%

A bit of recapping is in order here, but let's get the obvious out of the way first in that, no, each sample is not comprised of 100-percent equal groupings, but they are very close in stature and the results do not rest on this in any event. What makes perfect sense is the inverse relationship between the games started requirements and sample size. As we become more flexible, more pitchers qualify. Even with the different playing time parameters, we may run into a bit of sampling bias here in that each group is required to have appeared in four consecutive seasons. Thus, someone whose volatility is the product of injury and missed a year would not be included, nor would anyone who flamed out after two years. It might not be prudent to include pitchers like this in such an analysis anyways, but it is certainly worthy of disclosure.

What initially stands out is that the consistent group posted better park-adjusted ERAs from both the projection and actual standpoints. In other words, on average, the consistent pitchers posted better numbers, though both groups worsen as the playing time restrictions loosen up and any old pitcher is permitted to attend the sample party. Moving to the root mean square error, which helps measure projected to actual values, we can see that there is virtually no difference amongst those making 20 or more starts for four straight seasons. If the restriction is lowered to 10 or more starts, the consistent pitchers are ever so slightly tougher to project, which is neither a typo nor intuitive. Lower the bar even further and, finally, the expected results surface in that volatile pitchers were tougher to peg.

Shifting the focus to standard deviations in actual park-adjusted ERAs in that fourth season, the numbers jibe with expectations but the magnitude of the deltas should tickle your interest bones. It makes sense from a sampling standpoint that the 20-plus games started group will experience minimal discrepancies in any category aside from performance given that only pitchers with a good amount of talent will be called upon that frequently and consistently. However, the deviations are not much different amongst the 10-plus games started crop, and there isn't anything really to write home about with regards to the five-plus games started crop. Across a larger sample, a delta of 0.05 or 0.07 is certainly meaningful to an extent, but I entered this fray expecting to see deviation deltas along the lines of 0.15 or greater. Maybe I'm just an excitable person, but I cannot imagine being the only one whose expectations were not met.

Conclusion

Synthesizing the results, it is becoming clear that the differences between consistent and volatile pitchers in future performance deal primarily with playing time. If we expect Pineiro and Garland to each continue to toe the rubber for 30 starts, then it tends not to make a difference what they did over the last three years in terms of volatility, as neither is likely to end up any more consistent or volatile as the other. Sure, Pineiro in this instance might experience a boom-or-bust season, but the data suggests that decisions based on volatility or consistency are ill-defined if the underlying rationale centers predominantly on the recent past. Overall, this information does not alter the idea that volatile players should be attractive volatile teams, with the analog true for consistency, but it sheds light on the identification process of volatility and consistency.

A team that signs someone like Garland (stay classy, San Diego) should not expect any more consistency than from the lone member of the Pineiro Guild (hey, hey, L,A,A), even though the former has certainly been more consistent in the past then the latter. The idea of clubs deriving more utility from certain players remains intact, but our process of figuring out which pitchers could provide this increased utility is debatable.

Eric Seidman is an author of Baseball Prospectus. 
Click here to see Eric's other articles. You can contact Eric by clicking here

Related Content:  The Process,  Year Of The Injury

4 comments have been left for this article.

<< Previous Article
Premium Article On the Beat: Mid-Week ... (01/27)
<< Previous Column
Premium Article Checking the Numbers: ... (01/20)
Next Column >>
Premium Article Checking the Numbers: ... (02/01)
Next Article >>
Premium Article Transaction Action: Se... (01/27)

RECENTLY AT BASEBALL PROSPECTUS
Fantasy Article The Stash List: 15th Edition
Premium Article Minor League Update: Games of Monday, July 2...
Premium Article What You Need to Know: July 29, 2014
BBQ State of Mind
Notes About Baseball: The Logistics of Human...
Premium Article Transaction Analysis: The Migration of the B...
Premium Article Baseball Therapy: Trading Ryan Howard For No...

MORE FROM JANUARY 27, 2010
Premium Article Transaction Action: Senior Circuitry
Premium Article On the Beat: Mid-Week Update
Premium Article Hot Stove U.: Red Sox are the Best Farmers

MORE BY ERIC SEIDMAN
2010-02-09 - Introducing SIERA
2010-02-08 - Introducing SIERA
2010-02-01 - Premium Article Checking the Numbers: Drilling Down on Volat...
2010-01-27 - Premium Article Checking the Numbers: Preceding the Value of...
2010-01-20 - Premium Article Checking the Numbers: Valuing Volatility
2010-01-11 - Premium Article Checking the Numbers: Side Effects on Pitche...
2010-01-07 - Premium Article Checking the Numbers: Low Risk, Any Reward?
More...

MORE CHECKING THE NUMBERS
2010-03-03 - Checking the Numbers: Power Sapped
2010-02-24 - Premium Article Checking the Numbers: The Crystal Orb of SIE...
2010-02-01 - Premium Article Checking the Numbers: Drilling Down on Volat...
2010-01-27 - Premium Article Checking the Numbers: Preceding the Value of...
2010-01-20 - Premium Article Checking the Numbers: Valuing Volatility
2010-01-11 - Premium Article Checking the Numbers: Side Effects on Pitche...
2010-01-07 - Premium Article Checking the Numbers: Low Risk, Any Reward?
More...

INCOMING ARTICLE LINKS
2010-02-01 - Premium Article Checking the Numbers: Drilling Down on Volat...
2010-01-28 - Premium Article Expanded Horizons: Visual Consistency