BP Comment Quick Links
May 23, 2009 Prospectus Idol EntryCould you pick up some Kaopectate? I'm expecting runs.Introduction Doesn't it always seem to happen to your team? Down by two in the 5th, your team gets a leadoff double followed by a walk. First and second, no one out  this is the start of a big inning. Next thing you know, after a lazy flyball to shallow left and a strikeout, your only hope is a solid base hit. One weak grounder to short later, you curse your team for another wasted opportunity. The Run Expectancy Matrix helps us determine how much of a wasted opportunity the above scenario truly was. There are 24 unique states describing the position of runners and the number of outs in an inning. There are 8 unique runner positions (bases empty, men on 1st and 2nd, bases loaded, etc.) and 3 out possibilities. For each state, we are interested in determining the expected number of runs that are scored in the rest of the inning. If one goes to the Statistics page of Baseball Prospectus and clicks on "Run Expectancy Matrix" a table like the following will pop up: Runners Exp_Outs_0 Exp_Outs_1 Exp_Outs_2 000 0.526 0.281 0.108 003 1.520 0.951 0.362 020 1.165 0.708 0.334 023 2.017 1.425 0.600 100 0.908 0.536 0.228 103 1.772 1.566 0.496 120 1.558 0.944 0.461 123 2.349 1.596 0.803 Back to our scenario, this table tells us that, with runners on 1st and 2nd (Runners = 120) and no outs, the expected number of runs is 1.558. That's fine and good, but where did this 1.558 come from? More importantly, what were the assumptions? To create the Run Expectancy Matrix, the sabermetric gnomes (you thought real people did this?) look at the playbyplay data for some time period, typically a given year or set of consecutive years. At the beginning of each play, they look at the state and then determine how many runs were scored in this halfinning from the beginning of the play to the end of the inning. Then, they calculate the average of the number of runs that follow for all plays with the same beginning state. To illustrate the point, let's walk through an example inning. On April 5, 2008 in the bottom of the fifth inning of the RangersAngels game, there were six distinct plays:
The table below shows how the above six plays would be translated into the state and the additional runs to be scored that inning. Play Runners Outs Additional Runs Scored 1 000 0 1 2 100 0 1 3 120 0 1 4 120 1 1 5 023 1 0 6 123 1 0 At the beginning of the third play, we have the state of 1st and 2nd, no outs. After this state, one run scored during the rest of the inning (on the fourth play). Throughout 2008, there were 2,520 times a play started with 1st and 2nd and no outs. These plays plus all following plays in the inning led to 3,925 runs scored. If we divide 3,925 by 2,520, we get 1.558 expected runs. There is some filtering of which plays are included and excluded in the calculations. Typically, a Run Expectancy Matrix focuses on innings when the batting team's objective is to maximize runs. When the batting team's objective is to play for a single run (tiedgame in the ninth) or the inning is partial (a walkoff hit in the bottom of the ninth), these innings are typically excluded, because they underestimate the true run potential of a given state. In the numbers presented in this article, I have used each play in 2008 that occurred between the first and eighth inning as a simple filtering. Uses of the Run Expectancy Matrix There are two significant uses of the Run Expectancy Matrix:
In regards to improved performance measurement, imagine the following: a reliever enters the game in the middle innings with his team up by one, with the bases loaded and only one out. He gives up a sacrifice fly that ties the game and then strikes out the next batter to end the inning. With traditional statistics, this reliever would be charged with a blown save and allowing one inherited runner to score. Did he provide value to his team? Actually, he did. Based on the run expectancy matrix, in this situation, we would expect the opponent to score 1.596 runs. In this example, the reliever improved his team's game position by 0.596 runs, while the traditional statistic of blown saves suggests a poor performance. This type of situational analysis in comparing the actual runs scored to the expected runs scored is the basis of statistics INR, WX, and WXRL. The other important use is to analyze the value of certain strategies (stolen bases, sacrifice hits) based on a likely game situation. By comparing the beginning state to a few possible end states, we can determine if a given strategy is likely to create more runs. Joe Sheehan wrote an excellent analysis of the stolen base for the Baseball Prospectus Basics series back in 2004 using the Run Expectancy Matrix for 2003. Limitation #1: Expected Runs vs. Run Frequency Relying only on the Run Expectancy Matrix in evaluating strategy, however, can lead to flawed conclusions. We would dismiss the sacrifice bunt entirely, because in all situations a successful sacrifice bunt decreases the expected number of runs. If a team is playing for one run, however, a Run Frequency Matrix (something I first saw on Tom Tango's website) shows that it can be a sound strategy. The table below shows the probability of scoring at least one run given the situation: Runners Exp_Outs_0 Exp_Outs_1 Exp_Outs_2 000 28.2% 16.5% 7.1% 003 86.3% 65.5% 25.3% 020 62.8% 41.0% 21.9% 023 83.9% 69.0% 26.6% 100 42.4% 27.1% 12.7% 103 83.3% 62.5% 26.7% 120 64.5% 42.1% 22.6% 123 86.6% 67.0% 32.4% With a runner on 2nd and no outs, sacrificing him over to 3rd slightly improves the odds of his scoring from 62.8% to 65.5%, although the overall expected runs go down from 1.165 to 0.951. We are increasing the odds of scoring at least one run by 2.7%, but at the cost of lowering the probability of scoring more than one run. Interestingly enough, the very common situation of sacrificing with a runner on 1st and no outs is typically not beneficial, as the probability of bringing that runner home goes down from 42.4% to 41.0%. This counterintuitive result highlights one other key issue of using the Run Expectancy Matrix (or Run Frequency Matrix) blindly. It is important to understand who is bunting, who is on deck, and who is the runner. Limitation #2: Understanding Lineup Position The Run Expectancy Matrix essentially assumes that an "average" hitter is coming up to the plate in every given state. However, it's obvious that the average number of runs scored will be different in a 1st and 3rd, noout state with Mauer, Morneau, and Kubel due up versus Buscher, Gomez, and Punto. I looked at the playbyplay data for 2008 and calculated the Run Expectancy Matrix for each lineup position. In the table below, I pulled some examples that will help us determine the required breakeven success rate of a stolen base when a man is on first with no one out. In the first column, we see the run expectancies for the beginning state. The average is 0.908 runs, but this varies from 1.052 (for the #3 hitter) to 0.783 runs (for the #9 hitter). Lineup Expected Runs Expected Runs Expected Runs
Position (100  0 out) (0200 out) (0001 out)
Overall 0.908 1.165 0.281
1 0.923 1.192 0.323
2 1.018 1.254 0.325
3 1.052 1.232 0.328
4 0.938 1.201 0.311
5 0.827 1.162 0.270
6 0.813 1.090 0.260
7 0.819 1.022 0.234
8 0.873 1.098 0.217
9 0.783 1.108 0.225
If the steal is successful, we have a man on second with no outs (the 2nd column). If our base stealer is caught, we have the bases empty and one out (the 3rd column). The overall numbers (the first row) suggest that the benefit of a successful steal is an additional 0.257 runs (1.165  0.908). However, if caught stealing, it costs the team .627 runs (0.908  0.281). Therefore, the breakeven percent (after some simple math) comes out to be a 70.9% success rate. If we do the same math for each lineup position, we see that the breakeven percent varies based on who is up to bat, from a low of 62.4% when the #5 hitter is up to 80.1% when the #3 hitter is up. Lineup Benefit of Cost of Position Successful SB Caught Stealing Breakeven Percent Overall 0.257 0.627 70.9% 1 0.269 0.600 69.0% 2 0.236 0.693 74.6% 3 0.180 0.724 80.1% 4 0.263 0.627 70.4% 5 0.335 0.557 62.4% 6 0.277 0.553 66.6% 7 0.203 0.585 74.2% 8 0.225 0.656 74.5% 9 0.325 0.558 63.2% We are, of course, assuming that our current lineup positions match the average historical lineup positions of all teams. A better mousetrap would be for a manager to have these breakeven percentages based on his current lineup and the likely production from each spot. The best mousetrap would also have these percentages adjusted based on the current game state (are we maximizing runs or going for simply a single run), and the likely success rate based on our man at first and the current opposing batterymates. To create something like this, we would need to use more sophisticated tools from optimization theory like dynamic programming, but that is a different article altogether. Ah, to dream the impossible dream!
Tim Kniker is an author of Baseball Prospectus. 43 comments have been left for this article.

Real quick  the sabremetric gnomes comment was my first laugh at loud moment. After that, I have more positive things to say, as I REALLY like the fact that you took something we all know about and added a bit of your own research to make it even more accurate by measuring the quality of hitter. Nice overall job.