So how did the pitcher projections do?

I compared my pitcher projections to those of 4 other projection systems - Zips, Pecota, Marcel, and Hardball Times. The creaters of those systems are, respectively, Dan Szymborski of Baseball think factory, Nate Silver of Baseball Prospectus, Marcel the monkey (Tangotiger asks for the monkey to get the credit), and David Gassko and Chris Constancio of THT.

To compare I took a pitchers actual ERA, and compared it to the projected ERA. Sounds simple but there are a few things to take care of first:

1. Take pitchers who threw at least 50 innings
2. Adjust for league average. CHONE projected ERAs were lower across the board than the others. I figured the projected league ERA based on the innings pitched of the pitchers in my sample, and adjusted the projections accordingly. I multiplied CHONE projections by 1.008, PECOTA by .907, all the others somewhere in between.
3. There's a little over 300 pitchers in my sample, but not every system projected every pitcher. The ones not projected were mostly minor leaguers, or coming from Japan. What I did was if I had less than 3 projections for a pitcher, do not use him. If one or two systems do not project a pitcher but the others do, the pitcher gets a 6.00 ERA projection. Of the 5 systems, ZIPs projected the most pitchers and takes the fewest 6's.

Now, the evaluation criteria. I did two- the first is correlation, the second is average run error. Just take the absolute value of the difference between projected and actual ERA, and figure how many runs you are off. That way I adjust for playing time, it hurts more to be off by 0.50 in ERA for a fulltime starter than for a 50 inning reliever.

And the results:

Chone .405
Zips .370
Pecota .366
THT .340
Marcel .339


Not bad for my Chone. Now for the more advanced test, avg run error:

Chone 9.0
Pecota 9.2
THT 9.3
Marcel 9.4
Zips 9.5

Now what did my system do better than the others? For all I know it just got luckier. There isn't a huge deal of difference in the results. Last year Chone did not do so well with pitchers, and for all I know it was just unlucky, but it might be the use of batted ball data. From a pitcher's mix of grounders, pops, line drives, and flies, I attempt to get better estimates of a pitcher's batting average per ball in play and his homerun rate.