Wednesday, October 22, 2008

Evaluation of Defensive Projections

Here’s a question I’m curious to answer: What method better projects team defense, stat projections or scouting reports?

One thing I need to answer the question is an objective criterion to measure. I don’t think there is anything out there except team defensive efficiency record. If a team has very good defensive players, they should make more outs than average, and post a better DER. DER is not a perfect measure though. The ballpark has an effect on this, as does the types of batted balls your pitcher allows.

Here’s my method: Start with the league average DER. Park adjust this. Then look at how many groundballs, flyballs, line drives, and popups the team allows. The formula I used, along with the batted ball data from the Hardball Times team page, is this: expected batting average allowed = GB*.237 + FB*.16+ LD*.718 + pop *.01. xDER = 1- XBA. Then xDER/league average is your batted ball adjustment factor.

Now, let’s look at the projected defensive stats of the players on your team. I’m using preseason projections, but prorated to the actual 2008 playing time. For the stat projections, mine were made available shortly after the 2007 season on my site here. For scouting reports, I’ve converted the results of Tango Tiger’s scouting report by the fans into run values.

I need to use custom weights of the report’s attributes for each position. For 1st basemen and outfielders I ignore the arm ratings. It’s not that they are unimportant, especially for outfielders, but they have nothing to do with how a fielder turns a batted ball into an out, which is what DER, and most defensive metrics measure. For 2B, 3B, and SS, the arm as well as range determine how efficient a fielder is at recording outs. The weights used are:

1B: Instincts 44%, 1st step 44%, Speed 12%
2B: Instincts 22%, 1st step 22%, Speed 11% Hands 22% release 11% strength 6% accuracy 6%
3B: Instincts 15%, 1st step 15%, Speed 10% Hands 15% release 15% strength 15% accuracy 15%
SS: Instincts 18%, 1st step 18%, Speed 12% Hands 12% release 18% strength 12% accuracy 12%
OF: Instincts 17%, 1st step 33%, Speed 33% Hands 17%

Next I find the average of each position – I just take the simple average of all players at their listed primary position. Then I find every player’s rating at each position, and convert to runs. For most positions, (rating – lg average)/2 gives you ratings that appear to be on the same scale as the stat-based projections. For 1B, where a player gets fewer chances, replace the 2 with a 3, and use 1.75 for shortstop.

Next, I take both sets of projections, find out how many plays saved each team projects to, and modify the expected DER accordingly. For this exercise, in the interests of time, I only looked at players who played at least 250 innings in 2008. If a player did not have a stat projection at the position played, I used zero for his projection. I did the same for the fan projections, except that if a player was evaluated, I had a projection at any position he might play in 2008, regardless of whether he ever played there before.

Here are the results:

Team statDER fanDER RealDER
ARI 0.686 0.688 0.687
ATL 0.685 0.688 0.695
BAL 0.706 0.707 0.691
BOS 0.695 0.694 0.700
CHA 0.694 0.696 0.688
CHN 0.701 0.706 0.706
CIN 0.687 0.688 0.674
CLE 0.695 0.692 0.686
COL 0.678 0.672 0.679
DET 0.706 0.701 0.686
FLA 0.689 0.691 0.694
HOU 0.694 0.692 0.699
KCA 0.694 0.691 0.691
LAA 0.696 0.699 0.693
LAN 0.692 0.697 0.693
MIL 0.694 0.696 0.700
MIN 0.690 0.690 0.690
NYA 0.694 0.697 0.684
NYN 0.690 0.691 0.699
OAK 0.703 0.698 0.701
PHI 0.691 0.688 0.696
PIT 0.679 0.680 0.676
SDN 0.698 0.695 0.697
SEA 0.704 0.706 0.683
SFN 0.688 0.685 0.686
STL 0.685 0.685 0.697
TBA 0.712 0.712 0.712
TEX 0.678 0.680 0.673
TOR 0.696 0.695 0.706
WAS 0.685 0.685 0.690


By RMSE (Root mean squared error) the results are:
Stats: .0084
Fans .0087

Going without defensive projections, using the ballpark and batted ball adjustment only (BB), we get an error of .0091

Using correlation, the results are similar:
Stats: .552
Fans: .534
BB: .450

The fan report would probably do better if I used multiyear data to create the projections. I’m not sure what the proper weights by year should be. Most likely, an even better projection could be created by some combination of stats and fan reports. I hope that the framework used here makes sense, and can provide an objective measure to evaluate defensive projections.

I should note that this can only be used to evaluate defensive projections, not defensive ratings. By comparing to DER, a stat like my own TotalZone would probably look better than a far more detailed stat, such as UZR or John Dewan’s plus/minus. That is because TotalZone, once adjusted for ballpark and hit type, is pretty much the same as DER. Since the more detailed stats capture more fielding ability, and better strip out the luck, they should better predict next year’s DER than a simpler stat.

6 Comments:

At 8:38 PM, Anonymous Anonymous said...

This here is an interesting piece. I actually have to admit that I had some doubts about the reliability of Fans' Scouting Reports (mostly because of, well, the fans haha), but it's good to see that it is pretty reliable.
And since I've never formally told you, thanks for all your time and work. I'm looking forward to seeing the 2009 CHONES for the Giants. Ok, maybe I'm not.

 
At 11:13 AM, Blogger Chone Smith said...

Thanks, and if nothing else, I'm sure you'll love the projection for Tim Lincecum.

 
At 8:26 PM, Anonymous Anonymous said...

meet your sweet -
muscle gaining secrets -
muscle gain truth -
natural cancer treatments -
niche review templates -
nyhms warcraft guides -
pappa pc -
pc booster -
pc optimizer pro -
pc secret formula -
pda flix -
privacy control -
publicrecordspro -
public records pro -
push button ecover -
reg defense -
reg genie -
registry easy -
registry genius -
registry smart -
registry winner -
registry winner download -
reverse mobile -
richard mackenzie direct -
rocket japanese -
rotator cuff training -
save my marriage today -
secrets book -
silent sales machine -
silver lotto -
spyware cease -
super affiliate handbook -
the 11 forgotten laws -
thebadbreathreport -

 
At 8:26 PM, Anonymous Anonymous said...

the bad breath report -
thedietsolutionprogram -
the diet solution program -
the power pause -
tmj help -
tonsil stones remedies -
top secret fat loss secret -
turbulence training -
twitter rockstar -
uncle sams money -
underground hypnosis -
vincedelmontefitness -
vince del monte fitness -
warcraft millionaire -
warcraft wealth -
warp speed fat loss -
webcomp analyst -
wedding speech 4u -
win back love -
your software website -
zox pro -
zygor guides -
500 love making tips -
acid alkaline diet -
acne no more -
advanced defrag -
adware alert -
adware bot -
affiliate elite -
anti spyware -
anti spyware bot -
article bully -
art of approaching -
auto pilot profits -

 
At 8:26 PM, Anonymous Anonymous said...

bbq book -
blogging in action -
body building revealed -
burnthefat -
burn the fat -
carb rotation diet -
chopper tattoo -
conversationalhypnosis -
conversational hypnosis -
cure for tinnitus -
debt free in three -
driver checker -
duplicate file cleaner -
earth4energy -
earth 4 energy -
easy member pro -
easy tv soft -
eatstopeat -
eat stop eat -
end your tinnitus -
fap winner -
fat burning furnace -
fatloss4idiots -
fat loss 4 idiots -
final sync -
final uninstaller -
firewall gold -
fitness model program -
fit yummy yummy -
flatten your abs -
forex auto money -
forex auto pilot -
forex confidante -
game tester guide -

 
At 1:57 PM, Anonymous Organic Vitamins said...

Please check out my site as well and let me know what you think.
organic vitamins
::
best organic vitamins

 

Post a Comment

<< Home