I never knew that there was an elite way to walk down a ramp and sit in an airplane. Imagine that.

Posted by at 2:06 PM | Permalink | Comments (1) | TrackBacks (0)

Anybody who has played fantasy football through ESPN.com knows about Hector the Projector and Victor the Predictor. They are two fictional characters who represent the quantitative and qualitative factors that go into ESPN's weekly fantasy football projections. It's common knowledge that the projections are, for the most part, wildly inaccurate. This year my team's projections were particularly bad, so I decided to take a look at how much you can actually trust Hector and Victor.

To do the analysis, I gathered all the projected points and actual points from players on rosters for weeks 1-14 in my 10-person, head-to-head league. Our scoring system is very close to ESPN's default settings, and the league is very competitive. In fact, ESPN ranked our league in the top 10 percent of leagues last year in overall PF. While studying the number of points from my league is a kind of proxy variable to ESPN's yardage and touchdown projections that determine points, it's a good indicator of problems with the system, as you will see. Also, limiting the cases studied to actual rosters ensures that the numbers are not skewed by third and fourth string players who clearly wouldn't be doing anything in a given week.

Problems with Hector and Victor Projections

The obvious place to start is with the difference between projected H&V points and actual points, or how "off" all of the projections were.

Going left or right from the zero marker, you can see how many points off each projection was. Take a look at the shape of the bell curve. You can immediately see that the distribution of the differences is skewed to the left, meaning that ESPN's projections are more often than not higher than the actual point returns. The median of the distribution is 2, which means that, on the whole, a given player has an inflated projection by about two points each week, which works out to an inflation of 16 points per week for an active roster of 8 players. This is certainly not very earth shattering, as everybody knows their projections tend to be higher than the points you earn. More interesting is the plot of H&V projections to actual points:

You can see right away that the plot is all over the place. The solid line plotted is the linear regression of all the data points. The equation for that line is y = 0.555x + 3.469. Compare the solid line to the dashed line, which is the equation y = x, which would be the expected regression line of the projections were, on average, in line with actual point values. Seeing the two lines and the distribution of pairs is interesting for gauging how inaccurate the projections are, but they don't tell us much about each projection's actual predictive value.

However, looking at the standard deviation of how "off" all the projections were does give some insight into the predictive value of an individual projection. For all players on rosters for the 14 weeks, the standard deviation for how "off" the projections were was 7.5 points.

This means that you can predict with 68% confidence that a player will earn you a point value that is +/- 7.5 points from the projected value. So, if a player is projected to get 15 points, you can be 68% certain that it will be somewhere between 8 and 22 points. Similarly, you can be 95% certain that the same player will score between 0 and 30 points. You can see where I am going with this. The projections aren't particularly accurate.

When you break out the standard deviations by position, here is how they differ:

That yields these confidence intervals for each position:

Think about an estimate for each of the positions, look at the confidence intervals, and you can see how bad Hector and Victor are at their predictions. The differences between the standard deviations for each of the positions can be explained in large part due to the fact that the higher scoring positions are subject to more variation in projection vs. actual points. That doesn't explain everything, though. For example, the standard deviation for TE projection and actual point differences really speak to the erratic nature of tight ends and the fact that ESPN hands out TD projections to them like candy. Guess what I'd never put money on. Randy McMichael scoring a touchdown in any given week.

Similarly, take a look at how the standard deviations for how far "off" the scores are from projections week-to-week.

Look at the disparity in weeks 1-6 vs. 7-14. The projections are much further from accurate in the second half of the season, possibly due to injuries.

The most telling disaggregation, though, is the how standard deviation increases as projections increase:

The higher the projections go, the more inaccurate they become. Each week ESPN predicts too many players having big games, and it creates a lot of outlandish predictions. Far too often mediocre backs and receivers are slated for a touchdown that does not happen (see "How to Improve" for proof).

The Effect on Game Lines

One of the most entertaining features of ESPN Fantasy Football is the matchup tool, which calculates a game line every week between each of the teams playing. The problem is that the game line is created directly from Hector and Victor projections, which tend to be very inaccurate.

To examine how effective ESPN is in projecting games, I compared the line of each home team and their actual margin of victory or loss for weeks 1-14 in my league and got this:

As you can see, the points are spread out almost as much as the plot above. Again, the solid line is the linear regression, and the dotted line is y=x, the expected 1-to-1 regression. The lines for margins of victory are, thus, overstated. That's what happens when the point projections are inflated.

I was actually more interested in seeing how accurate ESPN was in predicting the winner of a game, regardless of the margin. Excluding the three games in my league that it projected as ties, Hector and Victor correctly predicted 66% of games. This was actually surprising for me personally because ESPN incorrectly predicted half of my games this season.

An interesting pattern emerges when you look at ESPN's game predicting accuracy by it's predicted margin of victory:

It seems that as you move from margins of 1 to 25 points, ESPN's predicting accuracy increases. As you go from 26-35 margins of victory, it decreases significantly. But, huge margins (36+ points) tend to be very accurate. The 1-25 and 36+ tiers seem pretty intuitive: the larger the projected margin, the greater your likelihood of victory. So, why the dip from 26-35? I would argue that it's those tiers that have too many players in games that have been given projections that are too high, thus creating upsets. But then why is the 36+ tier so strong? Those are probably the games where a very strong team is taking on a weaker one, and victory is almost a forgone conclusion. The trend is interesting, but view these numbers with some skepticism. We're only talking about 67 data points here.

How to Improve Projections

What good would this be if I didn't actually offer a suggestion for fixing the problem? Projections are too high because ESPN is much too liberal in predicting players' yardage and in assigning touchdowns. Here's the proof:

The first chart is the distribution of H&V projected scores for all players on rosters in weeks 1-14 in our league. The second is the actual amount of points earned. The spike in H&V projections on the right side of the distribution shows that mediocre players are projected too many yards and touchdowns. In fact, over the 14 weeks, there were 2,133 more points projected than actually earned. That works out to about 15 points per team per week.

Instead of having the inflated scores, ESPN should calculate the average number of touchdowns and yards per week and distribute only those quantities many among the players for their projections. The Hector and Victor team would need to make hard choices about who would actually make a touchdown and who would not, but in the end the projections would be much closer to accurate. Yahoo seems to have a system that does something close to this.

Comparison to Yahoo FFL Stats

I wanted to compare to Yahoo's fantasy football system to see if they have similar problems with their projections. The problem with making the comparison between the two leagues is that I could only get data in the yahoo league for starters -- not the benches, which I could get from ESPN. That doesn't mean that the comparison is meaningless, though. Take a look at the difference between the projected and actual point values in my Yahoo league:

The shape of these two curves is very similar, indicating that Yahoo isn't giving out too many points in their projections, like ESPN does. That turns out to be true. When my Yahoo numbers are weighted for comparison, ESPN gave away four times as many points as my Yahoo league did. Yes, the point systems are slightly different between the two leagues, but there is definitely something to be said for ESPN projecting four times as many extra points.

The standard deviation for how "off" the Yahoo predictions were, was 7.3. That's lower than ESPN, but not by much (recall that ESPN was 7.5). However -- that 7.3 number is just for those players who started in my Yahoo league. Recall also that starters have higher standard deviations than those players who sit on the bench. I did some estimating and think that I can reasonably put Yahoo's standard deviation under 6.5 when bench players are factored in.

If ESPN limited their projections even further than Yahoo does, their accuracy would increase significantly moving forward.

Download My Data [Excel, 2.6 MB]

Posted by at 9:17 AM | Permalink | Comments (4) | TrackBacks (0)

I got "tagged" on Lindsay's Book Meme. This one seemed interesting, so here you go...

1. One book that changed your life:

I don't think that any one single book has ever strongly changed my outlook on life. There is one book, however, that significantly altered my life path just because I read it. See, Wendy Kopp, the founder of Teach For America came to speak at UVA as part of our UDems speaker series. She gave out copies of One Day, All Children..., her book, and signed them as part of her presentation. I already knew that I wanted to apply to TFA, and did. But -- when I found out that I got an interview, I read the book on the weekend before my interview and it gave me a huge leg up. Had I not read the book, I don't think I would have gotten in, and I would be in quite a different place. So in that sense, it changed my life.

2. One book that you've read more than once:

I read Of Mice and Men twice to my sixth grade class. The first time I read it, the kids hated it. But, the second time I made up a voice for each of the characters. George had a really gruff voice and Lenny was completely goofy. Curley's wife was of course high pitched and highly suggestive. The kids were completely invested the second time I read it to them. It just goes to show that presentation is everything.

3. One book you'd want on a desert island:

OK, so the obvious answer to this is either a) a book on survival or b) your favorite book. I'm picking neither. I'm pretty confident that I'd be able to survive for a long time on a desert island, and wouldn't need a book to figure out how. I'd be much more concerned about the mental effects of being a castaway:

Real-life castaways were reduced to an extremely primitive condition, or lost the powers of speech, in a space of a few years. One report describes a Frenchman who, after two years of solitude on Mauritius, tore his clothing to pieces in a fit of madness brought on by a diet of nothing but raw turtles. Another story has to do with a Dutch seaman who was left alone on the island of Saint Helena as punishment. He fell into such despair that he disinterred the body of a buried comrade and set out to sea in the coffin.

So, for my only book on said island, I would choose Cannery Row. I choose it because the characters in it are all cut from a wide cross-section of society. There is the wealthy scientist, the young drunk guys, the prostitutes, the Asian immigrant store owner, etc. Plus, it takes place in California. It would want it to think about what all is back home, which would keep me going. Picking a book with a narrower cast or one from a time very far removed would not do me much good.

4. One book that made you laugh:

America (The Book) is pretty obvious. I actually haven't read the whole thing through yet, but what I have read is hilarious.

5. One book that made you cry:

I had to read Sofia Petrovna as part of my Soviet History class because it's a fictional account of the Stalinist purges. While most stories of the purges center on what happened to the purged, like One Day in the Life of Ivan Denisovich, this book follows the mother of a young man who is purged and her madness that ensues. The entire book epitomizes Soviet reality and the injustice therein. I didn't actually cry, but I was highly depressed.

6. One book you wish had been written:

I definitely think that someone should channel Faulkner to write a full length book about the Bush Administration. Here is a sample of what it would be like.

7. One book you wish had never been written:

I'm really not one for saying that a book should never have been written, because that's a slippery slope that leads to community book burning events (sponsored by the local GOP). However -- there are two books that perpetually piss me off. And, no, it's not some conservative right wing nut job's book. It's a well established fact that outrage over those books fuels their sales, so I won't bother.

The first is The Incredible Journey, which incredibly sucks. I tried reading this book with students, and usually I could quell rebellion over some students not liking a particular book. But when the book bores me and every single one of my students, it's really not worth fighting.

The second is The Catcher in the Rye, everybody's favorite book. Honestly, everybody needs to get over this book. IT'S NOT THAT GOOD! Most people read the book because it's a rebellious thing to do at the age of 16 and they somehow connect with Holden Caulfield because of their teenage angst. Then they grow up and still consider it as a favorite book. First of all, there are so many better books to read for venting teenage angst. Outsiders, anybody? Second, please let the book go. Holden isn't likable, or even noble. He just gripes about having to grow up. Honestly Holden, shut up and get a job. Oh wait, you can't because you're in an insane asylum.

8. One book you're currently reading:

I only got a heavily abridged version of Great Expectations in high school, and it always seemed like a cool story. So I'm reading it now. It's pretty good thus far. If you're looking for an abridged version, the creators of South Park made an excellent one. They even included the robotic monkeys.

9. One book you've been meaning to read:

This book would have gone in the "books I wish had been written" section, because I had been hoping for it for years. But, it just came out. Hannibal Lecter is by far my favorite fictional character. He's so interesting and I always wondered what made him who he is. Now I'll know ... once it comes out in paperback or is realeased in theaters, whichever comes first.

10. Six people to tag:

Nobody really updates their blogs anymore, but I'll tag these people anyway.

Starr, Therese, Lonely Eater, Ed Wahoo, Blue Cat Democrat, Jackie Viernes, Jennie, Shoa, et al.

Posted by at 6:23 PM | Permalink | Comments (0) | TrackBacks (0)

I just got done moving my blog to my Dream Host server. They are by far the best web hosting solution I've ever come across -- your dollar goes really far. Plus, it's all employee owned and their monthly emails are hilarious. If you need a website, definitely check them out. But, if their prices are too high, I can hook you up with some space on my server because I've got plenty of it. Just let me know.

Since I was moving all my files to the new server, I decided it was time to give the site a facelift. Again. Ever since I started writing this blog I have never been content with how it looked. Every year or so I would redesign the whole thing, but never got it to actually look good. Luckily, the folks at The Style Contest took some of the work out of it for me with all of their fancy templates. I'm pretty happy with this new design and don't think I'll need to update it for at least a couple years ... not that I'll have time to if I get into law school.

Anyhow, the site will continue. I was actually leaning toward retiring the whole blog thing because nobody reads blogs as much anymore. Well, mine, anyway. I used to have a decent readership, but it's plummeted since I went to Mississippi and didn't have a ton to write about. Maybe the new design will entice people enough to read my crap.

Posted by at 6:30 PM | Permalink | Comments (1) | TrackBacks (0)

Data for the Country

Posted by at 10:50 PM | Permalink | Comments (1) | TrackBacks (0)

The other day at work I was researching how Florida implemented Adequate Yearly Progress (AYP) provisions of the No Child Left Behind Act (NCLB). I was so confused by Florida's convoluted process that I had to make a huge flow chart of all that has to happen for a school to make AYP. When I got done, I decided to do the same type of chart for how states are supposed to implement AYP provisions of NCLB.

Despite all the bickering and distortions of AYP under NCLB, it's very simple: schools either meet the state achievement standard, or move a significant amount of students toward the achievement standard. If they don't make AYP for two years, then the school is put on an improvement plan, kids are offered access to another public school that does make AYP, etc.

Posted by at 2:34 PM | Permalink | Comments (0) | TrackBacks (0)