In Which We Take The Close Games Research Up About 10 Notches

Dudes.  Today's post, well, it's awesome.

I'm not one to toot my own horn, but that's cool this time, because I can't even take credit for it.  A reader had the underlying idea, another web site provided all the mathematical genius and my brother did some crazy Internet voodoo to make it all happen.  I just wrote it up.

I'm psyched about this one, folks. 

---

On Tuesday, we talked a lot about close games.  That was all good stuff, but one thing that continues to bother me about this discussion is that everyone -- including me -- defines close games by the final score.  It's an understandable short cut, but we should be able to do better.  (And we will, oh boy, we will.)

As a starting point, we could at least add in games that are close at the start of the fourth quarter, but then end up being big wins.  If it's tied going into the fourth quarter, that's a close game, even if a team ends up scoring 17 points to put it away.

Unfortunately, that kind of data is hard to collect.  As I mentioned in the first post, we'd need an intern around here to pull all those numbers.  It's too much hand collection for me to do.

Of course, later that day, a genie popped up my brother emailed me and said, "I read your post today and wrote a script to pull the last 10 scores from the last 10 years of Eagles games.  I don't know if you want to use it or not, but there you go."

And there was the spreadsheet.

After some playing around with the numbers, here's the result:

 

For the purposes of this chart, close games are defined as contests where the score was within seven points at either the beginning or end of the fourth quarter.  The column headers are pretty self-explanatory, but I did the two at the right because I was curious where blame / credit should go when the Eagles lose / win a close game.  My thought is that if you have a lead going into the fourth and you lose, blame the defense (mostly).  And if you're behind, but end up winning, that's a credit to the offense (mostly).

I'll give you a minute to peruse the chart.  Hmmm ... mmmm ... mmmmmmm ... bada-ba ...

Ok, ready?  Great. 

Now here's the most interesting thing about that chart, the way I'm looking at it.  Fully 74 percent of the times the Eagles were ahead going into the fourth quarter in a close game, they won.  And 68 percent of the time they were behind, they lost.

Obviously, this makes sense.  If you're losing after three quarters, you're going to have a harder time winning the game than if you were winning.  However, one of the problems with the overall close games conversation is that many, many people automatically jump from:

Eagles have a poor record in close games -->
Eagles don't play well in the clutch -->
McNabb just doesn't have it the way a great QB should

Now, all three of those statements might be correct.  But not one proves another.  The Eagles could lose 10 close games in a row, but if they enter the fourth quarter down by 20 in each of them, it's hardly their late game performance that's the issue.

Which got me thinking back to a comment made by bsencore (known around here as a commenter, but currently rocking it over on his single issue blog, McNabborKolb.tumblr.com):

"I wonder if instead of just point values, the analysis would be better (though probably more tedious) coming from WPA game charts — seeing who won games in which the outcome was in doubt for some portion of time, like the 4th quarter."

Now that would be something, wouldn't it?  Because then instead of arbitrarily drawing lines and saying, "This was a close game, that was not," we could use WPA as a measure of precisely how close things really were, then evaluate actual wins and losses based on predicted results.

---

Interstitial for the folks who just said, "What the hell is WPA?"

Win Probability is the brainchild of the brilliant Brian Burke, the man behind the website AdvancedNFLstats.com.  Brian is a former Navy fighter pilot who happens to be doing some of the most creative and interesting NFL statistical work you can find without breaking into Andy Reid's office.

Yes, we can all take a moment to ponder how much less of our lives we've made.

Brian laid out the concept of Win Probability in a series of three posts -- 1, 2, 3 -- back in 2008.  If you don't have any major deliverables today and you've never been to his site, you should stop reading right now and go check those out.

The work is based on similar analyses that have been done for awhile now in other sports, particularly baseball.  Basically, at any given point in the game, each team has a win expectation based on the score, time remaining, who has the football and what down it is.  Down 40 points with two minutes left, your chances of winning are close to zero.  Up 10 with first-and-goal on the one and five minutes left, your chances are much better.

There are many, many complications to the whole thing, but the beauty of the tool is that we don't really have to understand all of them to use it, much the same way I can't tell you how a drill works, but I can put a hole in the wall.  (I'm good on hammers, though.).

For our purposes, just understand that a WPA close to zero means you have almost no chance, a WPA close to one means you should win, and a WPA of .50 means both teams are equally likely to win the game, based on the current situation.

---

We can use WPA in a number of interesting ways.  On Brian's site, he has graphs that show us exactly how WPA fluctuates during the course of a game.  Here's the one for the Chargers' game this year.  Many people have cited that game as evidence of a lack of "clutchness" on the part of McNabb or the Eagles at large. 

But look at that graph.  At the start of the fourth quarter, the Eagles had a 5 percent chance of winning that game.  Even after they closed within five points, they peaked at a 23 percent chance. 

Failing to convert a one-in-four shot is not a sign of non-clutchness.  It tells you not to dig such a big hole to begin with.

So here's where this gets cool.  Let's say we want to isolate fourth quarter performance.  Rather than just pulling out a few "close" games and checking the results, what if we were able to look at the expected win probabilities the Eagles had going into the fourth quarter of every single game they played and then could compare those to the final results.

All we'd have to do is pull the WPAs at the start of every fourth quarter, code the wins as "1" and the losses as "0" (ugh, and the tie as ".5"), and then compare expected and actual values.

The Internet voodoo comes in again here, because that's a beautiful idea, but pulling the Q4 WPAs off each and every one of Brian's charts by hand was going to be a massive undertaking.  So I made a joke about that in another email to my brother, and a few hours later, ding, there's the next spreadsheet.

Now, Brian's working a pretty sweet creative commons license over there, so I'm quite content to use the data to serve our purposes here, but I'm going to be a little circumspect in terms of what I post beyond that.  I think there are some interesting possibilities for further study here, but I'm not going to blow out all the league numbers.

Here's what the data look like for the most current Eagles' season:

That bottom line is a preview of what's coming.  Based on how the Eagles played in the first three quarters, we would have expected them to win about 9.5 games (history rhymes and all that).  They actually won 11.  (Playoff games are included in all of these numbers.)

If you pull the same numbers for the last 10 years, you get the following chart:

 

The last column is the one that's most interesting.  I divided the number of actual wins by the number of predicted wins.  A team with a "Q4 Win Ratio" above 1.0 is winning more games -- and therefore outperforming in the fourth quarter -- than would be expected.  A ratio under 1.0 means the team is underperforming in the fourth.

This is another awesome feature of WPA.  If we just did a points-based analysis, we'd run into problems when, for example, the Eagles ended the third quarter with first and goal on the opposing team's two-yard line.  If they scored -- shovel, shovel, shovel -- then we'd be giving them full fourth quarter credit for a drive that really happened in the third.

WPA avoids that, by "assuming" the TD is about to be scored (so to speak), so only the actual culmination of the drive gets counted in the fourth quarter.

Back to the chart, it's clear the people who have been railing on the Eagles' late game clutchness have some 'splainin' to do.  Only three times in the last 10 years have the Eagles had a win ratio below 1.0.  And 2007 was really the only big discrepancy.  [UPDATE:  Brian's missing a game from 2007 in his database.  I've adjusted the chart and discussion accordingly.  It's not a big change, but see the comments for more.]

Again, I'm not going to paste up too much data, but based on the entire league's performance from 2000-2009, a Q4 win ratio of .91 .94 is at about the 33rd 40th percentile.  So not great, but not abysmal, either.

This year's win ratio of 1.16 is of course much better.  That's about 80th percentile, or more specifically, eighth-best this year:

Assuming the validity of this approach, the Eagles were a top ten team in terms of how they closed out games this year.  And yes, that was with Donovan McNabb at quarterback.

On a final note, the fact that the Colts and Saints are at the top of that list probably shouldn't surprise us that much.  Yes, yes, Manning and Brees are awesome, but this is really just one more indication of how it takes luck to make the Super Bowl.  For as much as fourth quarter performance may be a skill, it's impossible to define away the contribution of lucky bounces when you're winning 15 or 16 games in a season.

Told you it was going to be sweet.