Why Do We Believe That Top Draft Picks Are Overpaid?

For years, there has been a paper floating around by a couple of economists, Cade Massey of Yale University and Richard Thaler of the University of Chicago, which looks at the "value" in draft picks, and at the market for draft picks in trades. They have recently updated it (downloadable from SSRN) and actually have been promoting it a bit. The main finding of the paper boils down to this chart (Figure V in the paper):

 

Expected performance is highest for the best players in the draft, but at the top of the draft, the players are expensive, so the “surplus” value net of compensation is concave. How do they come up with this? To determine the value of a player, they look at player performance for guys on their second contracts in the league, and then look at what those guys’ salary cap hits are, and finally associate performance levels with those salary hits. So a pro bowl performance is worth $X, starting worth $Y, backing up worth $Z, etc. They then subtract what the draft pick’s actual salary cap hit was from that benchmark value, and the net is the surplus that the draft pick generated.

I will discuss their methodology more below, but the main reason for this post is that Thaler recently wrote an article in the New York Times, where he said:

It turns out that the N.F.L. and the players’ union are trying to renegotiate their collective bargaining agreement. One topic reported to be on the bargaining table is shifting some of the pay away from early draft picks.

The owners and players should find common ground on this issue, because it makes absolutely no sense to be giving so much money to unproven rookies, many of whom turn out to be busts. By reducing the premium paid to the highest draft choices, the league could restore the redistributive goal that the draft was created to achieve. And veteran players would probably agree with the principle that eight-figure salaries should be reserved for players who have already proved themselves on the field.

Last week, Brian at Advanced NFL Stats put up a solid rebuttal to that point. Look at the graph again: the surplus value is positive at all points. All draft picks are getting undercompensated, he observed:

Surplus value peaks shallowly at the bottom of the first round and through the second round. That's where teams get the biggest bang for the buck. But still, the draft pick surplus is significantly positive throughout the entire draft. Contrary to Thaler's remarks in his recent NTY [sic] article, draft picks are bargains compared to veterans.

According to Massey and Thaler, although the top picks appear to be overpaid relative to later picks, draft picks in all rounds are underpaid relative to veteran players. Thaler's own research contradicts his recent statement that it makes “absolutely no sense to be giving so much money to unproven rookies.” It makes perfect sense to me.

I think that Thaler’s point, though admittedly not well presented, is a bit stronger than Brian makes it out to be. Massey and Thaler measure total compensation to players. They don’t distinguish between guaranteed money and non guaranteed money. But for players at the top of the draft, most of their pay comes from guaranteed money. The veteran deals that create the benchmark are largely non-guaranteed. Therefore, there is much more leeway to cut costs for veterans if they stop performing than for draft picks if they bust.

This makes the risk of top picks far greater than the chart above makes it seem to be. As I will point out, later, Massey and Thaler fail to account for this on either side of the equation, but this guaranteed money increases the risk of the deal and doesn’t increase the payoff. That is the problem with the contract rules for draft picks.

Said another way, we shouldn’t be concerned that Matthew Stafford got 6 years, $72 million at the top of the draft, because in expectation, he was likely to be more valuable to the Lions than that – we should be concerned that he got $41.7 million of guaranteed money, which drastically reduces the team option to cut him if he winds up being horrible.

However, that isn’t to say that I like this paper. In fact, I have always had issues with it, for several reasons. One reason is what Brian called the brick-layer theory:

Now that I'm thinking about the Massey-Thaler paper again, I'm wondering if the overvaluation of top picks might be justified, at least to some degree. Massey and Thaler seem to assume a linear "bricklayer" model of football player production. By that I mean that a bricklayer who lays 10 bricks per minute and makes $10 per hour can be replaced by two slower bricklayers who each lay 5 bricks per minute and make $5 per hour. In this model, absolute performance is what's important.

But one great QB cannot be replaced on the field by two average QBs, and a team can only field 11 players on the field at a time. It can only carry 53 players on its roster. The bricklayer model does not apply. Further, relative ability is what matters in football, not absolute ability.

Beyond this, I have other problems with the paper as well, as I will describe after the jump.

Surplus for Top Veteran Contracts

This is really a restatement of the Bricklayer Theory, but from a slightly different angle: Massey and Thaler assume that veterans are paid their actual value in salary – what they get is what they are worth, in expectation. I think the salary cap makes the benchmark that Massey and Thaler are using incorrect for top players. The easiest example is a guy like Peyton Manning. Obviously, Manning wants to get paid. He also wants to win. Those are competing goals. He can’t make so much that a team can only afford 52 scrubs to put around him if he has any hope of succeeding. So realistically, he is limited in terms of how much he would make under a salary cap – he has to trade off the quality of his teammates with the quantity of his compensation.

Imagine, however, that there is no salary cap. First off, it is unlikely that Manning is playing in Indianapolis. He probably would have moved to Washington, Dallas or New York. For this example, let’s say he’s in Dallas. Second, he is limited not by the cap but by the team budget. As long as Jerry Jones has enough money to pay for a talented supporting cast, Manning can make as much as he wants. A guy like Manning would probably make a lot more as a percentage of the Cowboys’ budget than he makes as a percentage of the Colts’ salary cap.

More average guys don’t face this issue. Leonard Weaver, who is one of the top fullbacks in the league, we’d all agree, doesn’t explicitly have to trade of his salary demands with a desire to win. His extension isn’t squeezing anybody else. If the cap were eliminated, I’m not sure he’d get paid much more than he just got.

So while Massey and Thaler assume that the best players are paid their true value, I believe that the best players have their own surplus value baked in. Because those players are part of the reason the top end of the draft generates more value in expected terms before costs, there is surplus to teams that Massey and Thaler are not capturing.*

Study Doesn’t Measure Dead Money

Massey and Thaler only measure the salary cap hit for players in the league who make a roster in a given year. So if a player is cut for a big dead money hit, Massey and Thaler never measure that. When looking at veteran players, guaranteed money as a percentage of the deal increases with quality. And when that high quality, veteran player like Jevon Kearse underperforms and is cut early, the dead money from his signing bonus is never measured and is never counted as part of that player’s cost. So again, there is likely a non-trivial measurement error that disproportionally impacts the top end of the performance scale.

As mentioned earlier, this cuts both ways because the compensation cost for rookies is so heavily guaranteed-money based. But the top guys are rarely cut early for that very reason – the cap hit is so big for doing so, and so much is already invested. I’d expect the impact to be much higher on the measurement error of the benchmark value than of the rookie comp.

Study Doesn’t Account for Contract Length

Massey and Thaler look at performance of draft picks in their first five years. But a draft pick and the associated rookie contract has set terms. Currently, the top 16 picks can get a 6-year deal (though many only get a 5-year deal); picks 16-32 get 5-year deals; and every pick thereafter gets no more than a 4-year deal. In the period that the authors measure (ending 2008), in order to control that fifth year for guys taken in the second round or later, players had to be signed to a second contract. Even prior to the 2006 CBA, first round picks got considerably longer deals than lower round picks, which meant that teams got to capture more years of surplus for the top of the draft board than for the bottom. This was clearly to the team’s advantage – which was why the Eagles pushed hard to lock up their top picks to longer-than-average deals for their given draft slot prior to 2006.

Study Doesn't Consider Team-Controlled Option to Extend

The team has the option to extend their players early. It is not unilateral, of course, because the player has to be willing to sign, but there is a reason that few good players ever seem to hit free agency – most good players are willing to extend their deals. Because a team is only willing to extend conditional on a player succeeding, there is value in having those sole negotiating rights for so long that is again more heavily associated with higher-performing players than with low ones.

Assumption that the Cap is Binding

Massey and Thaler conclude that because the surplus for top picks is lower than for picks taken later in the first round, there is a value loss to teams, and the optimal strategy is to trade down. I agree, under two assumptions. First, assume that their measured curve is correct. Second, the cap has to be binding – teams spend up to the cap every year. If those three things are true, then yes, you want to maximize the surplus that they are showing. Again, if you get exactly what you pay for (on average) with veterans, the difference between teams will be how much better your players on rookie deals are than their compensation.

But the cap hasn’t been binding for several years. Teams aren’t spending up to the cap; far from it. Every dollar in savings on draft picks isn’t being matched by adding quality in the veteran market. So the best teams are the ones whose total value is greatest. Surplus helps there, but the goal is to have the absolute highest performance, because efficiency isn’t rewarded in the same way.

Of course, that point also has an embedded assumption: a team’s goal is to maximize wins, not maximize profit. In truth, both are motives, and related since revenue should be a function of team quality. With the exception of the few franchises that are perennial winners, I believe most teams’ goal in the draft is to maximize wins -- I would expect that incremental revenue from being a playoff team dwarfs any surplus savings from efficiency in the draft.

For teams that are perennial playoff teams, it becomes less clear cut. Which is why, for teams like the Eagles and Patriots, it probably does make sense to stay at that peak surplus point, because that is profit maximizing, while for other teams, they have to get the total quality of their team much better before they can worry about becoming more efficient.

In summary, I don't think that the overall findings of the study are necessarily wrong, but I disagree with the conclusion. It makes sense to me that top picks would capture more of their value, because they have more leverage in contract negotiations than later picks. But I guess that I disagree with the premise that teams should be maximizing the surplus that Massey and Thaler observe, for the various reasons outlined above.

-----

* Note that the 2010 version of the paper does a robustness check for this very effect. It doesn’t eliminate the value loss for top picks, as they measure it. Which is ok, because the point isn’t that their results are contingent on that, just that they are making an assumption that is invalid: that salary cap hit = value to the team.