To try to find out, I looked back at the 2005 and 2006 drafts and looked at the draft grades that he handed out the day after the draft by a variety of draft experts. To get the grades from 2005 and 2006, I used the Football Outsiders' draft grade report cards posts which pulled together all of the rankings from around the web. Evaluating the drafts is made possible/easy because of Pro Football Reference's draft history. PFR is an indispensable resource for any football fan. They not only list every draft pick, but also tally how many first-team All-Pro selections, Pro Bowl appearances, seasons as a starter and games played each draftee has.
Because of that, we don't have to be subjective in slapping grades on drafts now. Instead to try to be objective, I used a simple formula that awards points for ranking best in: the number of seasons started amongst drafted players, the number of games played and career approximate value (a formula developed by Pro Football Reference), with bonuses for Pro Bowl appearances (five points per appearance) and first-team All-Pro appearances (10 points). So first, we'll look at whether instant post-draft grades mean anything, and then we'll follow up Wednesday with a much simpler way to more accurately grade drafts immediately.
For each of the rankings, being first gave the team 32 points, second gave them 31 and so on down to one point for finish last. So a team that ranked first in all three categories would have 96 points while a team that finished last in all three would have 3. The current grades are assigned in order of rank, and are weighted to equal the same average grade point average as the post-draft grades.
Best Draft: The foundation of the Cowboys' moderate success in the second-half of the decade is built around this draft, which has to rank as one of the best of the decade. The Cowboys landed five significant starters (DeMarcus Ware, Marcus Spears, Chris Canty, Marion Barber and Jay Ratliff) and a sixth player (Rob Pettiti) who started for one season. Ware has been a dominating pass rusher and Ratliff has been one of the league's best nose tackles--which is quite a find for a seventh-rounder.
Worst Draft: If the Cowboys built their team around the 2005 draft, the Bills' long run of failure got a nice booster shot from this draft. Wide receiver/returner Roscoe Parrish has proven he can return punts, but he's the only long-term contributor Buffalo acquired. Center Duke Preston did get to start for most of the 2008 season, but the Bills were so unimpressed that they let him go before the 2009 season.
How The Experts Did? I'll defer to my friend, stats expert Greg Stonerock, my partner in putting together this analysis, for the following paragraph explaining how we analyzed the data.
We used a Spearman test to compare the two sets of rankings, and it looks like the draft experts did a pretty miserable job of predicting how the drafts would turn out. The Spearman test shows a correlation coefficient of .117. A coefficient of 1 (or -1) would mean post-draft grades predict NFL success perfectly; a 0 would mean the prediction is worthless. The correlation of .117 is so close to 0 that the grades cannot be considered any sort of predictor of future success. In fact, if you ranked drafts completely at random, you would get a .117 (or farther from 0) more than half the time. (52.3% of the time, to be precise.)
Two of the teams that got A's then (Dallas and Philadelphia) would still get A's now, but two of those A's are now Cs. It gets even worse as you move on. The Bengals were an A- in 2005, now you'd give them a D. The Vikings were a B+ back then, now their draft class would rank as a definite F, with Troy Williamson ranking as the best player the Vikings drafted that year.
Green Bay was rated by the consensus of draft experts as having one of the lowest grades in 2005. Now a draft that includes Nick Collins and Aaron Rodgers looks pretty good. But enough of me explaining it, here's a look at every team. The grades I doled out are based on the scoring system and are weighted to hand out the same overall GPA as the 2005 grades.
|Team||AP1||PB||ST||Car AV||G||Then||Now||Best Player||Score|
|San Diego||1||3||13||139||330||B||A||Shawne Merriman/Vincent Jackson||101.5|
|San Francisco||0||1||19||127||490||B||A-||Frank Gore||93|
|New England||0||2||18||150||333||B-||A-||Logan Mankins||92.5|
|St. Louis||0||0||23||112||472||C||A-||Alex Barron||87|
|Green Bay||0||3||12||111||313||B-||A-||Nick Collins/Aaron Rodgers||81|
|New York Jets||0||1||10||81||323||C-||B+||Kerry Rhodes||59.5|
|New York Giants||1||1||8||81||237||C/C+||B||Justin Tuck/Brandon Jacobs||52|
|New Orleans||1||2||7||64||207||C+||C+||Jamaal Brown||43.5|
|Kansas City||0||0||9||36||232||B-||C||Derrick Johnson||26.5|
Here's how it breaks down for the 2006 draft. Chosing between the Jets and Broncos for the top draft was a very close decision.
Best Draft: The Jets picked up two-fifths of their offensive line in the first round by picking left tackle D'Brickishaw Ferguson and center Nick Mangold. Second-round pick Kellen Clemens was the team's starting quarterback for the majority of one season (10 games played and eight starts in 2007) while fourth-round pick Brad Smith has been a useful receiver. Fellow fourth-rounder Leon Washington made the Pro Bowl and was a first-team all-Pro as a kick returner as well as being a contributor in the running game. Defensive back Drew Coleman has been a consistent contributor.
The Broncos draft is arguably even better (the Jets edged out Denver for the crown because of Washington's all-pro appearance as a returner, which is admittedly a very thin difference). First-round pick Jay Cutler, fourth-round picks Brandon Marshall and Elvis Dumervill and sixth-rounder Chris Kuper have all started for three seasons and Marshall and Dumervill have turned into two of the best in the league at their positions. If we re-evaluated this a year from now the Broncos would very likely end up on top. Of course, two of those draft picks have already been dumped by Denver, which doesn't count against them in this ranking (we're looking at acquiring talent not determining where they play), but does mean the Broncos draft hasn't helped them as much as New York's picks.
Worst Draft: First-round pick Jason Allen has been a significant bust for the Dolphins (he's started only 12 games in four seasons), and he's still the best player Miami picked in 2006. He's also the only player from that Dolphins' draft who's still on the team.
How The Experts Fared: Back to Greg Stonerock:
Even worse than 2005! The Spearman test shows a correlation coefficient of .054, even closer to 0 - but neither one is any different from 0, statistically speaking. As I'll show in a follow-up post, you would do a better job of predicting who will have a good draft down the line by just counting the number of players they drafted.
|Team||AP1||PB||St||Car AV||G||Then||Now||Best Player||Score|
|NY Jets||2||4||11||110||394||B||A+||Nick Mangold/D'Brickishaw Ferguson||120|
|Denver||1||4||13||149||326||B||A+||Brandon Marshall/Elvis Dumervil||116.5|
|Houston||0||5||16||135||307||B-||A+||Mario Williams/DeMeco Ryans||105|
|Green Bay||0||0||17||146||425||B||A||Greg Jennings||95|
|New Orleans||1||1||12||140||307||B+||A||Jahri Evans||90|
|San Francisco||0||1||13||82||362||B-||B||Vernon Davis||79.5|
|San Diego||1||3||9||92||238||B-||B||Marcus McNeill||71.5|
|New England||1||1||6||59||306||B||C+||Stephen Gostkowski||49.5|
|Kansas City||0||0||9||66||227||C+||C||Tamba Hali||38|
|Tampa Bay||0||1||9||54||216||C||C||Jeremy Trueblood||35|
|NY Giants||0||0||6||58||273||C+||C||Barry Cofield||31|
|St. Louis||0||0||5||48||266||C+||C-||Victor Adeyanju||25.5|