Subscribe to Annie's Substack

“The NFL Draft” – Annie’s Newsletter, May 4th, 2018

A grand decision experiment, wrapped in a pageant

Last week, the NFL drafted newly eligible players for the 2018 season.

You could argue that the NFL Draft is the most maddening sporting event: no one will know who won or lost for several years and, even then, it will be difficult to figure out the reasons for the outcome.

That’s what makes a pro-sports draft such an interesting decision puzzle. We know there’s a lot of noise in the selection process.

In this just-completed draft, four of the first ten picks were quarterbacks. If you go back to 1999, the first three picks were quarterbacks. The #2 pick, Donovan McNabb, made 6 Pro Bowls. The #3 pick (Akili Smith) and the #1 pick (Tim Couch), however, were major disappointments. In the next draft, 2000, zero quarterbacks were chosen in the top 10 picks, and the best quarterback of that draft, Tom Brady, was pick #199.

In a way, not knowing the outcome for years makes for an interesting study of the process.  Cade Massey (@BCMassey), a Wharton professor (and cohost of @WMoneyball), wrote an instructive article in @BehScientist on decision lessons from the NFL drafting process. I think it’s a great article, containing five lessons that apply to any decision maker.

Each of these lessons addresses biases that creep into evaluation decisions. Even if your decision doesn’t involve evaluation of talent, these are excellent pieces of advice.

I’ve included some highlights of the article here but I highly recommend reading the whole thing:

  • #1 – Understand Your Goal

  • #2 – Keep Your Judges Apart

  • #3 – Break the Candidate Into Parts …

  • #4 – … and Bring Them Back Together Mechanically

  • #5 – Keep Score

Lessons from “hilariously wrong” predictions

A recent article in @Inc by Jessica Stillman (@EntryLevelRebel) lists “12 Hilariously Wrong Tech Predictions.” (The list is part of an even longer list by @Rel_Interesting.)

Charlie Chaplin called motion pictures “a fad.”

Darryl Zanuck said people would “soon get tired of staring at a plywood box [television].”

Einstein even dissed nuclear energy.

Einstein predicting an atom could never be split? I’d say that is one of the best demonstrations of the uncertainty inherent in predicting the future.

Even the people who have had such a huge impact on how the future has unfolded can’t know the directions it will take.

Imagining the future is mainly a novel re-imagining of the past.

It’s difficult for even the best minds to escape the gravity of the status quo when they try to imagine the future. It’s hard to imagine big leaps, things that don’t exist yet, or that the world is going to be substantively different than the way it is now.

Ken Olsen, the legendary engineer and manager who founded DEC, a leading competitor of IBM in mainframe computers in the 1960s and 1970s, said in 1977, “There is no reason anyone would want a computer in their home.” DEC was one of the innovators in its time, so Olsen was speaking from a perspective of someone who had shaped the landscape as it looked at the time.

That position might seem unreasonable when we consider how things are now, where we can hold massively powerful computing devices in our hands.

But that would be hindsight bias and resulting.

After all, given the state of computing in 1977, was the prediction really that silly?

Why would someone in 1977 want the behemoth mainframe computers DEC and IBM were producing in their homes – or even some foreseeable evolution of those machines?

You’d have to be capable of imagining, in 1977, computers (and processors) becoming significantly smaller.

You’d have to imagine giant advancements in processing power.

You’d need an interface that didn’t require you to be a computer scientist to understand and operate.

The handful of computers for individual use were build-it-yourself kits for hobbyists, required programming skill just to play simple games, and some didn’t have screens or keyboards.

When we predict the future, it’s hard to get past the present.

That’s why “I’m not sure” is a more accurate prediction of what we think the future will look like.

“I’m not sure” also does the trick!

Another great read from @BehScientist, “The Benefits of Admitting When You Don’t Know” by Tenelle Porter, looks at how intellectual humility improves learning.

Porter and colleagues measured intellectual humility in high school students and found a correlation between intellectual humility and progress as learners.

To explore whether the relationship between “I’m not sure” and improved learning is a causal one, they took this finding the lab.

The researchers induced intellectual humility by having half the participants read an article describing the benefits of admitting what you don’t know. The other half read an article about the benefits of being certain of what you know.

More students (85% vs. 65%) who read about intellectual humility sought help for an area of intellectual weakness.

There is still a lot to learn about the role of intellectual humility, but the research in the article suggests some clues.

Embracing uncertainty and our imperfect knowledge seem like a good starting point for learning more.

A reminder about our difficulty keeping things in perspective

From the brilliant @NirEyal here’s a post on the Peak-End Rule, a cognitive bias that distorts our memory of experiences.

As the name suggests, when remembering an experience, we tend to overweight the peak and the end. 

For example, if we have a painful experience, like a colonoscopy, it matters less how long we are in pain and more whether the pain declines at the end of the experience.

Even though Patient A (shown below) is in pain for much less time, the experience is rated worse than Patient B whose pain declines toward the end:

The Peak-End Rule is a good reminder that our memories are biased.

It also reminds us that we can use those biases as hacks to make better memories.

It sounds like a good plan if you know gift-quality in advance.

Even then, the warm fuzzy feelings might not extend to the person whose gift got opened first, especially if they know about the strategy.


Prediction Machines: The Simple Economics of Artificial Intelligence, by Ajay Agrawal, @JoshGans@AviCGoldfarb.

Measure What Matters: How Google, Bono, and the Gates Foundation Rock the World with OKRs, by @JohnDoerr.

From @AkiyoshiKitaoka