I have been hosting The Decision Education podcast for the past few years where I get the privilege of talking to incredible thinkers in the decision-making space. This week, the first episode of the new season dropped with my guest, Spencer Greenberg. Spencer is entrepreneur, mathematician, and host of the Clearer Thinking podcast, which I have had the pleasure of being a guest on.
Always a delight to speak with, we explore how we can move from simply recognizing decision-making errors to applying evidence-based tools that actually improve our thinking.
We also discuss Spencer’s path from a Ph.D. in applied math to creating interactive tools that help people make better decisions. And we take a deep dive into why awareness of biases isn’t enough to overcome them and how AI can both amplify and challenge our blind spots.
Key takeaways from this episode include the power of precommitment contracts in shaping future choices, how Bayesian thinking can help us update our beliefs, and why—in a world increasingly influenced by AI—balancing intuition and analysis leads to the best decisions.
Guest bio
Spencer Greenberg is an entrepreneur and mathematician with a focus on improving human well-being. He’s the founder of ClearerThinking.org, which provides over 80 free, digital tools to help people enhance their critical thinking, make better decisions, and make positive behavior changes in their lives, as well as the host of the Clearer Thinking podcast. Spencer is also the founder of Spark Wave, an organization that conducts psychology research and builds psychology-related products designed to help benefit the world. Spencer has a Ph.D. in applied math from New York University, with a specialty in machine learning/artificial intelligence. Spencer’s work has been featured in numerous major media outlets, including The Wall Street Journal, the Independent, the New York Times, Lifehacker, Gizmodo, Fast Company, and the Financial Times
Transcript
Producer’s Note: This transcript was created using AI. Please excuse any errors.
Annie: I’m so excited to welcome you to the podcast, Spencer. Friend of the Alliance. Amazing human. Just for those who don’t know, Spencer hosts a podcast and I have been on that podcast. How long ago? I can’t remember when.
Spencer: It was a little while ago.
Annie: Yeah, yeah, yeah. One of my favorite conversations that I’ve had, and I’m so excited that we’re now doing the inverse, uh, where you come on this podcast. So what I would love to hear is, I think that you have a very interesting journey to getting to the place that you are right now. I was hoping we would just start off by you sharing a little bit of your journey, how you got interested in decision-making as a focus. Obviously been doing a lot of work with AI as well, and machine learning and really understanding kind of like what is that focus, what’s the intersection of those different areas of interest and how did you get here?
Spencer: Yeah. Thanks so much for having me on. I’m really excited to be here. So I do have a bit of an unusual background. I did my Ph.D. in applied mathematics with a specialty in AI and machine learning, but I always knew that I wanted to apply the math to a particular area. I just didn’t know what area. And quite a number of years ago, I eventually settled on psychology as the area I was most excited about. Partly because I think it’s just incredibly interesting and there’s a lot of room for impact in psychology because it touches on so many important topics like mental health and happiness, and how do we be productive? How do we make good decisions, how do we avoid biases and so on. So it just seems really important, but also because I think my mathematical background actually has some interesting tools to bear on the problem that aren’t usually applied in psychology.
Annie: I kind of want to start there, which I think is kind of interesting. I feel like in the sort of world of behavioral economics and psychology as it relates to decision-making, there was initially this idea, oh, people are rational. And then you know, Kahneman/Tversky along with a whole host of other people, Dick Thaler, John Baron, for example, Richard Zeckhauser came along and said, not so fast. When you actually look at their behavior in the economic sense within which you’re defining rationality, people do all sorts of things that appear to be irrational. And I feel like there was a very long period of time, decades long, as a matter of fact, which was proving and showing and demonstrating the ways in which humans veer from rationality.
So for example, oh, you know, humans take into account resources that they’ve sunk into something in deciding whether to continue on and spend more, the sunk cost fallacy, they were describing that, or availability bias or confirmation bias or so and so forth. And what I think over that period of time was that there was a real paucity in, okay, so that’s a problem. How do you fix it?
And in fact, when you sort of run across people in the wild who’ve read something like Thinking, Fast and Slow, the amazing book by Danny Kahneman. I think there’s a sense among people that once you recognize that there’s something called confirmation bias that you’re cured. Let, let’s just start on why do you feel like there was so long without a real focus on actual practical solutions to make this stuff better? And where do you kind of see the field, independent of you right now in that regard and where, where are you sitting in that push to try to say, okay, we know what the problem is, but how do we actually do something about it?
Spencer: That’s a great introduction and intuitively it makes sense that it’s important to understand the problem before you develop solutions for it. So I think the order of things is natural and you know, Kahneman really pushed for the feel of him and, and the others that you mentioned in understanding the way we go wrong. And now we now know so many of the ways that our minds can lead us astray. And it’s sort of now, I think, a new age of, okay, let’s figure out what to do about it.
And there are definitely some disappointing results where they try to de-bias people and they find it doesn’t work. And so I’m most interested in, okay, what could really work? What could we do that’s really effective? What actually works in practice? Help people make better decisions to be better critical thinkers and so on. And I actually think that there’s this phenomena where people learn about cognitive biases. They start seeing them in other people’s thinking and you really need to transition to seeing them in your own thinking, and that’s when you get the most benefit out of it. But that’s not something that happens automatically.
Annie: So I would love to get your thoughts on why is it that knowing about it isn’t enough? Intuitively people sort of feel like, look, if I understand that, searching for information that confirms me, it’s what I pay attention to. I’m really good at rationalizing away stuff that disagrees with me, well then I just won’t do that anymore.
I work with a lot of people in finance and I hear from a lot of them, something to this effect. Well, I know about the sunk-cost fallacy, and so all I do is I imagine, what if I didn’t own the stuff, would I buy it again? And as you know, that doesn’t actually work. But people are very confident that because they know about the problem, they are cured in some way, or, or they’ve come up with some intuitive workaround and then they’re cured. And I’m just curious as to why do you think that the biases are so durable and knowing about them isn’t enough?
Spencer: If we think about what causes you to avoid a bias, you have to be doing something in the world, like you’re about to, you know, decide to keep holding this stock. But then you have to have a thought, Hey, I might actually be being biased here. Right? So that’s the second step. Like you’re doing the thing in the world, then you actually notice you might have the bias, right? Third, you actually then have to override your intuition because your intuition is probably telling you, oh, I shouldn’t sell this because I, you know, and you have a kind of a sunk-cost fallacy, right?
And in some biases it’s much more difficult to override it, right? Like you, you might actually feel it so strongly, or you might not even know how to override it. And so I think knowing it is a first step, but knowing about the bias doesn’t necessarily mean you’re gonna think about it at the right moment. And it especially doesn’t mean that you know how and you’re going to successfully override it when you do notice it in the moment.
Annie: Can you give some examples of some tools that you’ve actually tested, that you’re offering to people as, Hey, this is actually gonna help you.
Spencer: Good question. One thing I will say is I think the evidence base is not very strong right now. There’s just not that much known. A lot more evidence needs to be generated. But on our website, ClearerThinking.org, we have over 80 free tools that people can use. And, um, it’s not realistic to run a randomized control trial in every case, obviously.
Annie: Of course, of course.
Spencer: So we take a staged approach. The first approach is we look at what is known when there is evidence and we kind of look at the research out there, we’ll spend a lot of time thinking about the problem. We then develop these interactive modules and we try to approach it from the point of view of getting you to apply the knowledge, because I think that’s actually a really key piece.
Learning about what the name of a bias is way less useful than learning by the name of bias and then having to do a bunch of examples where you actually have to apply that knowledge. So that’s a key aspect of our interacting modules. We then run two little mini studies on every module. The first is we recruit people from the general population, and we have them go through the module and we have them critique it in a bunch of ways, um, and give us feedback, both qualitative and quantitative. And we have like benchmarks we use asking questions like, do you think that you’re going to use what you learned? Do you think this was valuable to you, et cetera. And then we do a second little mini study where we test it on our core audience and our beta testers. While those are very far from a randomized control trial.
Annie: Mm-hmm.
Spencer: They’re a rapid, iterative feedback loop that allows us to catch issues like, oh, they didn’t understand the concept, or they don’t think they’re going to be able to apply this properly or they had trouble with this exercise we taught them. For some modules, we want to go further and run randomized control trials. The problem is it takes a really long time. It’s really expensive, so we’re looking for grants to do that. But we have gotten some grants in the past successfully. For example, this is a little bit less in the decision-making realm, but we designed a giant study on habit formation. We wanted to know what really works to get people to stick to healthy habits.
Annie: Can I just say that’s for me, firmly in the decision-making space?
Spencer: Oh, great.
Annie: Habits are processes that are running automatically that actually can be sort of taken out of the reflexive world, brought into a deliberative world, be examined, and then there are tools that you can use in order to actually effectively change habits so that you can shove them back into what Kahneman would call system one. And I think that’s actually a very good example of good decision-making in practice.
Spencer: Yeah, it’s so interesting because if you get something to be a habit, it’s sort of like you pre-made the decision and you don’t have to make it anymore. So if you’ve made the right habits, you’ve made a lot of good decisions for years to come, right? Yeah. So, uh, the approach we took is we didn’t know what was going to work to actually get people to stick to habits. We looked at the academic literature, we brainstormed, we crowdsourced, we considered a lot of ideas, and we ended up developing about 20 different micro-habit interventions. And then we put them to the test.
We ran a giant study, but the way it worked is you would be randomized to five out of these 20 different techniques. You’d have to pick a new hobby on the form, and then you’d go through whatever interventions you were assigned to, and then we track you to see how well you stuck to your habits. And it was really fascinating because most of the techniques totally failed. They did nothing. And even some that had really good literature support in the academic literature, it was kind of depressing. But we did find some standout techniques that seem to help.
One of those is a really interesting one we call habit reflection. It’s kind of a three-part process. The first thing you do is you think about a habit that you’ve succeeded at in the past. Then you reflect on how you actually succeeded at that habit. What did you do that helped you succeed? And then finally you think about how you could apply what you learned from that previous habit to the current habit. And we found that people who did habit reflection increased their chance of sticking with their habit over the eight weeks. And then we actually took the techniques that work the best, and we bundled them together into a tool called Daily Ritual. It’s a habit-formation tool. You can use it on our website. It’s totally free and it helps you form a new habit. It helps you apply these evidence-based techniques.
Annie: Well, so it sounds like that technique in particular is ensuring that there’s a tight feedback loop.
Spencer: Yeah.
Annie: Tell me if this sort of tracks for you. We sort of have the idea that we’re going to just sort of pay attention to stuff in the world, right? And I, I think this is like the foundation of, for example, a lot of Barry Staw’s work, which is like, we just assume that when you get signals from the world that things aren’t going well, you’re gonna pay attention to them. Likewise, I guess if you have signals that like, the habit is working or not working, that naturally, we’re going to pay attention to them. In general, that feedback loop is going to run. We’re going to notice it. Then we’re going to do something about it.
But I think what you’re saying is no, you actually have to bring that into the deliberative space, and you have to make sure that that feedback loop is explicitly being closed by having them say, what are the things that you’re noticing? How would you change that in the future? So you’re constantly getting this iterative loop going on, which we think, well, why would you have to write it down? Those things are happening anyway. But it turns out like the writing it down and making that part of the habit formation really matters. Is that a fair summary?
Spencer: Yeah, absolutely. And I think writing something down is really powerful because it forces you to actually clarify your thinking in a way that just having a thought doesn’t. If I have to actually write down, well, how do I apply what I learned from the last habit to this habit? It’s very different than merely loading into your memory the fact that you succeeded at the habit or even what you did.
Annie: Mm-hmm.
Spencer: Because now you’re like having to make your thoughts precise and set an intention to do things differently. But I don’t know if you know this, but I actually developed a framework to help people think about when they should trust their intuition, when they should go with their gut versus use their analytic mind.
Annie: Yeah.
Spencer: And I call it the FIRE framework. It’s an acronym, FIRE, and basically, I mean, I think our guts actually are accurate quite a lot, but we also know a lot about when they’re not accurate, and that’s sort of the key.
Annie: Right.
Spencer: On average, people tend to trust their guts too much, but there’s actually a substantial number of people that like don’t trust their guts. They constantly ignore what their intuition’s telling them. It gets them into horrible situations. You know, you can think about people who, you know, they have a series of bad relationships and they get a bad vibe about someone and they date them anyway.
Annie: Yeah. Right.
Spencer: Yeah. So the things I think about are F: is it a fast decision? Because often with fast decisions, we have to go with our gut. Like you just don’t have time to really analyze it.
Annie: Mm-hmm.
Spencer: Then I: is it an irrelevant decision, like if it’s really small and minor, like what movie you’re going to watch tonight, what you’re going to eat at the restaurant? Like don’t, don’t ever analyze it. You’re not doing yourself any favors. And then R, which I think is the most interesting one: repetitious decisions. So if we have a good feedback loop.
Annie: Mm-hmm.
Spencer: If you have a good feedback loop, your gut can be incredible. Uh, you think about someone who does something every day and they can see the result of what they’re doing. They, you know, like a martial artist or a chef, they become incredibly good at having intuitions about what to do. It’s just that in some areas of light, we don’t have the feedback loop. And I like to use a metaphor of archery. Imagine that you’re firing arrows at a bullseye, but you’re blindfolded. You never get to see if the errors hit. How long is it going to take you to become a good archer? Infinity, right? Like if you can’t see where the arrows are going, or imagine, okay, you don’t have a blindfold. You’re shooting arrows at a target, but you have to wait an hour to check if they hit. You don’t get to see for an hour. I mean, think about how long it would take to become a good archer, whereas if you can shoot an arrow and immediately see the result and then make an adjustment. You’ll get better and better and better quickly. And then the last one: E or evolutionary decisions. And I do think there’s a small number of decisions which we kind of have coding in our minds for. A great example is like that food smells off. I probably shouldn’t eat it, right? Yeah, you should probably go with your intuition on that.
Annie: I totally agree with you. I actually bucket things into sort of two categories, impact and optionality, which is going into what you said, right? So the lower the impact of the decision, I really don’t care if you go with your gut, be my guest. And the more optionality you have, in other words, the lower the cost it is to actually reverse the decision.
One of the costs of starting something is that you could make a bad choice and it stops you from starting other things because you can’t be two people in two different places, which is just opportunity cost. So the lower the cost to actually change your mind about it also, the more that you can go with your gut.
One of the issues that I think that we have on the repetition side is that everything isn’t archery, right? Where if you can see the bullseye and you’re just shooting arrows at it, you know for sure that you literally did the best that you could. A lot more things are sort of like, it’s not really clear how everything’s being scored, and there’s a lot of volatility, and so I think of things where you’re doing pretty well, but if you just always leave it to your gut, you can’t find out that you could be doing a lot better.
And a good example actually is in sports, right? That you had all these repetitions of fourth down decisions that were going on for a long time before people were really bringing analytics into it. And teams did pretty well punting on fourth down pretty much every single time, right? Like they were doing fine with that, and then someone was like, wait a minute, we should really question the coach’s intuition here about what you’re supposed to do on that play. And let’s actually analyze it and see what it says about that. And it turned out that they were leaving a lot on the table. They were, they were leaving a lot of win probability on the table. So then things started to change. Right?
And I think that a lot of things are kind of like that even in things where you’re getting like a lot of reps in it, right? Like even something as simple as like tennis, which is, you know, pretty highly skill-based. It’s good to step back and say, is there a better shot I could have hit there. Like that worked out really well and I’m winning, but could I still be increasing my win probability? And I think about that in pretty fast feedback loop things like trading, right, where you’re making money. But the question is, are you maximizing? Right? And sometimes I, I do think you want to sort of take things out of your gut in those spots sometimes.
Spencer: Yeah. I think a lot of times when we’re relying on feedback loops, we’re able to locally optimize, but if we want to jump to a much higher optimal point, it often requires careful analysis. What you’re doing is you’re actually trading information between your gut and your analysis.
Annie: Mm-hmm.
Spencer: Back and forth and back and forth, and kind eventually leading to a synthesis where ultimately they eventually agree.
Annie: One of the things that I really want to talk to you about is the work that you’ve done on personality tests, which is one of my pet peeves. The number of people who have come up to me and been like, I’m an INTJ. What are you? Like, uh, so, you know, we could think about personality tests, you know, as, as, as you point out the range of astrology, which is kind of a basic personality test, I guess. It’s like, when were you born? Oh. You’re a Virgo. Oh, I’m going to tell you.
Spencer: It’s a control group for personality testing.
Annie: It is. It is. But then you have personality tests that people really, really believe in. So if you have something like Myers-Briggs, right, so that’s one. You have something like Big Five. The thing that I think is really interesting with Myers-Briggs is that people literally will organize their teams around it. They, they, it actually changes the way that people operate in business.
So could you just start off by kind of explaining what is Myers-Briggs? What it’s sort of attempting to measure? And then what is Ocean or Big Five and what is that attempting to measure? And let’s just start sort of just set the table with that.
Spencer: Yeah, so I would say that the three most popular personality tests in the world are the Myers-Briggs personality test known as MBTI, the Big Five, and then Enneagram. We can put Enneagram on the side today.
Annie: Yeah.
Spencer: But the basic idea of Myers-Briggs is that it divides you into four dichotomies. So you, you’re either an N or you’re an S, which is intuitive versus sensing. You’re either an F or a T, feeling versus thinking. And each dichotomy is supposed to represent something important about you. Sometimes they’re represented scores, but much more often you see them as a single letter, like an F versus T. So most of the people think of it as this four letter code.
Annie: Mm-hmm.
Spencer: Like, I’m an ENTJ representing your personality as, as these four letters. Now, I’ll come back in a moment to what they actually measure, because I think introducing the Big Five gives some interesting insight into this. So the Big Five is a test developed by academics, and it’s considered the gold standard for personality testing. Scientifically, it uses five different measures to measure your personality. They go by the acronym OCEAN. So O stands for openness, which is about being imaginative and intellectual. C stands for conscientiousness, which is like being organized and perfectionistic. E is for extroversion, which is being social and energetic. A for agreeableness, which is being, being kind, being empathetic and being agreeable. And then finally, N for neuroticism, which is about being anxious, depressed, moody, things like that.
Now, what’s the relationship between these two? Right. So Big Five has five things. It’s measuring. Myers-Briggs has four things it’s measuring. Well, we actually ran a giant study looking at how these ideas relate to each other. And also we implemented our own version of a Big Five test, a Myers-Briggs style test and an Enneagram test. And by implementing them all together, we were able to kind of study how good they are actually making predictions about people.
And fascinatingly, the Myers-Briggs, the four traits that it measures, each of them relates most to one of the Big Five. So for example, the NS factor in Myers-Briggs relates most to openness and experience in the Big Five. It’s not obvious. So one way to think about what Myers-Briggs is doing is it’s measuring to some degree four of the big five. Well, what’s missing? Well, what’s missing is neuroticism. It doesn’t measure anxiety and depression and things like that. One thing that’s really interesting about that is that when you take a Myers-Briggs test, usually it tells you you’re good no matter what.
Annie: Mm-hmm. Mm-hmm.
Spencer: Right. It might have a little section on your flaws, but usually it makes you feel good about yourself. And in fact, we tested that in our study. We found, and in fact, people were happier with their Myers-Briggs style results than they were with their Big Five results, because the Big Five tells you, you know, that you’re, uh, close-minded and neurotic. And whereas the, uh, Myers-Briggs style test tells you, you know, you’re an E-N-T-J, that doesn’t sound bad.
Annie: Right.
Spencer: You’re, you know, right. So. But then I think the kind of most interesting piece is then we tested how well can we make predictions for each of the tests? And we found just as a control group, we used, uh, astrological sun signs, so are you, Pisces, an Aries, et cetera. We found using that, we were completely unable to make predictions about people’s lives.
So we tried predicting 37 outcomes about people’s lives. Not surprising to me, maybe surprising to some, but astrological sun sign had no predictive ability, so that’s kind of our control group. We then tried using the Big Five and our Myers- Briggs style test, and we found that the Big Five was about twice as predictably accurate predicting things about people.
Annie: As the Myers-Briggs.
Spencer: As the Myers-Briggs, exactly. Okay. Yep. And so that, funnily enough that put the Myers-Briggs style test literally halfway between astrology and the Big Five, which I think is a reasonable way to describe it.
Annie: One of the ways that we think about like personality tests is internal validity and external validity. So an internally valid test means that if I take the test on Monday and then I take it again on Friday, it should give me the same results. We’re supposed to be saying something pretty deep about people’s personalities here. And then external validity would be what you’re talking about, right? Like if you take it and I get a score, is it actually predicting things about life outcomes, for example. Right? So that would be external validity.
Spencer: Mm-hmm.
Annie: So one of the things that I think is interesting about Myers-Briggs, which, you know, business people are huge fans of, I, I actually sometimes think about it as the astrology for business people, but one of the things I found for myself was that depending on when I took the test, I was either an INTJ or an ENTJ, but that INE thing? Every time I took it, it was something different.
Spencer: Well, that’s actually very common. Yeah.
Annie: Yeah. And I think part of the problem, and I, it’s something that you’ve addressed in the past, is that I’m on the border between the two. And so depending on which day you catch me, I’m either going to feel energized by being around other people, or I’m going to feel enervated by being around other people. It’s going to depend because I’m on the border, I shift between the two things. And I don’t think Myers-Briggs allows you to sort of capture that variation because as you said, it’s like binary. It’s you’re, you know, there’s four things and you’re in one category or another. So can you talk about like the differences on that way in terms of internal validity and also like why are you not seeing that with Myers-Briggs versus Big Five?
Spencer: Yeah, it’s a great question and to answer it, we actually considered why is the Big Five outperforming Myers-Briggs style tests at predicting things about people? And so to try to figure that out, we first realized that neuroticism was missing from Myers-Briggs style tests. And so we decided, okay, well what if you remove that from the Big Five? And that closes some of the gap, right? So it makes the gap closer, but it’s still, there’s a substantial gap. Uh, Big Five is still predicting much better. Okay, well, what’s next? Well, then we thought about these dichotomies. Obviously if you dichotomize someone into, you know, E or I, you’re losing information and so well. Usually Myers-Briggs is using this dichotomized way. But what if you were using this continuous way and treated as scores? Interestingly enough, that closed a bunch of the remaining gap. And so if you were to remove neuroticism for the Big Five and you un dichotomize Myers-Briggs, we still found the Big Five a little bit better for making predictions. But now the gap was pretty small.
And so I think that sheds light on this. And we also looked at, well, why is dichotomization such a problem? And it turns out it’s because pretty much all of these traits are close to normally distributed, like a bell curve.
Annie: Mm-hmm.
Spencer: And that means that most people lie near the middle because the bulk of the people are in the center of the bell. Well, that’s the worst possible kind of distribution to dichotomize, because if you dichotomize it, many, many people lie just on the edge of the boundary, and so they retake the test and they might arbitrarily flip to just above or just below.
Annie: Gotcha. Gotcha. Actually, that’s such an interesting point, right? That if things are normally distributed, then having something as a dichotomy is really horrible because most of the people are actually going to be on that border where, where they’re going to arbitrarily be assigned to one thing or the other, and then all of a sudden you’re assigning people to teams based on the these letters where you think that it’s like, well, I’ve got an introverted person here and an extroverted person here, and so they’re going to be really good. They’re going to balance each other out. I have someone who’s more sensing and someone who’s more feeling, so that’s really good. It’s going to get diversity on the team. Meanwhile, those people are basically identical.
Spencer: Yeah.
Annie: And you, you don’t even know it because you’re losing that information. It would be like saying you’re tall or short.
Spencer: Yeah, exactly. This thing, you’re tall or short when it’s like, oh wait. Actually most people are fairly close to the average height actually. They’re not too far from it. And, and I, I’ll just say for anyone interested in this topic, we release something we call the Ultimate Personality Test that you can take, and it actually will measure your personality from these three different, most popular frameworks, show you how your results compare, but also we’ll show you the data that we collected on how accurate each of these are. So you can find it on our website ClearerThinking.org and it’s totally free to take.
Annie: First of all, people should definitely go do that, by the way. Because some of these things are actually like quite predictive and they do actually help you to kind of understand yourself. And I think one of the things for people to realize is even if something has a name that you kind of think of as good or bad, it doesn’t necessarily mean that it is good or bad. So one of the features that I think about that with OCEAN is agreeableness or disagreeableness. Agreeableness obviously has advantages, but can you talk a little bit about like what are the advantages to being higher on disagreeableness?
Spencer: Yeah. I think my view is that every personality trait is dysfunctional at the extreme. So if you’re like 98.9999 percentile, or 0.001 percentile, it’s probably not a healthy place to be.
Annie: Right.
Spencer: But that there are interesting trade-offs as you go up and down, like if you’re near the middle and you’re like, oh, I want to go up or down, like there often is a direction that tends to benefit people more, but it’s not costless. There’s actually trade-offs occurring. So if we take agreeableness in particular, agreeableness is correlated with being kind and empathetic, caring about other people. And I think we could, most people agree that that’s a good thing.
Annie: Mm-hmm.
Spencer: However, it is also correlated with not being able to say no to people and to being potentially taken advantage of. Also the highly agreeable people often have trouble saying, Hey everybody, you’re wrong. So I think if yeah.
Annie: They’d have trouble going backwards over the high bar.
Spencer: Yeah. Yes. So I think that we actually need some disagreeable people in society to say, Hey, everybody, you’re totally wrong. You’re doing everything wrong. To stand up against people without worrying too much about how they’re feeling, even though there’s a lot of benefits to being agreeable.
Annie: I think that we tend to put these value judgements on, I want to be one thing or the other. And I think it’s really good to understand like, no, but there’s value to being maybe in a different place on the scale. Maybe scoring a little bit toward the disagreeable side in certain situations is actually quite valuable. And I guess that that’s where, you know, you start to get into that term of like agreeable disagreement. Like how can you disagree with someone while also still being kind and empathetic and sort of standing there?
So don’t, you make it not a fight, for example. Right? And I think that there’s a lot that you can explore in there, once you say it’s not a dichotomy. These are all on a scale and we shouldn’t be judging things as good or bad.
So I’m imagining, and you can correct me if I’m wrong, that one of the reasons why people sort of felt better when they took the INTJ, is that the words in there, like the way that you’re scoring it, like introverted or not, or sensing or feeling, those words don’t have the same kind of valence as OCEAN does. Like what are you telling me? I’m not at all open-minded, then I feel very bad about myself. Yeah. Or I’m highly neurotic. I feel very bad about myself.
Spencer: Exactly. I do think it’s partly the words that are used because what does it mean to be intuitive versus sensing? In fact, it’s not even what you might think. Like you might think intuitive means an intuitive person. It’s not really even what it’s about.
Annie: Right.
Spencer: So they kind of just invented new words, or perceiving versus judging. What on earth does that mean? It’s actually linked to conscientiousness, but you wouldn’t necessarily think that. So I think that actually helps weaken the blow to our egos, which is actually a nice feature. Because I think a lot of scientific minded people want to say, oh, Myers-Briggs is totally useless. I don’t feel that way. You know, I think anything that helps you is valuable. I don’t think it’s the most accurate test for predicting things about people, but I do think it has some nice features, which is that everyone’s willing to share their results. And that’s actually pretty nice. Like, you know, who’s going to post about, I’m a neurotic, disagreeable, you know, close-minded person.
Annie: Right, right, right, right.
Spencer: And it’s like, no, there’s some value in having information you’re willing to share. Right. So, so something, something to that. I also think that it’s also about the way these profiles are written. You know what I mean? If you’re getting your Big Five results, often they’re just very like, here are your scores.
You know what I mean?
Annie: Right.
Spencer: Whereas uh, Myers-Briggs style tests very often have like a lot of just like exposition about you and here are the celebrities that you’re most like and you know, and so they kind of find ways to make you feel good, which is not necessarily a bad thing, you know? Because there is value, I think, in every personality trait.
I mean, even take something like neuroticism, which maybe is the most negative sounding of all the traits. And I do think that most people who are neurotic would be happier if he became less neurotic. However, like worry has a role in society, like we need some worriers. You know, even traits like depression kind of make sense in certain contexts, right? So I don’t think, um, you know, the answer is we want to max out in one direction or something like that.
We, as humans tend to be story-driven rather than data-driven. And you know, the reality is if you tell yourself stories about yourself, like I’m this sort of person, I’m like that other person. It’s not sort of as natural for most people to think in terms of scores. Like, I’m 70th percentile. I like to think that way, but I’m a mathematician. So, yeah. And I think there’s also an evolutionary pressure, right? Well, why are we talking about this Myers-Briggs style test? It’s because it won this evolutionary pressure of all the tests that existed and became sort of probably the most famous in the world. And it’s probably because it resonated with people, and partly that may be because it’s useful to people, but I think a big chunk of that is because it just resonates psychologically and with our egos as well.
Annie: Yeah. Yeah. When people come to your site because you’ve developed something that’s unique, are people willing to share their results? Do they feel like they’re sort of getting the best of both worlds from what you’re offering?
Spencer: Well, the way we approach that, because we do want people interested in sharing the results, is that we ran our own clustering algorithm across all of our data to discover clusters in the data. Now, if you look at popular systems, they love giving people a type, right? And we know that people want a type, but we didn’t want to just make it up. We wanted to be data driven. So we literally just took our enormous data set, ran a clustering algorithm. We forced it to have 16 clusters, because that’s a manageable number. And then we looked empirically, okay, what are the 16 clusters that come to emerge in the data?
And for each of them, we like studied them and tried to say, well, what traits do they have? What’s a reasonable name to give to it? So that’s actually what we give you. So we give you a nice infographic that you can share. But it, the big thing, the big bold thing is it tells you which of these 16 clusters you fell into.
And then it has below that your scores on Myers-Briggs style, Enneagram, and Big Five. But sort of what it highlights is this sort of archetype that is designed to make you feel good.
Annie: And given that you did the work on Myers-Briggs, astrological signs, and Big Five, where does the Ultimate Personality Test fall in terms of predictiveness in comparison to a straight academic OCEAN? Straight MBTI?
Spencer: Well, well, we measure OCEAN as well, the Big Five. So we measure all of them.
Annie: Yeah. Yeah.
Spencer: So because we give you scores in every single one of them, the idea is to give you kind of the best of all worlds, if you will.
Annie: Mm-hmm.
Spencer: And then our clustering system is less meant to give you the most accurate predictions as it is to give you a data-driven archetype that actually fits you.
Annie: Gotcha, gotcha. I’m just, I’m really interested to hear your thoughts on what are large language models—I’m assuming, you know, not AGI, but what are large language models? What do you see as their role in decision-making? Like if you thought about those questions, like, should we just have it make decisions for us? Is it gonna be less biased than we are? Like where do you see it fitting in the decision-making ecosystem?
Spencer: Yeah. I think LLMs and other AIs have a lot of potential to help us make better decisions, but we have to be very careful. Think about the way they’re trained, right? So they’re generating outputs and then they collect training data where they have people rate, well, which of these outputs was better?
Well, think about that for a second. That process of rating which one was better? If an AI tells you you’re right about everything and your worldview is correct and the other worldviews are dumb, you’re more likely to rate it as a good result than if the AI tells you that your worldview is flawed and there’s mistakes in your thinking, right?
So I worry that the LLMs actually pick up on subtle cues of what you want the answer to be, and then essentially manipulate you by spewing it back out, because that’s essentially what they’ve been trained to do. So one thing that I think is really powerful is talking to AI in a way where it knows that you really want to consider all the different sides of an issue.
I use custom instructions where you can build default instructions that get sent to the AI every single time. And one of my custom instructions is around making sure to consider all the major sides of an issue before coming to a conclusion and to call me out when I’m wrong about something and challenge my thinking.
Annie: Well, that’s really interesting because that feels kind of like a pre-commitment. If you ask most people, do you think it will be really good for you to consider all sides of the issue? I don’t think anybody would say no. Do you want someone to just tell you what you want to hear? No, not really. You know, I mean, I think that we end up answering no to all of these things, right?
We sort of recognize like, no, that would be bad. I wouldn’t like that. Except that in practice, when you’re actually in the moment, we do actually just want to hear what we want to hear and we don’t really want to consider all sides. So, you know, the question that I always really kind of try to dig into is, we have these intentions, these sort of stated goals and values, right, that we have for ourselves in terms of, like, our long-term well-being. But then in the moment, we don’t do a good job of actually aligning our decision-making with those values and goals. So how do you deal with that? Well, you have to sort of think about how do I put a process in place that’s going to protect that future version of me that just wants the AI to spew back what I want to hear?
And it sounds like that’s kind of what you’ve done is you said, okay, I’ve developed a set of instructions to not do the thing that it’s naturally going to do, which has caused me actually to be more biased. And I’m now making a commitment that every time I’m going to go and interact with an LLM, I’m going to give it this set of instructions and that’s going to help me to actually avoid this problem I’m going to have if I’m doing it on the fly.
Spencer: Exactly. And both ChatGBT, OpenAI, and Claude, maybe Philanthropic, actually let you permanently assign the custom instructions. So once you set it up, you’re good to go. So, it really is a pre-commitment mechanism, a way of protecting your future self. I think it’s very much the way that, you know, with social media, we might say abstractly, oh yeah, I only want to use it 20 minutes a day, or whatever. But then when you’re actually using it, you’re in the flow of things. You just get sucked in and then you end up scrolling for an hour. Right? So it’s a similar thing. It’s easy to decide in that cold state when you’re thinking about it abstractly that that’s what you want to do, rather than deciding in the moment when you’re talking to it about your beliefs.
Annie: It’s an interesting thing with these large language models. You know, I’m a big fan of Ethan Mollick’s work and you know, I’ve had discussions with him where he says, everything is just about prompting. And we sort of think that the AI is just going to give us the answer, but the answer is going to be completely dependent on the prompt that the AI gets. And we don’t spend any time actually training people on what are the appropriate prompts to actually help you to get to a more accurate place, to get you an answer that’s more objectively fact-based, as opposed to say what you want to hear. Period.
Spencer: Yeah. A real pet peeve of mine is when something’s being debated on the internet and someone goes into a chat bot and is like, prove the other person’s wrong. And then it will always generate arguments telling you, you’re right. And then they’ll go paste that in their Facebook comment or Twitter comment as though you know they have some knockdown argument. And you’re like, no, actually that provides essentially no information. Because you’re just telling it to argue one side.
Annie: So thinking about just like any kind of intervention, whether it’s AI driven or the kinds of things that you’ve explored, how do you think about balancing effectiveness versus, let’s call it practicality? You could have the most effective decision-making tool in the world, and if people don’t use it, who cares? If they’re not willing to use it, who cares? Right? So. If you’re going to intervene in order to improve somebody’s decision-making, the fact is that they’re going to have to do the thing that you’re telling them to do.
Spencer: The perfect tool you don’t use is worse than the oversimplified tool you use that actually gives you some benefit. And I think there’s not nearly enough research on these areas. So a question that’s really interesting to me is how simplified do you want to go with decision-making techniques? Because you know, we know that if it gets too complicated, it overwhelms people and they’re probably not going to really use it, versus if it’s too simplified that you lose a lot of the benefit and like, what is that trade off? And I don’t think that’s very well understood.
Annie: Yeah. Yeah. I mean, one of the things that I say to people all the time who are like, well, I’m going to go build a decision tree, and I’m like, that’s great. Don’t go more than two nodes deep.
Spencer: Yeah.
Annie: Right. And they’re like, why? And I’m like, eh, it’s going to be too complicated. You’re not going to do it. You’re like, just go. As long as you’re only going two nodes, I’m totally fine. Like, go give it a go. Right. But like when things get way too complex, it becomes too hard, and then you’re actually not going to be willing to do it. Whereas if you do some simple back of the envelope stuff that will allow you to write a little something down, you’re probably actually going to do that. It’s like you will actually do that and it will actually be helpful.
Spencer: There’s also a lot of value being able to keep something in your head. I’ve seen people make really complex decision-making models like for, you know, which project to invest in and things like that.
Annie: Mm-hmm.
Spencer: Where there’s like, you know, 25 variables and they’re all modeled out with different probability distributions and the whole thing like spits out some number at the end. If you ask them, well, why is it that number? Why is it? You know, they really can’t explain it to you. It ends up being very brittle and they can’t really even check that it makes sense because they don’t have an intuition for it.
So I think it’s useful going through an exercise like that if you know you’re willing to put in the time. But I think then you want to go back in and say, well why is this getting this output? Like what do the parameters that are really driving this, like, you know, which of the parameters is it really sensitive to? And you go back and then you build an intuitive model and then say, ah, there’s 25 variables, but it’s really driven by these three. And the reason it’s not higher is really limited by this one factor, and then you actually can reason about the model and can also very importantly catch mistakes. Because if you don’t really understand the full complex thing, it’s very easy to screw something up.
Annie: Yeah. Interestingly enough, that’s criticism I’ve heard about deep learning, you know, is that you have some input and then you have some output and it’s opaque. Everything that’s happening in between in terms of what that model is doing, particularly as you add a lot of different layers, on a deep learning model, it’s opaque. So you can’t go in and say, wait a minute, but why did I get this output given this input? And you just kind of have to trust it. So I love that way of thinking about it. It’s like trust but verify, right? Like if you can’t go in and like really understand why that is, first of all, like how do you know? And second of all, maybe you’re not getting to the heart of the matter where you can take something that a human being can actually do and understand and check, and then start to implement themselves. Because it starts to become part of their more intuitive process, and then you can allow that to go on its own as well. So I like the way that you put that.
What decision-making tool or idea or strategy would you want to pass down to the next generation of decision makers that would improve their lives?
Spencer: Well, I’ll talk about an oldie but goodie, which I love and I feel like most people don’t understand. But it’s easier to understand than you think, which is Bayesian thinking. The idea basically is that we, as a society, have actually figured out how to quantify evidence, and very few people are aware of that. And it’s insane to me that this is not better known. Like people argue with each other. Well, that’s not evidence. This is evidence. It’s like this is a settled science. Like we don’t need to be debating this. And, um. I think most people don’t engage in Bayesian thinking when they hear about it because it just seems like all these mathematical formulas.
Annie: Mm-hmm.
Spencer: But there’s a version of it that you can teach without any math that I love. And we have a module on website ClearerThinking.org called The Question of Evidence. It actually teaches you this, so if you’re interested, go check that out. But the way it works is you get some evidence and you want to know, is this strong evidence? Is it moderate evidence, is it not evidence? And you simply ask the question: how many times more likely am I to see this evidence if my hypothesis is true than if it’s not? So I’ll state that again. How many times more likely am I to see this evidence if my hypothesis is true than if it’s not?
And if you’re many times more likely to see it, if your hypothesis is true, than if it’s not, like three times more likely, then it’s moderate evidence. If it’s 30 times more likely, then that’s very strong evidence. If it’s equally likely to occur, this evidence, if your hypothesis is true than if it’s not, then it’s not evidence at all. It doesn’t actually move the needle. And I think this basic concept, it’s not that you really do the calculation in real life, but whenever you’re confused about like, well, is that evidence and is this weak evidence or strong evidence? You go back to the question, which we call the question of evidence, and that question actually tells you exactly what you need to know.
Annie: Totally love that. But I want to highlight something that’s embedded in that that I think is just important to bring to the surface, which is a lot of times when we’re looking at evidence, we’re thinking about whether it supports the thing we believe to be true, and we really need to always add in a question, which is, well, if the thing weren’t true, what would have to be true as well?
So I think that we see examples of this all the time. Like someone will be hiring engineers and they’ll be like, they score really high on disagreeableness. So I should go find disagreeable people to hire as engineers. And then the question is, okay, but like if that were a signal of being a good engineer, what would have to be true for that to be a signal for that? Well, it would have to be the bad engineers actually don’t score high on disagreeableness. So now we can follow your reasoning, right? Like if my hypothesis that it’s a signal of great engineers, and I see that great engineers happen to be disagreeable, how likely is it do I think that if I looked at engineers who weren’t so successful that they would also score high on disagreeableness? And I think we could all agree it was probably be pretty high in that case because it’s a particular engineering mindset. Nothing against engineers, I’m using this, like as a complete hypothetical. But you could see how you could then understand that through your framework. And the thing that I’m trying to highlight is we ask a lot what has to be true, but we forget to ask what has to not be true.
Spencer: Exactly. And that’s why the Question of Evidence is formulated as how many times more likely is it, it’s explicitly saying, how many times more likely am I to see evidence if my hypothesis is true, compared to if it’s not. And it’s the compared to if it’s not that’s critical. And so often when people look at evidence, they’re like, they’ll ask a different question. It’s not quite right. They’ll say something like, well, is this evidence likely?
If my advice is true, well, that’s not the right question to ask. Right. Or they’ll say. Can I explain this evidence with my hypothesis? Again, not the right question. You really have to say, how much more likely is it if my hypothesis is true than if it’s not. So it’s funny, and every other way of asking the question isn’t quite right.
Like what you’re essentially doing here is you’re taking the Bayesian mathematics, converting it to English, and it’s literally that question. There’s one question of evidence, which to me blows my mind that it’s not really known.
Annie: So that brings up another thing that I think about, right, which is, when you say like, oh, would my hypothesis explain this evidence, right? Like, I love Duncan Watts’s book, Everything is Obvious: Once You Know the Answer. And we forget that on the look back, you can kind of explain almost anything. This is part of the problem, right? So take my stupid engineer example, right? If all I told you was, Hey, what if I told you that great engineers are disagreeable? Like, give me an explanation for that. It would be really easy for you to explain it. Right? Like, oh, well, they’re looking for different ways to do things and they’re not necessarily going with the status quo or whatever. You could come up with all sorts of reasons for that.
Spencer: There’s a social scientist who used to present his findings by just saying, here’s what we found, and people are always like, yeah, that makes sense. That’s intuitive. Then he’s like, no, it’s not intuitive. And so what he did is he switched to, instead of giving them the answer, he said, here’s what we studied. I want you all to now take a moment to predict what we’re going to find. And then a lot of times people predicted wrong, right? So you actually have to make them guess before you tell them the answer. Otherwise they’re like, oh, that’s obvious.
Annie: The next question I have is a little bit of a wrap up. You obviously are an incredible friend to the Alliance for Decision Education. What do you think the impact on society will be when the Alliance succeeds in its mission to ensure Decision Education is actually part of every K through 12 student’s learning experience?
Spencer: I don’t think people realize the extent to which decision-making just imbues everything in your life and compounds, right? Making good decisions lead to future opportunities, and then making better decisions in those leads to even more opportunities. So that’s at the individual level. I mean, I think there’s profound implications to people making better decisions, but then there’s also huge societal implications. I mean, a population that makes better decisions, elects better leaders, has better people in charge of nonprofits, has better people in charge of companies. And so I just think there’s huge flow-through effects that benefit everyone. And then I think as we become more and more powerful with new technology, it actually becomes increasingly pressing, where like when we have the technology to destroy ourselves or to reshape the world, we better make good decisions or we’re going to drive ourselves off a cliff. So yeah, I’m a big believer that decision-making is critical to a great society.
Annie: Yeah, I’m gonna bottle that answer so I can repeat that to people for sure. Um, if listeners want to go online and learn more about your work or follow you on social media, tell me where you would like to send them.
Spencer: So if you enjoy this conversation, I’d really love it if you check out my podcast, it’s called the Clearer Thinking podcast. You can find it on Spotify or Apple, uh, Podcasts or any other podcasting app. So it’s Clearer Thinking with Spencer Greenberg. I have a personal site where I have, um, hundreds of essays. It’s spencergreenberg.com or you can check out our work at ClearerThinking.org.
Annie: Personally, I get your little newsletter. I got one yesterday actually in my email and it’s, you know, I immediately open it and read it every single time I get it. So.
Spencer: Oh, thanks.
Annie: I cannot recommend that, you know, highly enough. So subscribe. It will actually appear in your email. You will get it on the regular and it will make actually a big difference in your life. I feel like it makes a very big difference in mine.
As always, uh, you’ll find show notes, transcript, be able to find any books, articles mentioned today on the Alliance site. So please go there to find that. And Spencer, I could not thank you more for being in this conversation with me. This has been a total treat for me. I’m so excited that you agreed to come and explore decision-making with me. And I’m excited to have been on the other end of the conversation this time.
Spencer: No, thank you so much, Annie. I really enjoyed it.
Show notes
Books
Resources
Website
Podcasts
Social Media
Published September 10, 2025