Somebody recently asked me how I come up with ideas for experiments? And I decided to try and answer this in a more general way. By the way, I’m talking to you from a small hotel in Porto, Portugal. When I was first invited to the conference I’m in, I thought, I’d come a couple of days early, I’d walk around. Turns out it’s too hot. And I’m ending up enjoying this little room and working from it. Okay, but today is not about my inability to plan and the weather. It’s about how I think about ideas for experiments.
So, I think about ideas for experiments a little bit like art. If you think about somebody who is drawing or painting, they’re usually not representing the whole thing. They’re representing the essence of it. And they have to convince us, the viewer, that they have captured the essence. And I think about experiments in the same way. They are field experiments, but regular experiments are like that. So maybe I do an experiment on how much people are willing to pay for annoying sounds and how that changes over time. Or do people choose a chocolate that costs three pennies or one penny? Or what about building Bionicles? When I do these experiments, it’s not about the specifics. Like I would hate to say, “I have a conclusion about chocolate for one penny” or “I have a conclusion about building Bionicles.” No, we’re trying to create an idea and represent that idea in a way that the audience, hopefully you who are reading about it, will be convinced that we have captured the essence.
So very much like art, I look at the problem in the world. People seem not to make a lot of progress with something. People seem to not take their medication. People seem to not save enough. And I try to distill it down to some principles, and I try to create an experiment to test those principles. But between me, who is designing the experiment, and you who are reading about the experiment, I need to convince you that what I represent, and I leave some things out, but an experiment—not field experiments—regular experiments abstracts things, and only some of the information remains, some of it is outside the system.
Even if I say, “This is like a prisoner dilemma game,” it’s only a very small part of the whole system, but I have to convince you that it is the essence. So that’s what I think is the real nugget—is to look at the world and ask yourself, “What is the essence here?” And once you convince yourself that you know what the essence is, then you decide to prove that. Let me change the essence one way or another and see how the behavior changes.
So for example, I can take an experiment, let’s take the Bionicle example, and I can get people to build Bionicles and they stand or they build Bionicles and I take them apart and I basically say that there’s a difference between those two conditions. In one, people don’t feel their work being destroyed in front of their eyes. In the other one, they feel that it’s futile because they don’t see any kind of progress and let’s see how that changes.
That’s basically the story. Okay, taking reality, abstracting, focusing on the essence, abstracting it and then playing with the essence. And I think it’s a very good skill. It’s a very good skill because it tries to help us understand the essence of something, the social science essence of a situation.
Okay, so this is how we come up with experiments.
