Lecture 3: Social Experiments: Why and How?

• Download English-US transcript (PDF)

The following content is provided under a Creative Commons license. Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free. To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.

PROFESSOR: So I think we have a bit of background in mind. So now I'm going to summarize this discussion, and start asking about how do we go about answering some of the questions that have come about? So at the end of last lecture, we were left with our S-shaped curve, and saying that whether or not there would be a poverty trap was really dependent on the shape of the curve that relates income today to income tomorrow.

And we gave sent out examples where it could go one way or the other, or it could be that its S shape, it could be that it's increasing but inverted L-shape. And then one of you asked the question, well, how do we know? How do we even get about-- how do we get to know the shape of the curve?

This is not a question that is going to be solved theoretically. The debate we just had a moment ago suggested it won't be solved theoretically. Because, for example, take the question of whether if I give you a bed net for free today, you're less likely to buy the next one tomorrow. It depends on whether the entitlement effect is more important than the loaning effect.

And how will I know that? There is no way for me to know it without looking at the data. And the question is, what's a good way to look at the data? And today, we're going to look at one such way, which is a way which is increasingly often used by development economists, by policy makers, and which is the main approach that we are using in the Poverty Action Lab, which is the idea of on randomized evaluation.

So we'll start with our malaria example. I'm going to use the malaria example, because we had the debate, so we know the question we're asking. I'm going to use that as our example. But, of course, the point I'm making now are important methodological points as well.

So what are we talking about? We're talking about a pretty important thing. Almost 900,000 people die of malaria every year, give and take, most of them in Africa, most of them under five. That's the biggest cause of under-five mortality in the world.

Incidentally, you want to take every single statistic with a big pinch of salt, because it's not that all the malaria cases are recorded-- a lot of people who are not really diagnosed of malaria and are told to be dead of malaria, so it could be a gross overestimate or a gross underestimate. But I think it's fair to say it's a lot of people, one way or the other.

As we discussed, malaria is transmitted by mosquitoes who carry the parasite from an infected person to an uninfected person, mainly at dusk. So sleeping under a bed net, particularly a bed net which has insecticide woven into the fiber of the bed net is an effective way to prevent mosquito bites, and therefore to prevent the transmission of malaria.

And one thing that no one said, which was important, is that there is two effects of sleeping under a bed net. If you already don't have malaria, you're less likely to catch it, because the mosquitoes can't bite you. So this is a direct benefit that benefits you.

So when making the decision to buy or not buy a bed net when making the decision to use the bed net or not to use it, you are going to take this direct benefit into account, right? But then there is another benefit which is the externality, which is an indirect effect, which is, if you already have malaria, someone else is less likely to get it because they don't first bite you and then bite them.

And this benefit, unless you're an incredibly socially-minded person, is a benefit you don't take into account when buying your bed net. Because you don't care. It's not you. It's someone else. But as society, we are trying to maybe adjust the individual behavior to the social benefit.

There's one more thing which is important for us to keep in mind, which is this externality is not linear. So it could be that the externality is linear, which is one more person who sleeps under a bed net, say half a fewer percent is infected. So it could be like the more bed nets you have, the less malaria you have, but in a disproportional way.

But it turns out that it's not proportional. The reason why it's not proportionally is because once about half the population sleeps under the bed net, you have something called hard immunity, which is that the mosquitoes don't have time to bite someone else before either the mosquito or the parasite dies. So if you manage to cover about half of your population, you don't even need to carry the other half. I mean, it's better, but they already benefit almost as much.

So the externality benefits are like this, if you want. They're like, first they're linear, and then at some point, they have a big jump in the externality benefits. And then again, going from 80% to 85%, maybe less.

So that means that this would need to be reminding you of the S-shaped thing, which is there will be a distinct advantage in terms of cost per malaria averted, the benefit of going from a coverage of 25% to 30% of the population are going to be lower than the benefit of going, say, from 45% to 50%. Because when you go from 45% to 50%, you're in the zone where every extra bed net is really helping a huge deal. So that is going to be relevant when we do the cost benefit.

So that's the case for bed nets. So now, if we were traditional economists, we would make a traditional economic argument, which is to say, there is an externality. Whenever there is an externality, we want to try to do use prices or taxes to align the private incentive with the public incentive.

That's why, for example, we would like to have a congestion charge in London, or we have a congestion charge in London to make it expensive to drive your car in London, in the center of London, because when you drive your car in the center of London, that's a pain for everybody else. That could be one of the reasons why we have taxes on cigarettes, that smokers impose an externality on the rest of the people, here both because of secondhand smoking, and because they get sick a lot. So you would think that society has a right to impose a tax on them to try and discourage them from smoking.

So by using prices and taxes is where you can try and align the private incentive to the social benefit. Yes?

AUDIENCE: Can you do community incentives, like you tell the community if everyone uses their bed nets as bed nets, you'll give the community a certain number of fishing nets so that people will encourage or force other people to use bed nets?

ESTHER DUFLO: Right. So one way is the prices, but another way could be to have community incentives. Indonesia has a program where they are trying to give community incentive so if, say 90% percent of the community is immunized, and x percent to the community is in school, then they get some amount of money to build a road, or an irrigation project, or something like that.

AUDIENCE: I'm just curious. How much is given in foreign aid to fight malaria?

ESTHER DUFLO: That's an excellent-- how much is given in foreign aid to fight malaria. It's an excellent question, the answer to which I don't know off the top of my head, but it's findable.

AUDIENCE: Because I was going to ask if the cost of providing a country with bed nets is significantly less than the cost of giving [INAUDIBLE] money to fight malaria, then I don't even see why it's as big of a debate as it is. Clearly, they should figure out what's the most effective way to distribute the bed nets, make sure [INAUDIBLE]. But the option of not giving them just wouldn't make as much sense, in terms of cost [INAUDIBLE].

ESTHER DUFLO: Oh, I see. So providing bed nets would be part of the budget of fighting malaria. So your question is that, if we're already spending so much money on medicine against malaria, we would really be willing to spend a lot of money on bed nets. You're exactly right. And I don't think the debate is that we want to try and save money on the bed nets. I think the debate is always about what's the best way to get the money out.

So this would argue, so the fact that the private incentive to get your bed net is low, would argue for subsidizing the bed net, but only if the private incentives are not large enough that people are already not buying your bed net. So suppose, for example, that people hated being sick, which, presumably, they do. Then, they would get the bed net anyway. You would not need to subsidize the bed net.

So there would not be an effect of the price, because the demand would be very high at any price. So the fact that there is an externality justifies subsidizing, to the extent that the private incentives are too small for people to want to get them themselves.

Another traditional economics argument would be the learning argument that was made earlier, which is that trying your bed net once might give you the experience of it, and once you knew how it works, and you know how well it protects your children, you might be more likely to get the next one. And then there is an argument that mixes the two, which is another externality argument, which is, if try a bed net, and I realize that it protects my children and it's really not so unpleasant, then maybe my neighbors will also see my bed net, and say, oh, isn't this cool. We want also a bed net. So there is an externality in my trying, potentially. Yeah?

AUDIENCE: But if you were charged for bed nets and the borrower that could, again, come in to have the bed nets [INAUDIBLE]?

ESTHER DUFLO: Yes. So eventually, since there are those contingent benefits, someone who is very rational might feel, already more than 50% bed nets in this community. I don't need to get the next one. So the private incentives diminish as the coverage increases, which makes it more necessary either to subsidize it even more, or to tax people if they don't get a bed net.

So this is an argument that is made for immunization. So in this country, immunization is compulsory for a range of diseases. Except if you really don't want to do it because you have some religious or whatever reasons for not wanting to be immunized, then you can say, sorry, I refuse the immunization.

And people worry that as diseases such as polio, or diphtheria, or pertussis, these types of diseases become less frequent, people don't have the fear of getting those diseases anymore, and they want to free ride on the rest of the community. And that's, of course, bad, because being 100% immunized ensures that the disease goes away. As soon as a few people are not immunized, it could come back. And so there is this tension between once almost everyone's immunized, your private incentive to do it are almost zero. So you might need powerful inducement to get to do it.

So those are the traditional economic arguments. These are your traditional public finance-- let's use the prices. That indicates that we should subsidize the bed nets. And by the way, there is no reason to stop at zero in principle. We could pay people to use bed nets, for that matter, if this was the way to get the optimal coverage in the community. Yeah?

AUDIENCE: So one consideration that comes to mind is if there's a segment of the population that is really, really poor and cannot afford to pay any amount for the bed nets, whether they're subsidized or not, those people are also the least likely to be able to pay for health care should they get malaria. So they would have the greatest risk for everyone else and for themselves. So wouldn't you want to try to provide free bed nets for that set of people, no matter what?

ESTHER DUFLO: Right, so that's another consideration that you might want to include, a redistribution consideration, which you might feel that people who are particularly poor might have a very high private evaluation of the bed nets, but might be credit constrained and not be able to get the bed net. And those are the same people who will also not be able to cure themselves of malaria. So here, that gives you another argument for subsidizing bed nets, which is not an argument in terms of disease eradication, but an argument in terms of redistribution between people.

So those are traditional arguments. They are taken, say, by Jeff Sachs, or by the WHO, and say, we should subsidize bed net to a large extent, or potentially, up to giving them away for free. And then we have the less conventional economics, which we discussed, which has the argument that goes in the opposite direction, or the psychological sunk cost effect, which is, you have to pay for something to really appreciate its value.

There is a selection effect which is, if you give things away for free, even people who don't want it or who don't want it for its legitimate use are going to come and get it. They're going to waste a lot of bed nets. And finally, the entitlement effect, which counters the learning effect, which is, once you get something for free, then you expect to get it for free in the future.

And it might even have contingent things, which is you're not going to want to get chlorine to purify your water, unless someone is giving it to you. You're not going to want to send your kids to school unless they get a school meal, other things like that, which find that you become dependent. Yeah?

AUDIENCE: I have a question. Couldn't you argue that it's not sustainable, [INAUDIBLE] long-term policy and exit strategies, because if you're [INAUDIBLE] subsidizing and giving away bed nets, you might crowd out the private sector. And you won't [INAUDIBLE] your desired outcome, which is for people to be using bed nets.

ESTHER DUFLO: So in the traditional public finance argument, why would you ever want to stop to subsidize the bed net?

AUDIENCE: Want to not subsidize--

ESTHER DUFLO: Yeah. Why would you ever want to stop, given those arguments?

AUDIENCE: To create a market?

ESTHER DUFLO: Why would you? Take, for example, immunization in the US. Immunization is subsidized and compulsory. Is there any plan to make immunization not subsidized or compulsory?

AUDIENCE: That's true.

ESTHER DUFLO: There's no such plan. This kind of argument would tell you that you want to subsidize forever, because the private gains and the social gains are not aligned for a fundamental reason, which is my being protected by a bed net protects you as well, and that's never going to change. So in that sense, you're not doing a policy for the short run. You're doing this policy for the long run.

Now, again, we had this discussion earlier, which is the questions of who provides the bed net, whether they can be provided locally instead of imported, are all very legitimate questions, but in a sense, unrelated to the price you're going to charge. So that's an excellent question, because it comes back all the time, which is to say, oh, but don't we want to exit?

Just think, whenever you want to ask yourself this question about sustainability, ask yourself, why would I want to exit? Maybe I don't. Maybe it is actually socially efficient to subsidize something for eternity, because as Ben was saying, you save on the cost of curing people from malaria.

So the argument that goes against giving people for free are these arguments that giving bed nets for free or even subsidizing them is counterproductive, in terms of distributing bed nets. It's counterproductive because of the sunk cost effect, because of the selection effect, or because this entitlement effect that then nobody is ever going to want to buy anything, either a bed net or something else.

And these elements are very reasonable. Both sides of the arguments are, in principle, very reasonable. And they might apply here. We just don't know. It's a very heated policy debate. The laws that are involved in bed nets are nowhere near the laws that are involved in oil or anything, but it's still a fair amount of dollars, so people like those. So there is a lot of debate.

So Jeff Sachs of the WHO relies on the convention economics argument to say you should give the bed nets for free. Then you have people like Bill Easterly or Dambisa Moyo, who are making the argument based on this entitlement effect, sunk cost effect, the development of market, to say we shouldn't give them for free. And both arguments tell you, in a sense, we know the economics, and the answer is obvious. There is no reason to find out.

And I think there is value in both type of arguments that they are making. But there is no value in this last one. Because since there is value in both arguments, we don't know the answer. The answer depends on the balance of how they turn out in the specific application of bed nets.

So the true question we want to ask is the extent to which we should subsidize bed nets. And for that, what do we need to know? Well, first we need to know whether or not bed nets are really price elastic. So are we losing a lot of coverage in terms of people acquiring bed nets, if we start selling them instead of giving them for free.

Second question we need to ask is the immediate effect on use. So is it the case that people who pay for bed nets are more likely to use it than people who got it for free? Third question is the longer-term effect. Will it wreck markets?

And for that, that's the entitlement effect. For people who get it for free, in the future, are they more or less likely to buy one? And for people who are around them in the community-- this is something we didn't discuss much, but if I'm around you in the community, and I see you having a free bed net, it could affect me in one of two ways.

I could say, first of all, oh, she has a bed net. That's really nice. Her kids are doing well. I'm going to get one too. I could say, oh, she got a free bed net. Let me wait until I get-- I was going to buy a bed net, but now I'm going to wait until I get a free bed net. And so I wasn't going to sell a bed net, and now I'm going to have give it for free. So that's wasted money.

So again, we don't know which way it cuts. We have to look in practice. And those questions are pretty well-defined questions. They are empirical questions we could principally bring an answer to.

There are a lot of anecdotes. They are this one, which is bed nets used as fishing nets. That's a picture that was in the Malaria Journal, so it's all peer reviewed and all that. So this is the Easterly Dambisa Moyo argument, that you are going to see the used as a fishing net, or wedding veils, or what have you.

And then this is the other argument, of the other anecdote of the bed net being nicely hanged in the room, and the child, the healthy child who slept in there under the bed net. So that's not too helpful. So we want to go further that the anecdote.

There are many anecdotes, and they cut both ways. How do we go further than the anecdote? So the first thing we could do-- so suppose you were given this job. I tell you, find out for me whether people who get their bed nets for free are more likely to use it as fishing nets or more likely to use them as bed nets. And also, find out whether if I distribute the bed nets for free, I distribute many more of them or I distribute the same amount. What would your first response to what you should go out and do?

AUDIENCE: Yeah? I would find two similar communities, and then [INAUDIBLE] give bed nets would be one, and maybe help someone start a small business of selling bed nets in the other.

ESTHER DUFLO: So you would take two similar communities, and in once case, give bed nets for free, and in one case, start to sell them, maybe by finding a small business, or maybe importing them to the village. What would be an even--- we'll go back to that. What would be an even easier, even more immediate way to do it that we could think of to start with, if we needed to do this quickly? Yes?

AUDIENCE: Just look at the communities where it is already.

ESTHER DUFLO: Right. You would look at a community where this is already in place, and you would say, well, in this community, the bed nets are given for free, and in this community, the bed nets are sold. Let me try my thing. Yeah. In this community, the bed nets are given for free. In this community, the bed nets are sold. First question, do I see more bed nets where the bed nets are not given for free? Second question, do I see the bed nets more likely to be used as fishing nets in those communities? So that's, I think, what most people would-- that would be your initial instinct.

The problem you would have if you did that, as opposed to-- sorry, can you remind of your name?

AUDIENCE: [? Sif. ?]

ESTHER DUFLO: [? Sif. ?] The problem to do that, as opposed to do what [? Sif ?] suggested, is to know why some community happened to be the one that got them for free and some community happened not to have the free distribution. So let's think of one reason why there might be a free-- if you were to start a free bed net distribution program somewhere, where would you start it? You have a country and you want to start a free bed net distribution, what would you pick? Yes?

AUDIENCE: If there was a particularly bad increase, or something.

ESTHER DUFLO: Right. You would try to pick a place where there is a lot of malaria, presumably, or maybe a place where people are very poor. So let's work with the example where you would pick, to start your free bed net distribution, the region where there is a lot of malaria. It would make sense.

So if that's the case that free bed nets are distributed where malaria is a huge problem, then in these places, even if they had to pay for the bed net, they might have purchased them more willingly than in places where malaria is not such a big problem anyway. Because the private benefits of having a bed net is higher in those [INAUDIBLE].

So this is what we call the counter-factual, which is what would have happened in the place where the bed net got distributed for free if they had actually been sold. So let's look at it in the form of a simple graph. So suppose we have two-- we are going to do the argument for purchase, and the same kind of discussion we could have for use.

So suppose we have two regions, a high-malaria region and a low-malaria region. And suppose that God told us this is how many people would buy a bed net, the fraction of people who would buy a bed net in the high-malaria region and the low-malaria region, if the bed net was not subsidized. And this dot is higher, because since malaria is more of a problems, people are more willing to buy bed nets. OK? So that's what we would have if bed nets were expensive.

And if bed nets were free, we would have those ones. So compared to the previous dots, they are higher. Why are they higher in both regions?

AUDIENCE: It's because they're free.

ESTHER DUFLO: Because they're free, so you're more likely to get something free than to have to pay for it. But they are still higher in the high-malaria region, because why would you get a bed net in a region where malaria is not a problem? You really don't need it. If you don't have a fishing boat, you don't need a fishing net either. So they are still higher in a high-malaria region then in a low-malaria region when bed nets are free. So this is what God would have given you.

But sadly, God didn't give you that. God gave you either this point or that point. What you observe is the purchase of bed nets in the area where they are given for free, and the purchase of bed nets in the area where they are expensive.

And those regions are different because one is a high-malaria region and one is a low-malaria region. So what you observe is the demand for bed nets when they are free in a high-malaria region, and the demand for bed nets when they're expense in the low-malaria region. And what you don't see is those guys. What you see is this and this.

What is the true effect of price on purchase in the high-malaria region? Where do I find it on the graph? That's the vertical thing. That's the true effect of free versus expensive in the high-malaria region. And this is the true effect of free versus expensive in the low-malaria region.

So this is the true effect. That's what I'm looking for. Sadly, I don't have this, and I don't have this. I'm not in very good shape. What I do have is this and this. So if I just did to comparison that we started with, we would compare the high-malaria region purchase when it's free to the low-malaria purchase region when it's expensive. Take the difference. That would be our effect.

So is that the true effect? Is it biased upwards, or is it biased downwards? Is that too big, too small, or the right size?

AUDIENCE: Too big.

ESTHER DUFLO: Too big. And we can see it here, which we can re-put. We can say, well, suppose we had what God would have given all the numbers. In the low-malaria region, this is the effect of price and purchase. And the rest is the best. In the high-malaria region, this is the true effect of price and purchase, and the rest is the bias.

So if we knew this and this, we could calculate the effect, and we calculate the bias that we would get if we compare this to this. But suddenly, we don't have those. So what this is saying, in other words, is that when we compare purchase when it's free to purchase when it's expensive, in the high- and low-malaria region, we combine two effects.

We combine the effect that even at the same price, the purchase would have been different. So the counter-factual would have been different-- those are this versus this, and this versus this-- and the fact that there is actually an effect. So that's why it's not a very good answer for the question we're asking.

This we did for only two points, but we would have the same thing if we had many price points. So suppose this is the observed demand at various prices-- when it's free, when it's 10 shilling, when it's 20 shilling, when it's 30 shilling. That's the observed demand.

So someone could say, well, I'm going to trace a line here, and that's going to give you my elasticity of purchase with respect to the price of the good. Now, let's say if God had given us the observed demand, the demand we would observe in the region with the free bed net if the bed net were not free, that's the demand we would observe in this highest-malaria region.

It also decreases with price, but less than we see in reality. And why does it decrease less than we see in the data, the yellow dots? So the yellow dots are what we would see in the high-price region-- so in the free region, what we would see in the zero price region, if the price were in fact 10, 20, 30, 40. So why is it flatter than the observed curve? Yes?

AUDIENCE: Because it was free, [INAUDIBLE].

ESTHER DUFLO: So this is what is free? This is a bit more. This is the demand when it's free. It's very high. This is in the next region, where it costs 10 shilling. It's a bit lower.

This point is telling us, this is how many bed nets the people in this region would purchase if the price was, in fact, 10 shilling instead of being free. So this and this and this, we don't observe. But we just say, let's say I'm telling you this is God told you what it would be.

The reason why it is higher is because the region where it is free is a region where people need the malaria nets a lot. So even if they had pay 10 shilling, they would still buy them. And even if they had pay 20 shilling, they would still buy them. And even if they had to pay 30 shilling, they would still buy them, although less, because of course they are sensitive to the price.

But the difference between the yellow point and the red point is the yellow point is just the effect of the price, whereas the red point also has the effect of the fact that you're more likely to have to pay for bed nets in places where the malaria is not such a big problem. So the red dot, which is what we observe in reality, combines the effect of the price and the fact that the regions where people have to pay are different. Yeah, Ben?

AUDIENCE: The x-axis is price?

ESTHER DUFLO: The x-axis is price. Yeah, sorry. I should have-- it's like this, 0, 10, 20, 30.

AUDIENCE: It's still not obvious to me why if it's free, if you go from zero to, let's say, a million, why the demand for free bed nets would change [INAUDIBLE]?

ESTHER DUFLO: No, it's not the demand for free bed nets. This is always going to be the demand at this price-- 0, 10, 20. So it's still declining, you see?

AUDIENCE: No, I'm saying for free bed nets.

ESTHER DUFLO: Oh. For free, it's pretty high.

AUDIENCE: Why does it change at all, if you go from price 0 to 30--

ESTHER DUFLO: Oh, you're wondering about this one. You're wondering about this point. You're wondering, in the low-malaria region, if I give you a bed net for free, why don't you get it?

AUDIENCE: Well, no. Because the x-axis is high instances of malaria, low instances of malaria.

ESTHER DUFLO: Yeah.

AUDIENCE: You switched the price. I'm confused why changing how much a free bed nets costs would decrease the amount of purchases for free bed nets.

ESTHER DUFLO: So what is a little tricky is that there are two things in the axis. There is the fact that this is a high-malaria region, and the fact that it's a low-malaria region. And it turns out-- but this is what is very important-- it turns out that the price is related to the incidence of malaria.

So here, for example, in the two-region case, I have given the bed net for free in the region where the malaria was high. And the reason is because if I'm a good policy maker and I don't have that much money, that's what I will do. I will first give the bed net for free in the region where malaria is high. And I will ask people to pay for the bed net in the region where malaria is low.

So what I observe in reality is the demand for free bed nets in the high-malaria region and the demand for expensive bed nets in the low-malaria region. So if I compare the difference between the two, and if I call this-- I'm combining two effects. I'm combining the effect of the price, which is, given the same region, people are more likely to get a bed net for free than to pay for it, and the fact that people, given that bed nets are more useful in a region where there is a lot of malaria than in a region where there is little malaria. People are more likely to get bed nets at any price-- the red dots are for the same price-- at any price, people will be more likely to get bed nets in the high-malaria region than in the low-malaria region.

So if you compare within a line the yellow to the red dot, you get the effect of the price. If you compare across columns within the same color, you get the effect of being in high- versus low-malaria region. And you see they tend to go in the same direction, which is what creates your bias.

All right, do you get the two-region, two-prices example? If you get the two-region, two-prices example, then this one is the same, but with four prices. Now we have four regions-- a region where we give them for free, a region where we give them for 10 shilling, a region where we give them for 20 shilling, a region where we give them for 30 shilling.

And this is what I observe in reality, a very steep slope. At 30, almost no one buys it. But this is due, again, to two things. One is that that's expensive. So even if malaria was a real problem, people would be less likely to buy them for 30. And, in addition, those regions where people had to pay a lot of money for malaria is precisely because malaria is not such a big problem in those regions.

So in order to decompose those two effects, what I'm doing is that I'm playing God and I'm telling you, this is what the demand at various prices would have been in this first region, in the zero region, in the high-malaria region. So this I don't see in reality. In the world, I don't see that, but this is what it would have been.

And it's a little flatter than this one because this is like the bias. This is saying, these are places which don't really need malaria, and this is the effect of prices. The yellow one is a true effect of prices, and the vertical lines are the bias that are added by the fact that prices are expensive in places where a bed net is not very useful because malaria is not so much of a problem. Yeah?

AUDIENCE: I have a question. Why is there not a bias based on your environment and how much malaria there is when they're for free?

ESTHER DUFLO: Well, when they're for free, we observe the true demand at free. So that happens to be the same point, which is, I'm giving you what I observe. And the free region is the demand when it's free. What I would like to know, then, is what would the demand in the free region be if, in fact, it had been 10 shilling?

So this is this counter-factual, which is, there is something we don't observe, which is precisely what we need to compare to the region where, in fact, it is free. Ask as many questions as you have. The more questions, the better. Yeah?

AUDIENCE: I was wondering why [INAUDIBLE] more likely to go from zero to 10 shillings. Do you see that the [INAUDIBLE] decrease in the [INAUDIBLE]

ESTHER DUFLO: So you have-- so what is your name?

AUDIENCE: Katja.

ESTHER DUFLO: Again?

AUDIENCE: Katja.

ESTHER DUFLO: Katja. Katja has a question, which is, she doesn't believe that the elasticity is the way I drew it. You think that it's more like steep and then flatter?

AUDIENCE: Yeah.

ESTHER DUFLO: Possibly. That's the way I made that up. I consider that zero and 10 is kind of similar, but that has no reason to be true. And substantially, you're absolutely right. It could be that, in fact, there is a big drop in demand. But then the red point would probably be even lower. Yeah?

AUDIENCE: Why were the experiments conducted so that you only get the orange data points? Why did you not mix up different communities that have had malaria and [INAUDIBLE] malaria [? for each-- ?]

ESTHER DUFLO: That's an excellent point. So this is not supposed to have come from an experiment. This is the real world. [INAUDIBLE] a real world outside of an experiment. And outside of an experiment in the real world, it is likely-- it doesn't have to be the case, but it's likely that policy maker will make the bed net available for free in high-malaria region, because it makes sense for them.

But your question exactly brings to what the answer to the problem is. So we have a problem, which is that we observe the low prices in places where the need is high, and high prices in the regions where they need is low, so that leads to this bias. And the solution is exactly what you suggested, which is, let's have variation in prices in regions where the need is the same. Yes?

AUDIENCE: So on the x-axis, are you exactly correlating the price of the bed nets with the incidence of malaria, or--

ESTHER DUFLO: So what I'm assuming, I'm telling you this doesn't have to be true. And the problem is we'll never know, because we won't observe both in the real world. But I'm telling you this happens to be like that in this particular fictional world that I constructed, where this was like that.

AUDIENCE: But if you go along the x-axis to the right, you mean that bed nets get more expensive and malaria incidence goes up?

ESTHER DUFLO: Goes down.

AUDIENCE: I'm sorry, goes down?

ESTHER DUFLO: Exactly.

AUDIENCE: OK.

ESTHER DUFLO: I constructed it that way. You could construct the opposite example, and you could make up a story where the richer regions have more political power, and they also have less malaria. So the richer region will get the free bed nets, and they will also have less malaria. In that case, where would the observed compared to the true elasticity, where would the red dot be, if, in fact, the correlation was the other way?

AUDIENCE: They would go the other way?

ESTHER DUFLO: They would go the other way. So the red dot would be above, because the bed nets are more expensive where they are more needed.

And your question brings in an essential question, which is, sadly, we can't know. We can know some stuff, which is we can, for example, measure malaria prevalence. But we never observe the counter-factual. So either we observe the demand, where the price is high, in the places where it's high, or we observe the demand where the price is low in the places where it's low.

And we never get to see the demand where the price is low in the places where it's high. That's the missing data point. That's why this is what we are missing. In other words, we don't see those yellow points. We don't see what the demand would have been in a zero price region if the price, in fact, had been 10, 20, 30. We just don't know.

So the solution is what you suggested, which is to say, why don't we take one region and try to vary the prices in those regions? Or in fact, we could take two regions, a high-malaria region and a low-malaria region, and vary the prices in those two regions. And this is where the word experiment comes up. This is where the experiment is, which is the way you would do it to be completely sure that there is no difference between the places or the people which got the bed net for free, and the places or people which had to pay for it, is you would randomly assign the price of the bed net to different regions or two different people.

So this is what is done, for example, in medicine when you want when you want to find out the effectiveness of a new drug. You don't give the new drug to people who need it the most. Otherwise, you would find that the drug killed a lot of people.

You take some part of people who need the drug, and you randomly assign some people to take the drug and some people to get the placebo. You might also take a sample of people who need the drug less, and randomly assign the drug within them, to find out side effect if you misprescribe. But you wouldn't give the drug to people who need it, and not the drug to people who don't need it, otherwise all of the drugs would be sure to kill a lot of people.

So most evaluation of social policy is trying to estimate the effect of policies by doing these kind of comparisons, this point to this point. And that creates this bias situation. And in fact, you can try and make more progress on this question by creating experiments where you fix the region, or you fix the environment, and you randomly assign the prices. The randomization ensures that there is nothing systematically different between the people who got the free bed net and people who didn't get it.

In other words, there is no-- if we go back to this graph, for example, the places that got the free bed net and the places that had to pay for the bed net, if they had gotten it for free, they would be at the same level. And if they had pay for it, they would be at the same level. There is now no difference between the region who paid for the net and the region who didn't pay for the net.

One point before going into the results of what Cohen, and then Pascaline Dupas found for the bed net is that of course, this is not true if it's just one village and another village, like Steve suggested. Why is it, if we took only one village, and in one case we make free, one case we make not free-- we can flip a coin, and then we compare-- why is that problematic? Two villages-- why is two villages not enough?

AUDIENCE: Because you can't make sure that everything is the same [INAUDIBLE] like one village might have more [INAUDIBLE] actually care about each other, or one village might end up having a really rainy season, so all the mosquitoes happen to swamp around it.

ESTHER DUFLO: Exactly. Two villages, even if there is nothing systematically different, because you've thrown a coin, so you didn't choose one village rather than the other for their characteristic to give the bed net for free or to ask people to pay for them. Even in that case, there are things that are different across villages that you can't fully control.

So you need the samples to be large enough such that all the other differences between people or between regions average out, so that even though you do not observe what the demand for bed net would have been in the free region when actually you ask people to pay for it, you would not observe that. But you can be confident that what the demand you observe in the other region where it was free gives you the right counter-factual. And for that, the sample needs to be large enough so that all the noise and the idiosyncrasies get swamped.

So let's look at two sets of experiments. Both were Pascaline Dupas, who is a professor at UCLA, was is involved in both of them. The first one was conducted with Jessica Cohen. In and around this place, which is called Busia in Kenya. This hotel, you can see that the beginning of the hotel is called "Blue York," where I spent a lot of time.

And so what did they do? They randomly chose clinics, prenatal clinics, they gave them nets to give to people who come for their prenatal exam, either at zero or 20 shilling or 30 shilling or 40 shilling. So the unit of randomization is the clinic.

And then they compare how many bed nets they sold, and whether the bed net got used, and therefore the effective coverage in those clinics. And this is what they find. So the little vertical line here is the confidence interval. So it's taking into account all the noise that exists in the data.

And so what is the conclusion from this graph, at free, 10 shilling, 20 shilling, 40 shilling? This is the number of nets sold per week, I think.

AUDIENCE: Does that indicate if the customers actually use the bed net?

ESTHER DUFLO: No. So far, this is the first look at, which is just how many bed nets were given out. Yeah?

AUDIENCE: [INAUDIBLE] the demand curves downward.

ESTHER DUFLO: The demand curves downward, and it's quite steep, right? Basically, at 40 shilling, which is still heavily subsidized, because the full price of a bed net is more like 80 shilling, you distribute only about 10 bed nets per week, versus 40 per week, when you give them for free. So it goes down quite steeply. And I remember, because they were randomly selected, we know that it is due to the price, and not due to differences across those regions. Yeah?

AUDIENCE: Yeah. That's funny, because the error bars are pretty big. Like, if you if you looked at the lower error, versus the higher error, it's [INAUDIBLE].

ESTHER DUFLO: So you take this one compared to this one. That's the confidence interval. So we are 95% confident that the truth is somewhere in between there. That the 95 confidence interval, [INAUDIBLE]. Right? So you can know that this is systematically different from this.

You're right that this and this, the confidence interval overlaps. And in fact, the numbers are quite similar. This is why I don't answer someone who was asking why didn't I draw the elasticity that way I'd drawn it that way, because I already knew the answer to the question. I was ahead of the game, slightly.

So zero to 10 is not such a big difference. Maybe 10 is not that expensive. But 40 is definitely different from free. You're right. Certainly we would like those bars to be smaller. If we had more observations, they would be. They would be even more precise. Yes?

AUDIENCE: Were the clinics far apart from each other? It could be, say, I'd go to the clinic where I'd have to pay 20 shillings [INAUDIBLE]

ESTHER DUFLO: Right. That's a very good point. The people shift from one clinic to the other. So they couldn't really do that, because the clinics were quite far away.

On the other hand, what they did find is a lot more people coming to the clinic, presumably to get the bed net, which is a good thing, because it cost subsidizes other services that you get in the clinic, in particular a test for HIV. And if the mother is HIV positive, then you can give them Nevirapine just before the delivery to prevent the transmission to the [INAUDIBLE]. So that's another story, which is a reason why you might want to distribute bed nets if people like them.

Now, this was your question, which is, but are people are using them? And so far, the usage, you can ask people what they tell you. But they could lie. Or you can try and go to their home and see whether you see the bed net. And here, what's the conclusion?

So this is conditional, having gotten a bed net. So what they did that they took the register. They saw who got a bed net, and they went to their home and said, what are you doing with the bed net? Is it a fishing net or are you using it for a bed net? And what do they find?

AUDIENCE: What's the y-axis?

ESTHER DUFLO: So it's percentage divided by 10. So it's 0%, 20%, 40%, 60%, 80%, 100%. Ben?

AUDIENCE: I'm a little confused about the why. So 10 is interesting to me, like why 50% of the people who purchase the bed nets actually use the bed nets.

ESTHER DUFLO: So a bit more, it's more like 60%.

AUDIENCE: For 10?

ESTHER DUFLO: Yeah, for 10 shilling. It's about 60%.

AUDIENCE: Can you help me understand why that trend happens?

ESTHER DUFLO: Oh, why is it going down?

AUDIENCE: Right.

ESTHER DUFLO: It's not-- I mean, going back to the question earlier, it's not significantly different. So it's kind of all similar. So overall, about the trend, what do you have to say? This is pretty similar. It seems pretty flat. So there is no very big evidence that how much of paid for a bed net influenced whether or not you're going to use it or not. And it's not 100%. It's more like towards 60% to 70%, so not everybody uses the bed net. But how much they pay for it doesn't affect whether or not they use it or not.

Interestingly, when they went back six months to a year later, more people are likely to use them. So some people just took them because they were free, maybe, or they were not very expensive, and decided to store them for future usage, which is also something you can do.

So people do not seem to not use the bed net because they got them for free. And the result is that if you look at the effective coverage, the effective coverage is very strongly declining with price. So the conclusion from their study is that you should give the bed nets for free, because you get many more bed nets out, you're much more likely to go-- if you get them for free, you're going to be above 60% coverage. And once you go above 60% coverage, you're more likely to buy them.

But before you go, I want to give you the epilogue, which is-- so this is the reading I asked you to see, which is people-- so this got posted on Daniel Roderick's blog, under the title "Jeff Sachs Vindicated," which is to say, oh, yeah, we should give bed nets for free. This experiment showed that.

And then people complained, and said, this is only pregnant women, and pregnant women all need the bed net, so they are going to use it at any price. The sunk cost effect won't be there for pregnant women. And secondly, it's Kenya, where bed nets are still quite well-known.

So what are people objecting? People are objecting to saying, the experiment gives you a lot of internal validity. We know now that in the specific context of clinics in western Kenya with pregnant mothers, there is no effect of the price you pay on whether you use. But maybe this doesn't apply to Uganda, Maybe it doesn't affect non-pregnant women, et cetera, et cetera. So this is a question of external validity.

So what's the solution? When these kind of questions are raised, what's the solution to make progress? You've can't ignore these questions. They are very good questions. What's the solution to make progress? Yes?

AUDIENCE: Try it in other areas.

ESTHER DUFLO: You want to try it in other areas. So in fact, in bed nets, this got done. So this got tried first in different countries, so in Uganda, in Madagascar. And secondly, it got tried in Kenya, but as I was going to say, real people, not pregnant women, but people from the market, men and women.

And here, vouchers got given at various prices to individuals for them to pick them up in the shop. So now this is an environment where have you would have expected to be more likely to find a sunk cost effect. Moreover, the experiment was also done to look at these longer-term consequences. So after six months, they went back again to the same people, and tried to sell them a bed net for 100 shillings, so not the lowest price, and not the highest price.

And so the question is, the people who got the first bed net for free, are they more likely to buy the second one for 100 shilling, or are they less likely to buy the second one for 100 shilling? So I'll run you to the result quickly. So they find the same effect of price on purchase, that the more you pay, the less you get, quite steeply. They find the same lack of effect of price on whether you use.

And so this is not pregnant woman anymore. So this kind of generalizes a bit the result. And this is the effect on future approaches, the confidence bar, a little bit high. But if anything, what you find is that people are more likely to buy a second bed net when they got it for free or almost free than when they were offered the option to pay for one.

And this is despite the fact that at that time, they already had a bed net. So presumably, their need for a second bed net is less so. So it seems that the learning effect dominates the entitlement effect.

So if we go back to the wrecking market story, sustainability question you could say, well actually, giving a sweep of bed nets for free might be a good way to create a market. Because you create a demand among people who start learning and say, actually, it's kind of nice, that bed nets idea.

The last way in which you create a demand is by creating this first user of a bed net. So this is the last card that I'm going to show you. This is the purchase of a bed net if you have to pay for it if none of your neighbors got it for free. And this is a purchase of a bed net, if you had to pay for it, if all of your neighbors got it for free.

And that is randomly assigned, because depending on who your neighbors are, where they are located, what kind of voucher they get, this is just random whether they get this one. And what you see is that if your neighbor got bed nets for free, you're more likely to get one. So there is actually a positive social effect.

So the reason why I wanted detail into this experiment is that that's an experiment that shows us how we can use experiments not only to evaluate the impact of a policy, but the answer those questions that help us understand how people decide whether the entitlement effect is bigger or smaller than the learning effect, whether the sunk cost fallacy exists, whether how the big the price elasticities are, et cetera. And we need to design the experiment to answer all those questions. And then we can start informing the policy debate. Sorry for running over.