Transcript
Transcript: CSPS Virtual Café Series: A Conversation with Nobel Prize Winner Esther Duflo
[Le logo de l'EFPC apparaît à l'écran. Un texte apparaît à l'écran : CSPS Virtual Café Series / Série Café Virtuel de l'EFPC]
Myra Latendresse-Drapeau: [Inaudible]… and then to explore with them issues that affect us, that affect us as an institution, as public servants, as citizens, as individuals. Today's event will be held in French, but you have access to simultaneous interpretation. Simply follow the instructions included in the welcome email. Then, to optimize your viewing experience, I advise you to disconnect from VPN, if you're using one. Before I begin, I'm going to take a few seconds to mention that I am on the ancestral territory of the Algonquin Anishinaabe people. I know many of you are joining us from across the country, so if you would like to take a few seconds to acknowledge the territory you are on, I invite you to do so. Thank you!
So, without further ado, it's truly a great privilege and an immense pleasure to welcome Professor Esther Duflo and have her with us today. Esther Duflo is a Franco-American economist that many of you already know. She is the Abdul Latif Jameel Professor of Poverty Alleviation and Development Economics at the Massachusetts Institute of Technology. She is also the co-founder and scientific director of the Abdul Latif Jameel Poverty Action Lab (commonly known as J-PAL), which is specifically dedicated to combating poverty and evaluating related policies. In 2019, Esther Duflo, along with her colleagues Abhijit Banerjee and Michael Kremer, were also awarded the Nobel Prize in Economics, no less, for their experimental approach to fighting poverty. Professor Duflo has published a lot of books, but in 2020 she and Abhijit Banerjee co-authored Économie utile pour des temps difficiles [Useful economics for hard times], which is an important book that we are going to talk about a bit today. So, listen, it's truly my pleasure to welcome Professor Duflo and have her with us today. Thank you again for accepting our invitation.
[Un texte apparaît à l'écran : Canada.ca/School #GCLearning / CSPS Virtual Café Series : A conversation with Nobel Prize Winner Esther Duflo]
Esther Duflo: Thank you, thank you! I'm very happy to be here with you.
Myra Latendresse-Drapeau: Listen, for those who don't know a bit about your work and who you are, I'd like us to start a bit by maybe just telling us a bit about yourself, about why you became an economist, how someone becomes an economist, and why you dedicated your work to poverty reduction?
Esther Duflo: So, there are possibly as many career paths to becoming an economist as there are economists. But I took a bit of a winding path, maybe. My first academic love was actually history. But at the same time, since, really, a very young age, I was very concerned about the issue of poverty in a very personal way. In particular, I always asked myself the question how come I was lucky enough to be born into a family, in a suburb of Paris, to go to a good school, to have friends, to be able to study and play, while other children live either in war zones or in desert areas where they have to fetch water, etc. My mother was a pediatrician, and she was part of an organization of pediatricians who helped children who were victims of war, so she travelled to war-torn areas several times a year. And back in those days, we still had square slides, and she would do slideshows with those slides to show us a little bit about how the kids they were meeting lived. And even so, it still marked me very strongly at that age, from the age of 8, 9 or 10. And suddenly, without really having a very precise idea of what I could do, I had the idea that one day, I would have to do something in some way to thank the powers that be for putting me there rather than somewhere else. But then, well, I continued my studies. I did history, etc., and then halfway through my postgraduate studies, I realized that history doesn't really lead to anything very practical in line with that objective. And, you know, I said to myself that maybe I had to enter the public service.
Myra Latendresse-Drapeau: Really?
Esther Duflo: And so, I was potentially preparing to join Sciences Po, then the ÉNA, which is the equivalent of that school for training senior civil servants in France. And before doing that, I decided to take a year off, really, I'd say from the Parisian microcosm in which I was studying and go to Russia for a year.
Myra Latendresse-Drapeau: Okay.
Esther Duflo: And in Russia, I was able to work for teams of economists and as a research assistant. And it was through this work that I said to myself that there's another way to serve the public in the public service, which is to be a teacher, which had always been something that I thought kind of suited me. And I think it still suits me pretty well. But instead, I chose economics because that would make it possible for me to have relevant things to say on public policies and to drive improved public policies that way. So, it was during this trip to Russia that I decided not to go to Sciences Po or the ÉNA.
Myra Latendresse-Drapeau: To branch off?
Esther Duflo: To branch off from history to economics. I came back to do my Masters in economics, then my PhD. So, it was really this desire to have a real impact in the real world and in particular on public policy that took me from history to economics, and I went into economics for that reason.
Myra Latendresse-Drapeau: That's fantastic! And there's a very strong common thread in your work, in your research, and it's obviously this idea of action. And it's very interesting because it's not necessarily the traditional way that we perceive economics. Often, we see it as a bit of a theory. But you, you're doing economics in the field. How would you explain the differences? And how do the two compare, in a way?
Esther Duflo: There are different types of economists in the same way that there are different types of physicists or different types of mathematicians, so to speak. But it's maybe even clearer in the economics profession because economics includes a bit of everything. There are people who do the purest theory, which is similar to mathematics, which is really hard science, to the most applied work, which is my work. So, we cover the whole field, and most people think of economics as this purely scientific side, math, physics, etc., theorizing about how economic relations behave, how people make decisions, and so on. And it's true that it's part of the field. Then there's a part of the field of economics that is more like, say, engineering science, where we try to adapt these fundamental discoveries, say, to more practical but still well-understood subjects. For example, when someone like Al Roth, who won the Nobel Prize in economics a few years ago, works with a doctors' organization to organize how interns are matched with medical departments. It's using a lot of theoretical knowledge on how these allocation problems work in relation to a practical but still well-established problem since it can be done by a computer, by an algorithm, it can be tested, etc.
Then afterwards we go to the next stage, which I call the plumber stage, where we really get into the field and we realize, as you surely know in your day-to-day work, that the best theories and even the best engineering systems are faced with a reality on the ground that is never exactly what was expected, and that in the end, the policies that are the best thought out and even the best planned out on paper, face a whole myriad of problems on the ground that had not been thought of and that must be resolved as they arise. In the same way that when a plumber comes to your home to repair a leak, they'll try something and then well, it works or it doesn't work, and if it doesn't work, we try something else. And there's a whole branch of economics that takes, that embraces, let's say, this plumber job and that's the branch that I'm in. So, in that way, economics is a bit of a special field since it goes from Einstein to the neighbourhood plumber, and I'm more like the neighbourhood plumber. And so, the idea is that, okay, let's try. There are things, for example, if we want to encourage people to access their rights, for example. It can be a matter of information so we can carry out an information campaign. Then afterwards, we can ask ourselves how this information campaign should be structured. Do people need to see people they know in the videos we send? Or on the contrary, should it be experts talking to them? And that's a whole series of decisions that are the kind of decisions that you have to make day by day, that there's almost no guide for, often no theoretical guide for. And so that's where experimentation, testing, etc., comes in. And that's why a lot of my work moves very naturally from these issues of, say, plumbing to systematic experimentation.
Myra Latendresse-Drapeau: So, we're going to spend some time on your experimental approach. But maybe before we cover that question, really about the method, then how you apply it yourself, I really like your metaphor with the plumber. We say a lot in the world of public policy: it's 1% public policy and 99% implementation. And as you say, on the ground, there are surprises, and you have to know whether or not it works. But maybe I'll go back a bit to your 2019 accomplishments. The Nobel Prize in Economics, it's still a big deal, obviously. What aspect do you think really triggered that? What aspect of your approach brought about something that was revolutionary enough, that was promising enough, to merit this recognition for you?
Esther Duflo: Yes, thank you for asking that question because it's very important precisely in terms of the interpretation of this Nobel Prize. From day one, the day it was announced with my colleagues Abhijit Banerjee and Michael Kremer, this Nobel Prize was described as a Nobel Prize for a movement, not a Nobel Prize for the three of us and our work. I think if you took all my work away, if it disappeared from the libraries entirely and what was left was the rest of the movement, you wouldn't see any difference at all, in a way. None of my work individually is so revolutionary that it would have individually merited the Nobel Prize. Obviously, there are examples that were cited in the Nobel Prize, since examples must be cited. But what's much more important is that the movement has reached a magnitude that extends far beyond us. So, I think that's really what won us the Nobel Prize. It's the fact that there are now hundreds of researchers in the J-PAL network alone, it's almost a thousand network researchers, guests or affiliates who conduct randomized experiments that somewhat form the basis of our approach.
And then, behind these thousands of researchers, there are also tens of thousands of people who are implementing this work, whether in NGOs, in governments or even in companies as well, and also in relation to adopting public policies in terms of both the spirit of the method and the results. This means that today, there are a slightly over 500 million people who have been affected by policies that one of the members of the J-PAL network has found to be effective. In general, the policies are no longer implemented by J-PAL at all. They are implemented by governments; they're just adopted on a large scale, once we discover the results. So, it's an excellent question to know what was the triggering event that made us go from eight researchers, that we were the day J-PAL started in 2003, we were a network of eight people, to the situation we're in today. And I don't think there was a specific trigger. I think it's really a kind of snowball that took hold, firstly because the idea is ultimately quite reasonable that before, fittingly, in a plumbing context, we know so little that before launching things on a large scale, you have to try them out. Even if we don't experiment on the idea, as you say, 1% is the idea, but there's still 99% that's implementation, where there are grey areas where everyone ultimately agrees on that.
Myra Latendresse-Drapeau: Right.
Esther Duflo: And so I believe that the idea at the start is reasonable, especially if we leave the idea aside or precisely the ideology, etc., comes into play and it becomes more difficult to persuade, to experiment on flagship proposals like that. But on how it's been implemented, in general, there's a lot of consensus that it would be a good idea to do that. I think what happened is that several years ago, everyone agreed that it was a good idea to do it, but people, whether researchers or civil servants, were saying, "It's too hard, we can't do it, it's too complicated." And in fact, the way that we've persuaded people and changed their opinion is not by giving them grand speeches, it's actually by doing it ourselves and doing it with them. And we learned like that, by walking, we became a blacksmith by forging, and we built a network that made this type of work much easier. With J-PAL, with Innovation for Poverty Action, which is a bit like our organization, an NGO that works closely with us, and now with many other members of this ecosystem, whether it's ID Inside, whether it's among donors such as the Development Impact Ventures in the United States, the Fonds pour l'innovation et le développement in France, a whole ecosystem has gradually been built around them, around funding, around carrying out training which has largely gone beyond us, but that has carried this movement in an increasingly significant way, which means that it has become much more than it was for a part of the institution, let's say. And I think that's why, ultimately, it was possible for us to be awarded the Nobel Prize.
Myra Latendresse-Drapeau: So, the movement is huge now. You have branches all over the world, you started in developed countries, and then now in developing countries. And now you're also really established in more mature democracies, if you will. But before we talk about that, I would like to go back a little bit to just explain a bit what is… You talk a lot about this initial idea, but what is this initial idea? Because it's really linked to this notion of program evaluation, which obviously for an institution like the Government of Canada is crucial. How do we find out what works and what doesn't? How do we determine where to put our investments? When to stop and when to keep going, etc.? So, can you take us back a bit to that idea and how you came up with the idea of applying experimental methodologies to a field that, ultimately, well I think, before we hadn't thought would lend itself so well to, ultimately?
Esther Duflo: Yes, so the big question of impact evaluation, as you probably know in your practice, is that in the so-called usual world, we always have half the data we need, but not the other half. For example, when a new program was launched in an area, well, we know the results for the people who benefited from the program, but we don't know what would have happened to them if they had not benefited from the program. So, the traditional method has always been to compare regions where the program took place with regions where the program did not take place, or perhaps to compare before and after. And the problem with those methods is that, for example, when you compare two regions, if you take Quebec and Ontario, even if it's not very different, it's still not the same thing. And so, if Quebec decided to launch a new policy and Ontario did not, when we compare the results, we are not sure whether it's just that Quebec is fundamentally different or if it's that other things happened in Quebec, or what have you. So, we're still in this kind of uncertainty. When we compare before and after, there are always, of course, problems that happen over time that we can't control well, absolutely. Yes, obviously, the last two years have been fertile in this type of event, so it made things even clearer, that things change so quickly that the before and after is not very informative. So, the idea and the way that this problem has been solved, whether in agronomic experiments or in clinical trials for all new drugs, is to conduct randomized trials.
So, we take a whole population and we give the program, or an "A" version of the program to compare it to a "B" version, choosing entirely at random. This approach—the "A's" and "B's"—is also probably familiar to you now because this approach is used constantly and continuously in the industry for what they call AB Testing, which is comparing an "A" version to a "B" version, for example of a website, that you see, which are assigned at complete random. In fact, when we started, it wasn't yet as widespread as it is today. But on the other hand, clinical trials, of course, are the standard for drug adoption. A really clear example is the COVID vaccine trial. They took a cohort of 50,000 people whom they randomly selected to get the vaccine versus getting a placebo, and then they tracked the results and they looked at whether people got sick or whether they got very sick, and it allowed us to compare the two very, very, very clearly. When Abhijit Banerjee, Michael Kremer and I came in and found out about this idea of clinical trials and finding out about the possibility of applying them to public policy issues, we were actually not the first ones. There had been a very well-done experiment conducted by RAND on health insurance in the 1970s and a highly publicized experiment on a subject that is also quite familiar. It was the "Negative Income Tax Experiment," so the idea of giving people a negative tax, a minimum income which was therefore to provide people with a minimum income which was therefore… giving them transfers that were progressively taxed as they earned more.
The idea of the program was popular, but people were afraid, even those who supported it were afraid that the beneficiaries would stop working because they already had security. And what's more, their implicit tax on each source of income was very, very high since the transfer was taxed at roughly 50%, until we got to zero. So, there was that concern. And in fact, in the '70s, they did a series of randomized trials to test this program and it was very, very useful and very interesting, in particular because they found no effect on the supply of work, contrary to what they had feared. But hey, we concluded that it was still difficult to carry out this kind of experiment, that we were losing people, and it didn't really continue. It wasn't really picked up—not by economists or in public policy—as something that could be done on a large scale. The Manpower Development Research Corporation (MDRC), which still exists and is an excellent program evaluation organization, has in fact continued to do clinical trials, but somewhat on their own. And the idea was that it was too much, too expensive, too long, too complicated, too demanding. And I think the important innovation that Abhijit Banerjee, Michael Kremer and I did was to say no, it can be done in a much cheaper, much easier and much simpler way. We can work at the individual level for example, rather than at the level of trying to randomize whole groups, and that can be applied in developing countries where there is a lot of uncertainty about what can work and even sort of simple things like, are the textbooks going to be effective if we put more of them in the schools, will the children learn more? There's so much to learn that we can start there. And I think that's helped demystify the method …
Myra Latendresse-Drapeau: … okay.
Esther Duflo: … as something that could be done in practice. And so it's really a pretty rare case of kind of south-north innovation, where the north had forgotten that it was something you could do. And it went back to the south by showing evaluations that were faster, that yielded results fairly quickly, that were also more connected with academic literature and therefore with the problems that economists were posing as a profession. And it developed like that, and then afterwards, having shown the power and the potential to do that, it was able to swing back toward the north and therefore re-explode toward the north, and paradoxically, we are now back on developments that sometimes are very long, are very expensive, are very demanding, but where we agree that it is worth doing them. So, I think that's kind of what happened.
Myra Latendresse-Drapeau: So the pendulum really did swing back.
Esther Duflo: The pendulum really did swing back, yes.
Myra Latendresse-Drapeau: That's really interesting. So, you start from that idea. You do randomized controlled trials on a smaller scale. But there's always, once again in your work, this idea of having a bridge between results and research, but also public policies. Because the idea is to influence public policy on a larger scale. But that, of course, is a huge challenge. How do you approach it? What are your considerations around that when you start? Are you saying, for the time being, we don't focus on decision-makers yet, we focus on what we find, and then, gradually, we weave bridges or whether you still have this idea of influencing decision-makers, of influencing public policies?
Esther Duflo: So, there's really the whole spectrum now. That's why I was saying, paradoxically, we've gone back to experiments that are carried out on a large scale because, in fact, to know what we've discovered over time, it's that small-size experiments are in a better position to inform us about the 1%, which is the idea, and less well placed to inform us about the 99%, which is the implementation, or the plumbing, and that therefore if we really want to be useful for decision-makers, you also have to be directly in touch with the 99% of plumbing and consequently be directly in touch with all the problems that arise when you act on a large scale. And suddenly in the same way that there was a type, we started in the '90s, let's say, with rather small, well-controlled experiments. There are still people who do that, a lot of people who do that in economics, who do little well-controlled experiments. In a way, we are more on the engineer side of things, in relation to the description we were talking about before. That is to say, we try to control what we do to fully understand what we have to say. Experiments that are often inspired by fairly deep economic issues, for example, economic issues, or not necessarily economic issues as well. For example, I have a colleague who works on the impact of sleep deprivation on productivity, and so he wanted to know, because this is something that is very present in the literature of people working on sleep, it's more doctors who say that lack of sleep is a disaster, it makes people unfocused, unproductive, etc.
Myra Latendresse-Drapeau: We've all had that experience.
Esther Duflo: So, in developing countries, people often sleep very badly, especially people who sleep outside, who don't have air conditioning, and so on. So, they wanted to do an experiment to find out if this was true. And that's an experiment that they did in great, great detail by equipping people with a "Fitbit" to measure whether they slept well or not, putting people in conditions where they could take naps, by putting them in conditions where they made them work on a particular project so that they could measure their productivity properly, and so on. So, we have this type of very well-controlled experiment today, and so on. But we also have the kind of experiment—and that's more of what I'm doing today—that is on a much larger scale to try to imitate, let's say, public policy once it has been adopted on a large scale, which often takes ideas that are, let's say, promising in that literature, but then placed on a large scale. So, for example, I worked on the issue of vaccinating children against essential diseases. So, in India, where there are large regions where children are not vaccinated—not so much because people are resistant to vaccination, but because they don't do it—in the end, there is no … And so, I worked with the government of Haryana, who were really interested in finding out what they could do on a large scale. And so in that instance, we really worked on thousands of villages with systems that could be directly applied by the government. We had and we compared different programs, different versions of programs to really find the ideal version for them. So, for example, we had a small-incentives program in the form of telephone top-ups.
Myra Latendresse-Drapeau: Okay.
Esther Duflo: It's easy to do, it's easy for the government to send out. We had an SMS system to remind people that it's time to get their children vaccinated, which is very common in many countries. And we also had a newer system of going to ask people who are influencers in their community and and asking them and texting them every month saying, "Remember, the vaccination camp is coming, you can tell your friends about it." To spread the news a bit.
Myra Latendresse-Drapeau: We have seen that a lot in recent years, also during COVID, measures that are somewhat similar, perhaps not telephone outreach here in our countries, but this idea of going to community leaders and helping them to work with people. We've also used these methods a lot, haven't we?
Esther Duflo: That's right! So, we did an evaluation. I've also just completed an evaluation of exactly this type of method for COVID. It's very effective for the vaccination of children, but we've found absolutely no effect for COVID, perhaps because the contexts are very different. Because childhood vaccination, when people are asked, "What do you think about vaccination?" 98% of people tell us, "Yes, of course, I want to get my child vaccinated. It's just that I didn't have time right then, so I'll do it next month." Whereas when you ask people about COVID, they have had very, very, very strong opinions. Those in favour of vaccination have already done it, and those who aren't yet vaccinated are in fact people who are very, very opposed to vaccination. And in that case, it's this type of measure, let's say, a quite soft measure, which is sometimes called "nudging." It works less in circumstances where people have already formed their opinion. So, this experiment we had in the United States was also a very large-scale experiment, for this action, on a very, very large scale, since it was through Facebook. We sent out several million videos in January, so at the time of the peak of the Omicron wave in places, in states where vaccination was very low to begin with.
Myra Latendresse-Drapeau: Okay.
Esther Duflo: So, where there was a greater margin to have people vaccinated, but also where opinions on vaccination were very strongly linked to political opinions and perhaps difficult to change. And we did two things in that campaign. One, we sent direct messages to doctors, short videos explaining and demystifying various aspects of vaccination. So, that too is something that we saw a lot during COVID, and something that everyone did. And two: we used a slightly different message, which was that you can convince your friends to get vaccinated against COVID. If you want, go talk to them about it and if you want to do that, there are all these videos that you can share with them.
Myra Latendresse-Drapeau: Okay.
Esther Duflo: And then the third way it was handled was, friends are very important in vaccinating people against COVID. If you know someone who is particularly popular, send them this video and ask them to tell everyone to get the COVID shot. And in fact, none of these interventions had any effect. But really no effect at all, nothing at all, at all, at all. So, it was interesting, the contrast, both with our results on childhood vaccination in Haryana, which had worked very, very well, and then the results that we had gotten for COVID. In 2021, we launched an experiment to encourage people not to travel for Thanksgiving, which was in November in the United States, or for Christmas. This was before the vaccines, so really everyone was recommending that people stay home. And so similarly we had sent direct videos by doctors encouraging people to stay at home. And there too, we found that these videos had an impact, both on travel, where people travelled less, and even on COVID. And so, there was a contrast, and that's why we decided to launch this project on vaccination, thinking maybe that we can influence people, even with very simple messages. But I think opinions on vaccination were already much more firmly established.
Myra Latendresse-Drapeau: Cemented.
Esther Duflo: Yes, cemented and impossible to change.
Myra Latendresse-Drapeau: But that's where we see all the power behind your method because as a government, inevitably policy makers, decision-makers, probably said to themselves, well, hypotheses are made on the probable and possible behaviour of citizens, based on past experiences. And so, we build interventions, public policies based on these assumptions. And so you get to that point, you test them, and it doesn't work. Well, what do we do? We readjust! But that dimension is crucial, and it really translates into major decisions in terms of investments, obviously. The Government of Canada is very interested in that. In 2016, the Treasury Board minister was given a mandate to work with his colleagues across government to really develop experimental approaches to determining what works, what doesn't, and measuring the impact of the program. You're going to tell me, that's very late, 2016, but there's still this kind of groundswell where we try to adopt these methods which come from, among other things, which are inspired by, among other things, by your work. And of course, you mentioned nudges earlier, so there's also behavioural economics in there. Is this something you use a lot, nudges, behavioural economics in your experiments?
Esther Duflo: Well, it's something that I have to test often, actually because, again, in the 99%, which is implementation, there are what I sometimes call nudge aspects, in keeping with my plumber metaphor, the shape of the taps [overlapping voices] …
Myra Latendresse-Drapeau: … that metaphor.
Esther Duflo: There are aspects that aren't nudge aspects that are also important and often forgotten because they are also a bit less sexy. And then, if you're in public policy evaluation, I have to say that even the idea that the Department of Finance has to coordinate a whole process of evaluation, of trying things out, etc., 2016 isn't late! In fact, there are few countries, or even to my knowledge, there are not really any countries that have this systematically. So, I applaud you.
Myra Latendresse-Drapeau: We're getting there.
Esther Duflo: We are, more so the leaders than anything else, that's it. But in these evaluations, nudges are great, but it's not just nudges. There are also things like sewer pipes, pipes that you can't see, that don't necessarily interest people but that are very important too. And so for me, in my work, there's no one approach that I favour over another, so long as the objective is that it works. So, for example, to give an example on underground pipes, in India there's a program, which is, let's say, a redistribution program—a workfare program. So, people can come and work on construction sites. So, it's very complicated as an organization because someone has to do the work, the work has to be launched, there has to be money to recruit people and to buy the equipment they need for the building site, and so on. And so, traditionally, the way it happened was that each village received an advance.
Myra Latendresse-Drapeau: Okay.
Esther Duflo: Then, they launched a few projects, then when they finished early, they asked their district for more money, who then asked their units, who asked their district for more money, who asked for more money from the state, and then finally, after a while, it came back to them.
Myra Latendresse-Drapeau: Okay.
Esther Duflo: A process that on the one hand takes time and on the other hand does not lend itself to auditing at all because the time it takes for projects to actually be carried out is several months later. So, people left, and we can't check whether they were recruited or not, and so on. And as a result, it's a program that suffers from huge and varied corruption leaks. And we worked with the person who was in charge of implementing this program, who wanted to try a different way of doing things and who was saying that instead of the villages' needing to go through layers of approval like that to have an advance, instead they'll recruit the people and as soon as they have recruited the people, they'll send an invoice to the central bank directly for each person and the money will go directly to them. And it's not an advance, it's directly offset: "I hired Ms. Macha, I paid her for a week or two weeks; pay up!" So, they put it in place, and we evaluated it. And so that's really an evaluation of the pipes because it's happening from … the experience for the recipient doesn't change at all; it's the same as before. But what's underneath changes completely. And what we've found is that it doesn't solve all the problems. In particular, it doesn't resolve payment delays for individuals; on the contrary, it even tends to make them a little worse, precisely because the person, the official must first enter the information, then it comes back, then the money goes to the official's bank and then this person is the one who writes the cheque. So, it's a bit slow process, so it can be improved from the point of view …
Myra Latendresse-Drapeau: … of the process.
Esther Duflo: …from the point of view of the process for the final beneficiary. But that said, it's a revolution compared to what was happening behind the scenes, particularly removing the intermediate administration that was taking off the top and greasing their palms along the way, which has enormously reduced corruption in the program by approximately 25%. So, to achieve the same results in terms of the number of people recruited from what's done in the field, it costs 25% less, which is still good since it allows you to do more things later. So that's an example of, let's say, anti-nudging. Things that we don't see but that are also extremely important. So, otherwise. There are plenty of things that I do that are much more like nudges. For example, the example I gave you about childhood vaccination with text messages, small incentives, and so on. It's really the experience of the end-beneficiary trying to get people to do something that they probably would have done anyway, but there are obstacles in life that prevent them from doing it.
Myra Latendresse-Drapeau: But it's still interesting that you use an example not just of the direct impact on the final beneficiary, but also precisely the pipes, what you call the sewers—and there, the metaphor works moderately well—but in the federal government, our level of government, we have relatively few direct interventions with citizens. It happens a lot at the provincial level, so at the federal level, on the other hand, we have a lot of those pipes, if you will. And when you say a 25% improvement, the issue of corruption, it's perhaps a little less relevant in Canada, except that 25% efficiency, 25% improvement, we have a process as you say, it's important because it frees up funds to do something else, which, in the end, has an impact for our citizens. So, it's extremely important. I want to move on to the question of ethics. When we talk about randomized controlled trials, one of the obstacles that we run into a lot is the question, "Yes, but okay, if we give a treatment to a group and we have a control group that does not receive that treatment, how do we justify that?" How can we make sure that, in the end, we get roughly the same treatment or in any case that we don't create undue advantages or undue disadvantages between different groups? How do you approach that?
Esther Duflo: So, that's two separate questions. There is the question of ethics as such, of research ethics, and there is the question of, say, fairness which is separate from the question of ethics …
Myra Latendresse-Drapeau: … indeed.
Esther Duflo: … but that's also important, say, in the public service, in particular for government. So, for the question of ethics, it's a question that I don't need to answer in a way, since it's a question that is very much framed internationally on what can be done as an experiment and what cannot be done. And so, there have always been excesses and really unethical behaviour from researchers, such as the famous "Tuskegee trials" or even the Nuremberg trials before that, which revealed the need to regulate research. And there are international guidelines on this subject called the Belmont Protocol, the philosophy of trying to balance the risks to participants with the benefits to society. And in terms of the risks to participants, they include the risk of breach of confidentiality, the risk of ingesting substances that could be harmful. For drugs, that's how it is. And so, there's both a process of documenting what's planned at the start, there are independent committees that monitor the experiment, and follow-up is conducted throughout the experiment to verify that there are no issues related to the experiment. And these same procedures are applied to randomized trials for public policy.
Myra Latendresse-Drapeau: Okay.
Esther Duflo: So, nothing in these guidelines says that you can't treat people differently and in particular that you can't give someone a vaccine and not give it to someone else. It's completely ethically accepted for clinical trials, even if it means that there is someone who has access to the drugs beforehand. Why? Because the knowledge that we glean from it is so great in the future for both the people who participate in the experiment and all future generations. Could we have imagined the COVID vaccine if we hadn't done the trials, where would we be? I don't think anyone would've wanted to try an mRNA vaccine if there hadn't been major clinical trials on both the positive effects and the potential negative side effects of an entirely new product. So, the benefits of knowing if something is working or not working are for all future generations. So that justifies excluding some people. It's not necessarily easy. For example, at the height of the HIV/AIDS crisis, there were a lot of patient groups who were extremely against drug trials because they wanted to go home immediately, they wanted to be treated and they didn't want to be monitored, and so on. But all the same, on the whole, we accept this principle for medicine in the same way. I find it important to accept this principle for public policies in that, nevertheless, most of the things that we try don't work, and therefore we waste a lot of time and a lot of money doing things that are not useful, that could be put to better use.
Myra Latendresse-Drapeau: Right.
[Un texte apparaît à l'écran : CSPS Virtual Café Series : A conversation with Nobel Prize Winner Esther Duflo / Une discussion avec Esther Duflo, lauréate du Prix Nobel]
Esther Duflo: Take, for example, the example I gave you on the COVID vaccine. Okay, it doesn't cost any money to do this kind of … It's not very expensive to do this kind of campaign, but knowing that it doesn't work is really important because suddenly we say to ourselves that's not how we're going to persuade people in Alabama to get vaccinated. So, we have to find something else. So, if in addition we spend a lot of money on a program and then we realize that it doesn't work, it's very important because we can use that money and do something better with it. On the other side of things, if we find that something works, we can spread it. For example, the first conditional social transfer policies in Mexico, "PROGRESA," were accompanied by a randomized experiment that showed the positive effects of the program, which then spread like wildfire across the globe because there had been an evaluation. So, effectively, there were a hundred villages that did not have a program in the first few years, but they were then brought into the program because the program was effective, and millions of children were able to benefit from the program thanks to this. So, what's not ethical is doing things haphazardly without testing to see if it doesn't work, that's it. In fact, we experiment with people all the time, but without learning anything from the experiments. So, for me there's no problem. For me and for science, let's say, there's no ethical problem with randomized trials, but there may be equity issues and there may be political problems because people realize whether they are included in the program or not. And sometimes that leads to situations where it makes certain experiences undesirable. For example, the first transfer experiences of Universal Basic Income, or UBI, the first unconditional income transfers that were made in Kenya were made using randomization at the individual level in the villages.
Myra Latendresse-Drapeau: Okay.
Esther Duflo: And then afterwards, they realized that when you use randomization at the individual level in villages, there are people who are actually neighbours, some of whom will get help and others who won't. It's not very good for village dynamics.
Myra Latendresse-Drapeau: Consistency …
Esther Duflo: … or in fact the people who are excluded, not only are they excluded, they're not exempted because not only do they not have the funds, but they also don't have others in a context where others do have them. And so now, all these experiments are done with randomization at the level of the whole village, so either the whole village is included or the whole village is excluded. That way no one suffers from not being included in the experiment. So, we evolve like that, there are limits to what can be done, there are ethical limits to what can be done. And another important ethical limit is that if we do research on someone and we realize that they're in an extreme situation, there's an ethical responsibility to help them. So sometimes, there are programs that we might want to do, and that could potentially even be accepted by an ethics committee, but that I refuse to do. For example, a long time ago, the organization I worked with in India wanted to do a renutrition program for very malnourished children.
Myra Latendresse-Drapeau: Okay.
Esther Duflo: And so normally, we give them a kind of peanut-butter-like dough.
Myra Latendresse-Drapeau: Peanut butter, right.
Esther Duflo: Peanut butter. But they wanted to teach mothers how to make something. Okay, so why not, but that means you see a very malnourished child, you know that peanut butter works. Can we take the risk of giving them something when we don't know whether it works or not? So in that case, it's the people who are in the treatment situation that I found to be too vulnerable.
Myra Latendresse-Drapeau: Right, right.
Esther Duflo: For me, unless we've done more controlled things in the lab where people or children are in hospital and we can give them something else very quickly, and if it doesn't work, I'd rather not try. So, we're losing something because maybe we're losing that this peanut butter isn't available to everyone, and so on. It's still much simpler if the mothers can do something themselves. There were reasons to do this experiment, clearly, but the ethical cost for the participants seemed too high to me. So, actually, there really isn't an answer. The answer is really, on the one hand, that there are rules and procedures that we follow. All our J-PAL projects are overseen by an ethics committee and a local ethics committee as well. On the other hand, in terms of decisions, there are many cases that are absolutely obvious, where there is no question, but there are many cases where there can be a discussion, and one can be led to change the nature of the project or not to do a project because there's an ethical problem.
Myra Latendresse-Drapeau: Right, right, right. I can totally see how this kind of conversation could be extremely important in large institutions like the Government of Canada or any kind of somewhat governmental intervention. You were slowly coming back to J-PAL, and I would like to ask you a bit about how your strategic priorities for J-PAL have evolved. When you read a bit of your book Économie utile pour des temps difficiles [Useful economics for hard times], the book clearly has an opening on major current social issues, so beyond poverty, with a direct link to poverty, but really other things where you say, "But we can work on that too. "And so how do you position yourself in relation to some of these big issues, so-called wicked issues, that are really stubborn issues that we have to deal with? And it can really vary, from climate change to the digital economy, to inclusion and diversity, which is a fundamental issue here in the Government of Canada. So, I've heard you talk about that many times. Could you tell us a bit about how you see the future, the evolution of J-PAL, but also the evolution in relation to these big issues?
Esther Duflo: So, there's a difference between the evolution of J-PAL as an organization and my personal evolution as a researcher and an individual. That is to say, I believe that J-PAL has been well served in having a very clear mission, and what my staff or my collaborators always hear me repeat is, "creep mission, creep mission." So, on the contrary, I try to stick to a very clear mission, which is the evaluation of policies intended to reduce poverty with the ultimate objective that the poor of the world—whether in rich countries or in poor countries—are better served through more effective policies.
Myra Latendresse-Drapeau: And that discipline, was it fundamental to your success?
Esther Duflo: That discipline was fundamental for J-PAL. This was J-PAL's mission on day one, it's still J-PAL's mission today, and J-PAL as an organization does just that. That said, poverty is still a very broad topic because people who live in poverty are confronted with all issues. Take, for example, the issue of climate change. The main victims of climate change will be the poorest people in the world, since it's already hot in poor countries and therefore higher temperatures will bring us out of the comfortable areas for human existence to dangerous areas. We have already seen it in the spring with record temperatures for India and Pakistan, for example, which are more exceptional but that we are seeing more and more, and it will only go from bad to worse.
Myra Latendresse-Drapeau: And that last a very, very long time now. We're talking two, three months?
Esther Duflo: That last a very, very long time. So, in Canada, if the temperature of the Earth warms up by two degrees, in Canada, it will be rather more comfortable in winter, while in India or Pakistan, it's the difference between habitable and uninhabitable. Secondly, the technology isn't there in the poorest countries to allow us to adapt to these climate changes. For example, there's massive distribution in Texas, it's very hot too, but there's massive distribution of air conditioning and so people aren't dying of heat, basically. Very rarely there may be someone who has heat stroke. While in India and Pakistan, on the one hand, people don't have air conditioning in their homes, and on the other hand, they work outdoors a lot, out in the fields, and so on. So, the same temperature has a much stronger effect in poor countries than in rich countries on both productivity and production, and therefore on the standard of living and on mortality directly. For these two reasons, when researchers, for example from Chicago, Michael Greenstone, at the University of Chicago, combine the results of climate models that tell us how the climate will evolve, say until 2050, and these expected effects, according to the studies that have been done on the effect of a given temperature on mortality, and we see the mortality, the increases in mortality by 2050 completely concentrated in an area that is, say, around the equator, be it Brazil, north-eastern Brazil, the entire Sahel region in Africa, northern India and northern Pakistan and Pakistan. So, the issue of climate change is necessarily at the heart of J-PAL's concerns today because it is part of the issues of poverty. The biggest threat today to the gains that have been made against the ills of poverty is climate change. Which is why there is action both on what can be done in rich countries, which are the main contributors to the emissions that cause climate change, to limit them, and on what can be done in poor countries for adaptation and also for limitation.
Myra Latendresse-Drapeau: And does that influence your own work?
Esther Duflo: It influences my own work, it influences J-PAL, definitely. J-PAL is moving towards these subjects because they are absolutely essential. In the same way, at the very beginning of J-PAL, many of our evaluations were actually on health and on education and then we had an evolution toward governance, and so on. Because we realized once again, it's 99% implementation without the implementation. Without good governance, there is no good implementation. So, all aspects of governance are automatically included in J-PAL's concerns. Then, in the book, Économie utile pour des temps difficiles [Useful economics for hard times], there are many questions on these topics, but there are also many subjects that are not particularly based on randomized experiments but based on the whole body of economic knowledge. Because what we tried to do with Abhijit Banerjee was not necessarily to advertise J-PAL, but rather to advocate economics as a whole to try to show how economics has a lot more to say about the important issues that we're dealing with, about wicked issues, than many people realize. Many people think that economists try to predict what will happen to GDP or financial markets in the coming months. That's not what most economists do. They try to solve the kind of big societal issues, or at least think about them, as described in the book.
Myra Latendresse-Drapeau: Right.
Esther Duflo: But finally, J-PAL is not the sum total of economics, far from it.
Myra Latendresse-Drapeau: Well, I don't think we'll ever have access to the sum total of economics, but I think that referring to your work, reading your books, following a bit what you, personally, what your colleagues are doing, what J-PAL is doing, it's an excellent way to fuel our reflections and to continue to move forward in this direction. Professor Duflo, this has been great. Thank you once again. It's been an honour and a privilege. It has also been a very important moment for me, personally. So, thanks again for being with us today! Thank you to all our participants who once again joined us from all across Canada! We talked about a lot of things today, but I invite you all to maybe check out the directive on experimentation that was issued by the Privy Council Office and the Treasury Board Secretariat in 2018. That will give you some idea of how, here in Canada, we interpret this notion of experimentation. As well, please do not hesitate to consult the School's catalogue for other events. So, thank you again, Professor Duflo! Thanks, everyone!
[Le logo de l'EFPC apparaît à l'écran. Un texte apparaît à l'écran : CSPS Virtual Café Series / Série Café Virtuel de l'EFPC / La vidéo s'estompe avec le logo du gouvernement du Canada]
