Careful New Review of Randomized Trials of Microfinance

December 12, 2011

CGAP has published a nice summary of what we can learn from the growing collection of randomized studies of microfinance. I’ve been a bit slow to blog it because it did not generate sparks in my head. I neither found much to disagree with nor learned much from it with regard to themes I care most about—which I don’t mean as a criticism, since I am not the intended audience. The sponsors are CGAP, the Financial Access Initiative, IPA, and J-PAL.The authors are Jonathan Bauchet, Cristobal Marshall, Laura Starita, Jeanette Thomas, and Anna Yalouris. (Let history note that it was Laura who broke the story that randomized evaluation would challenge the claim that microcredit reduces poverty.)

Like other recent literature reviews, this one finds no rigorous evidence that microcredit, as that category of financial services is usually defined, reduces poverty. But that is less of a conclusion than a starting point for an inquiry into the subtler lessons from recent research. Thus, the review differs in narrowing the focus to just randomized studies; and by widening it to embrace both the hot-button impact question and how the design of financial products affects behavior. I’m guessing that one motivation for the work was to prevent the obsession with the impact question (of which I am as guilty as anyone) from completely overshadowing questions about how to design financial services to better meet the needs of the poor.

If you’re looking for a thorough, careful, and clear summary of the latest research, and one that gets to those important design questions, you can do no better than read this review. My own chapter 6 is not nearly as complete and precise.

You can feel the cautious tone in the conclusion:

The overall message from this body of work is that poor people face various limits, and their ability to capitalize on opportunities varies greatly. One of the next steps is to find simple ways to identify those differences and cater to them with the right products delivered with the right design.

Details matter. Purpose does as well—not all borrowers want to grow a business. The variable results seen can be as much a function of borrower intent as borrower ability. A one-size-fits-all product will not bring benefit to the borrowers or profit to the providers. Instead, the microfinance industry needs to continue to mature in ways that allow it to view poor customers as individuals. Some of those individuals will leverage financial services to smooth consumption; some to manage risk; some to make investments they have the skill and resources to profit from; some will do all of the above. With a view of serving all of these needs, microfinance providers may evolve a new generation of improved services and products that reliably and flexibly help poor people.

One difficulty in writing this literature review, I imagine, was positioning it within a polarized debate. After the first randomized studies of microcredit appeared in 2009, newspaper headlines cast the studies as proving that microcredit doesn’t work. Many in the industry fired back that the studies’ implications were overblown. The authors of this new report, I am sure, sought to distance themselves from both poles, which led them to accentuate the positive in some places and the negative in others. Alas, in a charged atmosphere, such signalling is easily interpreted as bias or inconsistency. Jonathan Morduch and I wrote in 2009 that lack of credible evidence from certain studies that we critiqued did not undermine the idea that microcredit reduces poverty, in the sense that absence of proof is not proof of absence. Our intention was to prevent a dichotomous interpretation of our paper as proving that microcredit does not work, a claim we simply could not support (or rebut) with the data in our computers. A literature review this summer interpreted this nuance as a wishful defense of microcredit.

To my taste, this review new does tilt just slightly to the positive, as in this overly qualified but hopeful sentence: “Despite the lack of evidence for positive effects on welfare from credit, the studies so far offer tantalizing evidence that there could be important potential benefits for some poor households to be gained by helping the poor reprioritize their expenditures.” But I see this tilt, if that is what it is, as a minor problem. It is slight. And the report carefully segregates evidence from interpretation. One can trust it on the evidence and form one’s own conclusions, which in my case are nearly indistinguishable from those in the report.

Possibly Related Posts

• Literature Review on Microfinance Impacts in Africa
• I Failed to Seriously Consider the Limitations of Microfinance as a Poverty Reduction Approach
• Great Grameen Foundation Report on Microfinance Impacts

Tweet  

1. Qazi Nazrul Huque Says:
   December 14th, 2011 at 7:22 am

   We those who have worked as grass roots level in a microcredit organization know very well that microcredit has the potential to reduce poverty.

   If microcredit would go on increasing the burden of loan for the poor (nobody would dispute the fact that many micro-borrowers borrow from more than one source and use one loan to repay the other), this could not go on indefinitely; one day the whole portfolio would fall. But this did not happen in most cases. Borrowers keep borrowing and repaying not only for years, but for decades. Grameen Bank has been maintaining a repayment rate of more than 95% for decades.

   There is no reason why microcredit can not improve economic condition of a poor family in a free-market economy. Borrowers know very well that they have to pay back their loans. So they usually invest in income generating activities (of course there are cases of over-borrowing, bad investment, accident, illness, loss of capital etc. resulting in failure to repay, but such cases are fewer in number compared to success stories). Cash flow of money increases demand in market and boosts economic activities that have tangible impact on the lives of all, not only the poor. Microcredit is like any bank loan; the only difference is that microcredit is targeted toward the poor with its own credit delivery- recovery system to suit the poor. It works in the same any commercial bank loan works. It creates demand first when it is invested, and then again when it generates income for the investor. At both levels demand for industrial products increases, boosting the industrial sector.

   It appears that evaluations of the impact of microcredit were never without bias, and methods of research works were not without limitations. Researchers went to the places of microborrowers with questionnaires, filled them up, and went back to their research institutions where the data were processed and analyzed and decisions were reached against some ‘null hypotheses.’ Once I asked such a researcher (she came from a university in the US for her thesis work and stayed at my branch for a few days) what was her impression – during her stay at the branch – about the impact of microcredit on the lives of poor borrowers. She said she could not tell anything until she went back to university and the data were ‘analyzed!’

2. John Cheong-Holdaway Says:
   December 19th, 2011 at 2:41 am

   @Qazi: Your researcher colleague is taking exactly the right approach. You do need to ‘analyse’ the data before you can make a determination about the impact of any intervention. Your exhortations to look at the stories and not at the data aren’t helping your cause…

   People who work in homeopathy know (or at least think they know) very well homeopathy can be used to treat any symptom they care to dream up; it doesn’t make it any more true.

   If microcredit works to reduce poverty then the data will bear it out.

   It’s possible that microcredit has beneficial effects even if it doesn’t directly work to reduce poverty. I’m not too sure about the methodology of these studies, but they generally work by measuring increases in monthly income. It’s possible that microcredit with its enforced savings and ability to gain credit at a reasonable price smooths out their consumption making their quality of life better for the same monthly income.

   That’s just one example of a potential benefit and there are many more. But even so, all of these need to be tested before we can say definitively that these exist.

   That microcredit is a force for good is very plausible, plausible enough that I support treating the hypothesis that it works as the null hypothesis* and that we should continue with existing programs. Even so, we shouldn’t be scared of impact studies.

   *(even though my stats lecturers would tell me you’re not allowed to have a positive as the null hypothesis)

3. Qazi Nazrul Huque Says:
   December 21st, 2011 at 7:42 am

   John: I never said statistical data are unimportant; data and research are essential for evaluating impact of an intervention, but a researcher must not be just a machine and must be able to say what her primary impression about something she has observed is. Stories are also very important. We need both data and stories to have a full grasp of the situation, and the change.

   In Bangladesh we have interesting experiences about statistics. There is an old saying here that statistics is the worst kind of lie (lie, damn lie, and then statistics). Some recent government statistics on number of population, percentage of population under poverty line etc. attest to this saying. Sometimes these claims are so absurd against common-sense observations that people are tempted to say that statistics are nothing more than lies.

   And researchers and research designs also have their limitations. The dynamics of the economy of the poor in Bangladesh may be too subtle to someone who is not well well-informed about the rural economy of Bangladesh. Even many urban educated people of Bangladesh cannot fully realize the dynamics of the rural economy of the country. For example, most poor families of Bangladesh, both in rural and urban areas, do not have one or two major sources of income; they have several income sources and some of these sources are seasonal, sometimes very transitory, in nature. A loan may be taken for a particular purpose, but loan installments may come from different other sources. A Grameen borrower once told me that he used to manage his weekly installments from four to five sources. I have seen many scholars miss this point.

4. John Cheong-Holdaway Says:
   December 21st, 2011 at 6:52 pm

   @Qazi: The old “lies, damn lies and statistics” saying is generally applied to politicians seeking to mislead populations by interpreting the same data differently, but you’re very right that it is appropriately applied to “international experts” who misunderstand the local context and design models that ignore important variables.

   Still, the answer to this isn’t less statistical analysis, it’s better statistical analysis informed by the stories you talk about.