Print
Share
Literature Review on Microfinance Impacts in Africa
February 9, 2011
An institution with a very long name (Evidence for Policy and Practice Information and Co-ordinating Centre (EPPI-Centre) at the Social Science Research Unit at the Institute of Education, University of London) just published a review of the academic literature on the impacts of microfinance in Africa. The study was funded by the U.K. aid agency, the Department for International Development (DFID).
I peer-reviewed the study protocol barely six months ago and a draft in early November, just as the reality of the Andhra Pradesh crisis was sinking in. A lot has happened since. Looking over the report today, I realized how it could become a football in the ongoing arguments about the efficacy of microfinance. Proponents can point to the preponderance of + signs in the tables that summarize findings about effects on outcomes such as wealth and health (pages 29–37) as well as to the citations of “high quality studies.” Doubters can invoke the negativity of the summary:
- We conclude that some people are made poorer, and not richer, by microfinance, particularly micro-credit clients. This seems to be because: they consume more instead of investing in their futures; their businesses fail to produce enough profit to pay high interest rates; their investment in other longer-term aspects of their futures is not sufficient to give a return on their investment; and because the context in which microfinance clients live is by definition fragile.
- There is some evidence that microfinance enables poor people to be better placed to deal with shocks, but this is not universal.
- The emphasis on reaching the ‘poorest of the poor’ may be flawed. There may be a need to focus more specifically on providing loans to entrepreneurs, rather than treating everyone as a potential entrepreneur.
- Micro-savings may be a better model than microcredit, both theoretically (because it does not require an increase in income to pay high interest rates and so implications of failure are not so high) and based on the currently available evidence. However, the evidence on micro-savings is small and further rigorous evaluation is needed.
- The rhetoric around microfinance is problematic and damaging. ‘Clients’ could also be called ‘borrowers’ or ‘savers’, and ‘micro-credit’ might just as well be called ‘micro-loans’ or even ‘micro-debt’. There is an obligation amongst donors and policy-makers not to falsely raise expectations with development aid in this way. The apparent failure of microfinance institutions and donors to engage with evidence of effectiveness perpetuates the problems by building expectations and obscuring the potential for harm. A growing microfinance industry may as easily be a cause for concern as one of hope.
Bear two points in mind in interpreting this report.
First, I would mentally italicize “some” in item 1 above. I think the authors mean to convey that microcredit, like all credit, appears to have a spectrum of impacts: it helps some, hurts others, and leaves many about the same. If I am right (as the lead author confirmed below), then the authors are emphasizing the negative in contradistinction with the prevailing, positive perception—or at least what was the prevailing perception when they started writing. They are not asserting that microcredit always or even on average does harm. I’m pretty sure none of the “high quality” studies suggests that.
Second, “high quality” is relative. The authors’ assignment was to glean what they could from the literature on Africa, and in this I think they did an impressive job. But as I have argued (blog post, book chapter), the pickings of truly credible studies are thin. That goes double when you restrict to Africa. For example, the Barnes et al. study of microcredit in Zimbabwe, one of the four “high quality” studies cited, is not randomized and not, in my view, credible. (See Beatriz Armendáriz de Aghion and Jonathan Morduch’s Economics of Microfinance.) [Correction from Ruth Stewart below: I had the wrong Barnes et al. study. The "high quality" one took place in Uganda---but this does not change my argument.] Roughly speaking, it is “high quality” because it has a control group. I trust the authors’ judgement that this puts the study a cut above. But since the treatment and control groups were not randomly constructed, it still doesn’t cut it with me.
So if you’re not exclusively interested in the impacts of microfinance in Africa, I would rely more on Kathleen Odell’s excellent, recent review for the Grameen Foundation. Even if you are only interested in Africa, Odell’s review of lessons from other continents is still relevant.
Overall, the texture and tenor of this fuller distillation of the findings feel about right to me:
The available evidence suggests that micro-credit has mixed impacts on the incomes of poor people. Microsavings alone appears to have no impact. Both microcredit and micro-savings have positive impacts on the levels of poor people’s savings, whilst they also both increase clients’ expenditure and their accumulation of assets.
The available evidence suggests that both micro-credit and micro-savings have a generally positive impact on the health of poor people, and on their food security and nutrition, although the effect on the latter is not observed across the board. In contrast, the evidence on the impact of micro-credit and micro-savings on education is varied, with limited evidence for positive effects and considerable evidence that micro-credit may be doing harm, negatively impacting on the education of clients’ children. Having said this, micro-credit does not appear to increase child labour.
There is some evidence that micro-credit is empowering women, but this is not consistent across the reviewed studies. Both micro-credit and micro-savings have a positive impact on clients’ housing. However, there is little evidence that micro-credit has any impact on job creation, and no studies measured social cohesion.
Possibly Related Posts
8 Comments on “Literature Review on Microfinance Impacts in Africa”
Post a Comment
We value frank and constructive exchanges and encourage you to use your real name in your comments.
February 10th, 2011 at 4:09 am
Thanks David for your comments on this review – a very fair assessment.
Absolutely, our message is very much that microfinance can lead a range of outcomes. For those who believe all is positive, we hope they will note that this isn’t always the case, but for those who are (particularly at the moment) all doom and gloom, we hope they will see that there is good evidence of positive outcomes.
As for including randomised controlled trials – we too would uphold RCTs as the gold standard for evidence of impact (the Ssewamala et al (2010) trial of micro-savings in Uganda highlights the value of good study design beautifully). But, as you suggest, we also wanted to deliver a pragmatic review which made the most of the available evidence in the region so we included all studies with a comparison group as a starting point and then weeded out those which weren’t good quality. We kept in those of ‘medium’ and of ‘high’ quality. (Without wanting to be too pedantic, the Barnes et al study in Zimbabwe wasn’t of the highest quality – but the Barnes et al study in Uganda was. Both were included in the review.)
Our inclusion of good quality non-RCT studies was therefore a pragmatic compromise – and I think it is worth us considering the implications of this. It has meant we have been able to report ‘what we know so far’ about a wider range of outcomes from more countries, settings, and populations. The danger is that ‘what we know so far’ is not quite right due to the lack of randomisation. So far we have included some reflection on this in the report (and note that the only differences we’ve found are in the evidence relating to income and accumulation of assets which is less positive in the included RCTs than the non-RCT evidence, whereas the evidence on the impact of food security and nutrition is more positive in the RCTs than in the non-RCTs). Colleagues at the EPPI-Centre have done some work on this methodological issue in the past which might be of interest (see http://eppi.ioe.ac.uk/cms/Default.aspx?tabid=2385) We hope to analyse this further in a paper before too long and would be interested in views on this issue.
That’s all for now, except to say, yes, we have a very long name, and to mention that some important people from an institution with a shorter name, University of Johannesburg, were a valuable part of our team.
February 10th, 2011 at 8:30 am
Dear David
Just a few comments in response to your recent blog since I am working on a systematic review (SR) on the impact of microfinance for DFID as well, which will be a sibling to the EPPI Africa one you just blogged about. Our review is geographically broader but does not include savings; we are concerned with the welfare impacts of MF as well as efficiency, which I see you emphasise in the blog. You can view our protocol at http://www.dfid.gov.uk/R4D/PDF.....EA_P1-7(3).pdf. The final report will be available in April, hopefully, but to give you a taste here are some thoughts arising from our work (without implicating my co-authors of this SR).
There are no high quality quantitative evaluations of MF (excluding savings which are included in the EPPI review) by the standards of the systematic review literature, and I agree with your assessment of the Barnes study, and its siblings in India (SEWA) and Peru, which means I agree with your overall assessment of eligible studies, which mirrors that of Armendáriz de Aghion and Morduch, of course.
However, while I too was surprised that the first point in the conclusions of EPPI was harm to some people, how confident should you be that Odell’s study (which you enthusiastically endorsed) is more robust and balanced than that of EPPI? Perhaps this question should be put in the context of the prevalence over the last two or more decades of largely uncritical, methodologically weak assessments of MF. Odell’s study, which follows that of Goldberg, 2005, and is an improvement on it, but is nevertheless from the same stable with a vested interest in finding beneficent impacts of MF – the Grameen Foundation. Perhaps the main contribution of Odell (I won’t discuss more than her to assessment of RCTs for reasons of time) lies in reporting the findings of three Randomised Control Trials (RCTs), which are summarised in the statement:
Based on my reading of the (not peer reviewed) reports of the two credit RCTs, setting aside the Dupas and Robinson study which evaluates savings, the other two are just not robust enough to warrant even this qualified conclusion. Further, even if warranted, the claim that expanded business activity is unproblematically beneficial overlooks that this could lead to failure, debt, and so on, as well as success, and that the crucial impact must be on well-being rather than putative means to it.
The limitations of these two RCTs are several; I will mention only a few to dampen any uncritical enthusiasm for their conclusions. Karlan and Zinman achieve only a 70% response rate, which, while similar between treatment and controls, does not exclude the likelihood of severe bias because, as common sense surely suggests, the treated who respond are likely to be very different to the untreated who respond, and no evidence to contradict this expectation is given. There are many other surprising, and disturbing from a methodological point of view, features to the Karlan and Zinman study (and its sibling conducted in South Africa) which will be discussed in our forthcoming review and lead us to rank it as moderately to highly vulnerable to bias.
The Banerjee et al. study was supposed to have a panel data set, but the baseline was unsatisfactory. Nevertheless, the apparently well randomised design should give robust estimates of treatment effects even with ex-post cross-section data. However, they only found modest business effects and no statistically significant well-being effects. In addition to the possibility that the business effects would be short lived, or indeed unprofitable possibly leading to debt, etc., there are, moreover, methodological concerns about their study, in particular whether there was unobserved population changes or attrition in treatment locations which the absence of the panel data leaves open. A second problem is whether the behaviour in the control locations might have been affected by the non-entry of the well-known MFI, which could have reduced borrowing and business development either because of the absence of a favoured credit source, or in expectation of the entry in the near future, biasing any estimates of impact.
I will not go on, but you can expect our SR to be rather more critical of the findings of RCTs and better designed pipeline and other quasi-experimental studies, and the results of the applying sophisticated analytical methods to analyse them than Odell. In this regard, we argue, as others have pointed out and indeed you and Morduch have shown in your replication of the iconic Pitt and Khandker, 1998, study of MFIs in Bangladesh, that “Elaborate analytic methods will not salvage poor design or implementation of a study” (Rosenbaum Observational Studies (2nd edn.) Springer-Verlag, New York, 2010, p343, quoting Meyer and Fienberg, 1992). And, there is clearly a paucity of studies of MF impacts in which we can be confident – on the basis of evidence, and understanding and acknowledgement of potential bias by researchers – that they were well-designed and implemented, even RCTs.
As we have argued with you before, David, an important question with regard to MF evaluations is whether you are trying to falsify the null hypothesis of no effect or that of expected beneficent impacts. In our view we cannot in a balanced way, on the basis of quantitative evidence, reject either at this stage, so whatever continuing beliefs you and others have in the beneficence of MF is just that – beliefs; let’s hope that they are not unduly influenced by interests.
PS – for interest, your readers might like to know that my replication of Pitt and Khandker and re-analysis with Propensity Score Matching is equally agnostic as to whether it can show any any impact, which I interpret as “evidence of absence” of effect, no evidence of no impact (see http://www.uea.ac.uk/dev/publications/WP27 for a summary).
February 10th, 2011 at 11:48 am
Hi David and Readers,
With the release of this new literature review and ongoing discussion of microfinance impacts, I have seen a number of suggestions that the literature review that I prepared in partnership with the Grameen Foundation last spring was either “commissioned” by Grameen Foundation or written with Grameen Foundation’s interests in mind. Neither of these suggestions is accurate. I wrote the paper independently at Alex Counts’ request. I wasn’t on the Grameen Foundation’s payroll, received no compensation from them, and was encouraged throughout the writing process to present my own interpretations of the impact assessment research. The report represents my own best efforts to understand and summarize the literature, not Grameen’s attempt to put a positive spin on that literature. I look forward to reading the Africa literature review as well as the upcoming broader survey mentioned above.
February 11th, 2011 at 8:21 am
Dear David et al.
Kathleen Odell’s disclaimer strikes me as typical of the blasé attitude of many economists to ethical issues in their work, which has attracted considerable attention in the aftermath of the financial crisis within the profession (http://www.economist.com/blogs.....conomics_1).
While traditionally professional ethics in the social sciences has been seen as a matter “protecting the subject” three other features are important – declaration of interests (which has been the focus of the professions recent panic), protection of the professional integrity of the profession, and respect for the interests of funders (http://www.the-sra.org.uk/docu.....hics03.pdf (which sometimes means speaking truth to power). Kathleen exculpates herself with regard to conflicts of interests and asserts she did her “my own best efforts to understand and summarize the literature”.
Without for a moment doubting the sincerity of this statement, this sent me scurrying back to her document for a more careful read. It is a very clear and clearly expressed popular guide to the impact evaluation literature with many appropriate qualifications, but nevertheless in almost all the cases reviewed puts what appears to my mind to be a relentlessly optimistic gloss on insecure (and mixed) findings. The most egregious example is surely the inclusion of the RBS Foundation India study, “Microfinance and Poverty: an Impact Evaluation”. Unfortunately I cannot locate an original for this document, but Kathleen’s summary is admirably clear as with the other papers she reports that I know quite well from our SR. This gives me confidence in commenting on the RBS study on the basis of her summary of this study.
This study – I don’t know who actually carried it out or their credentials – has a before and after design with no control group. It finds that, based on a survey in 2008 of “320 poor women who had taken and repaid at least five loans .. 66 per cent are no longer poor …”. The procedure for sampling the women surveyed is not given but we are told “the majority of new borrowers are below the poverty line”. On the basis of these findings Kathleen concludes that these findings “suggest[s] a relationship between borrowing and poverty reduction, though it is difficult to draw decisive conclusions in the absence of a comparison group”. Kathleen, how can you lend your name to any such statement given the obvious problems with such a research design? Even if the sample of women interviewed was randomly drawn from all borrowers and took account of dropouts (or graduates), it would not be a robust design (with respect to assessing the impact of MF on poor women) because, as you point out, RBS borrowers might be – to put it mildly – quite a biased sample of the same. And, we do not know what element of subsidy or other support is given to these women, and so on and on and on.
As in our draft Systematic Review of MF for DFID , Kathleen organises her paper by research design, giving most prestige to RCTs, then “the Coleman Design” (actually Steele et al., 1998, 2001, used a similar design about the same time and have panel data to boot, but this study is overlooked in Kathleen’s paper), which is often referred to a pipeline design; then quasi-experimental designs (with some sort of non-random control group); and finally “non-experimental” studies, including the RBS one discussed above. My views on the two credit RCTs that Kathleen reports are similar to those given by Maren (a co-author) in her contribution above; we are equally sceptical about the studies reporting pipeline designs, of which Kondo et al., 2008, is probably the best because it takes account of dropouts and graduates; however, to mention but a few problems, the clients in this case are largely not poor and the impacts on the poor are found more likely to be negative (as you report). Kathleen, nevertheless, concludes her summary of this paper “it may be the case that appropriate products lead to good outcomes while bad products lead to bad outcomes regardless of income strata”, which is not only something of an oxymoron and tautology, but also leaves the impression that MF can be done beneficently, in this regard. This is hardly the conclusion warranted by this study which we classify as moderately-to-highly vulnerable to bias not only because treatment and control locations are not randomly allocated (although it is claimed they are matched but no evidence is provided of how well this was achieved), and the absence of a pre-intervention baseline demonstrating their equivalence leaves the possibility of bias open. The other pipelines, including Coleman’s and several others we review, are equally if not more flawed in these and similar respects.
Kathleen reports several other quasi-experimental studies including those by Bruhn and Love in Mexico, and Kaboski and Townsend in Thailand. I won’t go into details in this already over-long comment, except to point out that here too (and in her summary of Coleman’s works) the conclusions seem to me more upbeat than are warranted given the limitations of the research designs (unrandomised pipeline and with or without control groups) and the limitations of sophisticated analytical methods used (e.g. their pervasive unreliability – of IV methods, as David and Morduch have shown in their demolition of the Pitt and Khandker work, and as Duvendack and Palmer-Jones do with respect to Propensity Score Matching using the same data, and as the demand for replication in economics more generally (Hamermesh, 2007), and the demand to take the “con” out of econometrics (Leamer, 2010) attest). It is here that I think the professional bias of econometricians towards over-valuation of their professional skills should be more clearly declared, since, to borrow an aphorism from the propaganda literature, “loose econometricks [sic] kills” when we legitimate treatments (MF for the poor) relying in large part on the results of sophisticated analytical methods, confidence in the conclusions of which is not warranted, thereby diverting attention from the search for other more potentially beneficial interventions.
In all, as David pointed out in his enthusiastic report of Kathleen’s study, the message is, quite cautiously and with qualifications, optimistic about the impacts of MF. My view is that the jury should still be out, especially with regard to the relentless advocacy of the MF industry, if we are to meet our professional obligation to maintain the highest standards of reporting and publication. In this regard it is not just a matter of declaring conflicts of material interests, but also of being aware of and declaring the relevant intellectual and work related commitments that one may have – for example to promoting the value of applied micro-econometric methods, or RCTs, or indeed of maintaining and enhancing access to routes of professional advancement that may reward certain types of text and discourse which frankly do not warrant the prestige awarded them. Is this perhaps one of the lessons of the sub-prime debacle and ensuing financial crisis in the light of the now recognised limitations of the financial econometrics that to some extent underlay and legitimised these phenomena – which have surely killed?
Kondo, T., A. Orbeta, et al. (2008). “Impact of microfinance on rural households in the Philippines.” Ids Bulletin-Institute of Development Studies 39(1): 51-70.
Hamermesh, D. S. (2007). “Viewpoint: Replication in Economics.” Canadian Journal of Economics 40(3): 715-733.
Leamer, E. E. (2010). “Tantalus on the Road to Asymptopia.” Journal of Economic Perspectives 24(2): 31-46.
Steele, F., S. Amin, et al. (2001). “Savings/credit group formation and change in contraception.” Demography 38 (2): 267-282.
Steele et al., 1998, The Impact of an Integrated Micro-credit Program on Women’s Empowerment and Fertility Behaviour in Rural Bangladesh, New York: Population Council, (http://www.givewell.org/files/.....201998.pdf)
February 14th, 2011 at 5:53 pm
Richard,
Thank you for your comments. As I said, I look forward to reading your upcoming study. While I don’t intend to respond point by point to your criticisms of my paper in this forum, let me just take up one or two items.
With respect for the RBS study, I find my summary to be appropriately qualified. In the excerpt that you’ve included in your comment, for example, notice that I do not suggest a causal relationship between borrowing and poverty reduction – only a relationship. I included this study mostly based on the fact that even with its methodological limitations, its conclusions are similar to the modest conclusions of the RCTs. Rereading that section, I don’t think a reasonable reader would come away with the impression that this study had a strong methodology or that its findings were particularly positive.
Regarding the “overlooked” Steele et al study, I limited my survey to studies published between 2005 and 2010 (see page 7).
Finally, regarding your stronger assertions that my work is ethically suspect and that the review inaccurately glosses up the study results to promote my own interests, which somehow coincide with the interests of the microfinance community, I disagree. What more can I say, other than to repeat what I stated in my earlier comment?
February 15th, 2011 at 9:10 am
Dear Readers
The point of the my comments about the RBS study were that it should not have featured at all, except perhaps to be dismissed as methodologically highly suspect. To make any favourable comments about it in the overall context of Kathleen’s piece is to lend an impression of value that it does not appear to warrant and to further the general impression that we can be convinced that MF has beneficent impacts. By the way, it would be useful to have a copy of this study – as I cannot locate a copy on the www – if Kathleen could point me to it.
Ok, Steele et al. is outside your domain – mea culpa.
I did not say Kathleen boosted MF to promote her own interests – deliberately or consciously; only she can know that. Bias can be unconscious and in my view the over-valuation of sophisticated methods of analysis is probably endemic among applied microeconomists – it is part of their habitus. But it is exactly this that I think should be given more consideration as part of professional ethics – we don’t have to act as a cartel of adepts promoting our wares beyond their reach.
However, I note that Kaathleen has written at least one other piece which boosts MF interests under the auspicies of the Grameen Foundation (http://grameenfoundation.wordp.....cumentary/) . She is right – in my view – that Heinemann’s film is stuffed with negative stories, but surely a balanced response would mention the numerous “docutainments” that are stuffed with positive stories and no negative ones, such as the http://nobelprize.org/mediaplayer/index.php?id=146 or http://vimeo.com/7476111. However, the backlash against MF seems well underway, and no doubt is as ill-founded – from an evaluation point of view.
February 15th, 2011 at 12:45 pm
Dear All,
I just wanted to add a couple of interesting observations to these discussions. I believe there is always an element of bias in all research, including systematic reviews, whether it’s in who does the review and for whom, or in our design or methods. I think (believe/hope) we’re all aiming to be fair and balanced and, given the inability of any of us to be entirely free from bias, transparency (and not perfection) is paramount.
Over the last ten years I’ve done reviews for different organisations using different approaches and synthesising evidence from a wide range of study types to address different questions. These have spanned health care, public health, education, and development. None of them have been perfect and all of them have been full of methodological challenges.
Systematic reviews in international development are relatively new and issues are complex (certainly compared to straightforward syntheses of drug trials). Clearly in trying to find out the ‘answer’ re the impact of microfinance there are interesting debates about whether particular trials do or don’t have fully randomised control groups, but if I’m honest I’m looking forward to exploring further the methodological challenges we are facing and reaching our way towards some solutions in time. I don’t think we’re there yet. I’m hoping that as the different reviews in microfinance are published (and I know of 3 others underway, as well as Maren and Richard’s) we will not only know more about the impact of microfinance, but also see more clearly the implications of our different decisions about quality and relevance and (perhaps most importantly) the impact (if any) of our reviews on policy and practice.
All the best
Ruth
February 28th, 2011 at 7:58 am
By no means an impact study, a recent Pakistan Microfinance Network publication titled “Microcredit Utilization: Shifting from Production to Consumption” (http://microfinanceconnect.inf.....on=article) brings out interesting stats on impact of loans in terms of income generation and extent to which the loans are used for non-income generation expenditures that may / may not contribute to the welfare of the borrower.