Global Development: Views from the Center
« Improving Access to Financial Services in Latin America | Main | Japanese Government Criticizes CGD Index »
September 14, 2006
The Evaluation Gap Hits Close to Home
Posted by Jessica Gottlieb at 04:29 PM
"Drowning in Data" (subscription required), an article in the fall issue of Stanford Social Innovation Review, gets at the core of key problems plaguing nonprofit attempts at evaluation. CGD's Evaluation Gap Working Group found similar problems at the global level: confusion over different meanings of the word 'evaluation', difficulties and expense of attributing impact, ethical issues with randomization, and the failure of decision-makers to use evaluation evidence.
The article recommends that nonprofits ditch "summative" (what others would call impact) evaluations. It makes sense that nonprofits shouldn't be asked to conduct lengthy impact evaluations at their own expense, especially when results of that evaluation might help the funder and the donor community at large do better the next time. However, this does NOT mean that impact evaluations should be dropped altogether.
Knowing which types of interventions generate what impacts is critical to guiding decisions made by donors and governments in the U.S. and abroad, and these actors must take the lead in advancing such an evidence base. It would be a waste to attempt to achieve this kind of learning from every intervention, but as the article states, "there are times and places for summative evaluations." Yes, they may be "few and far between," but so few are currently being done, we have a long way to go before exhausting our opportunities.
The article advises that certain programs be considered for impact evaluation: those that have a strong theory of change, are being implemented with fidelity, are transferable elsewhere, and generate confidence among the staff. These criteria should be taken into account by the Leading Edge Group--the designers of an international independent impact evaluation entity--when considering a priority agenda for evaluating the impact of development programs.
Trackback Pings
TrackBack URL for this entry:
/mt/mt-tb.cgi/620
Comments
The fundamental problem with evaluation as it is currently practiced is that the organizations doing the evaluations have no incentive to produce an accurate evaluation, especially if the news is bad. NGOs risk losing renewal funding from donors if their project did not do well. Donor reps risk losing funding out to other programmes and countries if they report bad news. Since the vast majority of project managers are expatriates on 3-year job cycles, they don't want to lose bureaucratic capital by taking accountability for what was really somebody else's project - so they put together a soft evaluation and focus on their baby projects... and their successor will do the same...
Getting into a practice of hiring independent evaluation teams with transparent outputs would go a long way toward producing information about what really has and has not been effective in development.
Posted by: paul at September 15, 2006 02:39 AM
I would like first to comment on confusion over WORDS. We have a "Social Innovation Review" we have an "Evaluation Gap Working Group" we have a "Confusion over different MEANINGS of the WORD (Evaluation) we have "Difficulties of attributing impact, ethical issues with RANDOMIZATION and the failure to use Evaluation Evidence.
Why dont we talk simply instead of Intellectualizing Randomizing Attributingizing Innovatingizing and Evaluationizing. WE TALK too much and so the people who need our help continue to SUFFER & SUFFER & SUFFER. The answer is simple let us go and live with the people at groundlevel and experience their hardships and sorrows and then make DECISIONS. WE need LEADERSHIP and we are not getting it and thats why the world today is SUFFERING. Little children are at the mercy of so called ADULTS and they are SUFFERING. Let us use simple words and do simple things like giving ALL little children their BIRTHRIGHT of an education because it is their world that we are taking to the edge. We adults have had an experience of LIFE let us not take that glorious feeling of living away from little children all over the world who are SUFFERING & SUFFERING each and every DAY.
Gandhi's God was TRUTH and the path to his GOD was NON VIOLENCE.Religion is: HOW WE LIVE OUR LIVES EVERY HOUR OF EVERY DAY 365 DAYS A YEAR AND EVERY 4TH YEAR 366 DAYS.
That is Gandhi's MESSAGE.
Sincerely
Garvin Brown
Mahatma Gandhi Awareness
Australia
Posted by: Garvin Brown at September 19, 2006 04:16 PM