In a perfect world, it would work perfectly. In this world, development goals are approached ” holistically”. Politics do exist and have the final word. This means that a government that fails dismally in one area (such as corruption, human rights, or just charisma and language skills of its leader) the country can be downgraded from donor darling to pariah.

Countries will not honour promises made 5 years down the line on girls’ education if meanwhile the president supports Khadaffi.

This means, if you are toiling in the education department, the holistic approach will deliver a totally randomised amount of cash on delivery, with no relation whatsoever to the actual delivery.

I would like to remind you that budget aid also went first to the leaders with good press, even if they were at war at the time.

Thanks Rakesh for the sharper suggestions.

1. I can’t agree with you more about the importance of setting the right outcome measure. In developing the education example we struggled with this question. The choice of paying for students who take an exam in the final year of school was a pragmatic one that balanced a number of concerns related to assuring the integrity of testing and assuring the feasibility of independent verification while still getting a measure that would reveal information about the quality of schooling to the public. We’ve done other work on measures like tobacco control, child mortality and access to water in other papers addressing this key question of moving from concept to workable proposal (see http://bit.ly/hqUeiq).

2. Independent verification is also critical as you point out. We identified it as one of the five basic features of the approach. I’m mystified as to why, for example, a proposal for a new results based operations policy at the World Bank (see P4R at http://bit.ly/gOR7A9) even considers the possibility that governments can be paid for results which they report themselves. Not only does it invite gaming, but it precludes the best way of improving administrative reporting systems – that is, checking them against independent estimates!

3. The one place I would differ, not disagree but open to complexity, is on the issue of “the right level.” When we’ve gone to different countries, we’ve heard different stories about what the major hurdles are to progress. In one country they were concerned with delays in Finance Ministry disbursements to the education sector; in another it was rural roads so kids could get to school; in yet another, it was the lack of incentives at the school level. Though your point about getting incentives to schools may be right in Tanzania and many other places, we generally have come to the conclusion that it is not the donor’s role to determine the right level – by providing COD Aid at the national level, the government becomes accountable to the citizens for using those funds in a responsible and effective way. It doesn’t mean a public only solution. The government could offer vouchers to children to attend private schools. If the right level is at the school or for teachers, other behavioral models have to be taken into account for how people respond to intrinsic and extrinsic rewards. I think the point is that there are promising ways to pay for results at each of these levels and hopefully learn the best (sharpest!) ways to do them.

Great posting by Rajani. His points about working at the sub-national level and working with the private sector are especially important, in my opinion.

Quick suggestions for people who are new to the concept:
–We have posted FAQs that answer many of these questions at http://bit.ly/fpDZbt.
–And if you have a bit more time, the book can be downloaded for free at: http://bit.ly/9ONNL2

Thanks all for your interesting comments.

@Bill Savedoff, as usual appreciate your thoughtful response. Specifically, on your three points:

1. I see the need to strike a pragmatic balance, but am not convinced by your choice in education at least, because sitting for an examination is a useful measure for not having dropped out of schooling, but not of achieving a learning outcome. The subsequent publication of examination results is likely to have only a weak effect, because a) national examinations are often a poor measure of learning, b)publication of consistently poor results in places like Tanzania and Uganda have not stimulated change, and c) the incentive effect between people most able to promote learning and the learning outcome becomes tenuous (see 3 below).

2. We agree. Note here I suggest that a simple independent test, such as Uwezo or ASER, is a better option than auditing of a government administered test.

3. I agree that constraints are at different levels, and specifics should be determined at country levels and not by donors. But from where I stand, as a practitioner trying to figure out how best to design this, I find it difficult to imagine how most countries in SSA would find a national government level COD incentive sufficiently powerful to drive the large changes and turn tankers around from an obsession with inputs to rewarding learning outcomes (save possibly few exceptions such as Rwanda and Ethiopia). I’m making two points here: that by setting the incentive at the school/teachers/parents level not only do we focus on the people most able to promote early grade literacy and numeracy, but that this approach may create the most powerful pressure up on national levels to do their part. This is of course only a belief now, and it may be really useful to see if one could design thoughtful evaluation experiments to test the effects of setting the COD incentive at different levels.

@Richard Morgan You are right that COD without qualifiers does not take adequate care of equity concerns. But one could design it to do so, at least in some regards, e.g. I understand that in Ethiopia the government and DFID are considering different levels of COD payments for girls and boys, and one could also vary it by districts/provinces.

Moreover, given how little money schools/teachers tend to receive (eg. see http://www.twaweza.org/index.php?i=501, http://www.uwazi.org/index.php?i=358 and http://www.uwazi.org/index.php?i=357) a transparent, well administered COD targeted at local levels may be a really effective way to get resources to poor communities at scale, and in this way have the greatest equity gain.

Still this would not take care of inclusion of e.g. children with disabilities, and that is why COD is not a panacea nor meant to replace other necessary interventions. Rather its one intervention on top of others that seeks to address a particularly large problem affecting children from poor households.

Regarding participation in outcome monitoring, having seen/ been part of many elaborate efforts (including some via UNICEF) I have come to believe that the best bet often is to do practical transparency and parents/teachers/community actors will figure out whats needed. In my view that is also the best way to address @Subrun’s concern above. Uwezo is an experiment to do that in East Africa that is being closely studied by researchers linked to MIT and Princeton, will be good to see what they find over the next 3 years.

@Sam Gardner and Steven Synyshyn: Indeed its politics, and a lot of aid is and will continue to be determined by concerns other than performance. But this does not mean that COD could be an instrument that steers money towards outcomes, and a contract based on payments only against independently verifiable results may better withstand donor jitters of the present sort in US, UK, Sweden and Netherlands. I have often found the opposite problem, that donors keep disbursing despite the evidence of poor/weak outcomes, eg witness Tanzania and Uganda (latter incidentally supports Gaddafi).

Rakesh, I could not agree with you more.

I would even go further: the current “development paradigm” mixes up outcomes for totally different objectives: you punish children by withholding vaccination when a minister is not complying with monetary guidelines. This is perverse. It is also called the Paris Agenda. As you don’t want to punish the children you will continue to fund everything, whatever happens.

COD is a very good instrument for a development world that has a budget and goals per objective (not neglecting the externalities and environment), while it will get muddled in this world, where the development budget is “in the general direction of good”. COD must get a chance, as it could be instrumental in the change from support to governments and heads of state, to the support for change on the level of the people needing it.

Thanks all for your interesting comments.

@Bill Savedoff, as usual appreciate your thoughtful response. Specifically, on your three points:

1. I see the need to strike a pragmatic balance, but am not convinced by your choice in education at least, because sitting for an examination is a useful measure for not having dropped out of schooling, but not of achieving a learning outcome. The subsequent publication of examination results is likely to have only a weak effect, because a) national examinations are often a poor measure of learning, b)publication of consistently poor results in places like Tanzania and Uganda have not stimulated change, and c) the incentive effect between people most able to promote learning and the learning outcome becomes tenuous (see 3 below).

2. We agree. Note here I suggest that a simple independent test, such as Uwezo or ASER, is a better option than auditing of a government administered test.

3. I agree that constraints are at different levels, and specifics should be determined at country levels and not by donors. But from where I stand, as a practitioner trying to figure out how best to design this, I find it difficult to imagine how most countries in SSA would find a national government level COD incentive sufficiently powerful to drive the large changes and turn tankers around from an obsession with inputs to rewarding learning outcomes (save possibly few exceptions such as Rwanda and Ethiopia). I’m making two points here: that by setting the incentive at the school/teachers/parents level not only do we focus on the people most able to promote early grade literacy and numeracy, but that this approach may create the most powerful pressure up on national levels to do their part. This is of course only a belief now, and it may be really useful to see if one could design thoughtful evaluation experiments to test the effects of setting the COD incentive at different levels.

@Richard Morgan You are right that COD without qualifiers does not take adequate care of equity concerns. But one could design it to do so, at least in some regards, e.g. I understand that in Ethiopia the government and DFID are considering different levels of COD payments for girls and boys, and one could also vary it by districts/provinces.

Moreover, given how little money schools/teachers tend to receive (eg. see http://www.twaweza.org/index.php?i=501, http://www.uwazi.org/index.php?i=358 and http://www.uwazi.org/index.php?i=357) a transparent, well administered COD targeted at local levels may be a really effective way to get resources to poor communities at scale, and in this way have the greatest equity gain.

Still this would not take care of inclusion of e.g. children with disabilities, and that is why COD is not a panacea nor meant to replace other necessary interventions. Rather its one intervention on top of others that seeks to address a particularly large problem affecting children from poor households.

Regarding participation in outcome monitoring, having seen/ been part of many elaborate efforts (including some via UNICEF) I have come to believe that the best bet often is to do practical transparency and parents/teachers/community actors will figure out whats needed. In my view that is also the best way to address @Subrun’s concern above. Uwezo is an experiment to do that in East Africa that is being closely studied by researchers linked to MIT and Princeton, will be good to see what they find over the next 3 years.

Dear all
I just want to this great debate one thing and that is the use of media in this whole concept of making aid work for the communities and helping rather guiding the governments to be an effective partner in this whole process. I have always advocated this and would like this community and the CGD to consider this as well. Use the media especially the local media as a social auditor. Given the fact that most of this aid is given to low income or third world countries where literacy is even less and where governments have nothing to do with governance and only with corruption. Therefore helping local communities develop a mechanism of accountability by involving them from planning, execution to sustainability. It will not only make locals responsible but also will make them feel part of the whole process which right now is considered as an outsiders intervention into their communities and that’s why repute of NGOs is getting a harder hit in Pakistan in the post flood situation. Where INGOs with the help of local NGOs are making money for their consultants and expats but are not delivering even half of what they present to donors through fancy power point presentations.