Print
Share
Priorities for AIDS Spending: Evaluating Interventions Individually Obscures the Benefits of Synergy
September 30, 2011
Last Friday I asked “How would you spend an additional $10 billion on AIDS in Africa over the next five years?” On Wednesday I learned how a panel of five distinguished senior economists who had never before worked on the AIDS epidemic would do so. Here’s how they decided to spend the hypothetical additional $10 billion dollars.
| Intervention | Cost (Five years, Million US$) |
| 1. Scale-up vaccine funding by $100 million per year | 500 |
| 2 Introduce medical infant male circumcision | 3,150 |
| 3 Prevent mother-to-child transmission | 140 |
| 4 Make blood transfusions safe | 2 |
| 5 Scale-up ART enrollment | 6,208 |
| Total | $10,000 |
This seems a surprising list in several ways. First, economists tend to give priority to government interventions which attempt to correct market failures, such as those caused by “externalities” (i.e. spillover effects) or “asymmetric information”. Neither the authors nor the panelists analyze the two interventions which target populations where spillover effects and asymmetric information enhance the benefit cost ratio of interventions: High risk groups and couple counseling.
Spillover effects: In situations where the behavior of identifiable subsets of the population make a disproportionate contribution to the HIV epidemic, a targeted intervention can have a much higher social benefit-cost ratio than an untargeted one. While the heterogeneity of sexual behavior is evident both within and across African countries and has been known since the 1976 paper of Anderson and May to accelerate a sexually transmitted epidemic, the only assessment paper that recognized this heterogeneity is that by me and Geoff Garnett on ART. In particular, the assessment paper on sexual transmission, authored by Jere Behrman and Hans-Peter Kohler, ignored the heterogeneity of behavior and therefore omits analysis of the potential benefit-cost ratio of interventions targeted at most at-risk populations (MARPs).
Asymmetric information: As pointed out in Susan Allen’s comment on my Friday post, Behrman and Kohler’s analysis of “large scale testing and counseling” omits any consideration of targeting HIV testing to couples, an option that reduces the asymmetry of information regarding HIV status and therefore should have a higher benefit-cost ratio than would the individual testing on which the authors focus.
Another surprising feature of the Copenhagen Consensus list is its inclusion of an intervention which was never analyzed by any of the assessment papers and only mentioned as an additional or supplemental idea by one of the discussants: infant male circumcision. It displaces the circumcision of adult men, which the panelists ranked in 7th place, and deprived of any of the $10 billion budget. What happened? How and why did the panel become so “creative”?
Here is the information that the assessment paper authors gave the panel on adult male circumcision (AMC) and on the intervention to which they allotted the largest budget share, ART. At the discount rate of 3% and the value of a life-year of $5,000, these two interventions would have the following benefit-cost ratios:
| Intervention | Cost per life year saved | Benefit -Cost Ratio | |
| Adult male circumcision | |||
| Three rigorous randomized trials have confirmed 60% efficacy in protecting a man from infection | – Optimistic Scenario (60% effective) | $41.50 | 120.5 |
| – Pessimistic Scenario (30% effective) | $83 | 60.2 | |
| Anti-retroviral treatment | |||
| ART not only protects the patient from disease but one rigorous trial shows it is 96% protective of partner. | – Optimistic Scenario (90% effective) | $780 | 6.4 |
| – Pessimistic Scenario (30% effective) | $1,020 | 4.9 |
So despite the fact that the authors considered adult male circumcision to yield from 12 and 20 times more benefit per dollar of investment cost, the panel of distinguished economists decided, on the basis of this analysis, that antiretroviral therapy ranked in fifth place among all the interventions, two places above the seventh place ranking of adult male circumcision. Since the panelists allocated their entire hypothetical $10 billion to the first 5 interventions, they provided no incremental funding, even hypothetically, for adult male circumcision. This decision is remarkably obtuse given the evidence on this intervention and the fact that this most promising intervention needs support to be scaled up in African countries today.
Part of the problem was that the authors of the assessment paper analyzing AMC, Jere Behrman and Hans-Peter Kohler, chose to use the 30% effectiveness assumption for their main results instead of the 60% result found in the randomized trials. This is a very conservative assumption. Given that male circumcision has been consistently and rigorously shown to prevent 60% of infections in the trials, field effectiveness could only be degraded down to 30% if either of two offsetting behavioral effects is extremely powerful: (a) Selection Bias: willingness to accept circumcision is much higher among men who would have otherwise protected themselves by condom use or having fewer partners or (b) Disinhibition: willingness to accept circumcision is much higher among men who, once circumcised, engage in many times more risky sex than they would have without the circumcision. The first effect posits that prudent well-informed men will disproportionately seek AMC, while the second posits that circumcision converts such prudent men into reckless thrill seekers. If the sexual behavior of African men were this sensitive to the risk of HIV infection, they would have ceased their risky behavior long ago, when awareness of the danger of risky sex became widespread in African societies.
But the panel of distinguished economists apparently thought Behrman and Kohler were not being conservative enough. At the meeting on Wednesday reported in USA Today, two of the panelists explained why they had downgraded AMC. They justified their decision partly by the selection and disinhibition arguments, but also partly based on the undocumented assertion that adult African men would simply find male circumcision unacceptable. Apparently in reaction to the cognitive dissonance produced by these unsupported beliefs and the authoritative estimate that the benefit-cost ratio of AMC is 60 to one, the panelists creatively introduced an 18th intervention to be added to the list they had been given: neonatal circumcision. Arguing that neonatal circumcision would be immune to the selection and disinhibition problems and would be more acceptable to African populations, they made this one of their top five interventions.
Now neonatal circumcision is not a bad idea. It can be expected to cost about the same as adult male circumcision and to eventually achieve, after a lag of 15 years, the optimistic benefits of adult male circumcision. Since the costs are incurred today, but the benefits accrue 15 years later, the benefit cost ratio must be discounted by multiplying it by (1/(1.03)^15 = .64, which yields a benefit cost ratio of about 77 to one, down from 120 to one. However, during the fifteen years we wait until the children reach maturity, the epidemic will continue to spread to a larger and larger proportion of the population.
So if neonatal circumcision crowds out adult circumcision before adult circumcision has been widely scaled-up, that would be a bad thing indeed.
Which brings me to a more general criticism of the Copenhagen Consensus’ application of benefit-cost analysis to HIV/AIDS. Epidemics are highly non-linear. They are characterized by a parameter called Ro, the “reproductive rate”, which is described so compellingly by Kate Winslet in the super new movie, Contagion. If Ro is above unity, the epidemic grows until it saturates the population. If it is below unity, the epidemic gradually disappears. The challenge then is to find the combination of interventions, which, when working together, will bring the value of Ro down below unity. That solution might well be a combination of ART, male circumcision, behavior change and an eventual vaccine. The Copenhagen Consensus process, by requiring each author to analyze only one of the 17 interventions at a time, makes it difficult or impossible to find the combination of interventions which can reduce Ro below 1.0. Thus, when applied to the control of an epidemic, the Copenhagen Consensus should modify its process to require each author to evaluate a different combination of interventions, instead of a single one. With this approach, the Copenhagen Consensus process might have had a better chance to produce sensible results for HIV/AIDS.
Possibly Related Posts
- How Would You Spend an Additional $10 Billion on AIDS in Africa Over the Next Five Years?
- Why Won’t AIDS Donors Confirm Their Best New Hope for Avoiding Future Treatment Costs
8 Responses to “Priorities for AIDS Spending: Evaluating Interventions Individually Obscures the Benefits of Synergy”
Post a Comment
We value frank and constructive exchanges and encourage you to use your real name in your comments.
October 2nd, 2011 at 12:47 am
This is a very curious set of recommendations. While the individual reports are thoughtful and well researched, the final conclusions seem to be based on very loose evidence. For example, the evidence of infant male circumcision for HIV prevention has not been established at all. The field of HIV prevention trials is full of apparently inconsistent results in randomized trials (for example, an arm of a trial with tenofovir for pre-exposure prophylaxis was recently stopped early for futility after great excitement about oral pre-exposure prophylaxis’ apparent efficacy), and extrapolating from adult male circumcision is irresponsible. Similarly, the recommendation for further investment in vaccine does not take into account the checkered history of HIV vaccine development. Past failures suggest that any future vaccine will require monumental investments and will not be fully protective. If ART’s preventive benefits are as HPTN 052 suggested, this is likely to be much more effective than any future vaccine. In some ways, the panel’s recommendations are so detached from the medical community’s perspective as to appear irrelevant.
October 4th, 2011 at 4:09 pm
Mead, Thanks for this post, though I have to admit that after reading it I was really depressed. If there were ever an ideal scenario under which to make optimal resource allocation decisions, this should have been it. But as you point out, the end result raises a huge number of questions. Doesn’t exactly instill confidence that less ideal setups can do better, or perhaps it does???
October 7th, 2011 at 11:02 am
Six papers, twelve peer reviews, and Five Wise Men to evaluate the whole she-bang. Then allocate $10 billion to five out of 17 (no, wait — 18) solutions (the Wise Men added infant MC). Regrettably, social policies and health system strengthening got labeled as ‘poor’ choices not qualifying for the incremental money.
Can we really believe that b/c ratios exceeding 100:1 are expected to happen? Are there no diseconomies of scale? Jere Behrman and Hans-Peter Kohler of Penn, and Charlotte Watts of LSHTM, has all written in with serious reservations about the results of this exercise, and voices from Geneva seem also be expressing doubts about the approach as applied in this case.
Should three-fifths of additional funds go to ART? Can the so-far unsuccessful search for a vaccine be rated as the best investment to make? Perhaps the Copenhagen Consensus will put all questions to rest by revealing how the Five Wise Men reached their conclusions. Stay tuned.
October 7th, 2011 at 4:18 pm
Erin, you may have noticed that Geoff and I made use of the HPTN 052 findings in modeling the benefits of ART. Although we optimistically assumed that the prevention benefits of ART would apply over 90% of the HIV-infected person’s lifetime (our parameter f = .9), the benefit cost ratio of ART improves only moderately when we push the average protective effect of ART from 0.7 (our focal assumption for the value of our parameter g) up to 0.9. The rules of the Copenhagen Consensus game required us to eschew other changes, such as the 40% reduction in risk behavior that the Granich et al paper assumes or the expansion of male circumcision and the invention and wide distribution of a vaccine after the year 2040 which we assumed in our work a similar model for Institute of Medicine report here. Only when we make these additional assumptions are we able to bring the reproductive rate of HIV below 1.0 and see the beginning of the decline of the burden of the disease.
Dan, I think it’s fair to say that none of the economist authors who participated in the CC exercise was pleased with the outcome – with the possible exception of Dean Jamison and Rob Hecht, authors of the “winning” paper. (Dean and Rob, do you want to weigh in here?) To me, and I think to most of the rest of us authors, the economists on the expert panel are heroes, some of virtually superhuman stature. But in announcing their results they revealed feet of clay. They sounded as lame as your average scientifically naive politician sounds when attempting to justify resource allocation decisions across technically complex alternatives within a tightly constrained budget.
So why did it come out so poorly? Does this experience suggest that even the smartest people with the best credentials in economics are rarely able to make good decisions based on a set of competing benefit-cost analyses?
Here are some thoughts.
Even the smartest people must have some technical preparation to be able to competently judge among competing highly technical proposals.
At one of the coffee breaks, Thomas Schelling expressed regret that he and the other panelists had not received some kind of technical briefing on HIV before the start of the two day meeting. It’s true that each of the “assessment papers” and many of the “perspective papers” tried to provide technical citations and background, but we did not have the time or space to lay out an introductory pedagogical presentation. The panelists might have benefited from exposure to Alan Whiteside’s HIV/AIDS: A Very Short Introduction, for example, or to Dow and Essex’s Saturday is for Funerals, which I recommended on CGD’s “What we’re reading” list not long ago. As a result, the panelists were struggling to absorb the technical information and to weigh the evidence on 17 interventions in real time over the two days. If this was the only problem, it would be fixable. Allow the panelists to spend some time learning about HIV before they consider the arguments in the assessment papers.
Benefit-cost analysis is most dependable when decisions are marginal or when the world is linear, which was not the case in this application
When fledgling economists first learn how to perform social benefit-cost analysis of proposed investment projects, the examples are of investments which are small relative to the markets in which they will operate. For small investments, we can plausibly assume that the project itself will not affect the opportunity costs of project inputs or the value of project outputs. In this application of the Copenhagen Consensus to the African AIDS problem, all of the investment decisions we analysts presented to the panelists are big and important enough to violate these assumptions, making benefit-cost analysis harder and less dependable.
Later in our studies we economists learn to apply benefit-cost analysis to larger projects or to other government interventions, which can suffer from diseconomies of scale, drive up the prices of factor inputs or drive down the value of the project’s output. But even in these more complex contexts, we typically assume linearity on every margin (e.g. constant second-derivatives of the production function, the wage bill and the total revenue function, so that all relevant first-derivatives are linear). However, in this application of the Copenhagen Consensus process to HIV/AIDS, the production technology, the supply functions and the social valuation of the projects’ consequences are arguably all highly-non-linear and sometimes might be discontinuous. In the last paragraph of my blog, I gave one example of extreme non-linearity. When an epidemic has a value of Ro slightly above 1.0 an extremely small change in infectivity, frequency of contact, or duration of infectivity can radically change the future trajectory of the epidemic from explosive growth to rapid decline. Other examples include the technology of research, in which chance coupled with very small investments can produce a game changing innovation, or the “technology” of behavior change, where large expenditures to convince people to stop littering seem fruitless, but somehow (perhaps as a result of crossing a “tipping point”) the next generation adopts more socially responsible social norms and abhors littering. One might argue that interventions to alter the course of the HIV/AIDS epidemic are simply too non-linear for the Copenhagen Consensus approach, as practiced heretofore, to have a chance of working.
If this extreme complexity is the explanation for our disappointment with the outcome of the CC approach to HIV/AIDS, two types of solutions suggest themselves: (1) change the CC approach to compare approaches in ways that fully take account of non-linearities; (2) switch our focus from the donor-driven project-oriented approach, in which supposed experts choose the investment that seems to have the highest benefit-cost ratio, to a results-based approach which rewards national and local innovators for measurable reductions in HIV infections and mortality.
Anyone else want to suggest a way to shore up the CC process?
October 7th, 2011 at 9:38 pm
Mead:
I am writing a book on public health and the changing political economies of Zimbabwe, Uganda and South Africa for the period 1980-2010. A key topic that I summarize in the book is the impact of Anti-Retroviral Therapy on people with AIDS in Africa.
Would you kindly indicate which 3 or 4 studies, in your professional judgment, best demonstrated the most successful clinical outcomes that the ART drugs had on ameliorating other AIDS defining symptoms in Africa such as diarrhea, persistent coughs, oral fungal infections, tuberculosis, or fever?
Thanks
October 10th, 2011 at 8:08 am
Charles, the evidence that combination antiretroviral therapy (ART) is effective at suppressing then opportunistic illnesses caused by HIV infection was published in the 1990′s. See my book with Martha Ainsworth entitled Confronting AIDS for a review of the evidence circa 1996. Perhaps more relevant for your purposes is the evidence showing the overall impact of successful ART adherence to productivity and the quality of life. I particularly recommend the papers by Harsha Thirumurthy and his co-authors, especially Thirumurthy H, Goldstein M, Graff Zivin J. The economic impact of AIDS treatment: labor supply in western Kenya. Journal of Human Resources 2008; 43: 511-552. and the papers by Bruce larson and his coauthors, especially Fox M. P., McCoy K., Larson B. A., Rosen S., Bii M., Sigei C., Shaffer D., Sawe F., Wasunna M., Simon J. L., \ Improvements in Physical Wellbeing over the First Two Years on Antiretroviral Therapy in Western Kenya, \ AIDS Care. 2010 Apr; 22(2):137-45.
October 10th, 2011 at 4:40 pm
Mead:
Many thanks for the excellent and wide-ranging references, including your own material. Much appreciated.
I was unable to locate a copy of your book with Martha Ainsworth, but did order *The Economics of Effective AIDS Treatment: Evaluating Policy Options for Thailand* and *Achieving an AIDS Transition: Preventing Infections to Sustain Treatment*.
I read the reports co-authored by Cootzee, Larson and Thirumurthy seeking evidence of the clinical endpoints improvements of patients on various combinations of ART interventions. I was disappointed to find so little evidence.
In the health economics studies by Larson and Thirumurthy, most of their findings concerned the household impact on children, number of days that tea pickers were able to work per month, cost-benefit analyses and various other non-health outcomes.
I was curious why the WHO’s Bangui Definition of a case of AIDS in Africa seemed not be used by them.
According to Thirumurthy, et al., “HIV-infected individuals are considered to have developed AIDS when they have one of several opportunistic infections or a CD4 count below 200/mm3. It is at this stage when functional capacity deteriorates and, according to WHO guidelines (WHO 2002), patients should be initiated on ARV therapy.”
Max Essex, et al. (eds.), *AIDS in Africa* (2nd edition, 2002) includes a Table 4 on p. 303, entitled “WHO Clinical Staging Systems for HIV Infection and Disease.” Among the 4 clinical stages and 31 opportunistic infections listed, there is no mention of CD4 cell counts.
To the best of knowledge, none of those 31 opportunistic infections ever cited by Larson or Thirumurthy.
It was the verifiable amelioration of any of those infections that I hoping to find in their works; I must have missed it somehow.
The important study by Coetzee, et al. focussed on outcomes from ART in the squalid township of Khayelitsha (outside Capetown). Yet it too never mentions which “opportunistic infections” were actually diagnosed and ameliorated with zidovudine, lamivudine and either efavirenz or nevirapine.
However, 13% of those enrolled died after starting ART. At the very outset, the authors acknowledge that Khayelitsha already suffered from an unusually high incidence of tuberculosis (1062 per 100,000) and pulmonary TB is considered by WHO at #13 on its list of clinical stage III AIDS.
I was hoping for evidence that perhaps the TB infection rate or prevalence had declined but that may not have been their primary objective.
So to sum up – I am still eager to find some random double-blind studies that demonstrate the clinical amelioration of AIDS infections, not just surrogate markers. I am eager to read your two books hoping that you covered that topic already.
Thanks again for your help.
October 13th, 2011 at 4:54 pm
Mead, thanks for this post. I share your puzzlement at the interventions selected, particularly as concerns circumcision. In addition to the other concerns you mentioned about the high rating of neonatal MC compared to adult MC, I see two more: It seems likely that adults seeking MC select themselves for the procedure based on their self-assessment of their high-risk behavior. With neonatal MC, such targeting is impossible. The risk profile of neonatal MC recipients when they become sexually active, is thus likely to be lower, perhaps much lower, than that of adult MC clients. The analysis appears also not to have taken into account that other, more cost-effective interventions including vaccines may become available before the current cohort of neonates becomes sexually active. Is it not contradictory to assign a high rating to both vaccine development and to neonatal MC? If effective vaccines become available in the next 15 years or so, neonatal MC benefits will be greatly reduced.