EA - GiveWell's updated estimate of deworming and decay by GiveWell

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: GiveWell's updated estimate of deworming and decay, published by GiveWell on April 3, 2023 on The Effective Altruism Forum.Author: Alex Cohen, GiveWell Senior ResearcherThis document describes the rationale for the decay adjustment in our deworming cost-effectiveness analysis. We have incorporated this adjustment thanks to criticism from the Happier Lives Institute.In a nutshellThe main piece of evidence we use for the long-term effects of deworming is an RCT in Kenya with follow-ups at ~10 years (KLPS-2), ~15 years (KLPS-3) and ~20 years (KLPS-4) after children received deworming treatment. While these surveys show decline in effect on ln earnings and consumption over time, we have typically viewed the different estimates across surveys as noisy estimates of the same effect and assumed the effects of deworming are constant throughout a person’s working life.We now think we should account for some decay in benefits over time. We incorporate this decay by making the following key assumptions:We put 50% weight on the interpretation that the different estimates over time are capturing true differences in effect size. While the data point to an estimate of decline, the confidence intervals are wide and there are differences in how data were collected over time, which make us reluctant to put full weight on KLPS 2-4 capturing true decay over time.We set a prior that the effects are constant over time. This is based on a shallow literature review of studies of interventions during childhood where researchers reported at least two follow-ups on income during adulthood. We find a similar number of studies finding a decline in effect as an increase in effect over time.We update from that prior at each time period (10-years, 15-years, and 20-years), using the informal Bayesian adjustment approach we’ve used previously.We then extrapolate effects through the rest of the individual’s working life based on the measured decline from 10-year to 20-year follow-up.Our best guess is that we should apply a -10% adjustment due to the possibility of decay in effects over time. While the decline in effects in later years leads to lower cost-effectiveness, this is partially counterbalanced by higher estimated effects in earlier years and by our putting only 50% weight on the interpretation that declines in measured effects across follow-ups reflects a true decline in effect over time.We have several uncertainties about this analysis:This decay adjustment builds on top of our current Bayesian approach for estimating the effect of deworming. As a result, it's subject to the same limitations of that approach. It’s possible that in the future we should overhaul our approach, which could lead to meaningful differences in how we incorporate decay.The model is sensitive to our prior on whether effects should decay or not, and our current prior is based on a shallow literature review. If we expected effects to decay, we would include a stricter adjustment because we would (i) be updating from a prior where decay was already occurring and (ii) put more weight on the decay interpretation. We could potentially refine this estimate with a more thorough review of the literature and additional data analysis.The weight we put on whether these are noisy estimates of the same effect or different effects over time is based on a qualitative and highly subjective assessment. Putting higher weight on the surveys capturing different effects over time, for example, would lead to a stronger discount.What we did previouslyThe main piece of evidence we use for the long-term effects of deworming is an RCT in Kenya that measures effects on income at ~10 years (KLPS-2), ~15 years (KLPS-3) and ~20 years (KLPS-4) after children receive deworming treatment.[1]Our typical approach has been to pool effects...