Ignorance Has High Costs

PROGRESA, Mexico. In Mexico, a change in government is a serious threat for the survival of social programs. Programs closely associated with the public image of a president are especially vulnerable, as are those perceived to have been abused by a specific political party for electoral purposes (Levy and Rodriguez 2005). 

The study has also yielded longer-term benefits in terms of improved knowledge and basic research. Academic researchers and others have used its rich dataset to study a range of topics from different aspects of job training interventions to evaluation methodology itself 

Credible evaluation uses robust statistical techniques to construct the counterfactual. These techniques include randomization or experimental design quasi-experiments such as matching, regression discontinuity design, or double differences and nonexperimental or instrumental variable methods (annex 6.3). All these techniques try to guarantee the comparability of the treatment and control groups by avoiding what statisticians refer to as bias (box 6.7). 

One of the major drawbacks of the quick and easy types of evaluation is that they tend to generate different results over time. For example, two separate evaluations of Peru s social fund completed a year apart and using different methodologies and data arrived at opposite conclusions on key impacts. This and similar examples reinforce the point that evaluation design is critical. 

Four main methods are available for estimating the impact of a program  

---