Many key stakeholders want to know how programs are performing, and some of these questions can be answered with data. Others are more complex. However, all inquiries should be measured for impact, not simply to determine the effectiveness of a particular program. Data collection and analysis must be focused on the question at hand to measure impact. For example, if the goal is to improve the cost-effectiveness of a program, data collection and analysis should include detailed cost data by site and product. However, such data collection and monitoring do not constitute impact evaluation.
Impact evaluations are a powerful strategy for identifying programs or policies that make a difference.
These evaluations are strategic and can influence critical policy decisions. However, several types of impact evaluations fall into two general categories: experimental and quasi-experimental. In experimental designs, a counterfactual is used to identify an alternative control group, whereas a quasi-experimental comparison group is not randomized. The primary advantage of an impact evaluation is its ability to uncover how a specific intervention or policy affects various dimensions.
A good impact and evaluation of outcomes has clear objectives and is based on measurable metrics. The purpose of the assessment can be any of the above. For example, it can determine whether a program should continue or not or decide whether it is making a difference. The purpose of the evaluation should be clear and agreed upon from the beginning so that the team and stakeholders will have clear expectations and know what to do with the results.
Impact evaluations are not appropriate for all interventions. Specific data and financial resources are required for impact evaluations. An evaluator must collaborate with the program planners and donors to develop the evaluation. For further information on impact evaluation, check out the resources below. The impact evaluation methodology is a rigorous process that can help identify the programs or policies that make a difference.
Impact evaluations can be undertaken only when the conditions are right.
Firstly, impact and evaluation should only be undertaken when the intended use is straightforward, resources are available, and the decision to implement an intervention is timely. This can be assessed by conducting an evaluability assessment. Once the conditions are right, a participatory approach should be considered to identify the most appropriate interventions for evaluation. Participation should occur at various stages of the impact evaluation process: from deciding to undertake an evaluation to design, data collection, analysis, reporting, and management.
Impact evaluations need to address gender issues.
The United Nations Guiding Principles for Business and Human Rights (UNGPs) were endorsed by the UN Human Rights Council in 2019. While they outline the duties and responsibilities of governments and businesses, they fail to give any substantive guidance on how to implement gender equality. These principles are vague and only acknowledge gender in general terms while failing to address intersectional gender issues. In addition, they do not address the role of ‘vulnerable groups’ or the intersectionality of these issues.
The 2030 Agenda for Sustainable Development outlines the importance of addressing gender equality. This goal cuts across all SDGs, including the SDGs on health and education. The i2i gender program identifies gender-specific market failures and constraints in two key growth areas. The report also identifies gaps in four areas where gender inequalities must be addressed to advance development.
Gender-responsive context analysis is an increasingly important area of focus in assessing project impacts. This is because gender-sensitive context analysis involves collecting and meaningful use of sex-disaggregated data, and gender needs to be embedded throughout the impact assessment process. By doing so, project-level evaluations will be more comprehensive. In addition, gender-responsive impact analysis will also improve the likelihood of success by incorporating the perspectives of women and men.