Program Evaluation in an Era of Equity

Program Evaluation in the Era of Equity: What this means for your work

Image Source: FreeImages

‍

The equity agenda has the potential to shape how we think about and evaluate programs for years to come. In response, program evaluators are asking new questions, developing novel techniques, and adopting new measures of success that align with equity principles. Program evaluation is a set of processes used to assess the effectiveness of programs or interventions in achieving their intended outcomes. Program evaluation can help us understand what is working and what isn’t working so that we can make informed decisions about whether to continue or end the program or take other actions. Programs often struggle with these questions: Are we reaching the people who need our services? Are they using them? Are they seeing positive impacts as a result? And if not, why not?

What is equity?

Equity is the idea that all people deserve to be treated fairly, with dignity and respect, regardless of their differences (e.g., income, race, ethnicity, gender, sexual orientation, language, religion, immigration status, disability status, etc.). In the context of program evaluation, equity is the extent to which a program is meeting the needs of the people it intends to serve or the degree to which a program is having positive impacts on its intended beneficiaries. Equity is a core principle developed for the field of evaluation by the W.E.B. Du Bois Institute for Social Research at Harvard University and the Evaluation & the Public Program (EPP) at the University of Washington. Equity is a broader framework for evaluating programs that goes beyond questions of impact and effectiveness to also include issues of fairness, inclusivity, and participation.

Why is equity important in program evaluation?

Programs that are truly successful expand beyond their initial reach and impact. For this to happen, we need to know who is being left out, why, and what we can do about it. Program evaluation can help answer these questions. But to do so, we must ensure that the evaluation is designed with equity principles in mind. Equity-oriented program evaluations include methods that enable us to identify and understand who is being left out and why, and also provide information on how to better include those people. A common problem in program evaluation is that people who are not served by a program are less likely to participate in the evaluation. This means that their perspective and experiences go missing from the evaluation, resulting in an incomplete and potentially biased picture of the program. There is growing recognition that many of the traditional methods used in program evaluation create significant barriers for people who are left out by the process. Equity-oriented program evaluation methods are designed to reduce or remove these barriers.

Key questions for evaluating equity-oriented programs

- In whose name are we doing this work? Whose experiences and perspectives are shaping our ideas, discussions, and decisions? Whose needs and experiences are we trying to address? - For whom is this work missing? Who is not being reached by the program? Who is not participating in the evaluation? Why are they missing? - What are the barriers that prevent people from participating? Are there ways to remove the barriers to participation?

Novel methods for evaluating equity-oriented programs

- Focus groups with people who are not participating in the program evaluation: One method for addressing the first question above is to invite people who are not participating in the program evaluation to participate in a focus group discussion. This provides an opportunity to learn more about their experiences and perspectives. People who have not been participating in the evaluation are often those who are being left out. - Mixed methods approaches: Mixed methods approaches combine quantitative and qualitative methods, such as surveys and interviews, to provide a more complete and holistic understanding of program impacts and outcomes. - Engagement-oriented evaluation approaches: Engagement-oriented evaluation approaches involve meaningful participation from program participants in designing, conducting, and analyzing the evaluation. Through these approaches, people who are served by the program play a greater role in the evaluation, creating a more comprehensive and complete picture of the program’s impacts and outcomes. - Creative and participatory methods: Creative approaches to program evaluation expand the ways in which the evaluation is done, including the use of art, poetry, and other creative methods. Participatory methods involve participants in designing and conducting the evaluation. Both creative and participatory methods are intended to broaden the reach of the evaluation by making it more accessible and open to a broader range of people and perspectives.

Conclusion

Programs are most effective when they reach a broad range of people and address their needs and experiences. Equity-oriented program evaluation can help us identify who is being left out and why, and suggest ways to more fully include these people. As the field of program evaluation evolves, we are seeing increasing interest in equity-oriented evaluation approaches. This is welcome and necessary because the equity agenda has the potential to shape how we think about and evaluate programs for years to come. In response, program evaluators are asking new questions, developing novel techniques, and adopting new measures of success that align with equity principles.