“I feel like our initial job is really to help people see the value of evaluation and see how they actually use evaluation processes every day in their lives but may not define it that way.”

– Sandra Silva

As 2020 made abundantly clear, changes and transitions can be difficult to manage. And while there is no one-size-fits-all approach to adjusting to new circumstances, evaluators provide a diverse array of strategies that aid clients and partners in achieving their desired outcomes. Evaluation is the process of systematic and intentional inquiry and the use of data to inform decision-making around a specific organization, program, strategy, or initiative. At Expanding the Bench (ETB), we pride ourselves on creating a network of diverse evaluators, but cannot forget the expertise of our in-house staff.

The Expanding the Bench Team would like to highlight Evaluator Sandra Silva. As the director of the Leaders in Equitable Evaluation and Diversity (LEEAD) program for ETB, she champions the practice of culturally responsive and equitable evaluation (CREE) and is uniquely positioned to discuss why this approach makes for better evaluation.

What drew you to the field of evaluation?

Evaluation is very systematic and that’s how my brain naturally works. I think that once I joined my first evaluation project, I was really interested in developing an evaluation plan and thinking about, “What are the questions that we’re really trying to investigate here? What will be most useful?” There are a lot of decisions that you have to make. And then you have to structure a process to collect that information, analyze it or make sense of it, and then report it out.

I also really like talking to people and collecting information. I love doing interviews and focus groups, making sense of the project, and asking, “What are the learnings coming out of this work? How can this improve the program/project?”

How did Change Matrix (the organization tasked with leading the ETB initiative) come to adopt a CREE approach?

When Change Matrix began managing ETB, we needed to define how we were approaching evaluation. I read the literature surrounding equitable evaluation and culturally responsive evaluation and drew from the literature to identify the key elements that we thought were important to include. Then the team workshopped the definition over time with input from the ETB Advisory Team.

You’ve worked for decades as an evaluator with agencies that practice a more traditional evaluation approach. What would you say are the biggest differences between a CREE-focused approach and what is standard?

I would say that the traditional aspect or approach to evaluation is that the evaluator is the “expert.” So that person is making the key decisions around the evaluation, maybe with some input from the client. That approach really centers the evaluator and his or her expertise. At Change Matrix [and Expanding the Bench], when we go into a community for any sort of project, we approach with humility. We’re really there to learn. We’re not there to say that we are the experts, because we really want to engage people in the community as a part of the process so that the decisions and the inputs that shape the evaluation processes are informed by the people who are impacted by the work that is happening. We want to understand and honor the context and the history.

What is the first step in working with communities?

We have this exercise that we do when we first go into communities and we say something like, “What is a word that comes to mind when I say ‘evaluation?’” And most responses are negative and fear-based. I feel like our initial job is really to help people see the value of evaluation and see how they actually use evaluation processes every day in their lives but may not define it that way. We use a cooking exercise to walk people through how you do make a lot of these similar assessments in your life and don’t think of it like evaluation. We break people up into groups and have them each select a recipe. The process of walking through a recipe is much analogous to evaluation because you have to have the inputs, the ingredients, and as you’re talking through the “program” then you’re walking people through preparing a recipe.

How do you build trust with new communities that are being evaluated?

It begins with relationships and finding ways to connect with people. I think people can sense when you come from a place of humility and are open to understanding their experiences, learning about the history and context of a place, and hearing their concerns. I think this fear of evaluation in communities is well-founded—the feeling that evaluators have been extractive. And, again, if you think about the standard approach of “I’m the expert,” you often see evaluators and researchers that have gone into communities, collected information, and then disappeared. At Change Matrix, we work hard to establish how an evaluation might support community goals and establish processes that will inform evaluations that are of value to communities and can help drive change.

Learn more about Sandra on the Change Matrix website or learn about the LEEAD Program and how you can get involved.