With 96 Evaluators and growing in the Network, we are highlighting an ACE Evaluation Network Member each month to share their experiences and current projects with the ETB® community.

Advancing Culturally-responsive and Equitable (ACE) Evaluation Network Member Nisaa Sahlem Kirtman, PhD, is a psychological and educational scholar and researcher as well as a program evaluator at Rockman et al Cooperative. She is also co-chair of the American Evaluation Association’s (AEA) DEI Working Group and is a Leaders in Equitable Evaluation and Diversity (LEEAD) Mentor. Nisaa is a social psychologist with experience designing and conducting research studies in formal and informal educational settings, and community-based settings. She uses an equity-centered approach to her research and evaluation. She studies stereotype threat; social cognition; historically marginalized groups in STEM domains (students, faculty, K-12 teachers); educational inequities; health inequities; diversity, equity, and inclusion and the science of diverse communities; climate assessments at institutions, organizations, or within post-secondary educational departments; and identity development and intersectionality. She has managed both long and short-term evaluations and several multi-year professional development projects at historically Black colleges and universities (HBCUs) and out-of-school programming for historically marginalized youth and girls of color, such as Black Girls Code. These projects are often funded by Minority Science and Engineering Improvement Programs (U.S. Department of Education), the National Science Foundation (NSF), or private foundations such as Google. Her doctoral research explored psychological coping mechanisms and ways to counteract stereotype threat at HBCUs and predominantly white institutions (PWIs) using narrative inquiry and case studies. Related, her master’s thesis explored stereotype threat at women’s colleges and co-ed colleges using experimental methods. Nisaa is also an artist and produced portraits for several decades, co-illustrating the book Restless Spirit: The Eyes of a Child by Joel Harper. She spent seven months in Zimbabwe studying Shona Stone sculpture and history. She is a mom of two and lives with her family in wine country, North Bay Area.

What first attracted you to the ACE Evaluation Network?

To put it plainly, the lack of diversity in the evaluation community and wanting to learn more from consultants, researchers, and practitioners that utilize equity centered approaches to their work. Specifically, I was interested in meeting, and networking with, other researchers of color who advocate centering research and evaluation approaches in particular around equity. A colleague of mine from REA recommended that I apply to become a member. I slowly began exploring the community and noticed that so many scholars I had heard of (and admire) were also Members, including Jara Dean Coffey, Geri Peak, and colleagues I had not communicated with in a while. The Network provides a plethora of resources that I cannot find anywhere else, including funding opportunities designed to improve the education and lives of those marginalized, articles, social opportunities, PD, and opportunities for professional service. I was given the opportunity to co-lead a panel on integrating culturally responsive and equitable evaluation (CREE) approaches into proposals and also serve as a LEEAD Mentor; these experiences have been transformative to my own professional development.

What do you value most about the ACE Evaluation Network?

The ACE community leads by example and education when it comes to equitable evaluation practices. I tend to hear often that CREE isn’t always feasible, and that research does not always have to be centered around equity in order to be a rigorous evaluation. ACE Members have taught me how equity should always be part of a program’s strategic approach, and that all research and strategies should focus on equity. And yes–we can do equitable evaluations when we are not evaluating “equity efforts” explicitly. I value the Network because it offers refreshing norms in practice that challenge us as Evaluators to implement, and also challenges Funders and community organizations. When I attended the Professional Development series in 2020, I remember listening to one Member (a Black female) describe from a recent study she was involved with in Washington, D.C. She talked about how she spoke with the program developers of a youth program about their use of the term “at-risk” and “urban” interchangeably, and that they associated being “at-risk” with being Black or Latino/a, impoverished, perhaps substance abusing, or from immigrant and/or illiterate families. Hence, the Funder and developers/grantees then designed the program and internal research questions to measure the self-esteem of participating youth, coping mechanisms and skills, and reductions in crime and teen pregnancy. Unbeknownst to them, or inadvertently, the promotion of youth deficiencies were simply perpetuating negative stereotypes about those who should be benefitting from the program. Such practices and lack of competency led to harm in the end, making youth feel ashamed to be who they are. When I listened to this Member speak about how if it weren’t for her competencies as a researcher, and her recommendations to discontinue the use of harmful terms such as “urban” and “at risk,” and promote positive youth development, that this particular program would most likely continue to design their program and internal research initiatives around group deficiencies. This is what I value about the Network and what motivates me as an Evaluator. This Network provides a space for us to reflect and share (I love the Coffee Breaks and opportunities to meet Funders). It points to the importance of diverse teams and that competencies are all core tenants to equity-centered approaches. If we look at federal programs and social policies, communities of color and historically marginalized groups are the primary beneficiaries, yet, program evaluators tend to be the dichotomy of those who either benefit the most or are left behind the most. Evaluators in the U.S. are still over 50% white and almost 70% female. This isn’t to say that one cannot be culturally competent if they are not a member of a group that has been historically marginalized, or that an evaluator of color is assumed to be competent in a culture outside of their own. I’m grateful for the ability to simply grow my (and REA’s) professional network, continue to learn, and continue to be inspired by ACE Members. The Network provides a community of researchers that recognizes our influence and how our research skills, lived experiences, and cultures can help shape the research questions asked and the inputs used to answer them.

What’s a current project you are working on?

First, my heart goes out to communities in Mississippi, primarily the Jackson area, dealing with the current water management crisis. This link includes ways to donate to Jackson relief efforts (the MS Rapid Response Coalition, Cooperation Jackson, the Immigrant Alliance for Justice & Equity of Mississippi, MS Food Network, MS Student Water Crisis Advocacy Team, and Operation Good).

Earlier this summer, a long-term partner, Jackson State University (College of Science, Engineering and Technology (CSET) & Chemistry, Physics, & Atmospheric Sciences (CPAS)), an HBCU located in Jackson, MS, was just awarded $3.6 million by NSF to fund an ACE (Achieving Competitive Excellence) Implementation Project. I designed a 5-year evaluation study and will lead this effort. The project is called Equity in STEM (ProjecTEST): Linking Faculty and Student Perceptions of Success Through Novel, Continuous, Evidence-based Assessment. ProjecTEST’s goals are to improve CSET faculty and CPAS program effectiveness in supporting student sense of belonging, persistence and success in STEM courses; completion of STEM degrees; and entry, contributions, and leadership in the nation’s STEM research and professional enclaves. Its objectives, which aim to better link faculty implementation of evidence-based approaches with improved student outcomes, are to: (1) Provide faculty with expert-led PD on the use of novel, fine-grained assessment tools, a collaborative model for their refinement, and support for implementing them in CPAS courses and provide students with targeted instruction in mathematics; (2) Increase sense of belonging, self-efficacy, and learning outcomes of STEM students through team-based research opportunities providing a mentored engagement of STEM exploratory research and problem solving to meet challenges and build readiness for STEM graduate programs and careers; and (3) Support dissemination of faculty- student team project findings to contribute to the development more equitable assessments, timetables, and benchmarks for sustaining a feasible progression to competitive STEM degrees for Black students. I’m excited to start both formative and summative evaluations, using a CREE approach throughout the study and adopting and developing instruments that measure culturally responsive STEM instruction, students’ critical thinking and problem-solving skills, and faculty-student relationships and collaborations. This is one of eight projects (funded by either NSF or the U.S. Department of Education) in which I have collaborated with Dr. Mehri Fadavi at JSU – all iterations of a similar PD model to support under-resourced schools across Mississippi, provide quality PD to K-12 teachers and undergraduate and graduate faculty, and prepare STEM students at JSU. The relationship with JSU and Dr. Fadavi has spanned over a decade. I’m extremely grateful and honored to be able to work with such an amazing team again and help inform effective PD models at JSU (and HBCUs in general), including faculty effectiveness in establishing students’ sense of belonging, perceptions that students matter in STEM gatekeeper courses, and evidence-based STEM teaching and mentoring, through research and evaluation. I miss Jackson and hope to visit the wonderful community at JSU soon.

To learn more about Nisaa, connect with her on LinkedIn.