Birthright AFRICA Impact Report & Measurement Tools

Birthright AFRICA inspires youth and young adults of African descent ages 13-30 to explore their cultural roots and legacy of innovation as a birthright (free) within the U.S. and nations in Africa. With impact data from 5 years of travel programs, Birthright AFRICA was poised to publish its first impact report in 2024 and to continue subsequent annual reports. Cohort 5’s Lisa Sargent worked with Birthright AFRICA to evaluate this data and co-author the impact report; they also provided expertise on how to better sync the organization’s theory of change with their assessment measures to tell the current story of program impact, while refining their assessments to convey that impact with greater accuracy moving forward. The impact reports also supported fundraising efforts as Birthright AFRICA conveyed its case for support — transforming the African diaspora for generations to come to be global leaders and entrepreneurs that are proud, confident, and connected to the continent.

• About Birthright AFRICA

Greenwood Initiative Debt Reduction Program – Bloomberg Philanthropies Historically Black Medical Schools Award Evaluation

In September 2020, Bloomberg Philanthropies made a $100 million commitment as a part of the Greenwood Initiative to invest in a generation of Black doctors by providing debt relief to medical students attending the nation’s four historically Black medical schools. Each school was awarded a portion of this commitment based on the number of eligible students and complementary wraparound services. Across the schools, 962 students were initially identified to participate in the Debt Reduction Program (DRP) at the start of the 2020-2021 academic school year. Cohort 5’s Shardae Osuna worked with Decision Information Resources to design and implement a project that evaluates DRP’s goals to enable Black medical students to attend school with reduced financial burden and to invest in future Black doctors toward building Black wealth.

• About Decision Information Resources, Inc.

Diversity, Equity, Inclusion, and Belonging Impact and Evaluation

The overarching goal of Dornsife School of Public Health’s (DSPH) diversity, equity, inclusion, belonging (DEIB) and antiracism assessment tools is to create and sustain the structures, policies, and culture changes needed to ensure both the success of diverse scholars and the production of scientific knowledge relevant to eliminating health inequities in our society. Cohort 5’s Endaisia Love worked with DSPH to update and design new metrics that better reflect their growth and emerging challenges in tandem with the development of the school’s strategic plan.

• About Drexel University Dornsife School of Public Health

Post-Grant Application Survey Analysis and Recommendations

In 2021-2022, the Hartford Foundation engaged in a grantee and staff-informed redesign of its grants process with the goal of ensuring that it is inclusive, accessible, and functional for both staff and grantees. One of the changes made to the grants process was the re-introduction and update to a post-application survey that would be sent to any organization that applies for a grant. Cohort 5’s Vanessa Segundo worked with the Foundation to apply CREE approaches to analyze post-application survey data collected to date; summarize findings, including identifying any strengths or needed improvements to the grants process; and identify potential improvements to the survey instrument, its administration (e.g., encouraging responses), its future use, and dissemination of results and results-based actions or grants process modifications with grantee partners.

• About Hartford Foundation for Public Giving

Deepening Diversity, Equity, Inclusion, and Accessibility Knowledge and Practice

Manhattan Strategy Group (MSG) has drawn from staff and subject-matter-expert understanding and skill to develop a diversity, equity, inclusion, and accessibility (DEIA) lens for specific initiatives, such as its National Charter School Resource Center and the Equitable Transition Model Technical Assistance Support. Cohort 5’s Sophonie Fenaud collaborated with MSG to catalog DEIA use across projects, highlight where they are excelling, and identify opportunities for deeper engagement.

• About Manhattan Strategy Group

Evidence Building with a CREE Lens to Improve the Lives of Children and Families Impacted by Domestic Violence

The Promising Futures National Capacity Building Center to Expand Services to Children, Youth, and Abused Parents is a project of Futures Without Violence that provides technical assistance and training to strengthen and advance prevention and intervention innovations, systems change, and policy solutions to improve the lives of children and families impacted by domestic violence (DV). MXM Research Group conducted a principles-focused evaluation of Promising Futures and supported the Promising Futures team to take stock of its TA approach and cross-site evaluation methods. Cohort 5’s Swathi Reddy supported a component of the cross-site evaluation to strategize, design, and execute an approach that answered the following questions: What types of evidence are important to spotlight and emphasize to decision-makers in the ecosystem to improve outcomes for children and families who experience domestic violence and address inequities? What are grantees measuring? What are their outcomes? What are some universal measures that every DV program should look to capture and design around? How can we promote these while holding context sensitivity and centering equity?

Developing the Theory and Groundwork for a Data + Soul Approach to Research and Evaluation

In recent years, the fields of social science, research, evaluation, and design have undergone a new round of critical examination with a lens of power, equity, ethics, and participation. There are a plethora of tools and frameworks available to researchers and evaluators around data equity and ways to incorporate CREE approaches to the work, however, shifting practice takes time, exercise to change muscle memory, and shifts in the way teams and institutions do this work. Building off a decade of research and evaluation practice, the team at MXM Research Group has spent 2023 exploring and developing what a Data + Soul approach to research and evaluation looks like at the individual and team levels. The Data + Soul approach is grounded in CREE, appreciative inquiry, data equity principles, emergent strategies, and trauma-informed evaluation. In 2024, MXM designed and rolled out Data + Soul Studio as a new project and service offering that aims to bridge gaps in the current evaluation ecosystem. Cohort 5’s Dr. Miranda Hill supported the team in establishing the pedagogical underpinnings of the Studio and documenting its roots by synthesizing literature and capturing ideas from team discussion and practice. Dr. Hill brought CREE and her professional and lived experience into conversation with the team as they designed and rolled out the Studio.

• About MXM Research Group

The Evaluation of the National Science Foundation’s Robert Noyce Teacher Scholarship Program

The National Science Foundation’s Robert Noyce Teacher Scholarship Program provides funding to higher education institutions via (a) scholarships, (b) stipends, and (c) other means of programmatic support to recruit and prepare science, technology, engineering, and mathematics (STEM) majors and other professionals to become K- 12 teachers. The overall goal of the Noyce program is to “increase the number of K-12 teachers with strong STEM content knowledge who teach in high-need school districts.” WhitworthKee’s evaluation was focused on understanding the recruitment and application process and perceptions of the Noyce program within the community. Cohort 5’s Eniola Idowu collaborated with WhitworthKee Consulting to support the engagement of an equity advisory committee, bring a CREE lens to the evaluation plan, support the development and refinement of qualitative data collection instruments, and develop a pilot testing approach.

• About WhitworthKee Consulting

Integrating Equity into Quantitative Evaluation Solution Toolkits

Over the years, Abt Associates has developed substantial internal resources for staff to facilitate the quality and efficient execution of evaluation research in practice. Cohort 4’s Dr. Diana Serrano worked to enhance Abt’s existing resources to provide concrete, nuts-and-bolts methodological practice guidance that ensures integration of equity throughout an evaluation project’s life cycle. Dr. Serrano produced updated resources that can serve Abt staff in their ability to integrate equity into the quantitative methods and approaches used to carry out major impact evaluations in practice and led a workshop that taught staff about the enhanced resources and how best to use them.

Updating Five Principles of Effective Financial Education

In 2017, the Consumer Financial Protection Bureau (CFPB) released its “Five Principles of Effective Financial Education” report. CFPB’s mission is to ensure that the consumer financial marketplace is fair, transparent, and reliable. To that end, CFPB conducts research on effective financial education to help consumers make sound financial choices and improve their financial well-being. In collaboration with DFPB, Abt is updating the report with an equity lens to reflect new research and best practices that have emerged since 2017. In this work, Cohort 4’s Dr. Paula Caffer reviewed existing financial education programs and conducted qualitative data collection with financial education practitioners.

• About Abt Associates

Greenwood Initiative Debt Reduction Program – Bloomberg Philanthropies Historically Black Medical Schools Award Evaluation

In September 2020, Bloomberg Philanthropies made a $100 million commitment as a part of the Greenwood Initiative to invest in a generation of Black doctors by providing debt relief to medical students attending the nation’s four historically Black medical schools. Each school was awarded a portion of this commitment based on the number of eligible students and complementary wraparound services. Across the schools, 962 students were initially identified to participate in the Debt Reduction Program (DRP) at the start of the 2020-2021 academic school year. Cohort 4’s Noemi Avalos worked with Decision Information Resources to design and implement a project that evaluates DRP’s goals to enable Black medical students to attend school with reduced financial burden and to invest in future Black doctors toward building Black wealth.

• About Decision Information Resources, Inc.

Reinforcing Learning and Evaluation for Foundation Staff — Strengthening CREE Approaches

The Hartford Foundation for Public Giving seeks to improve its staff’s understanding of culturally responsive and equitable evaluation (CREE) approaches and how to incorporate these into organizational learning practices and evaluative thinking. Cohort 4’s Elidé Flores-Medel served as a thought partner to create content for a learning session for Foundation staff to learn how to incorporate CREE strategies, concepts, and thinking in their day-to-day work (e.g., conversations with grantees, reporting expectations, funding programs such as RFPs, landscape assessment). Elidé also developed internal tools that the Community Impact staff will use in practice.

• About Hartford Foundation for Public Giving

Preschool Development Grant: Evaluation Design and Potential Implementation

The Preschool Development Grant was implemented to make it easier for children and families to get what they need to thrive. Embarking on its third year, the grant builds on the work of Early Childhood Systems reform and shares the same vision of focusing on racial, geographic, and economic inequity to ensure all children are born healthy and thrive. Cohort 4’s Dr. Ozen Guven Yildiz collaborated with the grant’s evaluation team to design and implement Black, Indigenous, and People of Color (BIPOC) family workshops with a culturally responsive and equitable lens.

Youth in Transition Longitudinal Study and Innovation Grant Program

The Minnesota Department of Human Services Child Permanency and Safety Division has participated in a longitudinal survey since 2010 to gather data about educational attainment, employment, public benefits use, housing, and incarceration of youth ages 19 and 21 who have experienced out-of-home placement. The survey gives Minnesota the opportunity to understand how it compares nationally; monitors youth outcomes over time, at different ages, and in different regions; and respond appropriately with funding and advising to local agencies. With The Improve Group, Cohort 4’s Dr. LaJoy Spears reviewed the design of the grant evaluation and longitudinal study with an equitable lens. Grounded in the knowledge of the systemic and structural racism that has resulted in the disproportionate separation of families, Dr. Spears engaged stakeholders in interviews and strategizing with the team on effective youth engagement.

• The Improve Group

Supporting Michigan Public Health Institute’s Learning and Evaluation Partnership With W.k. Kellogg Foundation and Their Racial Equity Anchor Grantees

Racial Equity Anchor Institutions prioritize outcomes addressing systemic barriers to opportunity that lead to a lack of access to early childhood education, employment, healthcare, and reduced community power; and, increase funding and partnerships in support of vulnerable children, their families and communities, to eliminate racialized outcomes. Michigan Public Health Institute (MPHI) is assisting grantees of this Initiative to build their evaluation skills, so they are better able to measure individual and joint efforts and ensure they are contributing to equitable communities where children thrive. Cohort 4’s Dr. Brittany Marshall, Dr. Janelle Armstrong-Brown, and Natalia Ibañez assisted the Institute with evaluation for annual grantee portfolio-level, and building CREE capacity amongst grantees through debriefing and discussing findings, providing tools and resources around CREE, updating the Racial Equity Anchors Portfolio Theory of Change, and providing technical assistance.

• About Michigan Public Health Institute

Using Qualitative Research to Understand How Medicaid Policy Informs Collection of Social Determinants of Health and Race, Ethnicity, and Language Data

This project seeks to understand how data on social determinants of health (SDOH) and race, ethnicity, and language spoken at home (REL) are collected in Medicaid claims and encounter data, and how federal and state Medicaid policy can better support collection of this data to advance health equity for Medicaid populations. Cohort 4’s Dr. Jovita Murillo worked with NORC to conduct an environmental scan to understand how state Medicaid policy informs documentation of SDOH and REL data. The scan will include a literature review of existing state policies around SDOH and REL documentation; virtual interviews with key states, providers, managed care organizations, and other key players to understand barriers and facilitators to collection of SDOH and REL data and strategies to address these barriers; and review of public use files in other data sources that could provide SDOH or REL data at the census track, zip code, or county level. Dr. Murillo also worked to produce data and policy briefs on the state of SDOH and REL data collection in Medicaid claims and encounter data, and describing states’ experiences collecting the data, challenges, and lessons learned to improve the collection of these data to improve health equity. This project sought to understand how data on social determinants of health (SDOH) and race, ethnicity, and language spoken at home (REL) are collected in Medicaid claims and encounter data, and how federal and state Medicaid policy can better support collection of this data to advance health equity for Medicaid populations. Dr. Jovita Murillo worked with NORC to conduct an environmental scan to understand how state Medicaid policy informs documentation of SDOH and REL data. The scan included a literature review of existing state policies around SDOH and REL documentation; virtual interviews with key states, providers, managed care organizations, and other key players to understand barriers and facilitators to collection of SDOH and REL data and strategies to address these barriers; and review of public use files in other data sources that could provide SDOH or REL data at the census track, zip code, or county level. Dr. Murillo also worked to produce data and policy briefs on the state of SDOH and REL data collection in Medicaid claims and encounter data, and described states’ experiences collecting the data, challenges, and lessons learned to improve the collection of these data to improve health equity.

• About NORC at the University of Chicago

Exploration and Advancement of Liberatory Evaluation: Learning Circle

Two Gems Consulting is facilitating a Liberatory Evaluation Learning Circle — a collaborative space for analyzing and envisioning liberatory evaluation — with committed members of the Global Majority. Cohort 4’s Chloé Greene, Elidé Flores-Medel, and Erika Gaitan worked to co-facilitate exploration of liberatory approaches to evaluation by organizing readings and developing discussion questions for the Learning Circle, as well as collecting and delivering insights in a way that builds collective power. Building on resources that Two Gems Consulting Services have either created and/or applied with their clients, the Scholars applyied their CREE expertise to the refinement of these strategies and tools to ensure they are empowering communities through data collection and evaluation.

• About Two Gems Consulting