By Arielle Levites and Alisa Rubin Kurshan

Philanthropists are driven by a desire to make positive change. But many well-meaning and sometimes expensive interventions don’t always produce measurable positive results. In some worst case scenarios they even unintentionally do harm.

A popular program to reduce teen pregnancy rates gives teens a “robot baby” to care for. A longitudinal study found that not only did the program not reduce pregnancy rates; in fact, it increased them. (These kinds of programs are still ongoing in about two-thirds of American school districts, in large part because students and schools like the program.)

The good news is that over recent decades, in the field of Jewish philanthropy, plans for serious evaluation have been included as part of many grants. In fact, evaluation has become an expectation of philanthropic boards; it is no longer enough to engage in what was affectionately called expressive philanthropy with a mindset of, “We care about Jewish education, so let’s give grants to the Jewish schools and camps. I am sure they will use it well.” Philanthropies, including Federations and private foundations, have professionalized. Grants are awarded after clear review of plans of action that includes goals, timelines, benchmarks, outputs, and intended impact. Final reports from grantees include program evaluation, recorded outputs, and measured impact.

Quality program evaluation has been an important step in continuing to improve the field of Jewish education. Going beyond program evaluations to design broader research studies governed by a spirit of transparency and shared knowledge will help our community learn and improve our interventions significantly. This collaborative approach elevates the entire field of Jewish education.

Research and evaluation are both forms of disciplined inquiry that can advance knowledge. Of course, evaluations are, by nature, local studies focused on specifics of particular programs. Still, in putting findings together from multiple studies we can find general patterns, relationships and idiosyncrasies. Fundamentally, research is governed by a set of norms regarding transparency and the commitment to building on the work of others. When foundations engage in systematic, disciplined inquiry, be it evaluation or research, they create the possibility for all of us to get smarter together.

Unfortunately, and perhaps inadvertently, program evaluations are sometimes favored over research. In failing to invest in a larger Jewish education research infrastructure, philanthropists can miss an opportunity to advance the whole field. Moreover, when foundations choose not to share findings that reflect poorly on their investments, they diminish our collective capacity to learn and make a difference.

Sesame Street’s early research found they were failing to meet their own goals for boosting school readiness for America’s most vulnerable children. They shared their disappointing findings with others, gained insight into where they were falling short, and retooled their program to meet their aspirations for early childhood education. In acknowledging their early shortcomings Sesame Street became a powerful tool for advancing the academic, social, and emotional development of millions of children around the world.

We can learn more from our mistakes than our successes. But we can only learn from mistakes that we acknowledge and seek to understand. Not every intervention works as intended. Sometimes well-meaning ideas don’t deliver on their promise. Some programs work, but only for some people, some of the time, in some contexts. When we begin to understand these patterns in Jewish education, we can let go of some ideas, re-design others, and even mix and match features of several. We can do so intelligently because we have real evidence, not just a desire to help.

We argue for making information gathering ongoing and systemic, rather than episodic and opportunistic, and for sharing findings and failures with others. In this way we will ensure that we engage in reform and innovation in Jewish education knowingly, with a clear grasp of where Jewish learners and teachers are and where they are going in a landscape of good information and evidence rather than in a field of dreams and wishful thinking (no matter how beautiful and well-intended),

Every philanthropic entity gets a gift from the greater society in that they are not obligated to pay taxes. In return they make a promise and incur an obligation to give back to society. At a time of crisis and an ever-growing scrutiny of the degree to which philanthropy can be a tool for social progress, one important way philanthropists can make good on the social contract is by supporting research designed to be transparent and advance shared knowledge that clearly demonstrates how their generosity grows the common good.

Arielle Levites, PhD is the Managing Director of CASJE, the Consortium for Applied Studies in Jewish Education, housed at George Washington University’s Graduate School of Education and Human Development.

Alisa Rubin Kurshan, PhD currently consults with newly established Jewish organizations after serving for decades as the senior planner of UJA-Federation of NY. She also serves on the advisory board of CASJE.