By Josh Miller
Within Jewish tradition, an appreciation for the importance of shared measurement tools dates back to biblical times. In Deuteronomy 25:15 the Torah teaches: “a perfect and honest measure shall you have, so that your days shall be lengthened on the land…” There are many reasons why it is valuable to have and to honor our universal systems for measurement. These fundamental tools enable us to describe what we observe, compare like items, and draw conclusions from these comparisons.
And yet, despite the implicit value of shared measures, the process of developing such tools is not always simple.
The Challenge of Establishing Shared Measurement in Jewish Education
In the field of Jewish education, establishing universal measures to assess learning outcomes is particularly challenging. Within the Jewish community, there are differing and often evolving opinions about the purpose of Jewish education and the desired outcomes. As a result, standard practice among Jewish program evaluators is to custom develop measurement tools for each new study based on one specific program’s stated objectives. While the findings from these custom studies are always informative, they leave us unable to answer a fundamental question – how do the results compare?
At the Jim Joseph Foundation, we have invested time and dollars over recent years exploring the role that we, as a funder, can play in moving the field of Jewish education closer towards the adoption of shared measurement tools. Grants to the Jewish Survey Question Bank, JData, and the Consortium for Applied Studies in Jewish Education have helped key Foundation partners from the research community advance measurement, assessment, and knowledge-sharing across initiatives and varied educational settings. Looking towards the year ahead, we are optimistic that two collaborative projects now in development will take this work to the next level, as key leaders from within the field of Jewish education endeavor to develop shared measurement tools for two important age cohorts – Jewish college students and Jewish teens.
Hillel: Measuring Excellence Pilot
Under the leadership of Hillel’s new CEO, Eric Fingerhut, the entire Hillel system has recently embarked on a multi-year process to define and assess excellence across its network of 500+ Hillels across the globe. Central to this effort has been stewarding a diverse task force that includes funders, volunteer leaders, and professionals from a range of local Hillels. The objective: help everyone involved in Hillel achieve better levels of success, and identify and adapt best practices, through a shared framework of measurement.
Hillel’s work with its Measuring Excellence Task Force began in early 2014 with scoping and goal setting. This was followed by an iterative process of identifying outcomes and indicators to use in assessing local Hillels. In its current phase of the work, Hillel is developing a draft set of qualitative and quantitative measurement tools focused on student attitudes and behaviors, participation numbers, staff satisfaction and financial strength. For the student outcomes part of the work, Hillel has engaged Rosov Consulting to help develop a student survey that incorporates new questions developed by experts in the field, while also building upon previous surveys used with this age cohort.
A diverse group of eighteen local Hillels representing a range of campuses are participating in the process of developing the shared assessment tools and will serve as pilot test sites over the coming year. During this phase, the student survey and other assessment tools will be put into the field and results will be analyzed. Learnings from the pilot will lead to revisions to the tools and how they are used, with the goal of a system-wide roll-out over the coming three years.
The Jewish Education Project: Measuring Teens’ Jewish Growth and Learning
Concurrent to the work happening at Hillel, The Jewish Education Project, under the leadership of Chief Innovation Officer Dr. David Bryfman, has launched a similar effort to develop tools to measure Jewish growth and learning among Jewish teens. They have undertaken this work on behalf of the Jewish Teen Education and Engagement Funder Collaborative – a coalition of national and local funders with plans to co-invest in up to ten new community-based teen education initiatives in the United States. The Jewish Education Project will conduct this work in collaboration with American Institutes for Research (AIR) and Rosov Consulting, the team of consultants recently hired to evaluate the collective work of the Funder Collaborative.
For the Funder Collaborative, developing and using shared measurement tools is essential to ensuring that outcomes of individual community-based initiatives can be compared and analyzed together. The group also hopes that the tools being developed and the resulting evaluation findings will be of use beyond the pilot initiatives in select communities. It is for this reason that The Jewish Education Project’s process will incorporate guidance and input from a very broad group of stakeholders: researchers, academics, leaders of major Jewish organizations that serve teens, rabbis, educators, funders, parents of teens and teens themselves. Similar to the work being conducted by Hillel, survey and interview tools will take into account new thinking about how to measure Jewish identity, while also building from previous tools used by others inside and outside the Jewish world.
Especially unique to The Jewish Education Project’s process is the emphasis on incorporating the voices of teens from a range of ages, geographies and Jewish backgrounds. To achieve this objective, they plan to conduct sixteen focus groups in four cities later this year. Pilot testing of new instruments will begin in spring 2015, with refined tools ready for use in fall 2015.
Early Lessons Learned
The Jim Joseph Foundation is proud to be involved as a co-investor in both of these initiatives, and to be learning from these projects alongside funder colleagues at Lippman Kanfer Foundation for Living Torah, the Marcus Foundation and the Schusterman Family Foundation. Here are four early lessons learned from our involvement in these efforts:
- The Field Is Primed – Increased interest in collective impact initiatives and lower barriers for collaboration (across numerous organizations and denominations) have set the stage for what we see as an unprecedented willingness among funders, practitioners and community leaders to embark on shared measurement work.
- True Collaboration and Broad Stakeholder Involvement is Critical – Shared measurement tools are most useful when designed with input from representatives of the target audience and with those who are positioned to act on the learnings. This includes funders, lay leaders, researchers, practitioners and participants. Involving stakeholders early builds a sense of shared purposes, develops trust, and ensures broad investment in the process itself.
- Commitment and Flexibility are Key – Like any collaborative effort, the development of shared metrics requires a readiness to commit to a multi-phase, iterative process. It also demands participants be prepared to compromise their individual interests for the sake of the group.
- Bring Outside Experts to the Table – A project to develop and implement shared measurement tools requires unique expertise – knowledge of the field, advanced project management skills, database development, survey design, etc. There are consultants who specialize in these areas who can bring important value to the work.
We at the Jim Joseph Foundation are looking forward to seeing the results of this important work, and learning from the triumphs and setbacks. We are also working proactively to open additional lines of communication between these independent efforts to encourage alignment, when possible, between the shared measures being developed for these two adjacent age cohorts. We are hopeful that such efforts will open up unprecedented opportunities to understand and compare the most effective Jewish education and engagement strategies at multiple stages of youth and young adult development.
We look forward to sharing more about this work as it progresses. In the meantime, if you have thoughts and suggestions for us and our partners to consider during the formative stage of the projects, we welcome your comments and feedback.
Josh Miller is a Senior Program Officer at the Jim Joseph Foundation, which seeks to foster compelling, effective Jewish learning experiences for young Jews in the United States. This is cross-posted from the Jim Joseph Foundation blog.