Early Family Engagement: Measuring Growth in Jewish Identity

Evaluation and measurement are important to any project that seeks to create social change and to those who make philanthropic investments to support such work. The inherent challenges to meaningful evaluation of Jewish engagement can be daunting, especially when we seek to measure something as changing and intangible as Jewish identity.

Even when evaluation indicates a program is making a positive difference, without expensive randomized controlled research studies, it is nearly impossible to claim causality. For many of us in the field of Jewish engagement and identity building, correlation is the best we can hope for. And yet, the rationale and theory of change underlying a new endeavor must be tested and the execution of it must be tracked – those performing the work and those investing in it both need evidence in order to make informed choices about the use of communal resources. Our experiences with conducting a quasi-experimental, longitudinal study in which the same individuals are followed over time may provide a useful model.

In 2009, in partnership with dozens of local institutions, Rose Community Foundation launched MazelTot.org, Connecting Young Families to Local Jewish Life and to Each Other, a new approach to engaging young families in Greater Denver and Boulder. We believe that if parents of young children could more easily explore all of their Jewish options in one place online, and if we could make trying new Jewish activities more affordable, they would more likely become Jewishly involved. And so, families who sign up for MazelTot.org receive comprehensive, organized and digestible information about local Jewish life for the prenatal through preschool crowd, along with significant discounts and free offers. In addition, the MazelTot.org Coordinator reaches out to welcome families and help them connect to and navigate the local Jewish community.

MazelTot.org provided us with the challenge and the opportunity of evaluating a new concept in Jewish early engagement. How could we go beyond tracking website visits and sign-ups to learn whether MazelTot.org actually made a difference in the Jewish lives of more than 1,700 participating families? And, what does it mean to help a young family build a strong Jewish identity when connecting Jewishly means something so different than it used to? Engagement can no longer be measured by synagogue membership and ritual practice alone. One look at the Slingshot Guide reveals a wide variety of Jewish organizations responding in new, creative ways to the diverse needs and interests of Jewish people today. And so, we needed to determine not only how to measure, but also what to measure.

When families request their first discount on MazelTot.org, they complete the “Engagement in Jewish Life Scale,” a measurement tool developed for MazelTot.org. The scale contains items that relate to families’ cognitions, emotions, actions, and ways of partaking in community. Jewish identity is far more complex than a discrete yes or no on a list of behaviors. Thus, the scale was developed so that the measurement of Jewish identity would focus on the intensity or frequency with which people engage in both Jewish ritual and non-ritual practices (e.g., lighting candles on Shabbat or wearing clothing/jewelry that shows that one is Jewish), and not only whether they do them or not. This provides far richer, and more accurate, information because the frequency or intensity with which families engage in Jewish behaviors is indicative of the place Judaism holds in their lives.

For example, families are asked to indicate how frequently they socialize with and celebrate holidays with other Jewish families and use Jewish, Yiddish or Hebrew words in their every day speech. While they are not asked whether or not they are members of a synagogue or JCC, they are asked to indicate how often they attend Jewish religious services and Jewish cultural or educational events. They are also asked how often they participate in Jewish programs for families, and with what frequency they read Jewish websites, books or magazines, say Jewish blessings, and tell stories about Jewish events or holidays.

Families also complete this exact same scale when they request their third and final MazelTot.org discounts and again as an integral part of our annual follow-up survey. In this way, the changing intensity of a family’s engagement in Jewish life, as well as the changing nuances in a family’s Jewish behaviors and cognitions may be tracked over time.

Responses to the Engagement in Jewish Life Scale are analyzed in three ways:

• First, the 13 behaviors that comprise the scale are analyzed one by one to see whether change occurs in families’ specific behaviors over time.
• Next, an overall mean score for each family is computed and changes in their overall scores are examined over time.
• A third analytic approach consists of calculating a “quotient” for each respondent. This is different than the previous analysis in that an overall mean score is not computed. The quotient was computed by assigning a number of points to each item on the scale (all items in the scale have the same weight). If, for example, a family were to frequently engage in 6 of the 13 behaviors, then the family would receive a quotient of 6. This “intensity” quotient can be understood as the summed total of a number of ways that Jewish families with young children may demonstrate their engagement in Jewish life.

The use of this scale has helped us to understand the extent to which families’ Jewish lives are changing over the course of their participation in MazelTot.org. At baseline, prior to families’ participation in MazelTot, the level of engagement in Jewish life tended to be low to moderate. The scores of the families on the First Discount Survey (“baseline”) and the Follow-Up Survey two years later were compared. Thus, the scores of the families on the quotient prior to participating in MazelTot and now were compared. In all cases, positive change was found; the “now” scores were higher than the “prior” scores, and the families attributed this change to their experience with MazelTot.org. (View the full pilot phase report at rcfdenver.org/reports/mazeltotpilot.pdf)

Moreover, according to their quotient scores, almost three quarters of the least-engaged families had increased the intensity of their engagement in Jewish life at some level since participating in MazelTot.org. The families who scored in the bottom third on the quotient at baseline agreed at a higher rate that the MazelTot.org discounts made it possible for them to afford participation in Jewish life in new or increased ways. Furthermore, these families actually used the discounts at a far higher rate than the more-engaged families. The families also agreed at a far higher rate that “I have become a member/regular user of a Jewish organization as a result of participating in MazelTot.” The families who (1) used the discounts at a high rate, and (2) indicated that the Jewish life of their families was growing at home had an important outcome: Their “intensity” scores on the Follow-Up Survey were higher. This is additional evidence of the impact of MazelTot.org on families who had room to grow in their connectedness to Jewish life.

MazelTot.org’s evaluation also asks parents to describe any impact their family has experienced in their own words and their comments tell a similar story:

The biggest increase in involvement came after my children were able to go to a wonderful summer camp because of the MazelTot discount. Attending the camp helped my kids make new Jewish friends and helped them be more comfortable with who they are.  (MazelTot.org parent)

We have learned a lot from our ongoing evaluation. We have learned that easy access to comprehensive information and cost-lowering discounts encourage parents to participate in Jewish life. We have learned that by embedding surveys into the initiative and asking the same questions repeatedly, at different times in a family’s tenure, we are able to come closer to measuring the extent to which MazelTot.org has had an impact on their Jewish identity. And finally, we have learned that carefully constructed evaluation can reveal valuable information about something as complex as the formation of a young family’s Jewish identity.

Perri Klein is the MazelTot.org Coordinator at Rose Community Foundation, pklein@rcfdenver.org

Michael Ben-Avie, Ph.D., is an outcome evaluator of Jewish youth-serving organizations. He is also the research director of Tag Institute for Social Development (taginstitute.org), which interweaves Jewish ideas and practices with the concepts and methodologies of the social sciences in order to impact children, families, cities, countries and global society. Michael.ben-avie@yale.edu