by Scott A. Shay

Charitable giving in the US is in the vicinity of $300 billion per year. While some types of giving lead to immediate improvements, others are more difficult to assess. Perhaps the hardest causes to assess are those which seek to shape identity or behavior, since they do not have obvious and readily available measures. Plagued with this lack of measures of success, discussions by philanthropists about American Jewish identity grants have often quickly devolved into either counting the number of people who show up for a particular program or making arguments about intermarriage rates. Neither of these criteria constitutes a useful measure for making a particular allocation decision. However, measuring success for identity programs is not impossible. The methodology I propose to measure impact is stylized toward the Jewish community but is conceptually adaptable to any charitable organization seeking to influence identity and self-definition.

American Jews are blessed with an unprecedented number of foundations focused on preserving Jewish Identity in the 21st Century. The Jim Joseph, Schusterman, Bronfman, Avi Chai, Steinhardt, Grinspoon and Mandel Foundations, among others, each grant tens of millions of dollars each year to Jewish identity causes. In addition many Jewish Federations and individuals are also focused in this area. Yet, all of these funders labor under the same profound conceptual limitation: they don’t have a metric to measure what they are doing. So far funders have relied on both unscientific and scientific measures of varying usefulness. Among the unscientific criteria, many programs are funded under the guise of Jewish identity building but are really just programs that the funders think are nice or whose organizers they think are nice and well meaning. These programs might be good at engendering other behaviors, but are not helpful in making a lasting difference to the participants’ Jewish identity.

Other more scientific methods are more useful. Some funders rely on reports critiquing the issues. This method is useful in identifying the context but rarely measures results. Alternatively, funders gather data from community studies that show a correlation (though not a causation) between life experiences and Jewish identification. This method is the most useful current approach and, as I will show, can tell us a lot about general trends that can help us focus make a first approximation as to program effectiveness. However, the results of these studies often make it difficult to ascertain which program, be it a trip to Israel, or participation in a Jewish camp, for example, had the most impact. Indeed, none of these methods therefore provides a metric for funders regarding the success of individual programs. Worse yet in many cases, as the saying goes in physics, it is even impossible to say that the hypotheses are wrong. The lack of a simple common metric to evaluate the success of specific programs is bad for the Jews.

In the business world, the metrics are pretty clear. Companies try to invest the least amount of money necessary to generate the most profit. This, essentially, is the idea of economic efficiency, captured by the term of “Return on Investment” or colloquially, “bang for the buck.” However, this general measure is not enough guidance for any one particular company. In “Good to Great” Jim Collins pointed out the immense power of getting a metric right for a particular organization. For Walgreens it was the concept of profit per customer visit that brought Walgreens from good to great. Jewish organizations don’t even have a ROI concept, much less a specific metric to go from nice to effective. I would like to propose such a concept. I call it SIDUR, an acronym that stands for Sustained Increase in Demand Unlimited Reach. SIDUR measures the increase in demand for doing Jewish things (however defined by that funder) at some point significantly (a year or more) after completion of the program in question. The farther away from the completion of the program, the harder it is to measure on a specific basis, but this is the conceptual goal.

The key to understanding SIDUR is that demand drives everything. Demand has unlimited reach to change the dynamics of the present status quo. In addition to the lack of a metric to measure the success of a specific program or set of programs, too much funding in the Jewish world is focused on supplying the perceived needs of the potential participant/user by the funder. The clearest current example of this approach is Hebrew schools. Hebrew Schools in the US are continually decreasing the numbers of their days, the hours per session, the years prior to bar/bat mitzvah, and of course, reducing the demands of the curriculum. The operative logic at play is that by requiring less and less commitment, more parents will have checked the box of Hebrew School. But the question then becomes so what if a child completes Hebrew School? Especially, if, as Jewish community studies have demonstrated, the completion of Hebrew School in its most watered-down form does not lead to demand for more Jewish involvement by the students or their families, and worse yet is often the end of Jewish involvement. In contrast to simply counting the number of participants in a program, as many funders have been doing with these least common denominator Hebrew schools, SIDUR would attempt to measure the increased demand for” doing Jewish” one year after completion of a given program.

SIDUR should be indexed from 0 to 10 with 0 being no impact and 10 being total commitment to being Jewish. To make the evaluation of a grant’s impact more powerful, the grantor would estimate the expected change in SIDUR among the participants who complete the program. This pre-program estimate would subsequently be compared to actual change in the SIDUR of the participants at the completion of the program and attempts should be made for further measurements post-program. The more the later evaluation of the SIDUR of the participants exceeds the initial estimate, the better. On the other hand, were the actual SIDUR lower than the estimate, then the grantor and program provider could consider whether either the estimate was simply over optimistic or whether specific program changes could have been better executed in order to have achieved the anticipated gain. The use of SIDUR should not just be limited to new programs. For example, there is no doubt, based on the information we have from community studies that the SIDUR of Jewish day schools in general is quite high. However, we do not have the SIDUR of individual day schools.

Specific schools should also measure their own SIDUR. Yet to fully evaluate any product, service or program in the world of limited resources, we also have to include the concept of cost. Thus, the next step is to take the SIDUR and compare it to the GELT (money in Yiddish) spent to subsidize each participant in order for it to reach more Jews. With these two components we can now put them together to create our useful metric. Here is where the rubber meets the road and the SIDUR/GELT (S/G) ratio becomes our metric.

To better understand how the S/G metric works let’s examine a potential formula and take several examples. Again to summarize the definitions:

SIDUR——–> SIDUR should be indexed on a 1 – 10 index. This index can be comprised of current behaviors be they religious, cultural, or community related. In the 2011 Jewish Community Study of New York, such an index was created by the authors using a variety of Jewish behavioral indicators. Other studies could refine these inputs or come up with their own index. However, I would recommend not paying too much attention beyond the decimal point to these indices since their power is in direction, not precision.

Gelt———> This is the amount funded by the funder per participant, which could be the total cost (if the cost to participant is zero as in Birthright Israel) or whatever the amount used to increase participation or to change the nature of the program.

Below is the metric I propose:

S/G = (SIDUR / Gelt) X $1000 For example if program A had a history of success and backward looking indicators suggested that it increased SIDUR by an average of 4 on the index scale and it cost a $2500 subsidy to bring in each new participants than the formula would be S/G = (4/ $2500) X $1000 = 1.6 . For clarity, let’s take the example of a subsidized Friday night dinner B which costs $25 per participant to subsidize. Of the participants who come to the dinner just 1% ever come to any other synagogue services or events so we might assume the SIDUR for the other 99% is 0. Let us further assume of those 1% who do later increase their Jewish involvement, there is an average change of 5 on the index for that person who becomes turned on by Judaism Thus assuming 100 participants, the weighted average SIDUR is (99×0 + 1×5)/100 and the Gelt to reach that person was the total subsidy cost of the program or $2500. For this program, the S/G = ( (.05) /$2500) x 1000 = 0.02.

Obviously, I made up the hypothetical S/G ratio of program A and of Friday night dinner B to make the point that this metric can be useful in comparing very different programs in terms of intensity and per participant cost.

Let’s look at a few more examples based on our knowledge about Jewish activities from community studies. The first being the type of watered down Hebrew Schools I referred to above which community studies have shown have no statistical impact on Jewish connections later in life. Based on this information, let’s say that the SIDUR measure of these Hebrew Schools in general is 0. Therefore no money should be spent subsidizing them as they are, unless individual Hebrew schools can demonstrate a better SIDUR. Alternatively, educators can create a different version of Hebrew Schools that show a higher SIDUR and apply for subsidies based on this data.

Let’s take another heavily researched example: Jewish Day School, especially for those students who attend through high school, for students in such schools the weighted average SIDUR ratio is likely to be in the range of 9 or more. Indeed we can call Jewish Day School the gold standard of SIDUR..Yet the cost of subsidizing day school education to the point that a significant number of parents would make the switch is also very high. Therefore, while the SIDUR of day schools is likely very high, the S/G ratio of subsidizing day school education may be lower than subsidizing other programs.

Another heavily researched set of programs are Teen Trips to Israel and Jewish camping. According to studies both will likely score well on the SIDUR metric (in general, since these studies do not give us information about specific programs), although not as highly as day school. Yet, their S/G ratio may be higher than the ratio of subsidizing day schools. Current research also shows that in general second trips to Israel and other longer bubble-type experiences would also have a high SIDUR and could yield high S/G ratios, since they cost relatively little to subsidize. The key novelty of the S/G ratio is that it will help funders move from simply computing the number of people who show up for a program to measuring their effectiveness.

The truth about Jews is that we don’t do epiphanies. All the currently available research indicates that longer-term experiences (read “more expensive programs’) have more impact on us and therefore will have a higher SIDUR than one time events. As a result, these are the programs funders should be evaluating first according to the S/G metric. We needed 40 years in the wilderness to form our first national identity. This is why study after study points to the strength and power of trips to Israel (and especially second trips), camping, and the like. Indeed, it is likely that longer teen programs may have the highest S/G ratios of all Jewish identity programs Intuitively, it is clear that connecting Jews during their key identity-forming teen years can have a logarithmic impact on later Jewish life. Further, as research has shown, a positive teen travel experience to Israel will inform a teen’s decision about where to go to university (presumably one with a better Hillel) and may inoculate participants against the sometimes anti-Israel atmosphere on some campuses.

All of this is another way of saying that a teen trip to Israel will lead to greater future demand by the participant and demand will almost always lead to supply for doing Jewish. While longer-term teen programs have good S/G ratios, they are by no means the only programs worth supporting. Other less studied programs show a lot of promise and should also be among the first to be measured according to the S/G metric. One example is longer-term programs for young Jewish families having their first child. The available research demonstrates that this is a crucial connection time for families It is likely that programs directed to this cohort would have very high S/G ratios since current research suggests that they would likely score a high SIDUR and subsidizing these programs need not be excessively costly.

Longer-term adult Jewish education programs like Chai Mitzvah are also likely candidates to successfully pass the S/G test. Chai Mitzvah is an adult bar/bat mitzvah renewal initiative that takes place over 9 months and reconnects Jews with their Judaism, through learning, ritual, social action, and celebration in a way that could not have been achieved during childhood. Chai Mitzvah programs can also be subsidized at a relatively low cost per person, thus increasing the SIDUR of a large number of Jews. Chai Mitzvah estimates that the weighted average current SIDUR for the program participants is 3 and that the current subsidized cost per participant is about $1,000. Therefore Chai Mitzvah’s S/G ratio is (3/$1000)x$1,000, a ratio of 3. The Chai Mitzvah board plans to get the cost down to the $200-$300 range as the program expands, this would result in a much improved ratio of (3/$300)x$1000, a ratio of10. Other programs like chavurah programs or the Conservative Movement’s Mitzvah and Sulam programs are also likely to have high S/G ratios. All of these programs need to be compared and measured against each other and against alternative initiatives.

In contrast, one-time programs will only have acceptable S/G ratios if they cause folks to begin their journey in sufficient numbers. Thus one-time lectures, one time Shabbat dinners, High Holiday events should be evaluated for their S/G ratio. If they do not, as experience suggests, lead to greater demand for doing Jewish, they should not be subsidized, rather participants should pay the whole cost. We should also re-classify these events as community-building experiences, rather than outreach activities. Funding programs with the highest S/G ratio should be our goal, but not our only one.

Another issue that analysts might have with regard to SIDUR is that it is hard to distill the impact of one discreet program There are indeed multiple forces that make it hard to disentangle all of the many causes of change in identity. Was it the camp, or the youth group, or the trip to Israel? Researchers typically use all sorts of complex statistical factor analyses to try to distill the causation. The reality is that this kind of statistical determination is sub optimal, its error ranges are too large, and it usually relies on too few differentiating cases. A benefit of SIDUR is that it abstracts away from this sort of fractal thinking and measures demand in a blunt but very practically useful way.

Like all good investors, the Jewish community needs to diversify its portfolio. So even though teen Israel and camping programs will probably yield the highest S/G ratios, it is a matter of common sense that funders should not put all of their eggs in one basket. Rather, funders should still consider subsidizing programs for Jews of all ages and different backgrounds. The point is to find the programs for each group that yield the highest S/G ratios. The S/G ratio can also help funders consider the proportion of dollars they allocate to specific age- or socially-defined groups. Further, expected S/G ratios should be a way for program creators to think more creatively and test their new ideas. At the very least, the SIDUR concept will encourage funders to be more explicit about the theory of change, improvement, increased participation, or other anticipated outcomes of the grant. Finally, S/G ratios can be a helpful measure for getting program creators to improve the programs in place. It is, however, important to note that while SIDUR is an appropriate measure for funding additional Jewish programming, it does not apply to granting funds for the necessary infrastructure of the Jewish Community.

SIDUR/GELT might not work in every case for every program, but it is a good starting point to conceptualize what Jewish Identity funders should want to achieve: greater and sustained future demand for “doing Jewish” among a larger number of Jews. It will also take a period of computing SIDUR to determine common standards and to be able to determine the ranges of good S/G ratios for different categories of programming. Funders who will want to continue doing things from “the gut” will argue that it is hard to figure out an incoming and outgoing SIDUR. It is true, it requires some work. A short questionnaire for participants at the beginning of a program with follow-up at the end is a good start. Questionnaires should also be sent out via email or other means a year or more after the program to provide a good enough rough answer regarding participants’ SIDUR levels. If no one answers the questionnaires, that is also an answer for funders. Today people are used to being surveyed about everything from their movie watching experience to their call center interactions over credit card disputes. We in the Jewish community should not shy away from gathering information. The results derived from such a method are certainly better than a non-answer or a very precise, but irrelevant answer.

Funders for causes outside the Jewish world will no doubt want to come up with their own acronyms, but the focus on changed demand in comparison to costs will be a new tool for all grant makers committed to the effective use of funds.

Scott A. Shay is the author of “Getting Our Groove Back: How to Energize American Jewry” (2nd Edition, Devora, 2008) and is the Chairman of Signature Bank and a Partner of Ranieri Partners LLC. He is also President of Chai Mitzvah.