In advance of its October 2012 Board meeting, the Jim Joseph Foundation Board of Directors invited a small group of community thought leaders to submit a madrich narrative addressing the following question: What “big idea” for supporting Jewish education would you propose the Jim Joseph Foundation fund?

These papers were presented by their authors at the Board meeting and were the basis for a forward looking, thought-provoking conversation among the Board, the Madrichim, and a select group of Jewish community lay leaders. The Board first conducted this activity as part of its August 2006 Retreat, shortly after the Foundation’s inception. Just as those madrich narratives continue to inform the Foundation’s strategy, so too will these ideas help guide the Board in future grantmaking decisions.

Over the next several weeks, eJewish Philanthropy will be sharing the madrich narratives with our readers.

*********************************

by Lee S. Shulman

In 2000, Bill Gates announced that his foundation was developing a new program of education reform built around reducing the size of comprehensive high schools in order to increase the quality of student learning. They accompanied the “small schools” program with careful research and evaluation. By 2005, Gates announced that the reform had failed. Shrinking the size of high schools hadn’t worked. The quality of the teachers and teaching inside the school mattered far more than school size. In light of that evidence, Gates announced a new initiative that focused on the improvement of teaching and teacher quality in schools. That effort is still underway. It is accompanied by a great deal of documentation and data gathering to clarify what kinds of teaching worked best with particular subjects and under particular conditions.

I’m confident that my colleagues among the madrichim have proposed an inspiring vision of the directions that Jewish education is likely to take in the next decade, and even more important, the directions that our efforts ought to take. If there’s anything we can predict with confidence, it is that the future just isn’t what it used to be … although we often act nostalgically as if we wish it were.

The longer I work in that corner of the world where philanthropy and education intersect, the more persuaded I have become of the wisdom that we should not aspire to predict the future; we should invent it. The track record of the great foundations is impressive in this regard. Carnegie, Ford, Mellon, Wallace, Hewlett and other foundations have created fields where they did not exist before and have invented institutions no one could have predicted. Similarly, we can look at the accomplishments of the most esteemed Jewish foundations and observe comparable accomplishments. But we are in danger of failing to learn from the experience of Bill Gates that our enthusiasms must be tempered by evidence, our self-confidence moderated by good data. And the best data will often come from unexpected sources. Surprise can be our most important teacher.

For the past 25 years, I have played many roles in the world of foundations, as grantee and as board member, as president and as advisor. I have grown increasingly impressed by the successes of foundations in shaping and creating new fields, and in anticipating markets and needs rather than sitting around and waiting for them to emerge. The great foundations are not afraid of failure; they are afraid of fear-of-failure. And far worse than fearing failure is the accumulation of experiences that don’t lead to learning. This is a lesson that the Jim Joseph Foundation has already learned in the active and responsible manner in which it commissions careful evaluations to accompany most of its programs and in its receptiveness to evaluations that do not always bring good tidings.

What lies ahead for the field of Jewish education? I don’t know … but can guess that the future will not bring comfort to the nostalgic. The next generation in all areas of education will certainly be more digital, more networked, less place-bound, more jurisdictionally diverse, and less centrally regulated. Educational innovators will increasingly reach out to populations that were once perceived as hopelessly disengaged. And empirical studies, once thought to be the esoteric province of arcane academics, will become commonplace, necessary and appreciated.

Precisely because the process of education is undergoing such transformation, from pre-Kindergarten through the doctorate, from summer camps to museums, and from cable TV to You-Tube, education programs will be more thoroughly studied in the next decade than in any generation before it. Educational programs will be documented and measured, tested and reviewed, assessed and evaluated as never before. Innovation will not abate; it will increase. But it will not be conducted in the dark.

We learn primarily from experience. We learn more from our mistakes than our successes. We can [learn] only from mistakes that we acknowledge and seek to understand. When we begin to understand why some innovations work some of the time, we can abandon some, re-design others and even mix and match features of several, and we can do so intelligently because we have real evidence, not self- fulfilling fantasies.

The strategy for ensuring that we engage in reform and innovation in Jewish education with the lights on, with a clear grasp of where Jewish learners and teachers are and where they are going, in a landscape of good information and evidence rather than in a field of dreams and wishful thinking, involves three initiatives:

• Making information gathering ongoing and systemic rather than episodic and opportunistic. And by ensuring that the data collected are analytic and explanatory rather than merely census taking and satisfaction monitoring;
• Taking full advantage in the most positive and extensive sense possible the support currently provided to the graduate preparation of the next generation of Jewish education scholars and school leaders. We must harness the applied research they already do in pursuit of their degrees and transform that work into components of a national Jewish education evidence- gathering effort; and
• Increasing the capacity of Jewish education leaders to integrate knowledge development with needed reforms of programs, curricula and institution building so that every funded project yields both clearly described outcomes and good data. In that manner, we become smarter about what we are doing as we do it.

Even the most singular successes of American Jewish education never taught us as much as they could. Camp Ramah is generally recognized as a great success (full disclosure: I spent six years as a camper and then a counselor at Ramah in Wisconsin, and met my wife there), but we have no idea which aspects of the experience were most critical or why the camp experience left an indelible mark on some campers and barely a smudge on others. In my northern California neighborhood, the Wexner Heritage Seminars are credited with transformational impacts on young adults who went on to significant lay leadership roles in Jewish education and the Jewish community. If this is so, why did it happen and to whom? I could continue with many other examples of both ostensible successes and apparent failures from which we have learned far too little because we have asked too little, too late, and too superficially.

At times, the need for careful data gathering and follow-up only become apparent many years later. In the 1970s, the City College of New York began an open-admissions policy for entering high school graduates in New York City. It was highly controversial and the short-term studies did not support the wisdom of the experiment. However, the female participants were followed up for a number of years and researchers (in an award-winning book “Passing the Torch”) made a stunning discovery. There was an apparent “sleeper effect” associated with the affirmative action program. The children of the “special admit” students had a dramatically more positive record of college attendance, academic success and subsequent careers. In a field like Jewish education, how many of our programs are seeking long-term impacts that may have larger sleeper effects than short-term impacts? Which programs affect the inter-generational bungee cord that pulls apparently disengaged Jews back into the community? (For a few other examples, see Appendix A)

I believe we need the Jim Joseph Foundation to take the leadership in developing a national information and evidence gathering strategy broadly with other collaborating foundations, institutions of higher education, evaluation agencies that do a substantial amount of data gathering, graduate programs and independent agencies such as bureaus of Jewish education and Jewish education non-profits. Such a strategy would create a set of baseline indicators of Jewish education that could be used as background and comparative information to enrich our analysis and interpretation of the meaning of other studies.

We have one example of such a strategy in the emerging Consortium for Applied Research in Jewish Education (CASJE), but it is not enough. I can imagine using Jim Joseph’s camping incubator project as a powerful model for actively stimulating such an approach to grant making. We may need to fund the Jewish Education equivalent of “brain camps” where the field of applied neurosciences has been able to make such dramatic progress over the past decade. Can we fund the development of a powerful set of Jewish Education indicators that will be like the National Assessment of Educational Progress (NAEP) or OECD’s PISA assessments (Program for International Assessment) which measure not only what students know but how they apply their knowledge in complex situations and also examines their sense of civic identity and responsibility? This is much more ambitious and analytically powerful than indicator projects like JData.

The Jim Joseph Foundation has had a remarkable six-year record of thoughtful, well-evaluated grant making with a healthy mix of risk and conservatism. It has used evaluations to cast needed light on its efforts. It has pioneered in collaborations with other Jewish foundations, a long-overdue virtue that must be continued and elaborated. Now is the time to move vigorously to ensure that the lights will be even brighter, and the evidence made possible by that will not give way to darkness.

Appendix A:

New York Times columnist David Brooks is particularly adept at using social science and educational studies to illuminate broad general questions of educational policy. Here are two examples from his Sept. 28th column in The New York Times:

1. The value of longer term follow-up of an apparently successful school reform program:

• Schools in the Knowledge Is Power Program, or KIPP, are among the best college prep academies for disadvantaged kids. But, in its first survey a few years ago, KIPP discovered that three-quarters of its graduates were not making it through college. It wasn’t the students with the lower high school grades that were dropping out most. It was the ones with the weakest resilience and social skills. It was the pessimists.
• What are the Jewish education equivalents of the non-cognitive qualities like resilience and social skills that may be better determinants of long-term consequences that the more easily measure cognitive outcomes? What is “Jewish resilience”? Why do some Jews undertake some of the best educational experiences like day schools and summer camps and still not “make it” while others seem to have no valuable early experiences yet turn out to be Jewish leaders?

2. The predictive validity of broad survey data collected earlier in life and their relationship with later life experiences, conditions and outcomes (with no particular intervention or reform)

In the 1990s, Vincent Felitti and Robert Anda conducted a study on adverse childhood experiences. They asked 17,000 mostly white, mostly upscale patients enrolled in a Kaiser H.M.O. to describe whether they had experienced any of 10 categories of childhood trauma. They asked them if they had been abused, if their parents had divorced, if family members had been incarcerated or declared mentally ill. Then they gave them what came to be known as ACE scores, depending on how many of the 10 experiences they had endured.

The link between childhood trauma and adult outcomes was striking. People with an ACE score of 4 were seven times more likely to be alcoholics as adults than people with an ACE score of 0. They were six times more likely to have had sex before age 15, twice as likely to be diagnosed with cancer, four times as likely to suffer emphysema. People with an ACE score above 6 were 30 times more likely to have attempted suicide.

Later research suggested that only 3 percent of students with an ACE score of 0 had learning or behavioral problems in school. Among students with an ACE score of 4 or higher, 51 percent had those problems.

What would such an index (or indices) look like for Jewish education? Can an “index of Jewish peoplehood” be developed and validated that will help us measure and then act upon the real factors that predict later “Jewish health?”

Lee S. Shulman is Professor Emeritus at Stanford School of Education.

The narratives will also appear in the forthcoming Journal of Jewish Communal Service, Vol.88, No. 1/2 Winter/Spring 2013.