Using Quantitative Data to Lead Qualitative Conversations
by Dan Hazony
Beginning with Rabbi Drew Kaplan’s excellent article, “Qualitative metrics for the Jewish Community?” on Aug. 7th, there have been several articles about the pros and cons of measuring qualitative data. Now more than ever, in this hyper-connected world, we live and die by numbers.
As Director of Information Systems at NCSY, I’ve had several experiences in this endeavor. In many ways it’s a confusing and daunting task: figuring out what needs to be measured, setting up an infrastructure that supports the measurements, overcoming staff reluctance to data-gathering and ensuring that the protocol for collecting data is followed. Despite all these difficulties, data-collecting is vital for us, not only because of the information we gleaned, but also because of the conversations we’ve been able to have because of it.
In NCSY, we started our data gathering attempts several years ago. Initially, we focused on digitizing our Shabbaton registration experience, which gave our staff the necessary tools to deal with the complex logistics that planning a weekend retreat required. Once most of those tools were implemented, we proceeded to start collecting regular attendance at our walk-in events such as our public school clubs, Latte and Learning programs and all our other events.
The reason for the data gathering in our case was simple: since we run on philanthropic dollars we often feel that we are not doing the best we can and we need to do more. The easiest way to understand what and how our organization is doing is to get raw attendance numbers and measure the number of teens that transition from low-impact programming to high-impact programming.
Our staff became nervous right off the bat. All of a sudden, they envisioned their higher-ups hitting “Refresh” on their browsers every five minutes in hopes of catching a staff member not meeting their attendance goals – they envisioned Big Brother monitoring them closely. We eased our staff into this new protocol, one regional office at a time, and soon enough the fear was gone. It required a lot of reassurance and support, but our staff realized that the transition to quantitative measurement did not translate into an axe coming down on their necks.
Our staff eventually became the biggest advocates for data collecting; they simply understood its power. Data became a war chest of information for them. When a local fundraising director went to the federation for funding, he was able to show the local chapter’s success at transitioning teens to higher impact programming – from in-school programming, to our regional weekend retreats and eventually to our month-long summer programs. Not only did it demonstrate our success, but it also showed our funders that we were looking at data and our results in a sophisticated light. Another regional fundraiser approached potential large donors to help fund local public school clubs. The ability to show our large volume of activity in many local schools with large Jewish populations demonstrated our ability to engage and interact with a significant part of the population.
Perhaps more importantly, the data has enabled us to have a much-needed conversation internally about our goals and the realities of our programming. In a nutshell, we can see what works and what does not. Over the past school year, some of our educators felt drained trying to recruit for one type of program. At a meeting of our regional directors, they presented raw evidence supporting their claim that the program was not yielding a substantial amount of teens transitioning to higher-impact programming. This led to a conversation of how to positively modify the program, and if need be, shut it down under certain circumstances. We are now able to have conversations like these on a regular basis, using hard evidence instead of relying on intuition, enabling us to gain the maximum benefit from our limited resources.
Dan Hazony is the Director of Information Systems at NCSY.