Using Quantitative Data to Lead Qualitative Conversations

by Dan Hazony

Beginning with Rabbi Drew Kaplan’s excellent article, “Qualitative metrics for the Jewish Community?” on Aug. 7th, there have been several articles about the pros and cons of measuring qualitative data. Now more than ever, in this hyper-connected world, we live and die by numbers.

As Director of Information Systems at NCSY, I’ve had several experiences in this endeavor. In many ways it’s a confusing and daunting task: figuring out what needs to be measured, setting up an infrastructure that supports the measurements, overcoming staff reluctance to data-gathering and ensuring that the protocol for collecting data is followed. Despite all these difficulties, data-collecting is vital for us, not only because of the information we gleaned, but also because of the conversations we’ve been able to have because of it.

In NCSY, we started our data gathering attempts several years ago. Initially, we focused on digitizing our Shabbaton registration experience, which gave our staff the necessary tools to deal with the complex logistics that planning a weekend retreat required. Once most of those tools were implemented, we proceeded to start collecting regular attendance at our walk-in events such as our public school clubs, Latte and Learning programs and all our other events.

The reason for the data gathering in our case was simple: since we run on philanthropic dollars we often feel that we are not doing the best we can and we need to do more. The easiest way to understand what and how our organization is doing is to get raw attendance numbers and measure the number of teens that transition from low-impact programming to high-impact programming.

Our staff became nervous right off the bat. All of a sudden, they envisioned their higher-ups hitting “Refresh” on their browsers every five minutes in hopes of catching a staff member not meeting their attendance goals – they envisioned Big Brother monitoring them closely. We eased our staff into this new protocol, one regional office at a time, and soon enough the fear was gone. It required a lot of reassurance and support, but our staff realized that the transition to quantitative measurement did not translate into an axe coming down on their necks.

Our staff eventually became the biggest advocates for data collecting; they simply understood its power. Data became a war chest of information for them. When a local fundraising director went to the federation for funding, he was able to show the local chapter’s success at transitioning teens to higher impact programming – from in-school programming, to our regional weekend retreats and eventually to our month-long summer programs. Not only did it demonstrate our success, but it also showed our funders that we were looking at data and our results in a sophisticated light. Another regional fundraiser approached potential large donors to help fund local public school clubs. The ability to show our large volume of activity in many local schools with large Jewish populations demonstrated our ability to engage and interact with a significant part of the population.

Perhaps more importantly, the data has enabled us to have a much-needed conversation internally about our goals and the realities of our programming. In a nutshell, we can see what works and what does not. Over the past school year, some of our educators felt drained trying to recruit for one type of program. At a meeting of our regional directors, they presented raw evidence supporting their claim that the program was not yielding a substantial amount of teens transitioning to higher-impact programming. This led to a conversation of how to positively modify the program, and if need be, shut it down under certain circumstances. We are now able to have conversations like these on a regular basis, using hard evidence instead of relying on intuition, enabling us to gain the maximum benefit from our limited resources.

Dan Hazony is the Director of Information Systems at NCSY.

Print Friendly
Send to Kindle

Comments

  1. Jackie says

    Great article!! I have been a volunteer in our Jewish community for many years. My background is sociology and statistics and I have been trying to help the organizations I volunteer for, understand these exact issues and the utility of great data for the bottomline and the strategic goals of the agency. However with limited resources, it is not always an easy argument. Fortunately I have the skill set to come in and design and sometimes even collect the information for the organization pro bono. I am still trying to find ways to convince workers in the Jewish community that the collection has to be done at the front lines and how to encourage them to implement these systems. Is there are community of practice for this topic out there?

  2. says

    On May 8, NEXT: A Division of Birthright Israel Foundation had a convening for Jewish professionals from around the south. One of the sessions we had was about how to measure success. The session was led by Robyn Faintich of JewishGPS. Robyn focused on evaluating the experience of a participant at an event not just counting the number that showed up.The feedback on the session was positive from those who attended because they were able to view success in a different way besides the numbers.

  3. Cindy says

    I was at the NEXT Convening mentioned above. The session Robyn led was a great way to see beyond just the numbers. While numbers can be important, knowing WHY someone had the experience they did is important as well. Why does someone choose to return after a first experience? Why might someone NOT return? Robyn taught us how to craft evaluations to get data beyond the numbers, and I found her session extremely valuable. If anyone is interested in learning more about evaluation, I highly recommend contacting Robyn Faintich at JewishGPS.

  4. says

    Thank you all for commenting on my article. The reason that I feel like it is important to put weight on measuring things as basic as attendance is because I think that Cindy is right. However, evaluations that attempt to answer the questions laid out in the preceding can be very time consuming and expensive, both of which are resources that most non-profits don’t have of excess. If not executed or processed well, then it can be a wasted endeavor. Focusing on easily quantifiable pieces of information will lead to the conversations necessary to yield the anecdotes that we all strive for — a better understanding of the emotions and thought processes that our programs evoke. No formal survey will get the full answer — it’s encouraging our staffs to have the necessary conversations with their consistency that will. Numbers-based conversations like these will put them in a position to do so.