Last month, the Los Angeles Times broke the news that a fly-by-night company, Median Strategies, had purposefully circulated fabricated polling data across at least three political races, including the heated Los Angeles mayoral run-off. Before any of that had come to light, some news outlets had uncritically reported on the “results,” helping to shape campaign narratives and shifting, even if modestly, the activity on election prediction markets.

Rahil Prakash, the recent college grad behind Median Strategies, said he did it as a “social experiment” to see if false data “could enter and spread through the political information ecosystem without independent verification.” Even he was surprised by how quickly the misinformation found an audience. “I was a bit shocked, perhaps at how fast this sort of information can spread,” he told The Guardian. “My overall takeaway was that if you just make it look a bit pretty, it’s rather easy for it to spread.” 

Adobe Stock

Not everyone fell for the hoax. Some news outlets mentioned in their reporting that they did not include the polls in their own election analyses because the polls did not meet their standards of transparency. For example, Scott Clement at The Washington Post’s The Pollwatcher explained some of the criteria he uses, which include whether the findings were published with a clear methodological report — a baseline threshold these polling results clearly failed to meet. 

Median Strategies is one of the more egregious examples of untrustworthy data shaping public discourse, but it’s far from the only kind. With AI already transforming our information consumption habits and a contentious midterm election on the horizon, the opportunity for bad actors to fabricate data on purpose is only increasing. Faulty data distorts public debate and electoral politics. Closer to home, it can undermine Jewish communal capacity to understand the communities we serve and the rapidly changing world in which we live. 

As a Jewish community, we are also vulnerable to misinformation and low-quality information, both of which can inadvertently shape how we understand the challenges facing the Jewish world and decide where to direct our interventions.

The fix isn’t catching hoaxes, whack-a-mole-style, after they start to spread throughout our information ecosystem. It’s building a habit of asking where a number comes from before using it to inform decisions and assumptions. 

Some of that habit already exists in Jewish communal life. It just isn’t universal yet. The Collaborative for Applied Studies in Jewish Education, a research lab in Jewish education based at the George Washington University, produces a research digest that regularly shares new research relevant to Jewish communal life. The digest carefully reviews the methodologies of the featured studies and also serves a pedagogical function by helping communal leaders become more savvy — even duly skeptical — consumers of research. 

In the early days of the digest, a Jewish communal leader once wrote to CASJE, worried that it was “undermining trust” in organizational research by teaching professionals to question data they hadn’t previously scrutinized. That’s a fair concern: Teaching people to ask questions of purportedly factual data can tip into reflexive distrust of anything they did not produce themselves. Even so, as a community, we are often too naïve about the trust we place in oft-circulated research that has not undergone a review process (often referred to as “grey literature”) and are not as thoughtful as we need to be about the quality and limits of data.

Whether one is producing or consuming a study, there is no single standard for how research earns trust. Universities have their own answer: institutional review boards and peer review. Yes, these approaches are slow and expensive, and they can be overly focused on questions that are not relevant to the world beyond the academy, but the bulky infrastructure buys something real: a clear set of commitments for ethics and integrity that define expectations for consent, privacy and rigor. This doesn’t obviate bad actors or the ever-annoying, hyper-critical Reviewer 2, but it does offer a clear standard that most academics embrace. 

Of course, academic research standards for ethics and integrity depend on a robust and stable infrastructure of research support that would be difficult to replicate in other professional settings. And so, it may not be fair to expect other forms of research to conform to these standards.

As it happens, public opinion research has its own industry standards, set by the American Association for Public Opinion Research. Importantly, these are not academic standards — they are professional ones. The AAPOR Code of Professional Ethics and Practices was designed “to support sound and ethical practice in the conduct of public opinion and survey research and promote the informed and appropriate use of research results.” Importantly, AAPOR’s standards demonstrate that rigorous shared expectations are not limited to a university setting. Key to AAPOR’s code is the insistence that findings are published alongside basic methodological information.

Polling and market research are two popular modes of study that the Jewish community uses to understand attitudes and needs and track trends. When done well, the insights these studies produce are critical for sound decision-making and planning, and we need more of them.

Most Jewish communal studies are produced by serious people working to provide high-quality information to shape the Jewish future. Jewish organizations that produce and disseminate research want their work to conform to the highest professional standards and are eager to access the necessary infrastructure and resources to make that possible. Even so, as a community, we are still vulnerable to fake data and unvetted information when asking basic questions is not the norm. That is what data literacy means in practice. Who did this research? How was it designed? How were people located and invited to join? What questions were they asked? Were the responses adjusted in any way and, if so, how and toward what end?

Asking questions like these isn’t discourteous or overly wonky. It reflects a basic habit of data literacy and one that belongs in Jewish communal life. Asking these questions can protect us from false or low-quality information on issues that matter. How do American Jews feel about the state of Jewish education? How do American Jews feel about their elected officials? How do American Jews feel about Israel? How do American Jews feel about AI respondents or survey farms aping Jewish perspectives on these sensitive questions? 

We don’t want to create a bar for information usage that is too high. Useful data comes from many kinds of sources, both inside and outside the university, and we need to know how to make the most of it while also understanding its limitations. This doesn’t require a PhD in statistics but rather a habit of inquiry — asking where a number comes from before relying upon it. 

Max D. Baumgarten is the director of North American operations at The Diane and Guilford Glazer Foundation in Los Angeles, where he oversees grantmaking strategies focused on strengthening Jewish life in the United States.

Arielle Levites is the managing director of the Collaborative for Applied Studies in Jewish Education, a research lab in Jewish education based at the George Washington University.