The Outcome Debate Continues
I suspect that most readers will not be surprised to learn that I am pleased to see that we in the blogosphere are beginning to see some of my fellow philanthropy bloggers plead for a more balanced and reasonable view of effectiveness measures for non profit success. For the last few years, most of the noise has been from those who push for ever more sophisticated metrics, outcome measures, effectiveness indices, ratios, and the like.
While well intentioned, read together, these attempts at applying some sort of “objectivity” to the grantmaking process and the results by the recipients have often served to straitjacket the process, force unrealistic and frankly meaningless and premature measures on npo’s/ngo’s, and distort the ability to get at what really matters.
In fairness, much of this pressure of the last few years was more than well intentioned. it was overdue. All of us on the funder side have met too many organizational leaders who tell us that what they are dong simply cannot be reduced to an evaluation. Too many continue to be unable to articulate the most direct and essential bottom line question: “what will be different if you are successful?” Many fields of service are so committed to their own processes that they have forgotten that funders typically care more about the product than how you got there. And many organizations are so busy filling program calendars or turnstile counts that they have lost sight of whether all of that matters.
It has been a welcome corrective to the quality of the sector that there are increased expectations that funding should have some demonstrable justification.
However, now that funders have learned that, they have a tendency to go too far: they [we] want quantifiables on everything, even when it doesn’t prove anything; they [we] want to measure social programs on scientific standards that are unrealistic and may even be unethical. They [we] want proof of short-term impact even when only time can determine if it is true. And too often they [we] want all grantees to demonstrate their effectiveness based on presumably objective and consistent measures, even if the categories measured may not fully apply. So: since many funders have not yet been students of mine or clients of Mirele’s, a few caveats:
Equality is not always equitable
It may seem reasonable to ask all grantees to meet the same reporting and evaluation requirements. After all, a funder has a legitimate right to monitor where the money is going and if it is having he desired impact. And what better way to guarantee that than to ask all grantees to have to do the same reporting. However, there is a profound difference between a grant to a small start up and a large established organization. A quarterly report and site visit may make sense in the latter but may hamper the ability of the former to do its work. A multi-page report, with documentation and full fiscal accounting is certainly expected if an organization has a grants manager; a 2 or 3 person operation is hardly equipped to do its work and meet your needs [and those of all of their other funders]
Moreover, there needs to be some proportionality: a $5,000 grant may be kind and welcome but if the amount of time involved over the course of the year to meet the needs of the funder exceeds the $5,000, have you really given them anything useful. Obviously, a $2.5 m gift is in a different category.
Evaluation should have some practical purpose for the funder and/or the grantee
There are many reasons grantees resist evaluations – and not all of them are pure avoidance. If the funder asks irrelevant questions, or wants info which will have no utility for either the funder or organization, or comes across as punitive, it should be no surprise that there is an instinctive self protective resistance. But if the process is structured to be a learning experience for both, if there is coherent agreement on what is worthwhile to be reviewed, and if the funder enables – read: funds – an evaluation process, it can be a major step in enhancing the quality of the institution and their programs, and can ultimately impact the social weal in profound ways.
Funders must not convey implicit threats
Sometimes these things are almost humorous. A few years ago, a foundation contracted us to assess the work of one ifs longtime grantees. The precipitating indicator was that something seemed wrong with their “results.” Not only did the nfp have increased numbers every year, but they were now reporting that they were serving more individuals than their total potential target market. It didn’t take a lot of sophisticated reading to know that something didn’t add up.
When talking to the head of the organization, he admitted that he was afraid that he would lose his funding if he ever reported a steady state or declining number of participants, and therefore acknowledged that, while he felt he was telling the truth, he was counting quite liberally. There is much to be said about this anecdote beyond this paragraph but what was most telling was the fear [that is the correct description] that funders are so judgmental and moody that it was safer for him to stretch the evidence than have a straight and direct talk about what he was and wasn’t able to accomplish. In fact, the style of one of the funders of this organization so overwhelmed this ceo that there was only salesmanship, and little true conversation. Funders have a lot of power – often more subtle than they [we] realize, and our conscious use of self can make all of the difference in abetting hype or getting at truth.
Patience is more than a virtue, it is often indispensable
Identity isn’t built in a day; poverty isn’t eradicated in a month; education is a complex combination of resources; policies need to trickle up and down. Funders do well when they help their grantees apply sophisticated self-awareness about their outcomes, but need to do so with a sense of reality of the do-able and the temporal. And funders need to apply it to their own methods as well so that projects are given the space they need to succeed and be evaluated.
Many of the currently popular methods of determining the quality of grants, the style of grantmaking, and the results of them are profoundly useful contributions to the field. But only when applied reasonably, responsibly, and ethically. Otherwise funders can intimidate, distort, and manipulate the funder- grantee relationship to the detriment of their mutual goal of improving the world – or at least some part thereof. And no one should want that to happen.
Richard Marker serves as an advisor to foundations, independent funders, and not-for-profit organizations; he is a Senior Fellow in Philanthropy at NYU’s George Heyman Jr. Center for Philanthropy. Richard specializes in strategic philanthropy and planning and regularly blogs at Wise Philanthropy.