Crowdsourcing the Search for Great Nonprofits

 

Noah Rimland Flower and David Ehrlichman of the Monitor Institute recently interviewed GreatNonprofits CEO Perla Ni on crowdsourcing knowledge about nonprofits:

One of the best-known and most-debated examples of crowdsourcing is Yelp.com, which brought a new rapid-cycle feedback loop to the relationship between customers and retail services. GreatNonprofits is built on the same model, offering a place for people who interact with a nonprofit to share their views about its performance.

To see them in action, check out their crowdsourced Top-Rated Nonprofits Awards for 2012.

But the problem Yelp is trying to solve is a simpler one than exists in the nonprofit sector. Many people who receive a nonprofit’s service aren’t paying for it, and many people interact with a nonprofit as a volunteer or a board member rather than as a customer. Since GreatNonprofits has been working on this thorny challenge since 2007, we interviewed its founder Perla Ni to find out what she’s learned so far. Read on to hear how she thinks about their evaluation model, her belief in the value of personal stories, and her experiences at the forefront of creating greater transparency and feedback in the marketplace for social good.

Q: When and why do you think people show up at your door, and what role do you think you play in their choices?

Nonprofits use GreatNonprofits to collect feedback from their clients, volunteers and donors. It’s simple, fast and free. They are able to collect amazing stories, written in the authentic voices of real people in their community, of how their work has made a difference. Many of these nonprofits use these stories in their annual reports, share it on their Facebook page, or in their grant applications. They also read the reviews to better understand if there are ways in which they can improve.

Donors and volunteers use GreatNonprofits to help them pick a nonprofit. 90% of donors who use our site says that reading the reviews help them better understand the work of the nonprofit. 80% of donors say that the reviews affect their decision to give to a specific nonprofit. There are so many nonprofits, and it’s hard to know exactly what they are doing because mission statements are pretty general. Volunteers want to know what other volunteers experience. Did they feel their time was well used? Donors want to know how their donations are affecting the community. The ability to respond to these questions requires a human-centered, yet crowd-sourced approach. That is why donors and volunteers use GreatNonprofits. It’s like Yelp or Tripadvisor for them.

So our role is two-fold: To collect feedback that helps both donors and nonprofits better understand the affect of the nonprofits’ activities on the ground in the local community and to bring the highly-reviewed nonprofits, as rated by their local community, to the forefront.

Q: How do you think about the value of personal stories versus numbers in evaluating nonprofits?

Let’s look at the decisions we make daily in our lives – where to go to get our hair cut, where to eat out, which vacuum to buy, which hotels to book for the family vacation, etc. We are generally not making these decisions based on statistics from double blind random trials! We ask our friends, we look online at Yelp or Amazon reviews, or frankly we just make decisions based on a photo. We need simple to use, easy to access information that will make our decision better.

The only type of data that all scientists accept is data from double blind, random trial experiments. Everything else is kind of in the grey zone – just look at the debate over the studies of organic vegetables. It is not feasible to do double blind, random trial studies of most nonprofit activities – for a whole list of reasons, among them, hiring a Ph.D., cost, decades of time, creating a control group of people who will be tracked but denied benefits of the program, etc.

Q: What do you bring by comparison to your partners at Charity Navigator, GuideStar, and GlobalGiving, and by comparison to traditional forms of nonprofit evaluation?

We offer a human-centered, crowd-sourced approach to understand how a nonprofit’s activities are making a difference. At the end of the day, we are all interested in real-world results. We want to know which nonprofit is making a difference, and which ones aren’t and which ones could do better. And through our syndication partners, it complements other types of useful data about nonprofits.

We are also starting to develop tools for collecting feedback about international development nonprofits via SMS feedback collection. We’ve seen that typically, feedback on World Bank projects for instance, may take up to 10 years for an assessment to be done. Now donors can learn that in a village in Somalia, for instance, “Oh, that market in the village never got built. The village council men took it all for themselves.” One of the leading groups doing this is the Danish Refugee Council. Donors to international nonprofits don’t have the benefit of seeing the work first-hand, so it’s doubly more important to get feedback from the local people in those countries quickly and easily.

Q: Many of the nonprofits have only a few reviews, and most of those with reviews are rated between four and five stars. How do you think about accuracy?

It’s still early in this marketplace, so the content is still growing. I would not focus on the top-level numbers. I urge people to read the reviews. Look for the reviews that are detailed. These narrative reviews are extraordinarily rich vehicles to communicate a lot of information. One mother for instance, described how the nonprofit organized harpists to play in the hospital room of her son, in between his traumatic chemotherapy treatments. She goes on to describe the calming effect it had on her son, and how it helped him through his hospital treatments. Or read the review by a man who is an ex-felon, and the nonprofit provided him a job, and now he’s a supervisor and he can provide for his two boys financially and can be a responsible role-model for them. Do I think these are accurate? I think, just as on Yelp or Amazon reviews, people can read the reviews themselves and see which ones they find credible and ring true to them.

Q: Your system asks each reviewer to label themselves by their role (volunteer, client served, board member, etc.). Tell me about that choice: are all opinions about a nonprofit equally important?

In addition to everyone wanting to know about the perspective of the beneficiary, volunteers and donors want more information specific to their experience. For instance, volunteers may also want to know from other volunteers if they felt that their time was well spent, or if they received enough staff support. Donors may also want to know what other donors experience. Many donors make site visits – for instance, what did they observe of the patients when they visited the medical clinic?

Q: You’ve been around since 2007. What have you learned in those five years about what makes a nonprofit great?

We did an analysis of the most popular words used by the community to describe nonprofits that were rated highly. The word “helpful” and “dedicated staff” were among the most frequently used words in these reviews. Our theory is that great nonprofits are those which listen to the needs, priorities and solutions proposed by their community. And they respond enthusiastically to meet those needs, priorities and solutions. One nonprofit which consistently gets exceptional reviews is North Hills Community Outreach in Pittsburgh, PA. They were providing a food pantry and they noticed that many of the people coming to them were people who had lost their jobs. Well, the staff started asking them, how did you lose your job? And it turned out many of them had cars that broke down and because they were late to their jobs, they were laid off. Their clients wanted reliable cars so that they could get a job again. So, North Hills Community Outreach then started a program which sold used cars with a 6 month service warranty to their food pantry clients. The result? Their clients have gotten jobs and held on to them and no longer are dependent on the food pantry. That’s an example of listening to and being responsive to the needs, priorities and solutions of the community. If more nonprofits did that, we would raise the excellence of the nonprofit sector.