Posted by Mary on Mar 20, 2017 Views: 3683

Simple Guide to Measuring Nonprofit Digital Impact

Originally posted on Whole Whale -Measuring Digital Impact

Impact. This topic is daunting for anyone working in the charity and social impact world. There are multiple frameworks and approaches when thinking about how to define what your organization does to impact the world. We have gathered some of the top resources to help you answer this simple question:

How do we measure a nonprofit’s impact?

There are two critical buckets to consider in this question: the what and the how.

What are we trying to change in the world?
How are we going to do it?
How are we going to measure that change?

Why does this even matter?

If you have chosen to read this it means you probably have been tasked with answering this question or refining it for your organization. Impact matters to the volunteers, donors and foundations that invest time and money into your cause, and the staff that pour their energy into making the organization run. Relying on intuition, assuming that since you ‘help people’ you must be achieving lasting social impact, is a recipe for waste.

‘We’re lost, but at least we are making great time’ approach to nonprofit impact doesn’t work. @leapofreason CLICK TO TWEET
– Chapter 1 of Leap of Reason.

This quote captures what many organizations with the best intentions may actually be doing: driving quickly toward an undefined destination. Driving quickly in the wrong direction, or to translate errant hard work can quickly take you off course and worse be wasteful or even detrimental to the cause.

An example of this can be seen clearly in the mentoring field, a century old tradition of youth intervention in the U.S. with hundreds of millions of dollars invested to date. Let’s say mentoring is a key intervention within your nonprofit’s approach. It is important to know that relationships under 13 months can be detrimental to youth development and those that last longer can “have varying outcomes but overall tend to show a positive, yet weak, impact on educational outcomes and delinquency (Jolliffe & Farrington, 2007).” In measuring and designing mentoring programs, the audience, training and implementation has tremendous impact on the outcomes as noted in many research publications – The Role of Risk: Mentoring Experiences and Outcomes for Youth with Varying Risk Profiles and The Journal of Juvenile Justice just to name couple.

Without outcome planning, measurement and consideration of existing research, it is not hard to imagine driving in the wrong direction in the mentoring field. The following framework is designed to help you think about that final destination and managing toward outcomes.

A Simple Social Impact Outline

There are three large areas we can break our thinking into. These sections represent the larger strategy of the organization, the theory of change, and the practical performance measurement. This outline compresses a lot standard nonprofit planning documents in attempt to distill the mission critical thinking needed to get started. Each area has questions a nonprofit should discuss and answer as a team.

1. Purpose and Intent
Determine purpose of the organization.

What is the long term goal?
What need in society drove the creation of this organization?
What would need to happen for us to (successfully) go out of business?

Define the intended impact, the more specific the better.

Who benefits from your organization? Define your audience.
How do you define success in your organization?
What does success look like for your organization?

2. Defining Outcomes and Indicators
Identify outcomes that demonstrate impact.

What specific changes and outcomes will demonstrate the success you outlined above?
Outline Key Impact Indicators — (warning it’s hard)
What indicators will you use to measure those specific changes?

Create impact indicators that are:

Specific
Measurable
Understandable
Relevant
Time Bound
Valid
Actionable

3. Track indicators over time

Choose the methods you will you use to track each indicator. No matter which types of indicators, make sure results are stored in a consistent way in an online database. Tracking and analyzing is an important role, make sure there is someone who’s job it is to do this, preferably a data analyst.

qualitative (how good), quantitative (how many)
randomized trial, double-blind longitudinal survey
online web measures via Google Analytics

How often will you measure each indicator?

daily, weekly, monthly, annually

Frameworks are nice, but what do I do now?

While frameworks are helpful, the next step is to actually bring this into your organization. There are many social change consultants that can help you do this and usually this process is done in parallel with rebranding work. You can also try leading this process internally by using the survey and discussion approach explained below:

Listen. Start with sending this nonprofit impact survey to staff and stakeholders to see how they view and define these core questions about your organization (feel free to edit ours since it is more focused on digital work).
Synthesize. Gather responses and aggregate points of consensus to see where people agree and where they may be confused. You will be surprised on what comes out. Your executive team can’t be expected to read every response. Condense the answers noting how many people had something similar, while noting any important divergent opinions. IE: do people think you should go a mile deep, spending lots of time with fewer people or a mile wide, spending less time with more people?
Deliberate and Define. This may be done at staff retreat or in a series of executive meetings with the goal of defining a single answer for each critical question while building consensus.
Create some paper. Build this into a logic model and plan for your programs. This can take many forms and can even be pulling out some of the questions in the survey and laying them out in the outline above. A common approach is to create a larger Theory of Change document, but don’t forget the supporting document of how and what you will be measuring. You might want to use Whole Whale’s BHAG Chart to get started.

The Theory of Change

This is a common planning document that foundations and funders may ask for specifically which is why we have included a short guide on it. The goal is to create a document that maps out the causal relationship between your long term outcomes and the daily activities. A good Theory of Change should answer the following questions:

Who is the target population you are trying to reach and help?
What are the short and long-term results?
When will these results be accomplished?
How will these results be accomplished?
- Detail on the activities and strategies that will be used.
- Note additional resources or teams members that will help.
What is the environment and context of where this work will take place?
Why will this approach work?
- Acknowledge the assumptions being made in the model
- External research- find related external research that supports your approach.

Sadly these kinds of documents have a way of being created once then retired to the file cabinet. Though the process is important, if it isn’t revisited the usefulness quickly fades.

This is why we like to build a more focused digital impact modeling map from the theory of change. Whole Whale focuses on digital impact and the document below is an example of how impact modeling looks for one of our clients – Power Poetry.

This article is an attempt to help a growing nonprofit critically think about their impact in a measurable way. This is a small primer that will hopefully kick-off deeper thinking, planning and action within your organization. If you are like us and want to geek out to nonprofit impact frameworks and thinking here are a ton of follow up resources.