5 Tips for Smarter Measurement Strategies

5 Tips for Smarter Measurement Strategies

My last post talked about the dangers of conflating email performance rates with the overall success of an online program and how industry benchmarks are a dangerous proxy for setting the bar for success.

So if benchmarking rates is a poor yardstick, what’s a digital strategist to do to evaluate overall performance? While every program’s different, here are a few tips:

–   Focus on You: industry benchmarks might be interesting, but real performance should be measured based on the history of your house list, not others. Track performance over a year based on different audience and action types and then use month over month and year over year metrics to compare. Set goals based on a theory of what volume and type of actions are needed to achieve your campaign objective.

–   Keep your Eye on the Prize: we don’t send emails because we want to brag about how well they perform.  Keep overall campaign outcomes as a key driving metric and incorporate email performance metrics as a means to that end. Set metric based goals up front with a strong rationale, keep sight on what you are trying to accomplish and how efficiently you were able to do so.

–   Maximize Efficiency: the goal of an online analytics program should be to track how efficiently an online campaign can meet its objective. The question is did you ultimately expend or build resources in reaching your goal? Track a net capacity count (new emails onto the file minus unsubscribes, social media subscribes etc.) from your outreach as key performance indicator in the success of a campaign.

–   Optimize: it’s still important to test and optimize! Don’t get me wrong, you should always be testing and striving for higher rates, just remember why you’re doing it. You want to optimize to squeeze as much out of your list as possible, not to measure the worth of your overall campaign.

–   Circle Back on Your Goals: I’m always amazed at how often a campaign will blow away engagement metric goals but totally fail to accomplish a policy objective or we’ll come up short on a projected engagement goal but still win the campaign. Learning about the relationship between grassroots activity and the outcome of a campaign is as much an art as science, but you have to make sure to be explicit about going back and evaluating overall impact.

Sifting Through the Policy Wonk

6 Tips for a Great Online Advocacy Plan

Strategic planning for online tools and tactics often will occur after an overall campaign goal or objective has been set.  It can be challenging to be brought into a planning process late, online organizers are often left having to think on their feet about where digital tools and tactics can fit into an overall campaign.

Below are a core set of criteria we often will use to wade through an overall issue advocacy campaign plan to identify the specific aspects that are most promising to focus on for online organizing.  ISMART is an easy way to remember it the next time you find yourself in an on the fly planning session.

  1. Integrated: look to bust down those silos between online, offline, communications and lobbying/legal tactics. Marrying online activities with the other major initiatives of an overall advocacy campaign is always ideal. Note that doesn’t mean digital should take a back seat or just be in a supporting role.
  2. Specific target/process:  this one is ignored a lot. A generic petition to a government agency to solve a broad issue is not a campaign strategy. It’s not going to build your grassroots power over the long haul and it certainly will have little impact on your policy objective on its own. Activists don’t need to know every single detail of policy process (please!) but they do need to see a clear target and process by which their commitment of time and effort will bring about change.
  3. Measurable: online organizers will get themselves into trouble taking on ‘awareness’ or ‘buzz building’ campaigns without specific goals up front. Drill down on specific volumes and types of actions or activity online that will be needed to achieve your objective.  Even raising the profile of an issue should have some hard and fast metrics.
  4. Achievable goal: activists want to get involved in a campaign that has a clear theory of change and an overall goal that can actually be accomplished. That doesn’t mean it needs to be an easy win or a sure thing, but it does need to be specific. A good way to gut check this is to think of
    the specific moment when the campaign will have won…can you describe what specifically has changed?
  5. Relevant to audience: look for the aspects of a campaign that are easiest to relate to as an activist who will spend at most a few hours or days in aggregate, not months or years, involved in the campaign.  It’s OK if down the road you need them to weigh in on more technical aspects of a campaign, but for planning, focus on your best, most powerful hooks to build your largest pool of engaged activists.
  6. Time bound with degree of urgency: a given at this point (almost over done one could say) but people want to know that their immediate participation is important.


An Honest Online Engagement Program

What’s Your Open Rate?

The Dangers of Online Benchmarking Reports

Senior managers want hard and fast ways to contextualize the performance of their organization’s digital program. And benchmarking performance reports provide an all too easy and tidy metric to track success and compare to other organizations. Whether it’s the M+R eBenchmark Report, the Convio Benchmark Index, or others, the rise of the interest in data, optimization and benchmarking seems to have everyone from online organizers up to executive directors asking ‘What’s Your Open Rate?’

Don’t get me wrong. These reports have great utility for tracking overall trends. And a culture of testing and optimizing online tactics to boost response rates is vital. However, industry benchmark rates are a poor measure of the overall success of an online organizing program.

Why would it be a bad idea to stack up against the competition? The danger of benchmarking reports is conflating high response rates with a high quality campaign. Good rates must mean a good campaigners and vice versa, right? Wrong!

Take email for example. There are a multitude of factors that impact the open, click, action and unsubscribe rate of an email that go far beyond the quality of the content, staffer or overall program.

Other factors that have statistically significant impacts on the overall performance rates of an email can include:

– Audience targeting: are you sending an email to your full file, people who have previously taken action or some other segment such as issue activists who have opted into a specific type of communication

– Type of Action: you should expect, and benchmark, different performance rates based on the type of ask in the email (online petition vs offline activity vs donation etc) and hopefully find some ranges of response rates to group different asks into tiers of performance, a sample grouping is below

– Email Arcs: messaging people a series of email asks all under the same campaign in a ‘ladder of engagement’ has also proven to be a statistically significant factor in email performance

A more detailed matrix of email performance could look something like this:

Email Performance Chart

So given the wildly varying rates depending on the type of action, audience and sequencing of email…does an overall open rate for a program really tell you anything? When you compare the hundreds or thousands of asks you will make to segments of your list over the course of a year, can it really neatly stack up against another organization’s to make a meaningful comparison into one single metric? I posit that there is very little information that is useful by comparing your organization’s online performance to others in this broad a fashion.

Digital campaigns should be focusing on maximizing overall engagement as efficiently as possible to reach your campaign goals, not on achieving the highest response rates. Too much emphasis on maintaining high rates can even discourage your program from making harder, higher value asks or reaching out to broader segments of your list.

Head on over to part two of this blog post for some tips on how to measure your online campaigns effectively.

This is a post with post format of type Link

Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Aenean commodo ligula eget dolor. Aenean massa. Cum sociis natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. Donec quam felis, ultricies nec, pellentesque eu, pretium quis, sem. Nulla consequat massa quis enim.

Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Aenean commodo ligula eget dolor. Aenean massa. Cum sociis natoque penatibus.

This is a standard post format with preview Picture

Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Aenean commodo ligula eget dolor. Aenean massa. Cum sociis natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus.

Donec quam felis, ultricies nec, pellentesque eu, pretium quis, sem. Nulla consequat massa quis enim. Donec pede justo, fringilla vel, aliquet nec, vulputate eget, arcu. In enim justo, rhoncus ut, imperdiet a, venenatis vitae, justo. Nullam dictum felis eu pede mollis pretium. Integer tincidunt. Cras dapibus. Vivamus elementum semper nisi.

Read more