A Tale of Two Analytics Programs: It Really Matters What You Optimize For

Steve Olson wrote a real barnburner at Medium last week, “DCCC, I’m pleading with you.” [h/t Personal Democracy Forum FirstPost]

He wasn’t the only one.  Jenny Lawson wrote a nice post about the over-the-top fundraising language, “Nancy Pelosi Is extremely disappointed in me for destroying the Democratic Party.” And, of course, there’s also the ubiquitous Emails from the DCCC Tumblr site.  For anyone who spends professional time reading or writing fundraising emails, the DCCC is an unavoidable topic of conversation.

Part of the reason, as Olson notes, is that the DCCC emails are as effective as they are annoying.  According to Shane Goldmacher at the National Journal, the DCCC is outraising its Republican counterpart by $41,000,000 this cycle. “Hey, it works, whatareyougonnado?” is a pretty effective conversation-stopper.

But Olson points out that the fundraising haul is based on some dirty, unethical, that-can’t-really-be-legal-ugh-why-is-it-still-legal techniques.  Like (probably) lying about small donations being matched.  Like making it near-impossible to unsubscribe.  Like auto-selecting the “make it monthly” checkbox, so that unsuspecting donors accidentally give a lot more than they intended to.*

And the broader lesson here is about analytics and testing. It really matters what you optimize for.  The DCCC email program is run with a single goal in mind: generate as much money as possible. Period.  That’s a reasonable goal.  But it can lead you into perverse habits.  Habits that turn your strongest supporters and allies into vocal critics.  Habits that degrade your image while you open that (digital) bank vault time and time again.

What would happen if the DCCC optimized for two goals?  What if they were trying (1) to raise a ton of money and (2) to improve the standing of the Democratic Party brand amongst supporters?

They would have to measure more results.  They would need to develop more sophisticated listening tools, which could measure how email recipients view the party organization, and monitor changes over time.  They would have to run more complicated email tests, but the DCCC has a huge list and talented staffers. They could pull it off.  Their emails would start looking different.  Over time, the Democratic Party could potentially become more likable.

If all of this sounds like a pipe dream, take a look at this new slide deck from SumOfUs.org.  SumOfUs announced a new metric last week: MeRA (Members Returning for Action).  Rather than focusing on the easy measurables like list growth, petition signatures, or donation totals, SumOfUs is going to track success internally based on “the number of unique members who have taken an action other than their first one.”

Why is SumOfUs rejecting donations, signatures, and list size as its main metrics?  Because list size =/= movement power.  And money raised =/= movement power.  And if you perform digital optimization on those easy measurables, you encourage and reward bad habits.

Upworthy.com made a similar metrics adjustment in February, switching from pageviews and unique visitors to “attention minutes.

Ever since the 2008 Obama campaign, digital politics professionals have been talking about the value of analytics and the “culture of testing.”  Rigorous testing programs can help you optimize tactics, compare the impact of competing programs, experiment with new strategies, and unearth member/supporter preferences.

But if many of the largest political organizations have learned lesson 1 (“you should test things”), they still haven’t begun to grapple with lesson 2: You should think hard about how your metrics match your goals.  It really matters what you optimize for.

—–

*Holy hell, that last one is just inexcusable.  I’m heading to Home Depot later this week, and ready to buy pitchforks and torches in bulk.