Epstein clearly wants attention for his work. So let’s go ahead and give him some.
(spoiler: it isn’t very good.)
His research centers on a series of lab experiments (also replicated through Mechanical Turk and with volunteer participants in India) where volunteers are asked their opinion of political candidates, then encouraged to spend 15 minutes searching the candidates through a fake Google setup (called Kadoodle), then asked their opinion of the candidates again. Epstein finds that, if his team artificially boosted the ranking of positive stories for a candidate in Kadoodle results, then opinion of that candidate would improve.
This makes basic sense. Participants in the experiment, when instructed to search through fake-google, click on the first few results they see. They incorporate that new information into their impressions of the candidates. It’s basically a digital-era update on the types of study that Iyengar and Kinder published in 1987.
But it’s a massive and unjustified leap to get from Epstein’s study to Epstein’s lede in Politico (“America’s next president could be eased into office not just by TV ads or speeches, but by Google’s secret decisions, and no one—except for me and perhaps a few other obscure researchers—would know how this was accomplished.”)
The basic problem is external validity. If undecided voters made voting decisions by Googling candidates and seeing what articles come up, then Epstein’s study would be relevant. But they don’t.
Undecided voters are overwhelmingly low-information voters. They aren’t watching political news. They’re mostly avoiding political advertising, when they can. They aren’t sitting at home Googling candidates. If they were, they wouldn’t be low-information voters.
What’s more, when actual low-information voters do encounter incidental information, it’s happening through social sharing, not google searches. That’s why search engine optimization has largely been overtaken by social optimization in the past 3-4 years. Social is where serendipitous discovery and incidental exposure actually happen today.
Facebook could potentially rig an election, as Micah Sifry and Jonathan Zittrain have both pointed out. It could fiddle with the newsfeed algorithm or selectively deploy its “I voted” functionality, in order to boost enthusiasm and turnout for one candidate or the other. (Facebook won’t do this, of course, because the company would invite a massive congressional investigation if it did. Lightly-regulated quasi-monopolies tend to rationally avoid behaviors that can invite major regulatory scrutiny.)
But Google? To rig the 2016 election, Google would have to tryreally hard. It would have to task dozens of engineers and social scientists with sorting through messy data, merging it with the voter file wherever possible, then apply aggressive nudges to expose low-information/high-susceptibility voters to information that they otherwise aren’t seeking out.
Epstein’s study doesn’t show any of what Epstein claims in his Politico article. Search Engine Manipulation Effects are just the digital equivalent of the traditional news media priming, framing, and agenda-setting effects that we’ve been aware of for decades. It isn’t some new dastardly digital disaster.
This research is an object lesson in why the trend in election research has been toward field experiments, instead of lab experiments, and why the best research also tends to feature observational research on how campaigns and voters actually behave. Elections don’t happen in a lab, and undecided voters don’t behave the way they would if we were paying them to participate. (…Epstein also doesn’t particularly bother to familiarize himself with the literatures on elections, voting behavior, media effects, or digital news, but now I’m just playing armchair peer reviewer.)
Social science gets a bad rap when researchers start making bold, self-promotional claims. Epstein’s peer-reviewed study isn’t great. But his Politico self-aggrandizement tour is downright embarrassing.
This is a follow-up to last week’s post on campaign microtargeting. I had the opportunity this weekend to read Eitan Hersh’s new book, Hacking the Electorate. It’s the most detailed, insightful account of how campaigns currently make use of voter data in elections. I learned a lot from the book, and strongly recommend it to anyone interested in the subject.
The core of Hersh’s argument is what he calls the Perceived Voter Model. “Perceived voters compose the electorate from the campaign’s-eye-view. They are not people; they are avatars generated from whatever data a political campaign, candidate, or party can surmise” (end of chapter 1).* His central point is that, if we want to understand how contemporary campaigns strategize, we need to pay attention to the actual data that they have access to. Campaigns are not omniscient. They go to war with the data that they have, not the data they would like (nor the data that salespeople promise they’ll have).
One of Hersh’s most important findings is that public data (the voter file + the census… stuff that the government collects and makes publicly available) is far more important than commercial data or social network data. He implements a nice research design to demonstrate this, using state-by-state variance in the public data (some states require party registration, some collect data on race/ethnicity, others do not) to show how differences in public data lead to significantly different voter contact strategies.
He also finds that, for all the talk about commercial data and network data, campaigns can’t put much weight on these data sources (chapter 8). The commercial data is incomplete and often out-of-date. While the most well-resourced campaigns certainly purchase this data, they gain very little added leverage from it. At best, they can use this data in states that are public data-deficient to try to model the same voter attributes they are tracking in states with rich public data policies.
Meanwhile, network-based strategies to reach undecided voters through their social networks (aka Facebook) have been severely limited, at least thus far. The network approach proves difficult because (a) it requires core volunteers to start awkward conversations with their least-political friends, (b) it proves hard to reach the whole electorate when starting from the networks of hardcore volunteers, and (c) committed campaign volunteers tend to have social networks that are heavily weighted towards other strong partisans. Again, this doesn’t mean that social networks are unimportant (see Ashley Parker’s NYT story today about Facebook in the primaries), but it does limit their value compared to other data sources.
Public data is reliable and relatively complete. Consumer data is patchier and less reliable. Network data is rich but constrained by the contours of your supporter base. When trying to determine and model their voter universe, campaigns mostly have to rely on public data.
The takeaway here isn’t “we don’t need to worry about microtargeting.” The point is that our normative debates about microtargeting ought to be grounded in an empirical understanding of the current state of the data.
That provides a lot of room for policy debates as well. The types of reliable and relatively complete data that are available to political campaigns are dictated by state policies. What sort of data should be available, in what contexts, at what costs, and to which users? Hersh ends the book by arguing that some of the real threats come when elected officials start using campaign data for governance. (Want help from your Congressperson? Just a moment while they check whether you’re in their supporter database or not.)
One of the reasons why microtargeting attracts so much attention is that it is surrounded by an almost alchemical or mystical sensibility. (“There are data wizards! They know so much about us, and they are using it to manipulate our democratic elections!”) Hacking the Electorate provides an insightful account of just what types of data the campaigns can currently rely upon. It’s an excellent, grounding contribution to the data and politics literature.
*I read the book on my iPad, so I have no earthly idea what page number this quote correlates to. Ah, technology…
[This post is part of an irregular series where I tinker with big concepts for my book. Comments and disagreements are extra-appreciated …and can earn you a spot in the acknowledgements section!]
I had to skip this year’s Personal Democracy Forum, and have slowly been watching archived versions of the keynote talks to see what I missed. One talk that really stands out for me as Cathy O’Neil speaking about “Weapons of Math Destruction.” O’Neil is writing a book about algorithms, and how social institutions cloak their decisions behind mathematical equations in order to obscure the choices that they make. It’s an important topic, and I’m looking forward to the book. Of the three examples she gives, though, one did not seem to be much like the others.
O’Neil provides three examples of algorithms as “weapons of math destruction.” The first is the Value Added Model (VAM) in public education. The VAM is an algorithm that is supposed to separate the good teachers from the bad teachers. That’s a laudable goal. We probably need a good model for grading teachers and incentivizing good teaching. But O’Neil explains that the model is a complete black box. No teacher, no administrator, no data scientist is allowed to look at the algorithm itself and determine if it is measuring the right things. When teachers and administrators ask to see information about the model, they are told “oh you wouldn’t want to know about it–it’s math.” We are evaluating teachers without explaining to them what answers they got wrong or how they can improve their scores. Let that pedagogical irony sink in for a moment.
This is a case where algorithms and Big Data take on an almost alchemical quality. “Put your trust in the data wizards,” we are essentially told, “they know things that you cannot fathom.” And as with all other forms of alchemy, if you dig beneath the surface you’ll quickly detect a faint scent of manure.
O’Neil’s second example is even more troubling: predictive policing and evidence-based sentencing in the criminal justice system. Judges rely on predictive models to estimate a “recidivism score,” which factors into their sentencing decisions. Likelihood of recidivism is, again, an important consideration. Policing, like teaching, is a massive public good, and it seems like better data would be a good thing. But the problem with these recidivism models is that they include factors (high school graduate? Currently employed? Did your father serve jail time?) which would be plainly illegal if they were brought to a judge directly. By cloaking these factors behind mathematics, the justice system becomes less just..
But then there’s her third example: microtargeting in political campaigns. And this is where I think the argument stumbles some. The first example she provides is Facebook’s 61 million person Get Out The Vote experiment. (Micah Sifry has written previously about how this experiment demonstrates Facebook’s implicit electoral power). But that experiment is not technically microtargeting. The second example she gives is a hypothetical: Rand Paul could highlight his positions on financial reform when she visits his website, while hiding other positions that she is less likely to agree with. “What is efficient for campaigns is inefficient for democracy,” she concludes.
This last example seems like a stretch to me. Political campaigns have always used targeting in their communications. Candidates spice up their stump speeches with local anecdotes and local issues. Mailings are targeted based on demographics, issues, and vote history. Broadcast political commercials are targeted to focus on the issues that swing voters (or base voters) find most appealing. Targeting and modeling in political campaigns isn’t particularly new. What we’re seeing with microtargeting is a difference in degree, rather than a difference in kind. The databases are becoming less terrible. The campaigners are taking testing and modeling more seriously.
The case of political microtargeting seems different from the VAM and predictive sentencing because of four general properties: let’s call them The Principle of Potential Harm, the Principle of Approximate Transparency, the Data Quality Principle, and The Principle of Potential Redress.
The Principle of Potential Harm asks “what (unintended) harms might befall an individual if this algorithmic model produces a faulty decision?” In the case of the VAM, good teachers could be unfairly punished. They could be denied raises or potentially fired. In the case of predictive sentencing, people of color and poor people could be sentenced to longer, harsher sentences than their white and well-off peers.* In the case of campaign microtargeting, an individual… might encounter less political advertising that they disagree with.
Within electoral politics, algorithmic models have also been used to purge voter rolls. There the potential harm is that an individual can be denied their right to vote simply because their name is similar to the name of a convicted felon. The Principle of Potential Harm states that we should be more concerned with algorithms in “vote cleansing” programs than with algorithms in political advertising.
The Principle of Approximate Transparency states “if someone asks why an algorithm categorized them as it did, they should be able to get a clear answer.” This is a rule that some of the leading netroots advocacy groups follow: if they are going to use predictive modeling to decide who gets what communications, then they should be prepared to explain what factors went into that decision. If they would be embarrassed to explain it, then they should not use predictive modeling in that case.
I call this “approximate transparency” because there are actually some quite good reasons to keep the details of a predictive algorithm obscure. If Facebook or Google were fully transparent about their algorithms, then malicious actors would be much more successful in gaming their ranking systems. If a predictive model is being used to make valuable decisions, then we should assume people will try to distort that model. A little bit of opaqueness can go a long way in helping the models to perform effectively over time. But if a model is completely secret, then we are unable to consider its merits and its flaws.
In the area of political microtargeting, political journalists enforce an approximate form of transparency. In the 2012 election, ProPublica set up a system that monitored emails from both presidential campaigns to see how they were microtargeting their messages. Political journalists and academics paid close attention to political advertisements as well. This was not full transparency — the Obama Campaign was not going to tell anyone its strategy for determining who got which messages — but it was enough of to keep the worst potential excesses in check. Any value the campaigns might get from extremely microtargeted advertisements would be washed away if it led to a front-page story about their deceptive practices.
The Data Quality Principle states that we should stay aware, and wary, of the underlying quality of the data going into the model. Again, the 2000 Florida voter purge is a helpful example. If that company had perfect data, then its computerized removal of names from the voter rolls would be a trivial matter. But their data was junk, and that rendered the model suspect.
The Data Quality Principle is a major reason why I am not particularly concerned about voter microtargeting. Even though the databases are better than they’ve ever been before, they still have lots of flaws and errors. Electoral campaigns (particularly the big ones that are flush with cash) lean towards overinclusion rather than overexclusion in their communications. So while they might use an enhanced voter file to help isolate the neighborhoods and households most in need of a door-knock, we are pretty far removed from the future dystopia where household A and household B receive entirely different messages at their door.
The Principle of Potential Redress holds that, since algorithms are flawed, there should be a clear avenue for redress when a person feels they have been algorithmically wronged. Teachers should be able to effectively challenge their VAM score. Convicts (or their lawyers) should have clear tools for arguing why the predictive sentencing algorithm is making the wrong prediction. Voters who have been algorithmically excluded from the rolls should be able to cast a provisional ballot, and that ballot should be counted after minimal procedural headaches.
The potential redress for citizens who receive microtargeted political advertisements is… read some political journalism! Electoral campaigns are awash in political advertisements. Better targeting of those advertisements is efficient for the campaigns and, for the most part, less of a headache for the citizens.
My main point here is that some algorithms are much more ethically dicey than others. It depends on what the data is being used for, how trustworthy it is, how transparent it is, and what pathways we have to challenge it.
Smart critiques of emerging digital decision-making often lump campaign microtargeting in with a laundry list of other, deeper problems. I, for one, have made my peace with campaign microtargeting. And I think the differences between it and other “weapons of math destruction” can help us understand which algorithms are the most dangerous.
*Note: people of color and poor people already face major sentencing disparities. So I suppose the potential harm here is that these disparities will be even more difficult to address.
Alex Howard reported yesterday on the release of Democracy.io, a sleek new tool for emailing members of Congress. It’s a nice tool, built with the support of the Electronic Frontier Foundation, based on open datasets created by the Sunlight Foundation.
I hate to sound like a broken record here, but… I don’t get it.
Here’s Sina Khanifar, in an interview with Alex Howard (emphasis added):
“Advocacy organizations that can afford it have long had access to tools for delivering bulk constituent messages, but those solutions are expensive and generally inaccessible for regular citizens. Democracy.io helps fill that gap by giving people an easy way to have their voices are heard in Washington.”
What’s the use-case here? Who are these “regular citizens” that want to share their thoughts and opinions with members of Congress unprompted by advocacy organizations? Where are they getting their information from, and what’s prompting them to write these digital letters?
Participating-because-we-were-asked is sometimes treated as non-“organic,” not as democratically healthy as spontaneous citizen participation that comes out of the civic ether. But let’s be real for a second: it takes a very particular type of person to walk through life believing that (1) they have all the answers, (2) Congress needs to hear those answers, and (3) writing an e-mail ought to do the trick. On season 1 of Parks and Rec, Leslie Knope referred to it as “people caring loudly at me.”
This isn’t to say that Democracy.io isn’t nice tech. It appears to be well-designed. Some future, nascent social movements might be able to deploy this tool on a mass scale, bypassing software vendors that they can’t afford. The codebase might be combined with something else to massively simplify some genuinely hard problems.
But, at least in its current form… I don’t get it. Citizen participation tends to be organized and mediated through networked advocacy groups. Those groups face a thousand different problems, some small, some big. How did simplifying the process for emailing congress out-of-the-blue rise to the top of the list?
I was out of the country last week, so I missed the initial launch of Brigade.com. Alex Howard and Nancy Scola have already covered pretty much everything you’ll need to know about the site so far. (If you haven’t read them already, I recommend you spend a few minutes with them for background.)
Brigade is the latest in a long line of ventures aimed at giving everyday citizens the tools to become more civically connected and involved in politics. Most of these efforts have fizzled. I generally take a pretty skeptical approach to them.
The site has a few interesting wrinkles — it’s clearly built for mobile, and it’s far less reliant on cumbersome profiles than many of its predecessors. It’s also at the beginning of a multi-stage launch, which makes it hard to tell whether some of the missing features (campaigning/organizing/mobilizing functions, in particular) are absent or just unfinished.
As I wrote a few months ago, there’s eventually going to be a tension between the users they want to attract and the eventual customers who will pay for the service. But that tension won’t surface in the early stages of development, while the site is trying to create traction with the inattentive American public. The current challenge (creating that traction/making the site “sticky”) is interesting in its own right. I don’t know if Brigade will succeed or fail. It has a lot of money, a lot of talent, and a lot of buzz working in its favor. But it’s also trying to offer a product that simply might not be in demand. That’s a hard problem to solve.
Now that I’ve had the chance to look around the site a bit, two main points stand out:
Banking on Gamification.
The core of the user experience right now is the stack of issue position cards. “The U.S. should remove the penny,” “Fracking should be banned,” “The voting age should be lowered to 16,” “Online marketplaces (like Amazon, eBay, Etsy, etc) should ban the sale of confederate flag merchandise.” You can choose between Agree, Disagree, and Unsure. You can type in an argument that describes your reasoning, or read and “upvote” other people’s reasons. And you can create your own topic for people to agree or disagree with. The more you participate, and the more people upvote your participation, the higher your “impact” score.
This is the basic logic of gamification. Create a simple reward structure that encourages people to engage in the actions you want them to take. Make it feel like a game, rather than a survey questionnaire. And gamification works… at least up until the moment when people get bored with your game, and find some other game instead.
The initial gamification play makes sense here. No one wants to set up yet another social network profile. The social-network-for-social-change model has failed too many times to generate much enthusiasm. So instead, Brigade is focusing us on easy civic tasks which provide a little fun and offer little psychic rewards. Along the way, they build a userbase, establish social ties, and amass data on those users. It’s a nice start.
But the big question is “what will keep people coming back?” I filled out 25 position cards at the dog park yesterday afternoon. It was brief fun. But will I open the app up tomorrow, and next week, and next month?
It reminds me of the early days of Foursquare, when I routinely “checked in” everywhere I went, amassing badges and mayorships throughout my Brooklyn neighborhood. Foursquare was a fun distraction for awhile. Then I found Angry Birds. That game was a lot more fun. I haven’t used Foursquare in years.
The gamified element of Brigade is an interesting start, but I’m not sure how long of a shelf life it provides.
“…the norms on Facebook dictate that ‘you obey a certain set of social rules on the kinds of things you share or don’t. When it comes to your civic identity or your political identity or your charitable identity, frankly you don’t want to express that side of yourself on Facebook. It’s not the right medium to do it.”
This strikes me as an important premise of the site: People don’t reveal their civic/political/charitable identity on Facebook. They’ll be more likely to do so on a separate site.
I’m not sure if the premise is correct, though. My Facebook newsfeed, for instance, is brimming with political articles. That’s because I have a lot of political friends, and they share political things. Facebook doesn’t discourage them from talking about politics or civil society. And those political friends also happen to be the people who are most likely to be early adopters of Brigade.
It sounds like Parker and his team are banking on the hope that the American public is filled with individuals who have a submerged civic or political identity, which they’ll happily reveal if given the right opportunity. But it’s also possible that (1) the public features a small subset of people who like talking politics and civics, (2) that subset is already doing so on Facebook, and (3) they’ll migrate to Brigade and poke around for a bit, providing an initial impression of early lift/viral success.
This means that if Parker is right about the public, we won’t actually be able to tell for the first few months. The early adopters of Brigade are already plenty comfortable talking politics on Facebook. So even if they flock to Brigade and get a kick out of the issue position card stack, it won’t demonstrate much about the potential demand among the broader, non-attentive public. The real test will come later.
I’m curious to see what the site’s initial growth looks like, and even more curious to see what features they roll out besides the issue position cards. Building a user base beyond the usual suspects is going to be a very tough lift, though. If my (admittedly cynical) read of the American public is right, then gamification and early adoption might produce phantom positive signals during this initial rollout phase.
Trust the agents of the State. Obey the agents of the State. If agents of the State behave inappropriately, that will be determined later, by other agents of the State. Your appropriate role is not to question. Your appropriate role is to comply.
The words above are fiction. They are a ham-handed attempt at depicting the language and ideology of a fascist state. I’m not a fiction writer, and you can probably tell. The language I come up with when I imagine a fascist ideologue is too brazen. Real fascists would probably be more subtle.
For those who haven’t followed these cases, Tamir Rice is a 12-year-old African American boy who was gunned down by Cleveland Police Officers while holding a pellet gun. The police account of the event did not match disturbing video of the event. John Crawford was shot dead by the Cleveland Police while in a Walmart, holding an air rifle that was available for purchase in that Walmart. A city prosecutor has cleared the officers involved in both cases.
And to Follmer, that should be the final word. Any citizen voicing protest or concern is wrong, and should have to apologize for their wrong opinion. At minute 7 of the interview, Follmer testily replies, “These two were cleared by a city prosecutor already. This shooting was justified, and […] it was a tragedy that it was a 12-year-old, but it was justified.”
This isn’t the first time we’ve heard this sort of language, either. This past August, as the nation grappled with the Mike Brown shooting in Ferguson, MO, Sunil Dutta wrote an Op-Ed for the Washington Post titled “I’m a Cop. If You Don’t Want to Get Hurt, Don’t Challenge Me.” Dutta is a 17-year veteran police officer with the LAPD. Dutta writes:
Even though it might sound harsh and impolitic, here is the bottom line: if you don’t want to get shot, tased, pepper-sprayed, struck with a baton or thrown to the ground, just do what I tell you. Don’t argue with me, don’t call me names, don’t tell me that I can’t stop you, don’t say I’m a racist pig, don’t threaten that you’ll sue me and take away my badge. Don’t scream at me that you pay my salary, and don’t even think of aggressively walking towards me.
If you don’t want to get shot… don’t threaten to sue.
It’s good advice, of course. Police Officers are agents of the state. They have the capacity to use deadly force, and they are placed in trying situations every day. As a rule for individual behavior, it is a good idea to be polite to police officers.
But Follmer and Dutta have stepped well past that rule for individual behavior. In the cases of Mike Brown, Eric Garner, Tamir Rice, John Crawford, and far too many others, it is abundantly clear that police officers are not held responsible when they make a deadly mistake on the job. Hell, they don’t even face a trial.
The American Public should be outraged. And Police Officers should be as well. The badge is not a license to kill without consequences. As we’ve seen recently, the justice system is biased against enforcing, or even investigating, those consequences. So we’re left with the weaker tools of public opinion and organized outrage. The police ought to be standing alongside that public outrage, engaging in a dialogue and looking for ways to do better.
The current state of affairs is that Police Officers can be immediately absolved for shooting a 12-year-old boy within four seconds of arriving on the scene, but the officer will then be forced to endure celebrities and citizens wearing t-shirts that call it an injustice. And officers like Jeffrey Follmer think the t-shirts are the problem.
Follmer’s ideology is too brazen for fiction and reality alike. It’s fascism, cloaked in the language of police solidarity. He’s telling us to trust the agents of the state, obey the agents of the state, and don’t dare raise questions when other agents of the state absolve them of wrongdoing. And he wonders why all these protests are happening…
Last week, Techpresident.com published a fantastic interview between Alex Howard and James Windon, President of Brigade.com. You should read it. I’m going to riff on one point in a long, fascinating interview. Please read the interview first, it’s better than anything I’m going to say.
Caught up now? Good. Okay, here goes.
I still don’t know what Brigade is going to be. It’s been on the horizon for about 6 months. I’ve kept my eye out. I still have no clue. It sounds a bit like Change.org 1.0 — a social network for civic participation. Change.org 1.0 didn’t work. Neither did Change.org 2.0, 2.1, or 2.2 (variations on an issue blogging platform and a one-stop participatory shop). Change.org didn’t succeed as a business until it stopped trying to be a civic social network and started trying to be a really, really good petition website.
Reading the interview hasn’t left me any clearer on what Brigade is going to be. And I’m going to try to keep an open mind when it launches. But two lines in the interview really stood out (bolded below) :
Q: I mean the fascination with ‘big data’ in the Obama campaign has subsumed the fact that both parties have a ground game. Both parties send people out with mobile apps. Both parties are trying to nudge people to use their social networks to target ads. So the campaigns are all about this. What’s the role of Brigade when the campaigns are all in that space?
A: I think it’s twofold. One, I think it’s about who’s your primary customer. And for Brigade, that primary customer is the citizen. The goal of this is to build a network that can connect citizens. And if we can do that, then we would invite and encourage candidates and elected officials to come on to that platform on the terms of the citizens who are there. So I think that’s one thing. We will ultimately interface with the existing structures of government, we hope. But there’s a difference between building tools for them from the get-go and then trying to plug the lists in, which is what this is really about, list management, versus building a social network.
You mentioned customers. Y’all are not a nonprofit.
A: We’re not a nonprofit.
Q: Where’s the money come from? Right now, it’s coming from funders, but down the road…?
A: I think that our best bet at how we will monetize is through advertising. That’s our best bet at the moment. We believe however that…
It’s become a bit of a cliche to say “if you aren’t paying for the service, then you aren’t the customer, you’re the product.” But it’s also a pretty damn important point.
Let’s think this through, using some other for-profits who operate in and around the non-profit advocacy arena as examples:
Change.org is a free service, used by millions of citizens. But those citizens aren’t Change.org’s customers. Advertisers (mostly non-profits and political campaigns) trying to cultivate new leads from among those active citizens are the customers. Change.org provides a sleek user experience because more users = more business from their customers.
NationBuilder is not a free service. It’s a CRM, used by thousands of campaigns and organizations, who in turn communicate with millions of supporters. Those campaigns and organizations are customers. The millions of supporters are not. NationBuilder works to meet the needs of the campaigns and organizations that pay them. If those campaigns want to send their supporters 50 emails per day, the supporters may hate it, but NationBuilder will make it possible. That’s their job.
Upworthy is a free service for the millions of people who visit the site, or view their videos on the social web. But those end-users are not Upworthy’s customers. Allied organizations, who sponsor Upworthy curators in specific subject areas, are the customers.
That last example, Upworthy, is the most interesting one. Upworthy doesn’t monetize through advertising. And that has led them to track alternate metrics of success — “Attention Minutes,” instead of page views or unique visitors. If Upworthy monetized through advertising, they would focus on maximizing page views and uniques. That’s what pays the bills. But Upworthy monetizes through convincing customers that visitors are deeply engaging, so they started tracking different things.
As Dan Ariely likes to say, “You Are What You Measure” (h/t Daniel Mintz). And, as an obvious corrolary, your metrics are derived from your customer demands.
With all that said, I still don’t know what Brigade is going to be. Right now, it has a big pile of VC money, and a lot of talented people on staff. That’s a nice opening position.
But I find it troubling that the company’s President is referring to citizens as “primary customers.” They aren’t the customers. The advertisers are the customers.
Balancing the interests and needs of citizens and advertisers is one of the serious tensions that Brigade will need to navigate if it is going to succeed. That’s hard to do. And if Windon and company have a plan for it, they’re doing one hell of a job keeping it a secret.
But Olson points out that the fundraising haul is based on some dirty, unethical, that-can’t-really-be-legal-ugh-why-is-it-still-legal techniques. Like (probably) lying about small donations being matched. Like making it near-impossible to unsubscribe. Like auto-selecting the “make it monthly” checkbox, so that unsuspecting donors accidentally give a lot more than they intended to.*
And the broader lesson here is about analytics and testing. It really matters what you optimize for. The DCCC email program is run with a single goal in mind: generate as much money as possible. Period. That’s a reasonable goal. But it can lead you into perverse habits. Habits that turn your strongest supporters and allies into vocal critics. Habits that degrade your image while you open that (digital) bank vault time and time again.
What would happen if the DCCC optimized for two goals? What if they were trying (1) to raise a ton of money and (2) to improve the standing of the Democratic Party brand amongst supporters?
They would have to measure more results. They would need to develop more sophisticated listening tools, which could measure how email recipients view the party organization, and monitor changes over time. They would have to run more complicated email tests, but the DCCC has a huge list and talented staffers. They could pull it off. Their emails would start looking different. Over time, the Democratic Party could potentially become more likable.
If all of this sounds like a pipe dream, take a look at this new slide deck from SumOfUs.org. SumOfUs announced a new metric last week: MeRA (Members Returning for Action). Rather than focusing on the easy measurables like list growth, petition signatures, or donation totals, SumOfUs is going to track success internally based on “the number of unique members who have taken an action other than their first one.”
Why is SumOfUs rejecting donations, signatures, and list size as its main metrics? Because list size =/= movement power. And money raised =/= movement power. And if you perform digital optimization on those easy measurables, you encourage and reward bad habits.
Ever since the 2008 Obama campaign, digital politics professionals have been talking about the value of analytics and the “culture of testing.” Rigorous testing programs can help you optimize tactics, compare the impact of competing programs, experiment with new strategies, and unearth member/supporter preferences.
But if many of the largest political organizations have learned lesson 1 (“you should test things”), they still haven’t begun to grapple with lesson 2: You should think hard about how your metrics match your goals. It really matters what you optimize for.
*Holy hell, that last one is just inexcusable. I’m heading to Home Depot later this week, and ready to buy pitchforks and torches in bulk.
1. I’m really excited about Write API. Mechaber writes “Beginning in October, third-party websites can submit signatures to We the People on behalf of their own signers, using our soon-to-be-released Write API (which is currently in beta). It’s the result of months of hard work, and we can’t wait to share it with the public.”
This looks like something genuinely new and different. One of the structural weaknesses of WeThePeople is that it doesn’t let petition-creators capture signup data and engage supporters in further actions. That creates a stumbling block. The government is both the venue for and target of these petitions, and limiting the ability of creators to build further connections with signers can short-circuit long-term efforts at political change. Write API could be a very powerful work-around. If it works right, it could be a bit like the ActBlue fundraising widget. Organizations can gather signatures, capture momentum, and then digitally deliver them to the government. The government gets citizen input without being on the hook for enabling follow-up citizen mobilization.
The big question will be whether Write API actually gets used. And it’s impossible to tell right now. I could imagine organized issue publics seizing the opportunity; I could imagine them yawning at the opportunity. But it’s definitely a worthwhile idea, and I’ll be watching with hope and interest.
2. The in-person summit is a lovely touch. Mechaber writes “To celebrate We the People’s third birthday, the White House will host the first-ever social meetup for We the People users and petition creators right here at 1600 Pennsylvania Avenue. It will be an exciting chance for users to meet with policy experts and connect with each other in person.”
I think the future of distributed petition campaigns lies in a move towards distributed organizing. Petitions are a nice, simple, flexible tool. But they’re one-dimensional if you don’t build something out of them. The first step to deepening member/supporter engagement is building new pathways for listening to them. And in-person listening, rewarding the most active participants, is an important step.
I’d be thrilled to see MoveOn.org or Change.org or Avaaz host a meetup where they connect in-person with some of their frequent participants and petition-creators. I would see it as a step towards building a deeper civic infrastructure.
Of course, I would then hope that one of those groups would treat these members as active stakeholders, and the relationship between the White House and its petitioners is fundamentally different from the relationship between MoveOn and its petitions (again, because the White House is playing dual roles as target and venue). So this social meetup has less long-term potential. But kudos for taking this step, I hope others choose to emulate it.
3. But now here’s the critique. Egads, that user survey… Mechaber reports the results from a 2014 user survey. He writes “…over the course of 2014, an average of response surveys showed a majority of signers thought it was ‘helpful to hear the Administration’s response,’ even if they didn’t agree. Nearly 80 percent said they would use We the People again.” (emphasis added)
80% sounds promising. But some quick arithmetic makes it look abysmal.
15,559,272 people have created accounts at WeThePeople. There have been 21,882,419 total signatures. That’s… an average of 1.4 signatures per person. By the most generous possible estimate, that would be around 9 million people who signed only once, and around 6 million people who signed two times.* At the very most, only 40 percent of users have actually used WeThePeople twice in its first three years. And the actual percentage (which they can calculate, but have never made public) is probably dramatically lower than that.
So here’s the friendly birthday critique. 80% of the users who took your survey have indicated that they would use WeThePeople again**. Let’s call that the potential participatory energy in the system. Let’s call the actual percentage of users who returned a second time the kinetic participatory energy in the system. …It’s currently somewhere between 1% and 40%.
Next September, when WeThePeople celebrates its fourth birthday, I hope the kinetic participatory energy has moved closer to the potential participatory energy.
That would make it a very good year indeed.
*And if we have a power law or other fat-tailed distribution of signatures (which we almost certainly do), then its more likely to be 12 million single signatures to 3 million multiple signatures, or 15 million to 1 million.
**Survey bias issue: Depending on response rate, this represents a much tinier portion of the user base. The people who would never use WeThePeople again are more likely to delete the survey invite than the people who love the thing.
Then I became convinced that there isn’t any such thing as the blogosphere anymore. Blogging is just a format for typing things and putting them online. In the early days of blogging (1999-2006ish), the subset of Internet-writers that used this format was small and relatively well networked. It made sense to talk about “the blogosphere,” because there were identifiable clusters of people using this digital tool, and they had distinct goals, priorities, and values.
But as blogging proved useful, it was adopted by more people, and adapted to a wider set of aims. Talking about “bloggers versus journalists” stopped making much sense once the New York Times and Washington Post started hosting blogs on their sites. Talking Points Memo used to be the blog of just-some-guy named Joshua Micah Marshall. Then he developed a business model and started hiring journalists. Then his site won the Polk Award for investigative journalism.
And then, of course, we started getting alternate digital formats that better supported some of the purposes that blogs used to be aimed at. Atrios (Duncan Black) and Instapundit (Glenn Reynolds) were two early influential bloggers who both stylistically chose to writes 20 or so brief posts per day. They were usually a sentence or two, with a link to something interesting. Today, most bloggers write longer posts. A couple sentences plus a link has become a tweet.
Andrew Chadwick calls this rapid dissolution of media genres “hybridity.” One of the major points he makes in The Hybrid Media System is that our newer, hybrid media system encourages nimble organizations that experiment with a wide assortment of tools and technologies.
Email action list. We’re no longer just a website, or a mobile site. Our email action list has grown so large, it’s now one of the largest in the (non-campaign) progressive movement. As of the end of August, the list is 1.6 million strong, which means it has literally doubled in size every year for the last three four years. That list gives us the ability to create massive pressure when necessary. For example, check out this report from the Sunlight Foundation on the 800,000 public comments the FCC received on its Net Neutrality plan. Of those comments that Sunlight could directly source to their sponsorship organization, fully 10 percent of them came from Daily Kos, making us the fourth largest source of pro-Net-Neutrality energy (behind CREDO, Battle for the Net, and EFF).
DailyKos.com has 1.6 million members on its email list. Those members receive daily updates on breaking stories and popular diaries at DailyKos. They also receive calls-to-action, urging them to participate in online activism. I’ve heard that DailyKos is building a field program as well, with a goal of supporting offline organizing.
There’s still blogging at DailyKos. There will always be blogging at DailyKos. And there’s still a community of diarists who use DailyKos to publish thoughts, opinions, comments, and reportage. But it no longer makes sense to talk about DailyKos as a part of “the blogosphere.” The blogosphere is a concept from ten years ago that seems to have already gone past its expiration date. DailyKos has succeeded because it has morphed from a community blog into a more complex digitally-mediated political organization.
Just when we researchers get comfortable talking about a digital phenomenon, the phenomenon itself morphs and changes into something new.