No, Politico, Google Can’t Rig the 2016 Election (without trying REALLY hard, at least)

Psychologist Robert Epstein has written a piece for Politico, titled “How Google Could Rig the 2016 Election.”  He’s trumpeting his recently-published study of “Search Engine Manipulation Effects” (SEME), stating with bluster that “Google has the ability to control voters.”

Epstein clearly wants attention for his work.  So let’s go ahead and give him some.

(spoiler: it isn’t very good.)

His research centers on a series of lab experiments (also replicated through Mechanical Turk and with volunteer participants in India) where volunteers are asked their opinion of political candidates, then encouraged to spend 15 minutes searching the candidates through a fake Google setup (called Kadoodle), then asked their opinion of the candidates again. Epstein finds that, if his team artificially boosted the ranking of positive stories for a candidate in Kadoodle results, then opinion of that candidate would improve.

This makes basic sense.  Participants in the experiment, when instructed to search through fake-google, click on the first few results they see.  They incorporate that new information into their impressions of the candidates.  It’s basically a digital-era update on the types of study that Iyengar and Kinder published in 1987.

But it’s a massive and unjustified leap to get from Epstein’s study to Epstein’s lede in Politico (“America’s next president could be eased into office not just by TV ads or speeches, but by Google’s secret decisions, and no one—except for me and perhaps a few other obscure researchers—would know how this was accomplished.”)

The basic problem is external validity.  If undecided voters made voting decisions by Googling candidates and seeing what articles come up, then Epstein’s study would be relevant.  But they don’t.

Undecided voters are overwhelmingly low-information voters.  They aren’t watching political news.  They’re mostly avoiding political advertising, when they can.  They aren’t sitting at home Googling candidates.  If they were, they wouldn’t be low-information voters.

What’s more, when actual low-information voters do encounter incidental information, it’s happening through social sharing, not google searches.  That’s why search engine optimization has largely been overtaken by social optimization in the past 3-4 years.  Social is where serendipitous discovery and incidental exposure actually happen today.

Facebook could potentially rig an election, as Micah Sifry and Jonathan Zittrain have both pointed out.  It could fiddle with the newsfeed algorithm or selectively deploy its “I voted” functionality, in order to boost enthusiasm and turnout for one candidate or the other.  (Facebook won’t do this, of course, because the company would invite a massive congressional investigation if it did.  Lightly-regulated quasi-monopolies tend to rationally avoid behaviors that can invite major regulatory scrutiny.)

But Google?  To rig the 2016 election, Google would have to try really hard.  It would have to task dozens of engineers and social scientists with sorting through messy data, merging it with the voter file wherever possible, then apply aggressive nudges to expose low-information/high-susceptibility voters to information that they otherwise aren’t seeking out.

Epstein’s study doesn’t show any of what Epstein claims in his Politico article. Search Engine Manipulation Effects are just the digital equivalent of the traditional news media priming, framing, and agenda-setting effects that we’ve been aware of for decades.  It isn’t some new dastardly digital disaster.

This research is an object lesson in why the trend in election research has been toward field experiments, instead of lab experiments, and why the best research also tends to feature observational research on how campaigns and voters actually behave.   Elections don’t happen in a lab, and undecided voters don’t behave the way they would if we were paying them to participate.  (…Epstein also doesn’t particularly bother to familiarize himself with the literatures on elections, voting behavior, media effects, or digital news, but now I’m just playing armchair peer reviewer.)

Social science gets a bad rap when researchers start making bold, self-promotional claims.  Epstein’s peer-reviewed study isn’t great.  But his Politico self-aggrandizement tour is downright embarrassing.



8 thoughts on “No, Politico, Google Can’t Rig the 2016 Election (without trying REALLY hard, at least)

  1. I think you’re missing the point, and taking down a straw man. Epstein is correct to point out, that Google can have an impact on elections. This is a statement of fact. You have to justify your decision about “undecided” voters. They are not all “low information” unless you have some kind of proof to the contrary.

    “Swaying” an election can be done by a relatively small intervention.

  2. Really? Did you read Epstein’s politico piece? If I’m taking down a straw man, it’s because he carefully built it himself, set it in the middle of the road, put a fancy hat on it, and then started pointing and shouting, “hey everyone! Come look at this TOTALLY REAL HUMAN BEING I FOUND!”

    Epstein has shown that Google search ranking manipulation can have an impact on voter opinion reports if and when voters access those rankings in a limited choice environment. That’s entirely consistent with a long history in political communication research on media effects and voter behavior. He ignores that long history in order to make his findings sound more novel.

    The situation he approximates in his experiments is by definition an undecided voter scenario. He introduces his research subjects to candidates they’ve never heard of (running for elections in other countries), and provides them with a paragraph of background information on each (that’s “low”) before asking them to search the candidates in his fake Google.

    The findings on low information voters are among the most well-established in the field. Read Philip Converse. Read John Zaller. Honestly, pick up just about any article on American voting behavior written in the past 30-40 years. If Epstein had presented his research at an APSA or MPSA meeting, his discussant would gladly have pointed this literature out to him.

    I’m not his discussant and I’m not his peer reviewer, so it isn’t my job to introduce him to the literature he’s pretending to contribute to.

    And his article doesn’t say “sway” an election. It says “rig.” The guy is massively puffing up his findings. I’m deflating them. He’s asking for it.

    Google can have a small impact on an election, but so can every other mass medium. The guy wanted headlines, so he reached way beyond what the findings justified.

  3. Totally agree with you on this, David. Great post. But I prefer the term “straw person”.

  4. David,

    I think you focus too much on Epstein’s motives and you miss his points, weakening yours. The argument about “Undecided voters are overwhelmingly low-information voters” is not a strong one because we cannot be sure who will be the undecided voters in 2016. Maybe they are misinformed voters getting their information through facebook which could be compromised. Maybe they are those trusting Google regularly and Google is misleading them. Google has been spammed in the past, and still is – that’s what SEOs (Search Engine Optimization companies) do.

    One should note, and this is important, that Google has been fixing its search results when it comes to political searches. There is no much influence to its search results of politicians because it has a strict formula on how to allocate the top-10 results. And we have evidence of that. See, for example, Network Manipulation (with application to political issues)
    It tries to be “fair” and consistent in the top-10 results so that no negative or unofficial information appears. If Google suddenly started “unfixing” its political search results, I would be alarmed.

    For the record, I have had this discussion in a sequence of emails with Epstein in the past, some of it reported in the Washington Post Interestingly enough, Epstein chose not to reference my paper in his, and even misspelled my name in the “acknowledgements”. Unfortunately, the PNAS review process is not as thorough as it should be.

  5. Thanks for your comment, Panagiotis. I want to push back on the topic of undecided voters, though. We actually know quite a lot about who the undecided voters/low-information voters will be!

    The big advances in voter modeling over the past couple of elections (see Eitan Hersh’s “Hacking the Electorate for an overview) have mostly come through modeling data based on the voter file (who has voted in past elections + party registration in some state + demographic data in some states). They create predictive models that rate individuals on a scale of 0-100 for the likelihood to vote and their likelihood of voting D vs R. Particularly in the states that collect party registration data, these models are really quite good.

    Since we know a fair amount about who undecided voters will be (and this is a well-established literature dating back to Philip Converse’s pathbreaking work in the 1960s), we can also make some well-founded claims about how often and in what ways those voters interact with political information (Delli Carpini and Keeter wrote an award-winning book about this in 1996, “What Americans Know about Politics and Why It Matters. Also see Bimber and Davis’s 2003 book on the web’s use in political campaigns).

    These are central works in the literature on political communication and elections. They represent established, bedrock knowledge. Voters don’t approach each election anew and start gathering information about the candidates. They develop civic habits, preferences, and beliefs over time, and those habits structure how they engage with politics and political campaigns.

    So, quite apart from the technical aspects of what Google can or does do with its political search offerings, if an experiment is going to have external validity, it has to not violate our core assumptions about political information-gathering. Epstein’s study mimics a low-information voter scenario (because he’s intentionally using candidates from other countries so his research subjects won’t have strong existing opinions), but then presumes that these voters will spend their time googling for election information. That heightens the statistical significance of his findings, but creates a scenario so artificial that it would virtually never occur in the real world.

  6. Pingback: Reading recommendations: Twitter, Han Solo, and Ashley Madison | Aske Kammer

  7. Politico article on How Google Could Rig the 2016 Election

    The study behind this and the advocacy essay written by the researcher are incredibly, even ridiculously flawed. Here are just some of the issues. The results don’t withstand even passing scrutiny.

    1. In his study, he used a search engine that provided only 6 results on a page. Artificial and much different from Goigle itself.

    2. Please the researcher. Give ’em what he wants. Especially likely when seeing such a big impact in a short time with little input.

    3. Increasing skepticism over Google results, so limited impact in real world.

    4. Only positive stories on candidates?

    5. Assumes Google results are the primary source of information for voters–silly! People go to news sites, candidate sites, etc. They may not be using Google much at all. So, even if true, the hits are only a small part of information–and info is in turn only a small part of what affects voting.

    6. If using Google results, doesn’t say what voters do with info they get to.

    7. Ignores that negative stories can trend and lead results–key is what is found and what is made of it.

    8. Indian example is correlation; a candidate doing better will get better and more coverage.

    9. Recent report showing nearly 2/3 of major studies in leading psychology journals not replicable!

    10. Political bias in examples! Each one raises the spectre of Google conspiring to manipulate mass opinion in Democratic directions. Reveals a lot about his motives in doing the research. Did Roger Ailes and Fox News fund him?! 🙂 Be more worried about outright electronic voting machine hacking.

    Bottom line: have fun talking about this around the water cooler or online, but don’t believe it for a minute!!

Comments are closed.