Social media has played an augmenting role in these protests.  There’s the pizza orders, which are pretty cool.  There’s the twitter- and blog-based information diffusion.  There are the solidarity events planned around the country, occurring throughout the past week and also this Saturday.  There’s the $300,000 raised by DailyKos, DFA and PCCC to support the “Wisonsin 14.” And of course there are the “mundane mobilization tools” used to coordinate events themselves.  But in general, whereas the focus on social media in the Arab protests has been so intense as to border on self-parody, no one has really spent much time talking about the internet’s role in this saga.  Nor should they.  The story here is pretty simple.  Walker is trying to destroy the central organizing structure for working class interests.  This isn’t about reduced benefits – the unions have already agreed to cuts – it’s about power, plain and simple.

The new generation of internet-mediated organizations can achieve a lot of things.  They’re optimized for the new media environment, both in terms of organizational overhead, staff structure, membership communications, and rapid tactical repertoire.  But they can’t organize workers in a specific industry or location to increase salary, working conditions, or benefits.  MoveOn isn’t going to sit across from management at the negotiating table.  Your local Meetup group sure doesn’t have that capacity either.  This is the most obvious space where “organizing without organizations” comes up short.  You need to build power if you’re going to confront power.

What’s more, MoveOn, DFA, PCCC, DailyKos, Living Liberally, New Organizing Institute and the rest of the netroots are fully aware of this.  Professional organizers, old-school and new-school, understand that the Wisconsin fight is about power.    Take away the unions, and the super-wealthy will be the only interests in America capable of aggregating massive resources to affect policy change.  It looks an awful lot like a coordinated, multi-year strategy to knock out every significant organization of the left (first ACORN, now Planned Parenthood and the Unions).

I feel the need to point this out because the short-version summary of my research is “the new media environment is transforming the interest group ecology of American politics.  We’re experiencing a ‘generation shift.’”  I want to be absolutely clear about this: what’s happening in Wisconsin isn’t about digital media or about generational displacement in the advocacy group system.  It is a coordinated Rightwing assault on the rights of citizens to organize in the workplace.  Internet-mediated organizations are doing all they can to support the unions in this fight, because they know full well that the unions fill a niche that internet-mediated issue generalists, online communities-of-interest, and neo-federated organizations cannot.  I cannot think of a single serious online organizer who believes otherwise.

I’ve written once before on this blog about Moore’s Law, the surprisingly accurate 1965 prediction that computing capacity would double every 18-to-24 months.  What I’ve noticed recently is that, while Moore’s Law is common knowledge within the tech community (you see it mentioned in almost every issue of Wired magazine). it’s much less well-understood in the political and social science communities.  Those crowds are aware, of course, that their computer from 4 years ago now seems ancient, slow, and lacking in storage space, but it appears to me that the deep political implications of Moore’s Law (which I’ll be calling “Moore’s Law Effects” in the book, unless someone wants to earn their way into the acknowledgments by suggesting a catchier name!) have largely gone overlooked.

I checked through the indexes of several major internet-and-politics books and, sure enough, there’s no mention of Moore’s Law.  Bruce Bimber’s Information and American Democracy, Matt Hindman’s Myth of Digital Democracy, Bimber and Davis’s Campaigning Online, Phil Howard’s New Media Campaigns and the Managed Citizen.  I’ll check a few others on Monday when I’m in the office, but I’m pretty sure there’s no mention of it in Kerbel’s Netroots, Davis’s Typing Politics, Chadwick’s Internet Politics or either of Cass Sunstein’s books either.  …These are good books I’m talking about here — award-winners that rightly deserve the praise they’ve received.  I’d be thrilled if my book ends up half as good as many of them.  Yet Moore’s Law doesn’t earn a single mention, nor does it show up in most of the influential articles in the field.  It just hasn’t entered the discourse.

The one exception I’ve found is a Berkeley Roundtable on the International Economy working paper by Zysman and Newman that eventually became the lead article of a co-edited volume, How Revolutionary is the Revolution.  It’s a political economy treatment of the digital era as a whole and seems pretty promising (amazon should have it to me by mid-week).  I really enjoyed the following quote in the working paper: ”…Information technology represents not one, but a sequence of revolutions.  It is a continued and enduring unfolding of digital innovation, sustaining a long process of industrial adaptation and transition” (pg 8).  That “sequence of revolutions” line is what I think we’ve largely been missing when talking about digital politics.

Take Bimber and Davis’s Campaigning Online for instance.  They conducted first-rate research in the 2000 election cycle on citizen access to campaign websites.  The central finding was that, by and large, the only citizens who visit such sites are existing partisans.  The sites are useful for message reinforcement, rather than message persuasion.  As a result, Bimber and Davis conclude that the impact of the internet on political campaigns is pretty slight.  Web sites simply don’t reach undecided voters, so they aren’t of much use in determining election results.

Their book was released in September, 2003.  By that time, the Dean campaign had already attracted overwhelming media attention, leading observers everywhere to rethink the importance of mobilization.  It was an unlucky sequence of events, having a definitive work on the internet and American political campaigns come out just as the Dean campaign was overthrowing everything we thought we knew about the internet and American political campaigns.

Here’s the thing, though: Bimber and Davis weren’t wrong. The Internet of 2000 wasn’t particularly useful for mobilization.  John McCain raised a bit of online money around his primary, but online bill paying was still in its untrustworthy infancy, and the social web was still restricted to the lead adopter crowd who had heard of Pyra Labs.  The suite of technologies making up the Internet changed between 2000 and 2003.  It changed again between 2003/04 and 2006.  [Pop quiz: what was John Kerry's YouTube strategy in the '04 election?  (A: YouTube didn't exist until 2005.)]  And it continues to do so.  The internet of 2010 is actually a different medium than the internet of 2000.  The devices we use to access it have changed.  Cheap processing power and increasing bandwidth speeds let us access video and geolocational aspects that were prohibitively expensive and technically infeasible or impossible in 2000.  We’ve traveled through five iterations of Moore’s Law, and that means that the devices and architecture of the earlier internet have been overwritten (html to xml being just the tip of the iceberg).

The internet is a sequence of communications revolutions, and that is entirely because of Moore’s Law.  It makes the internet different than previous revolutions in information technology.  Consider: as the television or radio moved from 10% household penetration to 80% household penetration, how much did the technology itself change?  I’d argue it wasn’t much at all.  A television set from 1930 is fundamentally pretty similar to a television set from 1960.  The major changes of the 20th century can be counted on one hand – color television, remote control, vcr, maybe a couple others.  It is frequently noted that the internet’s penetration rate has been faster than these previous communications technologies.  But what rarely gets mentioned is that the internet itself has changed pretty dramatically in the process. (Need further convincing?  Watch the 1995 movie Hackers and listen for the reference to one character’s blazing-fast 28.8 kb modem.  LolCats and YouTube aren’t so fun at 28.8kbs speed.  Or read James Gleick’s 1995 New York Times Magazine essay “This is Sex?” in which he explains that the internet is a terrible place for pornography because search is so complicated and the pictures upload so slowly!)

Transitioning into the political sphere, it bears noting that every election since 1996 has been labeled “the internet election” or “the year of the internet” by a set of researchers and public intellectuals.  The paradox, of sorts, is that they have been right every time.  2012 will be different than 2010, 2008, 2006 2004, 2002, and 2000.  It will be a different medium, in which users engage in modified activities, and this will create new opportunities for campaigns and organizations to engage in acts of mobilization and persuasion.  The cutting-edge techniques of last year become mundane, encouraging organizations to maintain a culture of ostentatious innovation.

Now I’m not suggesting that the internet exists in some state of quantum uncertainty, where we can predict basically nothing in the future based on the past or present.  In fact, as Rasmus Kleis Nielsen points out, the tools that will have the biggest impact on campaign organizations will be the ones that have become mundane, reaching near-universal penetration rates and no longer subject to a steep learning curve.  (As we recently learned with Google Wave, e-mail is much a settled routine at this point.)  Indeed, one of the lessons here may be that we are on much safer grounds when studying individual internet-mediated tools that have reached near-universal adoption (within a given community).  The techno-centric studies of facebook, youtube, and twitter that are a recent fad of sorts are on much weaker ground, because those tools are themselves still pretty dramatically changing thanks to increasing adoption and the ongoing influence of Moore’s Law.

The other thing it tells us, however, is that we should focus attention on the new organizations and institutions being built out of the digital economy.  The continual waves of innovation made possible by Moore’s Law mean that existing industries do not solely need to adapt to a single change in communications media.  Rather, an existing market leader who hires the best consultants, purchases a fleet of state-of-the-art hardware and software, and spends two years developing their plan for the digital environment will suddenly find that the internet has changed in a few important ways, their hardware and software is outdated, and the plan those consultants developed has collected more dust than accolades.

Communications revolutions (or changes in “information regime,” if you prefer to avoid talk of revolution) create a classically disruptive moment for various sectors of the economy.  Rather than advantaging existing market leaders, whose R&D departments let them lead the way in sustaining innovations, disruptive moments tend to lead to the formation of new markets that undercut the old ones (this is classic Christensen).  Startups do better under those conditions, because they have low operating costs and no ingrained organizational routines.  And while individual areas of the internet eventually give way to  monopolies (particularly if we lose net neutrality and let major firms capture markets and tamp down on competition),  those monopolies aren’t as secure as they were in previous eras.  Just ask AOL, Compuserv, Microsoft or Yahoo. The wrong policy decisions can still basically kill the internet, but Moore’s Law creates a scenario in which ongoing disruptions continually advantage new entrants, experimenting with new things.

That, frankly, is why my focus has been on the rise of these internet-mediated advocacy groups.  It’s because they represent a disruption of the advocacy group system.  They embrace ostentatious innovation, keep their staffing and overhead small, and otherwise continue to act like a start-up (and are often founded by technologists with a background in startup culture).  They fiddle with membership and fundraising regimes, and develop new tactical repertoires unlike anything found among the older advocacy groups.  And Moore’s Law suggests that the internet is still in a state of becoming, that the emergence of these new institutions is much more substantial than the mass behavioral patterns found among citizens in the internet of 2010, which may very well be altered as Moore’s Law allows the internet to become something else in 2012.

Moore’s Law, disruption theory, and new developments at the organizational level.  That’s what I think has been missing from our understanding of the internet and American politics thus far.

What do you think?

I’ll note as a caveat that Research 2000 is launching a counter-suit.  The facts will be revealed in time, and the way things look today may not turn out to be the reality of the situation.  I don’t mean this blog entry to prejudge the results of this trial.

That said, it appears as though the progressive political blogosphere has just claimed a second scalp within the polling industry.  The first occurred back in the fall of 2009, when Nate Silver at 538 raised serious concerns about Strategic Vision.  Noticing serious anomalies in their data, as well as a lack of public information about the company itself, Silver asked some very public questions about whether they were fabricating their data.  The head of Strategic Vision cried foul and claimed he’d see Nate in court, but he then beat a hasty retreat and hasn’t been heard from since.

DailyKos has contracted with R2K since the 2008 election cycle, and has sent them a lot of business.  After Nate published his inaugural pollster rankings last month, Markos announced that he’d be rethinking the partnership with R2K (who fared poorly compared to other pollsters).  That apparently led to a few statisticians deciding to take a deeper look at R2K’s numbers, which revealed anomalies that would be consistent with mild cooking of the books and/or outright fraud.

Talking Points Memo took a deeper look at the head of R2K, Del Ali, and found that his background consists of 2 degrees in recreation.  That’s really pretty odd, to say the least.  You would expect the head of a major polling firm to have a background in, well, statistics.

And that leads us to the point I’d like to make: how is this possible?  Professional polling is a competitive and lucrative business, with longstanding industry leaders and standard-setting organizations.  Neither Strategic Vision nor R2K was a minor player — both were significant pollsters whose findings were reported by mainstream media sources.  Both (it appears) were somewhere between shady and fraudulent.  In a well-functioning market, incentives should exist for shaming and discrediting such actors.  The field of professional polling involves enough statistical wizardry and high enough stakes that, if such incentives operate anywhere, they should operate there.  And yet we now have seen two occasions in which, essentially because Nate Silver and company have made a hobby of advocating for responsible polling practices, major irregularities have been uncovered, with field-transforming impacts.

There’s a lesson here about just how robust the market mechanisms in various knowledge industries actually are.  Even in a field that has incentives for self-policing, even in a field tied to academic institutions like AAPOR that are full of people who have the means and motive to investigate such irregularities, there has been a distinct lack of accountability for years.  The lowered transaction costs of the internet has enabled skilled hobbyists to dramatically affect that market.  The internet itself doesn’t magically improve the polling industry (far from it), but it did create a new opportunity structure through which motivated volunteers could challenge and affect existing institutions.

Bravo to Nate Silver for his nearly one-man quest to improve the polling industry.  He didn’t have to take on this challenge, and I’m sure it’s made him plenty of enemies in the process.  Kudos to Markos Moulitsas as well for partnering with statistical researchers and readily admitting it once he learned there was a problem with his data.  The internet doesn’t make the industries perform more responsibly, it just creates new opportunities for motivated outsiders to mobilize knowledge/people/resources in new and interesting ways.  Between R2K and Strategic Vision, we have a good example of just how poorly the “statistical wizardry” industry was actually functioning, and also a case study in how networked volunteers can transform such industries.

As the old proverb goes, “may you live in interesting times…” Interesting times, indeed.

I hope all my progressive groups and friends remember #Haiti today. I’d be bummed if they didn’t.

Mattzie has backed up the talk himself, pledging to match up to $1,000 in disaster relief donations from his fellow twitter-donators.  As the day has progressed, I’ve already seen online appeals from Color of Change and MoveOn (both urging their lists to donate to groups such as Oxfam and Doctors without Borders).  Nothing so far from the single-issue political advocacy groups, though of course Red Cross and others have appropriately sprung into action.

I don’t mean this to be a critique of the single-issue groups, but it does bring one point to mind that bears examination.

In the presentation that I’ve been giving about my research, I use the phrase “Headline Chasing” to describe the distinctions between MoveOn-style targeted fundraising and the direct mail funding appeals that fueled advocacy groups for the past 40 years.  It’s an intentionally provocative term.  The new generation of advocacy groups organize around whatever issue is at the top of the public agenda, whereas the earlier generation of groups mobilize around specific issue topics, regardless to their immediate salience.  That proves very effective as a fundraising tactic, but it implies a sort of nimbleness and fluidity that may or may not be such a good thing.

I think today’s fundraising appeals are an important example of the unquestionably positive side of this “headline chasing.”  MoveOn isn’t making a buck off this tragedy.  They are mobilizing their large supporter list and asking them to help out through other organizations.  When tragedy strikes, tragedy rules the headlines.  And in that moment, unless the tragedy impacts an issue group’s central focus, the large majority of organizations remain silent, clearing out of the way while the red cross and others take center stage.  The new political economy of advocacy organizations allows the progressive netroots to get behind the red cross, doctors without borders, and other center-stage organizations and quietly help out.  Internet-mediated organizations are performing mitzvahs right now, because their structure allows them to.  Older organizations, progressive or not, remain sidelined because the logic of their structure demands it.

Let’s hope that organizations, governments, and individuals do all they can to come together in the wake of this tragedy.  A 7.0 earthquake is a reminder of just how fragile many social institutions actually can be.  Unpredictable tragedy like this can happen anywhere, and national boundaries should not stand in the way of efforts to aid our fellow human beings.

Professor Lessig likes to introduce three ideas to frame his talk today: 1) influence, 2) independence and 3) responsibility.

Relying on his framework of the four modalities of control that he used in Code, Professor Lessig explains how the law, markets, norms and architecture together exert influence, and that depending on your policy objectives, these four forces can be complementing or conflicting. He suggests that together they form an “economy of influence” that we need to understand if we want to make effective policy.

He continues to explain “independence”, in the sense that something is not dependent on something. Independence matters, because it means that you try to find the right answer for the right reason, as opposed to doing so for a wrong reason you might be dependent on.

Independence, however, does not mean dependence from everything. Lessig reframes independence as a “proper dependence”. In legal terms, it means that a judge depends on the law for her judgment. So independence is about defining proper dependence, and limiting improper dependence.

Responsibility is the third concept Lessig goes into. He tells us about a case he represented in 2006: Hardwicke vs ABS. It was a case that focused on a series of events concerning child abuse, all perpetrated by a single person. The question that was raised: Who is responsible? Lessig makes the argument that responsibility does not lie with the individual, that this individual has no power to reform, and that this is pathological. Instead, he makes the case that responsibility in this case is all the people who knew about the wrongdoings, but refused to pick up the phone. Nevertheless, the focus of the law was on the one pathological person. Lessig suggests it is more productive to focus responsibility on those who have the power to make changes, instead of those are pathological and are not in a position to reform. He notes it is ironic that the one person who is least likely to reform is held responsible, while the one entity who could do something about it, was immune.

He raises another example of “responsibility” gone awry. He cites Al Gore and his book “The Assault on Reason”, and lambasts its narrow perception of responsibility. It focuses on former president Bush, arguably the man least likely to reform, and instead forgets those who could have done something about it, suggesting that they also have been critically responsible.

His argument is one of “institutional corruption”. What it is not: what happened with Blagojevitch; it is not bribery, not “just politics”, not any violation of existing rules. Instead, institutional corruption is “a certain kind of influence situated within an economy of influence that has a certain effect, either it 1) weakens the effectiveness of the institution or 2) weakens public trust for the institution.

He explains the system of institutional corruption using the White House. Referring to Robert Kaiser’s book “So Damn Much Money”, he argues how the story of the government has dramatically changed in the past fifteen years and how the engine of this change has been the growth of the lobbying industry. He illustrates this with numbers: Lobbyists pay with cash which members use as support for their campaigns. The cost of campaigns have exploded over the years, and subsequently, members have become dependent on lobbyists for cash – he cites that lobbyists make up 30-70% of campaign budgets! This is not new, he carefully explains, but citing Kaiser again, what is new is the scale of this practice has gotten out of hand. Members /need/ and take /much more/, becoming /dependent/ on those who supply. This is only during the tenure, but institutional corruption also needs to be understood as something after tenure: 50% of senators translate their senate tenure into a career as lobbyist, while 42% of the house do the same. This suggests a business model, focused on life after government, that perpetuates itself, and influential people who end up becoming dependent on this system surviving, both during and after their time in Congress.

He goes on to give example after example of institutional corruption. He mentions the important work done by maplight.org that tracks money in politics, who have shown that members who voted to gut a bill had 3x times the contribution from lobbyists than those who voted against. Simply put, policies get bent to those who pay. He cites a study by Alexander, Scholz and Mazza measuring rates of return for lobbying expenditures, who conclude that ROI is a whopping 22,000%! He again cites Kaiser, who suggests that lobbying is a $9-12 billion industry.

Why does this matter? It matters if it
1) weakens effectiveness of institution or
2) weakens public trust of institution

In the first case, he argues how lobbying can shift policy. He cites a study by Hall and Deardorff “Lobbying as Legislative Subsidy” on how the work of congresspersons shift as a result of lobbying. Imagine you’re a congressperson and you see it as your goal to work on two issues: one is to stop piracy, the other is to help mums on welfare. The line of lobbyists that will happily help you with stopping piracy is long, whereas not so many will help you with the latter – so work of the congressperson shifts, and thus work of Congress shifts.

Lessig suggests it also bends policies. Does money really not change results? Citing the Sonny Bono case of October 27, 1998, he shows how in copyright lobbying power had a powerful influence in getting the copyright term extended for another twenty years. Does this advance the public good? A clear no. Lessig backs this up by telling how in the challenge at the Supreme Court, an impressive line-up of Nobel Prize winning economists, including Milton Friedman, supported this and that it would be a “no brainer” to sign the support that copyright extension did not advance the public good. But he concludes that there were “no brains” in the House. An easy case of institutional corruption. There are two explanations: Either they are idiots, or they are guided by something other than reason. He suggests of course it’s the latter. It is not misunderstanding that explains these cases.

Lessig continues to explain how corruption can be seen as weakening public trust. He tells us about how the head of the committee in charge of deciding the future of healthcare is getting $4 million from the healthcare industry. Or how a congressperson ended up opposing the public option even though the majority of his constituency supports it. The idea is not that there might be a direct link between the money and the vote, but that if you take money to do something that is against the public interest, people will automatically make that link, and this weakens public trust. If you don’t take money and you go against the popular vote, that won’t reek of corruption.

Lessig goes on to discuss different fields: medicine and the healthcare industry, citing research by Drummond Rennie from UCSF that shows how there is an overwhelming bias in favor of sponsor’s company drugs. How there are 2.5 doctors to 1 detailer (a detailer being someone who is like a lobbyist for the pharmaceuticals, promoting the drugs to doctors, often giving “gifts”). How the budget for detailing tripled in the past ten years.

Lessig asks us: how can we find out whether these claims are true? Do detailing practices either weaken the effectiveness of medicine, or weaken the public trust for it? What would it take to know?

There is also the issue of “the structure of fact finding” that Lessig suggests is corrupt. Again, he argues we need to understand whether this is a process by which results are affected or trust is weakened. He cites how sponsor funded research can cause delay, and mentions the case of “popcorn lung”.

Lessig makes a strong case that we need more than intuition. That we need a framework or metric to know for sure. Because we all have ideological commitments, that we need to escape this in order to have a proper understanding of corruption. This is, in short, the aim of his new project: The Lab. It should be a neutral ground with a framework that determines whether and when institutional corruption exists, to develop remedies for institutional corruption when it exists. He sees the initial work having three dimensions: 1) data – necessary to describe influence and track its change; 2) perception of institutional corruption and understand how it has changed;
and 3) causation – what can we say about what causes what in these contexts in alleged corruption. Having this information, we can then design remedies.

Consider: when I was in college a decade ago, I carried around a spare floppy disk for saving files and transfering them from one computer to another.  It could store about 100 megabites, which was a lot more storage than you’d find on campus webmail.  A few friends of mine had new computers with a whole GIGAbite of space.  It wasn’t entirely clear to me what one would do with all that space.

Today, my iPod has 120 Gigs.  I save my files to gmail and they reside in the clouds.  Wireless networked computing means I can access them anywhere, even on the road from an iPhone.  My computer, bought 4 years ago, is antiquated and slow.  It’s a mac that doesn’t even have embedded video.  And therein lies a disconnect for businesses and governments in the digital age.

Let’s say that the University I work at decides to invest in a complete upgrade of their computer systems.  This includes hardward and software, a new fleet of computers; custom-built e-learning tools.  (full disclosure: this post was inspired by the “MyCourses” e-learning training I attended this morning)  In so doing, they incur some substantial sunk costs.  Those computers and those software programs have got to be useable for several years.  Organizations and governments don’t move seamlessly up the line presented in the figure above.  They move in a stepwise fashion, investing in new tools every few years which they are in turn saddled with as the technology continues to evolve.

Why is this important?  Well for one thing, it helps to explain why large bureacracies will virtually always have outdated websites.  Purchasing a fleet of new computers or moving into a new area of web-related activity caries a cost, and no organization can afford to keep pace with the rapid expansion.  Sunk costs are a reality of egovernment and organizational adoption.

Particularly in terms of software, however, it seems to me to be a strong argument in favor of partnering with organizations like Google or, better yet, relying on open source software platforms.  Learning today about Brown University’s “My Courses” software platform (it is similar to the “blackboard” site at Penn… a password protected site for students to access the syllabus, readings, grades, etc), I couldn’t help but think that the more advanced functionalities could be accomplished for free with a Ning site.  I’m sure “My Courses” was cutting edge when Brown first made the investment, [well... I'm not sure, but I'm willing to believe it] but companies like Ning have been free to innovate — incentivized to do so, in fact — while Brown has incurred the sunk costs.

In a digital environment that evolves as quickly as the internet, we can draw a hardline distinction between the entities who have an incentive to continually create and innovate (tech giants, startups who hope to challenge tech giants, and open source communities) and the entities who proceed in stepwise fashion, incurring sunk costs, staying pat for several years, and then incurring another round of sunk costs.  It then follows that one of the benefits afforded to informal networks (policy networks, activist networks, political networks, communities-of-interest… Clay Shirky’s “organizing without organizations”) is that their lack of resources also leads to a lack of sunk costs, meaning that any online action they engage in is likely to be, and essentially remain at, the cutting edge.

Those are just a few early thoughts on the topic.  E-gov hasn’t been one of my major research areas thus far, but I’m warming up to it.  Thoughts/reactions/feedback would be appreciated (as always)

Do you want to know how Wikipedia was able to become such an incredible success? Who the people behind its success are? The best book to learn about the history and the culture of Wikipedia is Andrew Lih’s new book “The Wikipedia Revolution“, launched last week. He was at Harvard last night to give a talk and do an interview with Berkman Fellow and distinguished internet scholar David Weinberger.

Andrew shares with us his story of how he first came across Wikipedia – in many ways, it was a very different experience from most people. On February 9, 2003, Andrew was looking for his next research project – he has been studying online journalism and new media for a long time – and has been instrumental in creating the new media program at the Columbia J-school – he was told that he should take a look at this new site called Wikipedia – this amazing site that “anyone can edit”. Contrary to most people, he heard the principle first, before he saw the actual website. When he took the time to explore the site, he was immediately taken away with it, thinking “the crowd could not have written this” He looked at more pages, started using Wikipedia in class assignments, and became so fascinated with the project that he wanted to study it full-time.

“It works in practice, but not in theory” is often said of Wikipedia. And that’s definitely true if you consider its origin. Wikipedia started out of a project called Nupedia – in many ways this was projected to be a conventional encyclopedia. Started by Bomis, it envisioned a 7 step rigorous peer review – it would recruit volunteers to write its articles – and the hope was that most of these volunteers would have a PhD degree. That is, the original vision of an online encyclopedia was one with very high stringent requirements.

The big problem: after one year, Nupedia had the grand total of twelve (count ‘m) articles. Even worse, they were written by someone on the payroll. This was clearly not sustainable. Larry Sanger decided to intervene – realizing they needed something radical to at least get seed material. He turned to this thing he saw called wiki software – created by Ward Cunningham – wiki was a way for programmers to share best practices – it would be an online resource for programmers. The name came from the wiki wiki bus in Hawaii – meaning ‘quick’. The wiki software indeed produced quick results – as of recent, there are over 2.8 million entries in the English Wikipedia alone. So why does Wikipedia work? Andrew suggests five key factors: it was free – open – neutral – timely and social.

Andrew describes the piranha effect – the idea that one change in one corner can inspire other changes and create a torrent in the community. For example, in one particular week, 33,800 (count ‘m) articles were added in Wikipedia. This was largely from a huge body of census data from the US – a software robot was written to extract relevant information from this data and inject every possible town and city in Wikipedia. One such town was Apex, and it just happened that on one day SethIlys visited this page. It was a dry article – what he decided to do was – hey, why not put a map on there? A few keystrokes later, he had added his own handmade map – and in his own way, was able to contribute his knowledge to the world. Useless perhaps? Perhaps, but if he visited this page, why not someone else as well? This experience was really empowering to him. Once he started with one map, he figured, why not add others? And once he started, it did not make sense to stop – so like Forrest Gump – he kept on running. The strange thing was, others started running, too. Nearly all the US census location articles now have maps.

There’s a famous saying: “if you have a hammer, everything looks like a nail”. Andrew adds to that: “If there was ever a project that had lots and lots of unhammered nails, it was Wikipedia.” The dot map project was an inspiration – an exemplar – encouraging people to do things they never thought possible. And in many ways, Wikipedia itself is such a project as well – an exemplar.

David starts his interview with Andrew.

David: Let’s get this out of the way first, are you neutral about Wikipedia?

Andrew: No I’m not. I analyze neutrally. But I’m a big fan. I believe Wikipedia is one of the most fascinating creations man has ever made. But that doesn’t mean it doesn’t deserve scrutiny.

David: You think it was important enough to write a book about – an endorsement in itself. But let’s go to its origin myth – as all super heroes have one – the myth is often that idealists came together to do this democratic experiment and that the world’s greatest encyclopedia is the result. Is that right, where did it go wrong?

Andrew: Telling Nupedia’s story helps debunk a lot of this. It started as a failure. There was no way anyone knew how to do this. Even though the founders were very internet savvy and big fans of open source, it was not apparent that doing an encyclopedia in that style was the way to go. Only after a full year, did they decide to try it this way.

It’s also interesting that Wikipedia is always cited as an example of democracy, but the community itself never uses that word. It assumes good faith, it likes consensus, but it never ever uses the word democracy. As a matter of fact, a key thing in wiki is NOT to do voting. They discourage voting – they rather decide through discussion, not to rely on hard measures like voting.

David: What’s wrong with hard measures?

Andrew: The problem of gaming the vote without having meaningful discourse. One of the most contentious issue was the Danzig/Gdansk edit war. An edit war is what happens when you don’t converge on a neutral point of view – the result is that there is a constant flipping back and forth between different revisions of one article. This edit war was the catalyst of a lot of policy change – for example, the Three-Revert-Rule. But in this case, after a year of brutal edit war, voting was inevitable – it was a defining edit war in English Wikipedia history.

David: Can you talk about the flatness – that supposedly every voice is equal and there is no hierarchy – and its rules, the anti-rules and emergence of rules?

Andrew: The rule is that you shouldn’t have that many rules – having too many rules, you start to game the rules. There are rules nevertheless – neutral point of view, assume good faith, – the idea that your next contributor could be the most prolific one, so don’t bite the newbie. But these rules are soft ones and established during the early days – the community has changed quite a bit since 2001.Today it is no problem to get people to contribute. The problem is to get rid of bad stuff. The concern: is the community is still as vibrant as the early days?

David: There is an antipathy towards rules – the idea that rules tend to breed bad behavior – yet at the same time it is a warm-hearted community – assume good faith. To what extent is Wikipedia free of a certain political mindset in the structure of Wikipedia as an emergent community?

Andrew: The English Wikipedia, it’s a liberal progressive community, or libertarian. It is reflected in the early roots of Wiki – they met on Objectivist mailing lists. Jimbo (Jimmy Wales) is a straight forward libertarian – common in the geek community. The articles are generally of good quality nevertheless. But if you disagree, you can fork. One such response is Conservapedia.

David: Is it built along the same principles?

Andrew: No, but I wish it were. Articles are often written in direct opposition to Wikipedia articles.

David: Is it open to edit?

Andrew: Hmm, hard to say. More people are in control, they are not as inclusive.

David: What I like is the pragmatism of Wikipedia – a general dislike for rules, but if you need a rule to build an encyclopedia, then it’s fine.

Andrew: There are really five pillars, one of them is that Wikipedia is an encyclopedia. That might sound silly, but that wasn’t so in 2004. Wikipedia had grown as a community with lots of social aspects – there was a gaming lounge for example where people were playing virtual chess games. We had to shut that down – it was pretty cruel – but we are here to write encyclopedia articles and not to support MySpace activities.

David: It’s also a discussion about which articles are deleted – that Wikipedia is not an art project. It’s an encyclopedia, but sort of different – so the question becomes what an encyclopedia is in a digital age? It’s a sharp edged debate between the deletionists and inclusionists – what side do you fall on?

Andrew: The inclusionists’ argument is that wiki is not paper – why not have articles about anything under the sky? An article on an obscure issue does not take away from your general experience. The deletionists, also called exclusionists, argue that the value of an encyclopedia is that it is a set of articles. It’s no good to have an article where every single word is cross-linked, or that are not reliable – the key test here is – should we have an article on what we had for breakfast?

In the early days, I was considered an exclusionist. I argued that it does matter how selective you are – that articles need to be verifiable, high quality. Over the years, the community standards have shifted, to the point that I don’t think I have changed my stance that much, but where I am now being considered an inclusionist.

Now it is crucial to keep out the bad stuff – Wikipedia is now high profile – and recent policy changes are all about restrictions, restrictions, restrictions. It provides a much more different atmosphere than the early days – now much more stringent.

David: What gets people so passionate about this particular issue?

Andrew: It’s not just within one language – it’s across cultures as well – for example, the German Wikipedia has 900,000 articles – a long way to go before you hit the 2.8 million articles of the English Wikipedia. But the Germans are very happy with their 900,000 articles – they generally have a much more stringent standard. Wikipedia used to be known as the definitive guide to Pokemon – that would not fly in German Wikipedia. That’s their style. The German Wikipedia is more traditional – but also has a great reputation – the German government, libraries, and universities are all interested in working with Wikimedia Deutschland because their quality is so high.

That is to say, the inclusionist/exclusionist argument also varies widely depending on the cultural lens you use.

David: Is it a problem that neutrality happens only if there is enough homogeneity in the community? Or they will have to break off? Does Wikipedia reinforce a prevalent domain of discourse that everybody agrees on? And thus excluding other views?

Andrew: Certainly in some languages – the first twenty languages – the largest languages – are fairly well educated and multilingual – especially contributors for the English Wikipedia span the whole world – and there is diversity of view points. But after the twenty languages – the drop off is bigger – and people are more homogeneous.

David: Isn’t this the case in English Wikipedia as well? That is, neutrality hides a fork – people fork.

Andrew: Yes, but they create meaningless forks, that nobody links to, they fade away.

David: That is exactly the price that it exacts – marginalization of points of view out of mainstream – that they cannot get on the same page – lots of groups accuse Wikipedia of this.

Andrew: Jimbo said once that Neutral Point of View is a term of art – most things that work are not razor sharp. There is a lot of faith in the actual ground troops – that they stay within directive – and that hopefully the diverse community will take this in account and create reliable content.

David: Lets talk about the changing roles of authority. Being a big prof doesn’t matter – its bad form even if you say this.

Andrew: Editorial authority is even more interesting in Japanese Wikipedia – most are anonymous – this is because the dominant internet culture in Japan is based on anonymity. You could be discussing with anyone, a housewife or a prof, what matters is the quality of edits.

David: Let’s talk about Essjay.

Andrew: That was one of the bigger crisis. Essjay was a pseudonym – and on his user page it said that I can’t tell you who I am but I have a PhD in Theology and I work at an academic institution but would get into trouble if I tell you my real name. Was an incredible prolific contributor – over 10,000 edits and everybody generally accepts that they were good quality. He eventually got access to admin privileges – that is, he could check IP addresses of users behind the scenes, and only a dozen people can do that, had access to private data.

What happened was that the New Yorker was doing an article – by Stacy Schiff, a Pulitzer award winning reporter – she did an interview with Essjay – wrote a long piece. Then Essjay took a job with Wikia Wikimedia Foundation (EDIT: Andrew corrected me: he took a job with Wikia, the for-profit firm founded by Jimmy Wales and another Wikipedian Angela Beesley) – and to do so, he had to come clean – that he was a 20-year old with no PhD degree. This was a huge embarrassment to the New Yorker – it seemed that Stacy never even asked Essjay’s name just to fact check it.

Some people argue that Essjay lied to a reporter but had good contributions. Others pointed to the fact that he sometimes used his credentials to win arguments. It was a real soul searching for the community – a prized Wikipedian would lie to the outside world, to a Pulitzer award winning reporter, and raised issues with regard to having faith in each other in the community.

David: The increasing use of credentials – or the German system that now allows for the marking, a flagging of pages that are considered reliable – is this a trend that will continue?

Andrew: Germans lead on quality issues – they have a tighter community of admins, who almost act like a council – whereas the admins in the English Wikipedia function more like janitors. So why not have a flagged version – you could flag the last version of an article that is stable – and you show people the latest checked version. You get better quality but you lose that they are instantly updated. The Germans implemented this last year – quite a success – flagged 89% in first year. The English Wikipedia has interest to implement this but it is hard to get the community to reach consensus on anything at all. Right now it’s a total stalemate – it had a surge of initial support but now trickled down.

David: The common complain is that students go to Wikipedia and simply believe what is there. What is it that readers need to do not to be fooled by occasional vandalism? How scared should we be?

Andrew: Wikipedia should be the starting point, but not ending point. It should not be in citations, just like entries from the Britannica should not be cited.

David: How confident should we be when we use it to look things up>

Andrew: The critique that it is dangerous when 14 year olds take it as gospel is not fair. Most people are media savvy. And then there is a whole range of things the community implemented – for example, requiring sources – in 2003, 2004 you never had any article that was tagged ‘citation needed’, now you do everywhere – there is a team called the ‘citation needed patrol’. Standards have improved – but ultimately I think flagged versions should be put in some way – right now it looks like it will be used for entries of living persons – this is for libel reasons. We start there and see what happens.

Audience Questions

Question: Can you discuss failed Wiki projects?

Andrew: The battlefield of failed wikiprojects is vast. Wikitorial from the LA Times was a real disaster. There is an assumption that you put up a Wiki and the Wiki Magic will happen. The LA Times learned the hard way – if you have no robust community with admins that fight vandalism, it’s a recipe for disaster.

What you realize after all this failed projects – wiki is perfectly suited for encyclopedia. It’s like a bento box of writing.
Very structured writing and lends to crowdsourcing. Very modular. This is not true for a novel, for example. Penguin had a contest where they put up a Wiki and expected that the magic wiki crowd would write a novel – did not happen.
Those that do work: lots of sharing, step by step, modular structured style of writing. Certain type of content are like this, but lots don’t. A lot of other organizations learn the hard way.

Question: Why not make people use full names?

Andrew: There is always talk in community – now do we don’t need anonymous people anymore – they give us more problems than they are worth – lets start requiring higher standard. In the beginning – the original culture dominates – Wikipedia tends to be inclusive – anonymous users are the core value of “anyone can edit”.

David: What about pseudonyms?

Andrew: It makes you to be able to converse with this person, it allows interaction, although you don’t know the authenticity. You can still see all the edits. Interestingly, pseudonym users give less information than anonymous users – with anonymous users, an IP address is recorded, and that often provides geographic location, what organization you are part of, etc. The Wikiscanner used this to its advantage – found out that people in Congress, Ogilvy, all kinds of organizations were editing articles they probably should not be editing. It was a typical example of sunshine being the best disinfectant – it was a kind of watchdogging the crowd.

Question: If Wikipedia would have been run by company, would it have been different?

Andrew: If Wikipedia was a commercial company, no way it could have been successful – people contribute because it is a free license – same like with Linux – people know it wasn’t making a company rich. Example is the Spanish fork – in the early days, there were some rumours about the possibility of advertisements – the Spanish community went ballistic on the mention of ads – they literally took the ball and went home – started Encyclopedia Libre – convinced all contributors to leave. This incident set the tone for the community since then.

Question: Is the bulk of content made by a small number of people?

Andrew: The idea behind the 80/20 rule is that 80% is done by 20% of the people. But this is not necessarily true for Wikipedia – Aaron Swartz’s research shows that there is a wide swath of people that edit Wikipedia. While the distribution is still non-linear, it’s just not the case that there is an elite crowd who edits over hundred hours a week.

David: Aaron’s work shows that the creation of new articles, the bulk of it is done by a broad range of users – which makes intuitive sense.

Andrew: As far as where the community is now, we don’t have good numbers. Since October 2006, there is no authoritative dump of Wikipedia anymore – it takes more than a month to do a monthly dump. This leaves Wikipedia vulnerable – and you also can no longer do statistical analysis.

David: We should each download one page!

Question: Can you talk about Larry Sanger?

Andrew: Sanger has an odd role – he did set up most of the basic rules of Wikipedia – but over time also encouraged Wikipedia to be more elitist over time – and some started seeing him as a pariah, as the anti-Wikipedian. Citizendium is supposed to be Wikipedia done right – with a layer of expertise but still largely open. His main criterion seems to be maintainability. He thinks a lot of what is going on in Wikipedia is just bs – trying to turn vandals into productive members – he is saying, cut that out, work with experts who can cut through the junk. We’ll see what history will say about that.

(Question about the vote on license migration – got lost in the details)

David: Wikipedia experienced exponential growth – but what got us there may not be the right set of tools to move ahead.

Andrew: That’s why flagged is inevitable – not to grow further, but to maintain quality.

Question: How did the power structure evolve?

Andrew: The number of privileged positions have grown but tend to be technical rather than editorial oversight. As an admin – you can block users – but only in narrow situations. You can lock articles – but only temporary – for combating vandalism. Promotion is community decision, there are no hard metrics. Things considered include the number of edits, activities you engage in, social capital – these are all intentionally left vague – the decision is made on an interaction human human basis – it’s not like there is an eBay rating or Amazon ranking.

Question: Why are there different forks and how do they exist – is there a possibility to have one global Wikipedia instead of all these divides?

Andrew: You’re right that it is too easy to see the 2.8 million English entries as the super set from which other Wikipedia languages should be translated from. This set is missing lots of things on Chinese arts, history – things the Chinese Wikipedia has. But the problem is, you need bilingual folks, tools to discover which article is good in one language and has a bad counterpart in another ..

Question: Will the WikiMedia foundation do this?

Andrew: They are a great engine to raise funds.

They are currently looking for donations that will help them sustain the incredible valuable and good work they do. I ended up donating $77 dollar – why $77? It’s my birth year. It’s a small sum with a symbolic value that I hope will encourage others to chip in as well.

Why should you donate?

Donating to Global Voices helps tell them that they are doing a good job. The value here is symbolic, rather than material. This is not unimportant – they would never have gotten so big if most of their work was not ‘free’, free as in volunteer labor. Getting appreciation for the volunteer work you do is incredibly important. Viviana Zelizer has called this the crowding-in effect of money on volunteer work.

Donating to Global Voices helps them stay a bit more independent from big donors. And allow them to write about topics they think are important, as opposed to topics that will attract the biggest crowd. The question of how media organizations get funded is not a trivial one. Global Voices get funded through a combination of support from foundations, corporations and individual donations. Political economy, particularly work by scholars like Robert McChesney and Oscar Gandy to name a few, has pointed out how money shapes what media writes about, and what not. In a perfect world, media organizations would all be funded by many individual donations, so that they can maintain independence and write about topics without constraint. In reality, media organizations will often not write about topics that might offend their owners or advertisers. Also, they will write especially about topics that will get the attention of a lot of audiences in order to attract more advertisers. These are topics people might want, but not necessarily what they need. Consider how much words are devoted to Britney Spears and the iPhone, which are great topics, but they tend to drown out other regions, areas and topics.

To sum up, giving a donation is a good idea because they are great people that do important work nobody else is doing – we want to make sure they can continue to do this work as well as let them know we appreciate the work they do. Please consider making a donation.

Besides donating, there is another way to help and show your appreciation: by spreading the word. They have made some cool – and cute – badges you can use to put on your blog.

cross-posted from global voices, one world

Explaining the economic meltdown as an example of a gridlock economy, he suggests that in the past there was a direct one-on-one relation between lenders and borrowers, that they knew each other. Banks lost money on foreclosures, they’d rather work out a deal with you. But in the new world, there are potentially several thousand owners, and it is much harder to re-negotiate a loan. Too many owners fragment mortgages.

The second example he gives is drug patents. He tells a story about an invention, a treatment for Alzheimer. But producing this treatment would touch upon a dozen patents. Imagine a room with a dozen start-ups, each of them thinking they are sitting on the patent that is key to treating Alzheimer. Imagine having to negotiate with all of them. The inventor decided not to go for it and put the treatment on the shelf. The deal could not be made. Heller is making the argument that this is not an isolated case. There is a huge increase and investment in invention and patents in the last thirty years, but there has been a decrease in discovery of major classes of new medicines, what he calls the drug discovery gap. Forcefully, he argues, we have drugs that should and could exist, but don’t. And it is not just drugs, but this problem exists all across the high-tech frontier.

He starts his third example by asking a question: what is the most underused natural resource in the United States? The answer is: spectrum. About 90% is dead air. The licensing of spectrum dates back to Coolidge and basically hasn’t been updated since. We have created a system of geographically fragmented licenses that are non-transferable, making it extremely difficult to assemble public or private networks. The United States has fallen in broadband from number 1 to somewhere number 15. Cutting high-tech will not occur in the United States, the next generation technology cannot emerge here, because it is so hard to find spectrum to facilitate high-speed transfer.

Fourth example: airports. Why do we have to spend so much time at airports? Why not build more airports? Thirty years ago, air traffic was de-regulated, yet Denver is the only new airport that has been built in the United States since 1978. The reason we don’t have more new airports is former real estate law, that allows every community to block the assembly of land you need to build new airports.

Heller concludes that these are essentially all the same problem. There is a change, a shift in the nature of innovation. There used to be a one-on-one relationship between patent and invention, between copyright and song. That is the old style economy. Now the new style economy is much more like a funnel, from many to one. That is to say, assemblage is needed for innovation. Big breakthroughs come from assembling multiple parts into one. Cutting edge is found in mash-ups, remixes, even in the case of resources like land.

He calls this the tragedy of the anti-commons. The tragedy of the commons is that no owners lead to overuse. It was a huge turning point for the environmentalism movement, a metaphor leading to a change in framing, of thinking about public good problems. It was key to a spur to privatization, seen as the solution to the tragedy of commons; that is to say, private property is a great engine to conservation.

But privatization can overshoot.The tragedy of the anti-commons is when too many owners lead to too little use of scarce resources. This is, contrary to the tragedy of the commons, an invisible tragedy – you don’t see when something is not appearing, is not being invented, is not being built.

Edit: See also the much more lucid and detailed blogging of this event by my esteemed colleagues David Weinberger and Ethan Zuckerman. In particular, check out the fascinating exchange on the nature of “property” between Michael Heller and Yochai Benkler that I was unable to capture in my blog. Too many bloggers clearly did not lead to the underblogging of this event, although clearly some blogs were better than others.