In a blog post over at The Publius Project, Judith Donath asks “Is Reputation Obsolete.” It’s a provocative piece and well worth a read. Honestly, I’ve spent the past week trying to dip into the literature on reputation systems and to call it the shallow end of the pool would be an insult to pools. It’s shocking how little attention has been done on the topic, and Donath raises a lot of interesting points about the ill-fit between present day reputation systems and the total availability of online information.
It seems to me that her post could be best rephrased as “Is Reputation Tracking Obsolete?” In that case, the answer would be a clear and definitive yes.
Reputation in its purest form is deep, contextualized, complex, and local. I have a very different reputation with my colleagues in the Sierra Club than I do with other academics, and still another one with my drinking buddies. All of those reputations are linked to different dimensions of my identity, and each is accurate in its own way. They accrue over time, and they are exceedingly difficult to scale up from local context to general form.
Online, reputational data is put at a premium, because the purer the anonymity, the worse people are bound to act. I haven’t seen any studies on this yet (I’ll get around to doing one someday, I suppose), but it’s pretty clear that when you require people to login before posting comments to a blog, they self-moderate a bit more, and when you add a Mojo system like they have at SlashDot and DailyKos, and “superuser” status contingent on high Mojo ratings, people behave better still. That’s standard “Shadow of the Future” stuff, a basic finding from game theory, and replicated in a host of experimental settings. So reputation systems incentivize good behavior while distributing the costs of punishing bad behavior. As a basic example, consider how costly eBay would be if they had to provide top-down monitoring of all transactions. Actually, you don’t need to bother considering it: without reputation tracking, there would be no eBay. Period.
So is reputation obsolete? Yes and no. The thing we need to recognize is that when you divorce reputation assessments from their local, complex, and contextualized settings, you have to rely on rough proxies to fill in the gap. Those proxies are not, themselves, reputation. When an eBay buyer ranks the seller, that tells us relatively little about the seller. When a DailyKos user contributes to a diarist’s “tip jar,” that functions as a “thumbs up.” But real reputation isn’t the aggregate of online clapping and booing. And as more diverse information becomes available online, the simplicity of aggregating clapping and booing seems like a coarse and outdated tool for measuring reputation.
I would suggest that the quality of reputation tracking is always going to hinge on three elements:
(1)relevance of the proxy data. How good of an approximation does the online rating mechanism provide?
(2)Traffic levels. I’m always entertained by low-traffic blogs that include recommended diary structures and such. Online reputation tracking assumes huge inputs, but given the power law distributions of web traffic, we know that there are only going to be a select few webspaces that obtain that level of traffic.
(3)Gaming of the system or lack thereof. This last one is long-term problematic. Any high-traffic webspace is going to represent valuable online real estate. The perverse incentives are there for actors to try to figure out the rules of the game and then innovate ways to get around them. We haven’t seen a lot of innovations in reputations systems for years, and most of the literature seems to be focused solely on eBay. So reputation tracking systems are probably obsolete at this point, simply because every system is going to have weaknesses and vulnerabilities, and there haven’t been many new developments (at least that I’m aware of — which is a decent indicator that if something great is out there, it sure hasn’t diffused very widely yet).
What we really need is reputation systems that take advantage of Metcalfe’s Law. As processing speed and memory continue to double — as Information Abundance becomes still more abundant — we need to develop reputation tracking systems that use better proxies. Donath asks whether “in a world where all action is recorded, is there still need for reputational information?” I would respond, “Yes, all the moreso!” If we broadly understand reputation data as a form of filtering and content management, we have little choice but to rely on reputation assessments, but we also need them to evolve along with the rest of the web. In a world where all action is recorded, reputational information is all the more necessary so we can sort through the mess. But likewise, as more types of data become available, we need to diversify the types of proxies we use for assessing reputation. This will be particularly true as the mobile web comes into wider use, rendering whole new classes of data available.
The real challenge lies in figuring out how to sort and use that data, particularly keeping in mind the competing needs for reputational assessment/filtering and privacy. The weaker the privacy norms, the stronger the reputation tracking can be. I don’t think I particularly want my academic or Sierra Club colleagues to know my reputation among my drinking buddies, though (or vice versa, for that matter!). The tradeoff has steeply decreasing returns at some point, and there’s an important role for public scholars like Donath in helping to identify what that point might be.