18
Feb 11

The problem with online reputation

When it comes to seeking out knowledgable others about our work, we have a reputation problem. The people who have the highest reputation scores are usually the people who are the most public.

Offline we all know who to turn to ask advice on specific things. Specific people we work with now and people we’ve worked with in the past. Yet when thinking about turning to people online who we may not have worked with before, it’s nowhere near as straightforward. In fact, some of the best people I’ve worked with, the people whose opinions I most highly respect, have really low online reputation scores – as measured by emerging services such as Klout and PeerIndex.

These services are tackling an immensely challenging problem, credit to them. But this is a human problem, not a technology problem, and I’m not sure that measuring what people are saying publicly (or semi-publicly) and how others are responding is the right approach.

In my experience there is little correlation between people who are public, or people who are loud, and people who are knowledgeable. There are some very knowledgeable people who are public and loud. I follow them. I read their stuff. But there are a far greater number of people who are equally (and often more) knowledgeable, but simply prefer not to engage in public discourse. These are the people that matter.

Often, social media tools afford a lack of critical thinking. See something, half read it, tweet and reweet it. It’s easier than crafting a thoughtful response. We’re all guilty. The problem is that it amplifies the connected group, without much analysis of the quality of the content. Through the lens of our online measurement tools the author looks knowledgeable, the content looks good, and the retweeters look influential. But none of those may be true. Being part of a clique community on Twitter does not make someone knowledgeable.

Moving away from the tweets and retweets, “official” approaches like LinkedIn recommendations are also problematic when determining who is knowledgable. Due to social pressure, people only give positive reviews. Everyone is an expert. I’ve seen job adverts where employers ask for at least three reviews on LinkedIn. That’s not a job requirement, that’s a game.

Another approach is to compare people, for example Cubeduel lets you vote between two coworkers. The problem with this approach is that very few people are directly comparable. You can’t compare people who do completely different jobs. More importantly, we’ve built good friendships with many of the people we’ve worked with, and comradery driven obligation takes over. Our friends get the vote over someone who might be better than them.

Honestly.com tries to solve the social pressure problem that exists on LinkedIn by allowing people to leave reviews that are anonymous to others but attributed to the real you behind the scenes. However, the lowest average score I’ve seen is 4 out of 5 stars – so something is clearly not working. I’m guessing that people are afraid of their review being accidentally leaked – and the risk is much greater than simply giving people a good review.

So given this mess, how do we figure out who is knowledgeable by measuring reputation and describing it online?

As with most people problems, I feel the roots of the solution lie offline. From our ongoing face to face interactions, we learn who is knowledgeable, who to turn to for an informed opinion, who is likely to say it like it is, and who has hidden agendas. There is no substitute for that. Reputation is built conversation by conversation at the desks, halls, cafes and meeting rooms of businesses all around the world. Any attempt at measuring reputation needs to map this.

The first step is to map who worked with whom. Not that we both worked at Google at the same time, but that we worked in the same team for an extended period. Then we need to map how much interaction we had. We collaborated almost every day. Or we were in the same meeting every week but not much more. We then need to map areas of experience and expertise. That Paul knows about doing research. Or design. Or he knows about online identity. Or designing communication tools. Finally we need to cross reference those areas with other people who also worked in the same team. Does Paul really know about “online identity”? Do you really trust him to help you create new communication tools? In effect, it’s like turning an internal peer review process inside out.

Mapping this will be challenging, it will involve incentivizing people to rate or describe the people they have worked with, the people they trust. But until we do, we’ll continue to hear about the loudest, most public people, and miss out on the people making the greatest difference inside companies around the world.