85% of user reviews are genuine says Gartner

Except that it doesn’t. The headline on the press release just received is:

Gartner Says By 2014, 10-15 Per Cent of Social Media Reviews to Be Fake, Paid for By Companies

The full paid-for report is here.

I think we can agree that fake user reviews are a Bad Thing; but the current agitation about sock puppetry among book authors and fake reviews in general seems to be missing the key point, which is that user reviews in general have had a transformative and beneficial impact and should be applauded.

Examples that come to mind are tripadvisor, which has hotel and restaurant reviews, user reviews in general on Amazon, and restaurant reviews on Yelp.

Since the emergence of sites like these, I have had fewer disappointments and more pleasant surprises in travel, books, music, technology and more.

That said, there are plenty of flaws in the various systems for gathering user opinions. The problems are not confined to the most extreme examples of paid-for reviews masquerading as genuine.

You often get what I think of as a smiley face effect, where across the spread of reviews there are disproportionate numbers that are highly favourable, because fans are more likely to post reviews, and disproportionate numbers that are too negative, because users who have a bad experience are more likely to post.

After all, if you use a product or service and it is so-so, why bother posting a review?

Another problem on sites like Amazon is competition for top reviewer status, which means reviews are upvoted by friends and downvoted by rivals, a kind of meta-sock-puppetry.

On balance, I reckon 85% is far too high, if you want a measure of what proportion of user reviews on social media are both genuine and useful. Certainly, Gartner’s headline seems back-to-front in terms of what would be more surprising.

Those reading the reviews should also be credited with some common sense. Most people will look for a balance of opinion over a quantity of reviews where possible, and observe which reviews seem to be based on a solid analysis of facts rather than on bland opinions.

Long live user reviews.

One thought on “85% of user reviews are genuine says Gartner”

  1. Obligatory xkcd reference:

    Surprisingly true to life (and summarising the issues you mention with user reviews) – a rating of 5 stars likely means too few reviews to be reliable. A rating of four or less means a high proportion of very negative reviews. Leaving 4-4.5 stars as a genuinely good rating…

Comments are closed.