I’m reluctant to throw another stink-bomb into the debate about sentiment analysis, but I have a view of the tech that I don’t often see from others (I’ve worked extensively with sentiment analysis tech, in both development, while at Infonic, and as a power user of dozens of enterprise social media monitoring tools at The Conversation Group).

So here they are: the two reasons I think sentiment analysis automation has no place in social media monitoring:

  1. Granularity. Here’s the reason that people often find that their experiences of sentiment accuracy don’t match the claims on the boxes: training and testing of the tech is done against a brief that doesn’t match the typical user needs. It used to be that testing (and hence reported accuracy) was done at a document level (ie the sentiment of a piece of text as a whole), not at an entity level (ie the sentiment towards the brand, person, or concept that a user is trying to track). I suspect it still is, but because vendors are so fluffy (defensive?) in the way they market sentiment analysis, I can’t tell you for sure. Reasonable expectation is that the sentiment reported by a platform relates to the keyword(s) or search string in which sentiment is reported, not just towards the aggregate sentiment in the whole document returned by that search. For typical users of enterprise social media monitoring tools, it’s hard to see how document level sentiment is of any use at all – it makes the score given to any multi-entity documents not just inaccurate, but probably actively misleading (and rules out comparison of sentiment towards competing brands). I guess monopolies ought to be good to go though
  2. Fuzzy definitions. Here the problem is on the client side. As it’s currently sold in enterprise applications, sentiment analysis means something quite specific – in a nutshell the presence of words and phrases deemed culturally positive or negative in a body of text, and a judgement of the overall tendency within that body of text. It feels like customers of monitoring platforms frequently have a slight cognitive blind-spot: they see “positive” and read “that which benefits us”; they see “negative” and read “that which harms us”. But as anyone working in comms worth their salt knows, those two concepts really aren’t the same thing at all, or press embargoes wouldn’t exist and Nudge would never have been written

This isn’t to say that I don’t believe in sentiment analysis of social media – I do, in fact I can’t think of a recent case in which I haven’t recommended that sentiment is tracked as part of a core set of metrics. I just believe that technology doesn’t cut it for enterprise social media search applications, and I’m sceptical that it ever will. It’s simply too context-dependent to work for every (any?) individual case straight from the box. I’m supportive of tools like ScoutLabs, which include a user-override for sentiment scores, so it can be used a production console for human coding, but I wish the machine generated sentiment score wasn’t there to start with.

So where is it useful? In high volume, high stakes text mining, where the potential payoff of doing-clever-stuff-with-data can justify the huge effort needed to massage and contextualise the data and the technology to suit specific project circumstances. The technology was originally commercialised for use in HFTS type applications after all, not marcomms and customer service support work.

  • Share/Bookmark