At the beginning of June I attended the Inspire Conference, London, a great event organised by a the very smart and thoroughly charming Adil mohammed, whom I have the pleasure of working with of late.  It was a great day that lived up to its name. There were many great presentations but one that stood out was by Nick Halstead, CEO of MediaSift and creator of Tweetmeme. His presentation was called Big Data and centred around growth of data and how important and valuable 'Big Data' management is.

Perhaps one of the most interesting things he mentioned was that it has now been proven that the average sentiment of everyone on twitter correlates ahead of the stock market to an accuracy of 87%. Consequently an increasing number of hedge funds are now using twitter sentiments as their guide to investment.   This was news to me so I feverishly tweeted it.   Now on the face of it, it seems ludicrous – fund managers speculating with my pension fund based on the global ramblings of a load of social butterflies talking about what they had for breakfast?

But the more you think about it, the more it makes sense.  The 'sentiment' of millions of people, with their varied networks, individual views and collective intelligence, all flowing naturally and broadly and incorporating influence from all sources. Most importantly, freedom of expression and thought.  It's realtime market research on a massive scale, with the most comprehensive and diverse research pool to draw from.  Contrast this with the average bunch of analysts and financial 'experts', sitting in their financial institutions, surrounded by other financial experts, in the centre of the financial district.  For up to 18 hours a day.

When you consider that a good number of these chaps probably spend the little recreational time they have left snorting cocaine, quaffing champers and chatting up lap dancers its not that difficult to appreciate that they live in a bubble and are probably far too close to the numbers and too engrossed in their world to tap into the broader collective intelligence.  All of a sudden, betting on twitter starts to make sense.

I didn't give it much more thought until recently when I came across a conversation going on amongst some of my US HR friends at the Ohio SHRM conference.  Charlie Judy sent out a tweet that caught my eye:

I think he has a point and I tweeted my agreement.  For me, surveys are a tool that were once the only real way to get an understanding of what is going on inside the organisation. But for many years I have questioned their value and results.  Ask a leading question and you will get a leading answer.  See this recent article in HR magazine by Michael Silverman who challenges the current status quo on employee surveys.  He talks about "distilling the essence of conversations" which is exactly my point about sentiment.  Things have moved on and whilst I can see annual surveys playing a useful role in benchmarking, the emergence of social and community tools allow us to "open up the watercooler conversation" and get a real sense of what is on the mind of our fellow employees. Better still, for the small number of companies that are adopting these technologies internally, there are additional benefits including significantly increased engagement and product/service innovation.

Measuring employee sentiment rather than surveying them across a number of structured questions would appear to unlock far more than the traditional survey route and I'm hoping to see a much wider use of this approach as more and more organisations realise that the watercooler is where the insight is.