One of the few things that captured the public imagination from Chaos theory (a subject now making its second appearance on this blog (!) after this) was the idea that a butterfly flapping its wings in the Amazonian Rainforest might ultimately be the trigger for a hurricane which devastates the Eastern seaboard of the US. The principle being that the butterfly's actions have effects, which create other effects, which create others. Starting very small, these effects are magnified throughout the system that is meteoreology, over and over, growing each time.
Ultimately, the tiniest, most incidental of activities can thus be the start of a chain of events which culminates in the biggest.
This phenomenon has been long something with which the market research industry has failed to grapple - principally because it's Too Hard*. Big on deterministic solutions, and attempts to break down markets, consumer behaviours, brand activities into their constituent parts - from which it still (often) makes linear, literalistic assumptions - the concept of 'emergence' has failed to gain any real traction in mainstream research thinking. Emergence is the unfortunate fact that when systems operate as systems, they don't actually behave as you might expect from a simple study of the activities and reponses of their individual parts. A termite colony is an emergent system. The human body is one. The collective response to the death of Princess Diana was one. The world economy is another (and the absurd primitivity of mainstream economics in failing to recognise this - still - has been laid bare these last two years).
Many mainstream research methodologies - I mean quantitative ones chiefly, and so called 'predictive' ones specifically - complete ignore this question. And they have done for decades. And made a lot of money from so doing too.
But here's another thing.
If you are going to try and draw some conclusions of sorts about what's going on, or what something means in a collective sense by exploring the tiny parts of a system and then rolling what you find into some big ole' conclusion, then you'd better make sure you the data you collect is accurate and means what you say it does.
If you don't, then the little errors you make will be compounded over and over and over as you scale your conclusions up. Like building a house with rotten joists, the most dangerous moment comes when you put what you've built under pressure - by constructing, say, several storeys on top of them. The rotten wood itself isn't dangerous per se. What you do with it, and what happens when you sell the house to some unsuspecting buyer, is.
But, hey, who cares? You're outta there. And when the whole thing falls down, who's to say it was the joists that caused it?
Yesterday I spent the day at the snappily titled Social Media Monitoring Bootcamp.
I've been to plenty of conferences in my time. This was, in fact, one of the best I can recall for sometime. But not because the news the speakers had to give us was good.
Social Media - talked about everywhere in marketing, and particularly within, er, Social Media - is trying to come of age. As we speak, companies large and small are queuing up to start Tweeting, or to invite fans to their FB pages. They are starting to sweat a little about the real effect of all those blog posts and forum comments. They are wondering just what the hell FourSquare is. They are worrying that the brand snapping at their heels is doing this stuff and they aren't. And so on.
Ultimately, most don't have a clue why they need to do it, they just somehow feel they do. Some are using the opportunity of this new world as essentially PR for the 21st century - a chance to tell people things they might or might not want to know, and sell them things. A few - a very very few - are understanding that potentially what's going on is a game changer. That they are now on a two way street and that what's on offer is the chance, and the opportunity, to develop an entirely new dialogue with 'real' people - a dialogue which can help them make and sell the right things, at the right price, to people who really want to buy them, who feel loyal to that brand and who actually might give a damn.
Naturally, as the avenues expand, and the technology improves, a 'Measurement' sector has emerged. It's certainly needed. The absence of it has been one of the biggest brakes on the growth of this area, as Boardrooms look puzzled about it all and wonder whether any of their company's time and investment actually achieves anything.
Yesterday revealed for me the fairly ghastly truth on this question - confirming a hunch I'd had since I first starting looking into all this. There are many many companies out there offering funky software you can download, offering to measure the prevailing sentiment (positive, negative, neutral) pertaining to your brand. Giving you data on the number of mentions you have had, and where. Even purporting to tell you how much 'influence' you are having, via the 'influence' of those who are writing about you.
And guess what? Almost all of it is smoke and mirrors. Bullshit, as the excellent talk by Philip Sheldrake put it, in fact.
Why? Because much of the data on which these conclusions are created is flawed. Incomplete. Subjective. Even incoherent.
The query you use to define what you're looking for is usually immensely difficult to form accurately. It takes considerable crafting, and patience.
The spiders which go out looking for reference to what you come up with may or may not cover anything like the 'entire' net (think about it...Google can't even do that, and is a long way off from doing so, when you include the almost infinite amount of data that is stored within elements within pages within trillions of sites).
The algorhythms that are used to define whether the person who is posting or tweeting about you is 'influential'...on this topic, in these circumstances, to this group of others...are nonsense.
The 'automated sentiment analysis' which claims to detect whether you are getting good or bad coverage is rudimentary, still unable to decode real human writing properly, understand context, intent, sarcasm, irony, or vernacular (and any focus on single word or phrase analysis is useless). Nor can it adequately unpick complex remarks, or those which might reference several brands. Nor can it take account of the fact that human beings don't always say what they mean, or do what they say.
And on top of all this, everybody does it in a completely different way, with their own proprietary software, about which they are probably not going to tell you very much (check the answer to question 2 here). As Marshall Sponder pointed out, you need only put the same query into five different platforms, and wait for their sentiment analysis to come back. You'll get five entirely different sets of data. He did it.
And on top of this, it all eventually boils down to the requirement of human beings to sit down and actually read all this stuff and then work out what everyone is really saying. Read the data that, of course, may have already been entirely, subjectively, ruined. And there's not a whole lot of time and money spent on that bit of the equation in the rush for 'intelligent' web crawlers to do it all.
There are no common, industy-wide, protocols for collecting data, nor for defining terms, nor for providing results. What there are, are hundreds of sexy looking dashboards which will produce very plausible looking graphs, pie charts, and ranked tables. All of which you can download easily, and cut and paste for the Marketing Director. And many of which aren't worth the electricity used for the pixels which display them. Nice.
So there we are. Hyperbole? Overclaim? I'd go further. Snakeoil.
Or if you like, the rotten joist, nicely painted, buried in the masonry?
* You know the old joke? This man is walking home from the pub one night and he sees another guy peering at the ground by his car. So he stops and asks him what he's doing, and the other chap says "I'm looking for my keys".
"Right," says our man, "Where did you lose them?"
"Over there in that bush" he says. "But the light's better here".
