Blackbeard Blog

2. The Process Is The Story

Some of the case studies on offer were more interesting. Kodak, who sponsored the whole event and have put together a free guide to social media practise, talked about how they’d taken a product idea straight from Twitter – implementing specific suggestions, like flexible USB ports and mic jacks, and then crowdsourcing a name. This kind of thing is becoming more common – taking design and useability improvements straight from the user’s mouth online.

You might argue that they’d have got the insights and information anyway, but that’s not the point – the process here is the story. It’s like the three young filmmakers who got up to tell us about the crowd-funded film they’re making of a Jules Verne novel. You’re not buying a good film, or even the expectation of a good film. You’re buying the experience and warm feeling of participating in something crowd-y.

That’s not to say the film – or the camera or the Axe pick-up tips Twitter – won’t end up being good. It might be magnificent! But we’re still in talking-dog territory here, where the fact of socialness matters more than the outcome. This won’t last forever, of course. It probably won’t last out 2010.

1. Social Landfill

I’ll start with the marketing stuff, because I work in that field, though in some ways it seemed the most sluggish of the various criss-crossing streams of thought. There was an awful lot of familiar stuff being trotted out – people have unprecedented power to influence brands, you have to join in the conversation, the customer does the marketing for you, you need to give people something they can’t get elsewhere… nothing that wasn’t on some level true, you understand, but also no longer inspiring by itself.

Especially when the first concrete example was Unilever speculating that maybe its Axe brand should start a Twitter account with a daily tip on how to pick up women. Later on a rep from Warner Music, after explaining that none of the majors were really experts in the digital space, talked enthusiastically about how they’d got Madonna onto trending topics with some hashtag or other when her Greatest Hits record was out. As my notes said: “Oh joy.”

So here’s the problem – I’m coming at this conference as a researcher, as someone not unsympathetic to marketing, but also as a user of Twitter, someone who enjoys its ecosystem for the info it feeds me. And as a researcher I appreciate that one of the demands of social media on brands is a shift away from creating advertising and towards creating content and experience.

But as a user I know some of that is junk content and junk experiences and I hope we’re now at a stage where we can call that out. Something like the Axe Twitter idea is a bit uninspired but would only affect anyone following it – hashtag spamming, on the other hand, deliberately drives attention away from something potentially more important or interesting or just funny.

0. On The Disparate Edge Of Now

I went to the 140 Characters conference on Tuesday and took a lot of notes, expecting to write a quick summary. Once I started on it last night, the quick summary turned into a 3,500 word monster, so I’ve split it into 10 parts, plus this intro, and will be posting them over the course of today.

The 140 Characters conference – tagline: “The State of NOW” - is an exhausting but rich experience. With 50+ speakers in a single day the scheduling is a deliberate jumble, refusing to coalesce into a marketing event, or a cool stuff event, or a tech or theory one. There are very few audience questions, the turnover is rapid, there are no real breaks apart from an hour for lunch. So the vibe is very much that of the firehose, the real-time brain-pummeling info-stream: not interested in Twittering taco trucks? There’ll be Facebook-wielding policemen along in a minute.

The EasyConference style of 140 fits this to some degree – no coffee, no lunch, not much legroom - you’re here for the INFO, dammit, not to schmooze. That’s how they want to play it and fair enough – the lack of WiFi was a no-frills touch too far though.

A paper-by-paper summary of the 25 or so speakers and panels would be ridiculous, and you can see most of it on video around the web anyway: this is an attempt to pull together the day into some kind of narrative, or at least pick out important themes, dancing about from panel to panel. It’s a mix of what people actually said, the implications as I understood them at the time, and further thoughts of mine post-conference.

The Future Of Research: 10 Odd Ideas

Here are ten things I think might happen in and to market research before too long. As usual, I’m chasing ideas that interest me. They aren’t supposed to be sober predictions. They are not representative of the views of my employer, they are not wholly representative of the views of ME, and some of them contradict one another. Here, in other words, be dragons. I hope you like them.

1. Respondents, RIP: With enough behavioural data and the right algorithms to analyse it, there is nothing we cannot know about people. This statement is one (extreme) conclusion of a tendency to move away from self-reported data and claimed attitudes, towards a kind of post-respondent research, in which information is crunched out of the datamass with never a question asked. What matters isn’t whether the statement is right but that powerful people are acting and investing as if it’s right. The giants of the internet – Google in particular – seem to have a near religious belief in the power of accumulated data to drive decisions.

2. Social Meteorology: What then of social media? We’re already seeing the metaphor appearing of social networks and communities as a kind of weather system of human attitudes and feeling. Studying this highly complex weather system (and mapping it, since social-local doesn’t map to geographic local) – with the ultimate aim of learning to navigate and predict it – will be a preoccupation of marketers, researchers, academics and quite possibly politicians in the 10s.

3. Dawn of the replicants: I’ve written on Research’s site about experiments creating demographic twitter bots – replicant individuals who retweet content along particular demographic, attitudinal or interest-based lines. John Griffiths calls them “furbies” for clients – the target consumer digitally manifested and personified, like a 21st century pen-portrait. And as the technology advances, the ability of replicants to predict or typify groups of people will only increase.

4. Data brokers: What about flesh-and-blood respondents (assuming they’re needed at all)? Also in Research I’ve predicted that participants – particularly in-demand or hard-to-reach ones – will have a growing awareness of the value of their data and opinions, and that the incentivisation system designed to reward them for these may be broken. Why shouldn’t they bargain – collectively or individually – for better incentives, or sign up to new kinds of panel which can act as their information brokers: a tilting of the research bargain back in the participant’s favour.

5. Business class research: For the real high rollers and super-hard-to-reach participants, incentives aren’t enough: you need to retool the whole research experience to make it as seductive and delightful as possible – a kind of first- or business-class research to set against the economy experience we usually offer. Whether this is a bespoke, beautifully designed and smooth online questionnaire or a luxurious one-on-one interview, there will be some participants who are definitely more equal than others.

6. Spontaneous surveys: The rest of us might find ourselves faced with authorless surveys. If we can be served ads based on our searches and interests, why not questions? Researchers will be thinking in terms of short, highly modular questions anyway – release those as streams of individual attitudinal questions and let Google serve them individually. Eventually it might auto-generate its own questions and let you buy “information terms” like you buy ad words.

7. Goodbye community, hello swarm: “Community” may be the hottest word in research right now but in terms of the wider web communities are a staging post – a destination-based solution to the problem of group formation. The direction of the net is towards flows of information, not destinations, and the temporary, ad hoc swarms of interest around a Twitter hashtag or Facebook group are nimbler and less demanding for participants than community membership. Future research communities will be task or goal-oriented swarms, not carefully-built MROCs

8. Peak crowd: Just as crowdsourcing is becoming wholly mainstream as a creative tool, the spectre emerges of “peak crowd” – the point at which the effort demanded by all the creative competitions, open innovation platforms, brand communities, crowdsourcing opportunities etc. out there exceeds the creative ability of those who could give a monkey’s. The result, hopefully, will be a big flushing out of limp activities and lame techniques.

9. Play Power: So the lesson is, amuse people. To get all Herd-y, bring people together and give them something interesting to do. Actually interesting. Researchers are going to be learning a lot more from game designers, who are not only good at building stuff people want to do (individually or together), they can get those people to actually pay for it.

10. Social sabotage: At the moment we’re in a happy era where in the excitement of getting our hands on social tools people actually believe they can trust what other people say using them. Information gleaned via social media is natural, authentic, the true voice of the customer, etc. How we will look back and laugh. We’re already seeing people gaming review systems – the iPhone App Store, for instance – and we’ll see a lot more social sabotage: from anti-consumer groups spreading misinformation, brands seeding social media with breadcrumb trails of false insight, refund-chasing customer complaints, and simple trolling.

I would love to hear some of YOUR curious ideas for what’s going to happen.

Black Hat Research

Some phrases that jumped out at me from the #esoc (ESOMAR Online Research Conference) tweetstream:

“a tsunami of unstructured data”

“no mentions of representativeness, why not? Fit for purpose, not for rep.”

“ticks on the back of the information hippo”

“Just listen”

OK, I admit it, the hippo one was mine. But this was a theme of the online conference - data, data, everywhere, data that must be listened to. It must also conform to certain standards of quality, at least when we create it ourselves via surveys, but the definition of quality is subordinate to questions of usefulness - did the information help its buyer make a better decision?

Nothing wrong with that! But at conferences it’s also worth listening for the dogs which don’t bark - in this case privacy, ethics, consent, that kind of stuff. These have been growing topics at UK conferences, it surprised me they weren’t more under discussion here. At one point a guy from Facebook explained that no, they weren’t planning on opening the platform up to research too quickly because the user and her privacy came first. Given how keen Facebook are on advertisers this may not be as high-minded as it sounded, but even so the statement was a rebuke: you lot, he was implying, are not an industry that respects your participants terribly much.

Anyhow, it seems to me we have a nice little research storm brewing. First of all we have an increased pragmatism: as the industry has become more and more client-centric, we’ve moved to a point where the value of the information we sell lies in its ability to help a client make good decisions. It always has been, of course, but the information could be rated another way - how valid was it? It’s all very well saying “research was never representative”, but you don’t get off that easy - representativity might not have been achievable, but it was the ideal to strive for as best you could as you looked for validity. “Fit for purpose” as a test of validity is a big shift towards, essentially, deregulation.

Now, I love social media, I think it’s stuffed with valuable information, and in a sense deregulation suits me just fine. But the industry isn’t actually embracing ‘deregulation’ as such - quite the opposite: it’s bristling with new definitions, taxonomies, codes. And this is the second part of the storm: codification - because the landscape of research is in flux, industry bodies are trying to update their rules and principles, often for the very good reason that they want to stay out of any potential legislative trouble. After all, marketers are increasingly a target for government, why wouldn’t researchers be too?

And the third part of the storm is the sheer abundance of information that now exists about people. People are sharing more information but they’re also leaking more information all the time - not to research companies, but it’s out there and a lot of the time it’s public. This element WAS talked about at the conference but generally in the sense of it being a happy opportunity.

So what we have is a collision of pragmatism and codification in a context of abundance. And I’m saying what that’s going to lead to - it is already - is black hat research. It won’t be called that of course, it probably won’t even refer to itself as research and it won’t be at many of our conferences.

What is it? Well, you have the information gathering which happens within industry codes and conforms to industry quality standards. And you have the law which lays out what information you can actually legally gather but which is full of loopholes and grey areas and stuff. And black hat research is what happens between the codes and the law. Scraping and mashing up personal data, rogue PR surveying, push polling, communities that mix research with marketing… you can add your own, I’m sure. Black hat research is going to get bigger and bigger and more important, a shadowland of information and intelligence provision which passes the “fit for purpose” test but maybe not other ones we might set it.

Call For Opinions

When Alison Macleod of The Human Element blog posted her plea for market researchers to be more opinionated, I took it as something of a challenge. So here, quite unsupported by anything other than grumpiness and prejudice, are some of my research- (and social media) related opinions.

1. “Insights” aren’t zen koans. If you can express something that briefly, it’s probably banal.

2. Between “data” and “recommendation” comes a little thing called “argument” which we neglect rather too readily.

3. Making two different pieces of information talk to one another is the most important skill a researcher (or almost anyone) can have.

4. We may not be as good as we ought to be at interpreting information, but by god we’re better at it than 90% of the people who end up blogging that information, so we have to get the message right from the start.

5. We are really bad at making celebrities out of our great practitioners: where’s the research Rory Sutherland? Why isn’t she or he at TED?

6. An online community is a factory for unintended consequences, and most of the people using them don’t understand how they work, let alone how to analyse them well. (I am not saying I understand how they work either.)

7. I know some very bright people who do semiotic work but on stage or in case study format it almost always ends up looking like a qual black box, one set up to produce undergraduate cultural studies essays.

8. I am always scrupulously polite about neuromarketing but if you were to WIRE UP MY BRAIN at a convention you’d be able to tell my true feelings. Or I might just be thinking of how nasty the canapes were.

9. The engine powering social media isn’t “influence”, it’s favouritism. Most talk of ‘trust’ is a way of justifying cronyism. This is not always bad, still less is it avoidable, but it’s not a brave new social arrangement either.

10. Hypocritical this in the light of much of the above, but the very worst thing about market research is its unending tendency to flagellate itself and envy people who are a great deal less informed than it is.

I’m not saying these are controversial, or consistent, and certainly I bet they’re not all correct, but there you are. Opine away on your own blogs!

Goodhart's Law and Market Research

This weekend I came across Goodhart’s Law. Like a lot of “laws” it’s more Goodhart’s Observed Tendency: it basically says that when you start basing policy around an economic indicator, the information value of that indicator falls to zero. It’s a kind of decision-making equivalent of the observer effect, as seen in physics. Or, if you like, a fancy way of saying “a watched pot never boils”.

Now in its original form - talking about government policy - it’s a sly way of asserting that government intervention is useless and so economic agents should be left to their own devices. But I suspect Goodhart’s Law - or something like it - also applies at every other level of measurement activity, including the firms the majority of said agents operate in.

How might this actually work? We design metrics to simplify complex systems. But when a value designed to describe a system becomes a way of assessing success within that system, two things can happen:

- the system adapts to reflect (and therefore game) the metric

- the stuff not reflected in the metric goes unnoticed, becoming a big breeding ground for potential unintended consequences.

And the more important the metric is, the more it gets gamed.

Anyone who’s spent any time working with social media - looking at “buzz”, reputation systems, measuring “influence” etc. - won’t find these ideas particularly foreign: I wasn’t surprised to find good blog posts from a couple of years ago talking about it. You can see it happen on Twitter, say, with “follower counts”.

Over in the world of market research, though, these ideas aren’t quite as commonly expressed and I suspect many wouldn’t agree with them. In this business an aphorism I’ve heard quite a lot is “if you can’t measure it, you can’t manage it”. What Goodhart’s Law is implying is the inverse: “once you measure it, it manages you”

Let’s assume Goodhart’s Law is at least poking at a greater truth about the unreliability of metrics to assess complex systems. What can decision-makers do about it?

1. Abandon the metrics.

2. Double down on Goodhart’s Law and increase the importance of a metric until removal of the metric would crash the system.

3. Increase the opacity of the metric so gaming it becomes significantly more difficult.

All of these, of course, have their risks :)

Snark and Stability

Nitsuh Abebe is one of the smartest guys on the Internet (my patch thereof). In this post he offers a defense - or at least the best explanation I’ve seen - of how snark became a default mode of online discourse.

This bit is really crucial I think (emphasis mine):

Flippancy is more fun. The work of reaching out and explaining things is potentially dull and time-wasting; it’s just plain funnier and more exciting and more gratifying to be on the inside of shared assumptions. (We like talking to friends, not strangers.) The histories of a lot of message boards and comments boxes can be traced out along these lines: they begin with a few people earnestly explaining themselves to one another, finding common assumptions and common ground and welcoming newcomers; then they grow, and their shared assumptions solidify, and they get flip and concise and referential and giggle at newcomers who stumble in and Have to Ask.

This is exactly right, and is what I was getting at in my conference paper a couple of years ago when I talked about the shift from “content motivation” to “social motivation” within communities (of course, Nitsuh and I are thinking primarily of the same community, so beware!).

At an individual level community is all about having interesting conversations and meeting new people and suchlike. But at a group level all those conversations and interactions create a complex system out of which emerges social convention and shared knowledge and the perhaps unpleasant glue of snark. And this happens pretty much beyond the level of one or two individuals to change (well, OK, I’m not sure I believe that, for reasons I’ll come on to).

Anyway, occasionally on social media blogs you get someone saying “LOL Twitter is all about what people had for lunch” and someone else then says “No LOL @YOU because we really need this phatic stuff and it’s good for us.” And snark is kind of like next level phatic communication - we’ve invented something which combines the two crucial primate group activities of picking shit out of each others fur AND chasing other apes off the territory. Good for us! (Seriously!)

You can tell I’ve been reading Herd recently, I’m sure.

Which book also reminded me of Philip Ball’s terrific Critical Mass, and the stuff he writes there about metastable states and their applicability to social behaviour. An example of a metastable state - forgive my rubbish layman’s explanation! - is when water stays liquid well below freezing point, until it gets disturbed, and collapses into its stable state (ice) all at once. What I took from Critical Mass is an appreciation not necessarily for the science of metastability but for the concept as a wonderful metaphor for fragile equilibria.

So what I wonder in relation to communities is - and here’s the research-relevant bit - what if generosity is a metastable state of online discourse? With snark being the stable state - the stronger equilibrium that generosity (by which I specifically mean - welcoming to newcomers) is likely to tip into as a community grows and creates its social glue.

If so this would have some interesting implications. One largely unexamined baseline assumption about communities among marketers is that they stabilise around generosity: with a certain amount of light moderation they are basically self-sustaining. But if generosity is metastable we’d expect its maintenance to require more and more effort as the community grows - with consequences for the cost and energy of running one.