Blackbeard Blog

140 Characters (London) Report

Sun, 29 Nov 2009 13:44:48 -0500

On Scribd - this is simply an easier-to-read compilation of my 11 posts on the conference the other week.

Climate Sceptics Seize On Leaked Emails

Tue, 24 Nov 2009 10:20:53 -0500

Climate Sceptics Seize On Leaked Emails:

Reading excerpts from these emails didn’t make me think of a vast conspiracy. It made me think of a market research debrief. A couple of times I’ve encountered the situation where you’ve done some research, and all the data is pointing towards a particular recommendation, except for a couple of glitches that don’t stack up with that interpretation.

Fair enough, you think, present everything, draw your conclusions, and the client will on balance ignore the glitches. But the problem is there’s someone on the client side who doesn’t like market research very much, and who doesn’t want to spend any money based on it, and has frankly already made up their mind that it’s a waste of time. And you know - you just know - that they will derail the entire debrief chivvying away at the one or two bits of data that don’t stack up, pretending it invalidates the rest.

So what do you do? You might present the data, you might look into it a bit more closely, you might frame it differently. You’d certainly have a conversation about what your tactic would be.

10. Being There

Mon, 23 Nov 2009 13:24:40 -0500

Certainly if there was one background note of the conference, it was authenticity: rarely defined, often present. For instance, the most useful service the X-Factor twitter staff provide, one mentioned, was sorting out the multitude of fake Twitter accounts that pop up for even the most transient of nine-day wonders. Chris Brogan put a lot of stress on trust, and one telling quote came from the music panel: “Where you used to have quotes from the NME on adverts, now you’ll have actual people’s opinions on a product” (emphasis mine!). Digital presence enriches digital information: one speaker talked about how users of Twitter quickly scan the page when it loads, looking first for avatars they know.

Andrew Keen’s point about the performance of authenticity doesn’t just mean faking authenticity – social media communities are reasonably adept at spotting that. It’s also about the presentation of some aspects of yourself as more authentic than others. Chris Brogan’s advice on how to present yourself on Twitter chimed nicely with this – put up a photo of yourself, make sure your background looks like this not like that, and so on. It might make sense on an individual basis but in my opinion it reduces diversity – the rows of well-scrubbed, smiling, best-face-forward pictures on a Twitterstream reinforcing a sort of picket fences mentality.

But for now the pendulum seems to be swinging against creative fakery – though Betty Draper cropped up a couple of times as an example, the media panel generally dismissed fictional twitter presences as pointless: Twitter is currently more about exposing the machinery (via performers and backstage crew tweeting) than enhancing a mystique. Kevin Slavin went further, predicting that the day of marketers personifying their brands through stories and characters is coming to an end – instead the ‘face’ of brands in the realtime media world will be inhuman: local active systems that build non-narrative presence, like a Taco Bell Truck tweeting its journeys around an area.

Meanwhile the idea of digital presence is something we’re increasingly born to. One of the day’s most interesting comments came from an audience question in the education panel – mentioning that he knew of 5 and 6 year olds who were tweeting. The reaction to this from the media might be concern – where are the safeguards? – but the questioner was more positive: Tweeting for these kids is learning to write for a purpose and an audience, not as an exercise. It was one of the moments in a rambling, exasperating, fascinating conference that cut through the rhetoric to provide a bit of real optimism.

(And that’s the end of my conference write-up! I will put the whole thing up on scribd or somewhere tomorrow)

9. Disintermediation

Mon, 23 Nov 2009 12:56:00 -0500

The risk of confrontation posed by hashtags sits at odds with another great theme of the conference – disintermediation. Stephen Fry, in his keynote, was the first to raise this – his comments became the lead story for most reports on the event. Fry’s point is that Twitter allows celebrities to break away from the journalist/interview cycle. They can communicate directly to their millions of followers, and the journalists’ difficult or intrusive or mocking line of questions can go hang.

Fry presented this as a democratising force. Andrew Keen, presenting directly after, was far less sure. The “flattening of power” that Fry had presented is an illusion according to Keen. Power is never flattened, it simply shifts, and now it’s shifted from the organisation to the individual. Fry is simply giving a new performance of power.

This shift is best illustrated, I think, by thinking about the difference between corporate brands and ‘personal brands’. The corporate brand entering social media is urged to give up control, to surrender some of its autonomy. But Twitter’s most popular users – its A-Listers, the celebrities – are using it to regain a level of control over their presentation and perception, through disintermediation. The celebrities themselves are happy with this – so are their core fans, who get to feel closer than ever to their stars. Its the uncommitted, the casually interested, the non-fans who lose out – they might have preferred interviews in which journalists got some chances to be prickly.

There’s not much harm in it, even so, if the disintermediation is happening at the level of the celebrity PR puff-piece. But it goes further than that. At the Police Who Tweet panel, one member pointed out that the collapse of local news has hit police reporting – it used to be they could take a local journalist with them on, for instance, a drug raid, but there are far fewer reporters to spare. No matter, thanks to social media the police can put up their own video of the raid on YouTube or Facebook. This was particularly beneficial, said the policeman, because the journalists might have reported the police action in a “negative manner”. Disintermediation at work again!

If power shifts to the individual, says Keen, and authenticity becomes something power is arranged around, then the risk is that will lead to more charlatanry not less, as the key is simply to perform authenticity well. Individual digital power is also highly unequal – on the telecoms panel, talking about how these large, highly regulated companies could become more agile, one member talked about profiling subscribers based on their online influence: the unspoken implication being that the more influential you are, the quicker and better your service will be.

8. A Quick Digression On Hashtags

Mon, 23 Nov 2009 11:45:00 -0500

Twitter of course has evolved a mechanism for creating a breach in your social graph – the hashtag. Hashtags were in one sense a dominant presence at the conference, the #140conf tag creating a wall around the discussion and an impromptu community (who might or might not be physically present). But there was little explicit talk about them.

Boyd Hilton, of HEAT magazine, described how the hashtag-powered backchannel for the edition of BBC’s Question Time featuring neo-fascist Nick Griffin felt “empowering” – 99.9% of the posts were against Griffin. It “made the experience more real”, in part I’m guessing because of this sensation of instant, righteous consensus – which Griffin would probably spin as ‘mob mentality’.

One young activist on the e-democracy panel, on the other hand, mentioned the #saysorrybrown hashtag, started by a conservative blogger and listing things the British PM should apparently apologise for. He held this up as a ‘bad’ use of Twitter, which seemed debatable to me even though I didn’t agree with the hashtag’s sentiment. Topical tags like #welovetheNHS have been popular before and will be again – if you don’t like the content the tag is up there for you to counter or mock.

This idea of hashtags as a confrontational space within the Twitter network was explored by Kyra Gaunt in her talk on “racism as a resource” – frustratingly cut short by technical difficulties. One of the things she focused on was the controversy around hashtags created by and largely used by Twitter’s black users – her example was the #thatsAfrican one. But anyone can step into the hashtag stream, and these tags have been plagued by racist content. This illustrates the dual nature of the hashtag, probably the most fascinating part of Twitter culture: they are the mechanism for building impromptu communities, but they’re also rifts in the twitter stream continuum of anyone using them, portals that can lead to co-operation or confrontation.

7. The Graph Is The World

Mon, 23 Nov 2009 11:03:51 -0500

At the root of the Breakfast Barrier is the great problem of Twitter (and by extension other social tools) – there is no such thing as a typical Twitter experience, because one’s experience of it is so determined by one’s social graph: the followers, the followed, and how many of each there are. But people talking about Twitter often talk about their benefits and their experiences as if they were universal benefits.

One speaker, for instance, talked about the benefits of Twitter in terms of the three As – alerts, advice, and assistance. But to get useful advice you need a certain number of followers, and to get useful real-world assistance you need a certain number more: the example he gave was leaning on the Irish government to give him a Visa, hardly something open to Joe Fifty-Friends!

This is an understandable effect – what makes the Twitter experience so seductive (and here I’m generalising from my activities) is that it gives you the impression of a multitude rushing past around you while actually limiting your experience to a small and self-selected part of it. The graph becomes the world. Even a piece of advice as sensible and generous-minded as Chris Brogan’s suggestion to promote other people’s content 12 times as much as your own carries the limitations of personal experience: Brogan’s followership is so large that he need never actually promote his own content at all!

6. Them And Us: The Breakfast Barrier

Mon, 23 Nov 2009 08:38:57 -0500

There is still, in other words, a Breakfast Barrier for Twitter. As in, “It’s just people talking about what they had for breakfast.” This meme – mentioned a few times during the conference - has a double purpose. It’s a shorthand way for the media to dismiss Twitter users as trivial. But it’s also a shorthand way to dismiss the media’s criticisms of the service and site, lumping more nuanced views into a big silly mess of not-getting-it-ness.

(The idea itself found its best answer in Stephen Fry’s speech, where he pointed out the similarity between the knowing triviality of “Twitter” as a name and the deceptive frivolity of the 18^th century press – The Idler, The Spectator, The Tatler. You can quite imagine a bewigged Lord Carter-Ruck scoffing at them as purest trivia – until they bit him.)

But this them and us mentality can lead to a certain utopianism on one side as easily as a dismissive technophobia on the other. Jeff Pulver, in his introduction, suggested that politicians who use Twitter then drop it wouldn’t get re-elected: this seems to me an overclaim. The Breakfast Barrier also acts as a mode of exclusion – perhaps because of companies’ best efforts in using it. One speaker on the small business panel praised BT’s BT Care program as an example of great customer service – an angry tweet dealt with in 30 seconds. But if Twitter is a way to turbocharge your customer service, what happens to the 80-90% who aren’t on it? Does their lack of social media savvy mean they don’t deserve that superior treatment?

Of course reaching around the breakfast barrier can be very rewarding: for the police, a Facebook presence has made a difference in contacting hard-to-reach audiences – the kind of people, especially young people, who don’t usually come forward to them. But again, the problem of scale looms: if everyone was on Twitter, for instance, would the kind of service BT Care offers still be practical?According to another recent survey, over half of social media users now expect brands to intervene after an angry mention – the Candyman Effect (say their name and they appear) which poses a serious problem for overstretched customer service departments.

5. Realtime's Growing Pains

Mon, 23 Nov 2009 07:55:45 -0500

(Continuing my series of posts on the 140 Characters Conference)

According to one speaker on the communications panel, the future of realtime – represented by Google Wave – is the end of conversation, replaced by a kind of type-telepathy, a flux of perpetually interrupted and always provisional collaborative thoughts. But Google Wave, its complexity and its stumbling launch also illustrate a wider realtime issue – how do you make it scale?

It was 3 and a half hours into the first 4-hour session before a speaker – from The Creative Store – became the first to hint that with a lot of the current realtime architecture we simply don’t know yet. Twitter, for instance, isn’t representative. It’s not a mass medium. Maybe 20% of people have tried it, in its core territories. According to a survey that came out the day after the conference, three-quarters of Brits don’t even want to try it. In other words, it’s still very much a creature of the early adopters, and its culture reflects this.

The point here isn’t to say that Twitter is a fad – it may be, it may not be, but the real time information streams it represents are something more lasting (as a lot of speakers stressed, it’s not about the tool). The point is that what happens to the use and culture of tools and technology after mass adoption is often very different to what happens to them before.

A lot of the talk at 140 Characters seemed to imagine that the expansion of Twitter, or other social media tools, was largely a matter of people “getting it” or not. But it’s equally a matter of the tools “getting” what a wider audience wants from them. At some point “not getting it” becomes a problem for the tool, not the people. Mass adoption will change these services in ways we aren’t trying to imagine, except through rose tinted lenses where everyone who uses them is Just Like Us.

As someone on the educationalists’ panel put it – to the day’s only boos! – the problem with Twitter is Twitter. It’s a proprietary service, not a universal protocol: “it needs to be more like email”. Meanwhile there was plenty of evidence of barriers to adoption: between a quarter and a third of higher and further education institutions in the UK still block Twitter access, and 70% of schools do. On the Police Who Tweet panel, one panellist admitted that 18 months ago they couldn’t look for evidence on Facebook – because it had been blocked. The problem of people internally who don’t “get it” arose throughout the day. Realtime social media is at an awkward stage in its development – no longer a novelty for its users, still a mysterious continent for doubters.

4. Meaning In The Stream

Fri, 20 Nov 2009 10:50:34 -0500

Issues of immediacy and agility were in the background whenever the talks turned to using the realtime web as an information source. The Venture Capitalist panel pointed out that extracting meaning from realtime flows was still very hard – a real unresolved problem for real time search. Text and sentiment analysis, meme tracking, social graph searching – these all went some of the way to cracking the issue but not currently far enough. In the future, said conference organiser Jeff Pulver, hedge fund “black boxes” might use social media as a source of arbitrage – exploiting the gap between information being created and confirmed.

Which wasn’t to say that people aren’t using realtime information already. Environment blogger Vikki Chowney turned the issue on its head by talking about how live blogging had become the only way she could deal with the sheer amount of information coming through to her at international conferences like the G20. By getting the information out there and leaving synthesis and analysis to the crowd, she’d been able to manage the overload without trying to filter it.

On the media panel, meanwhile, website publishers talked about how social media and Twitter let them pre-empt and predict interest in a story, quickly re-prioritising content to meet audience needs. The crucial role realtime media plays for TV or Radio is as a backchannel of ongoing, immediate response, and Endemol discussed plans to create apps to structure and legitimise those backchannels (phone companies seem to have similar plans).

One thing the backchannels can do is restore some power to the TV networks and schedulers – for much-discussed shows they have brought back “appointment TV”, since the experience of watching and talking about a show within the impromptu realtime community is lost if you choose to watch it later. (Not to mention the fact that on Twitter, policing spoilers is impossible!)

And still the information keeps coming. The most stimulating talk of the day, Kevin Slavin of area/code on “Things That Tweet”, talked about how “sensor aesthetics are streamy”. In other words, data from objects, be they Tower Bridge, your fridge or the Earth itself via seismic data, naturally works best as a real-time information flow. The problem is identifying who needs this information and making sure they know how to get it.

3. The Thing Is The Idea

Fri, 20 Nov 2009 10:13:01 -0500

A glimpse at the more sustained changes realtime is ringing came from the panel of Venture Capitalists. Five years ago, they said, if someone had a great idea it might be funded. Now? No chance, unless there’s something already launched. In the realtime web, the important thing is presence – doing stuff, getting ideas launched as soon as possible and finessing them later.

This is reflected elsewhere in the culture. On the music panel a DJ and promoter talked about how there’s been a shift from the idea of the finished album – dead in 5 to 10 years, he claimed – to “dripfeeding”, a stream of activity (recorded music, shows, interviews, stuff) keeping the brand-cum-band alive in people’s minds.

This is the realtime web being used as a key to agility. The best business advice of the day didn’t come from a business speaker at all, but from the “e-democracy” panel. Because tools are rapidly changing and potentially unrepresentative, it’s very unwise to get lured into a focus on one particular service (like Twitter). Instead the strategy should be lightweight but rapid – engage quickly but don’t commit a lot of budget. Don’t get hung up on process, don’t put too much stock in mistakes or successes.

2. The Process Is The Story

Fri, 20 Nov 2009 09:39:11 -0500

Some of the case studies on offer were more interesting. Kodak, who sponsored the whole event and have put together a free guide to social media practise, talked about how they’d taken a product idea straight from Twitter – implementing specific suggestions, like flexible USB ports and mic jacks, and then crowdsourcing a name. This kind of thing is becoming more common – taking design and useability improvements straight from the user’s mouth online.

You might argue that they’d have got the insights and information anyway, but that’s not the point – the process here is the story. It’s like the three young filmmakers who got up to tell us about the crowd-funded film they’re making of a Jules Verne novel. You’re not buying a good film, or even the expectation of a good film. You’re buying the experience and warm feeling of participating in something crowd-y.

That’s not to say the film – or the camera or the Axe pick-up tips Twitter – won’t end up being good. It might be magnificent! But we’re still in talking-dog territory here, where the fact of socialness matters more than the outcome. This won’t last forever, of course. It probably won’t last out 2010.

1. Social Landfill

Fri, 20 Nov 2009 07:24:00 -0500

I’ll start with the marketing stuff, because I work in that field, though in some ways it seemed the most sluggish of the various criss-crossing streams of thought. There was an awful lot of familiar stuff being trotted out – people have unprecedented power to influence brands, you have to join in the conversation, the customer does the marketing for you, you need to give people something they can’t get elsewhere… nothing that wasn’t on some level true, you understand, but also no longer inspiring by itself.

Especially when the first concrete example was Unilever speculating that maybe its Axe brand should start a Twitter account with a daily tip on how to pick up women. Later on a rep from Warner Music, after explaining that none of the majors were really experts in the digital space, talked enthusiastically about how they’d got Madonna onto trending topics with some hashtag or other when her Greatest Hits record was out. As my notes said: “Oh joy.”

So here’s the problem – I’m coming at this conference as a researcher, as someone not unsympathetic to marketing, but also as a user of Twitter, someone who enjoys its ecosystem for the info it feeds me. And as a researcher I appreciate that one of the demands of social media on brands is a shift away from creating advertising and towards creating content and experience.

But as a user I know some of that is junk content and junk experiences and I hope we’re now at a stage where we can call that out. Something like the Axe Twitter idea is a bit uninspired but would only affect anyone following it – hashtag spamming, on the other hand, deliberately drives attention away from something potentially more important or interesting or just funny.

0. On The Disparate Edge Of Now

Fri, 20 Nov 2009 07:17:00 -0500

I went to the 140 Characters conference on Tuesday and took a lot of notes, expecting to write a quick summary. Once I started on it last night, the quick summary turned into a 3,500 word monster, so I’ve split it into 10 parts, plus this intro, and will be posting them over the course of today.

The 140 Characters conference – tagline: “The State of NOW” - is an exhausting but rich experience. With 50+ speakers in a single day the scheduling is a deliberate jumble, refusing to coalesce into a marketing event, or a cool stuff event, or a tech or theory one. There are very few audience questions, the turnover is rapid, there are no real breaks apart from an hour for lunch. So the vibe is very much that of the firehose, the real-time brain-pummeling info-stream: not interested in Twittering taco trucks? There’ll be Facebook-wielding policemen along in a minute.

The EasyConference style of 140 fits this to some degree – no coffee, no lunch, not much legroom - you’re here for the INFO, dammit, not to schmooze. That’s how they want to play it and fair enough – the lack of WiFi was a no-frills touch too far though.

A paper-by-paper summary of the 25 or so speakers and panels would be ridiculous, and you can see most of it on video around the web anyway: this is an attempt to pull together the day into some kind of narrative, or at least pick out important themes, dancing about from panel to panel. It’s a mix of what people actually said, the implications as I understood them at the time, and further thoughts of mine post-conference.

The Future Of Research: 10 Odd Ideas

Tue, 10 Nov 2009 10:15:00 -0500

Here are ten things I think might happen in and to market research before too long. As usual, I’m chasing ideas that interest me. They aren’t supposed to be sober predictions. They are not representative of the views of my employer, they are not wholly representative of the views of ME, and some of them contradict one another. Here, in other words, be dragons. I hope you like them.

1. Respondents, RIP: With enough behavioural data and the right algorithms to analyse it, there is nothing we cannot know about people. This statement is one (extreme) conclusion of a tendency to move away from self-reported data and claimed attitudes, towards a kind of post-respondent research, in which information is crunched out of the datamass with never a question asked. What matters isn’t whether the statement is right but that powerful people are acting and investing as if it’s right. The giants of the internet – Google in particular – seem to have a near religious belief in the power of accumulated data to drive decisions.

2. Social Meteorology: What then of social media? We’re already seeing the metaphor appearing of social networks and communities as a kind of weather system of human attitudes and feeling. Studying this highly complex weather system (and mapping it, since social-local doesn’t map to geographic local) – with the ultimate aim of learning to navigate and predict it – will be a preoccupation of marketers, researchers, academics and quite possibly politicians in the 10s.

3. Dawn of the replicants: I’ve written on Research’s site about experiments creating demographic twitter bots – replicant individuals who retweet content along particular demographic, attitudinal or interest-based lines. John Griffiths calls them “furbies” for clients – the target consumer digitally manifested and personified, like a 21st century pen-portrait. And as the technology advances, the ability of replicants to predict or typify groups of people will only increase.

4. Data brokers: What about flesh-and-blood respondents (assuming they’re needed at all)? Also in Research I’ve predicted that participants – particularly in-demand or hard-to-reach ones – will have a growing awareness of the value of their data and opinions, and that the incentivisation system designed to reward them for these may be broken. Why shouldn’t they bargain – collectively or individually – for better incentives, or sign up to new kinds of panel which can act as their information brokers: a tilting of the research bargain back in the participant’s favour.

5. Business class research: For the real high rollers and super-hard-to-reach participants, incentives aren’t enough: you need to retool the whole research experience to make it as seductive and delightful as possible – a kind of first- or business-class research to set against the economy experience we usually offer. Whether this is a bespoke, beautifully designed and smooth online questionnaire or a luxurious one-on-one interview, there will be some participants who are definitely more equal than others.

6. Spontaneous surveys: The rest of us might find ourselves faced with authorless surveys. If we can be served ads based on our searches and interests, why not questions? Researchers will be thinking in terms of short, highly modular questions anyway – release those as streams of individual attitudinal questions and let Google serve them individually. Eventually it might auto-generate its own questions and let you buy “information terms” like you buy ad words.

7. Goodbye community, hello swarm: “Community” may be the hottest word in research right now but in terms of the wider web communities are a staging post – a destination-based solution to the problem of group formation. The direction of the net is towards flows of information, not destinations, and the temporary, ad hoc swarms of interest around a Twitter hashtag or Facebook group are nimbler and less demanding for participants than community membership. Future research communities will be task or goal-oriented swarms, not carefully-built MROCs

8. Peak crowd: Just as crowdsourcing is becoming wholly mainstream as a creative tool, the spectre emerges of “peak crowd” – the point at which the effort demanded by all the creative competitions, open innovation platforms, brand communities, crowdsourcing opportunities etc. out there exceeds the creative ability of those who could give a monkey’s. The result, hopefully, will be a big flushing out of limp activities and lame techniques.

9. Play Power: So the lesson is, amuse people. To get all Herd-y, bring people together and give them something interesting to do. Actually interesting. Researchers are going to be learning a lot more from game designers, who are not only good at building stuff people want to do (individually or together), they can get those people to actually pay for it.

10. Social sabotage: At the moment we’re in a happy era where in the excitement of getting our hands on social tools people actually believe they can trust what other people say using them. Information gleaned via social media is natural, authentic, the true voice of the customer, etc. How we will look back and laugh. We’re already seeing people gaming review systems – the iPhone App Store, for instance – and we’ll see a lot more social sabotage: from anti-consumer groups spreading misinformation, brands seeding social media with breadcrumb trails of false insight, refund-chasing customer complaints, and simple trolling.

I would love to hear some of YOUR curious ideas for what’s going to happen.

Confessions Of A Moderator

Thu, 29 Oct 2009 09:06:49 -0400

Confessions Of A Moderator:

This is the paper I wrote for the MRS Research 2008 conference, about my experiences moderating an online community (ILX), now uploaded to scribd.com*. Given that it was the first time I’d ever written anything for a conference it did rather well: nominated for Best Paper and Best Newcomer, and winner of Best Presentation.

I read it again last night before uploading it to see how well it stands up. Given that I wrote it before I had any experience of actual research communities, I was expecting to cringe at a few bits. But quite frankly I think it stands up gratifyingly well. Obviously, it’s extrapolating from personal experience of ‘wild communities’, and the industry practise surrounding MROCs (horrid word) means that some of my observations aren’t relevant for those. But the differences themselves lead to unintended consequences. Expect an updated look at the topic on this blog in the future!

*you have to sign up to scribd to see it, but it’s a painless process - I signed up ages ago, had forgotten I did so, and have never had a moment’s bother from them.

Black Hat Research

Wed, 28 Oct 2009 20:00:26 -0400

Some phrases that jumped out at me from the #esoc (ESOMAR Online Research Conference) tweetstream:

“a tsunami of unstructured data”

“no mentions of representativeness, why not? Fit for purpose, not for rep.”

“ticks on the back of the information hippo”

“Just listen”

OK, I admit it, the hippo one was mine. But this was a theme of the online conference - data, data, everywhere, data that must be listened to. It must also conform to certain standards of quality, at least when we create it ourselves via surveys, but the definition of quality is subordinate to questions of usefulness - did the information help its buyer make a better decision?

Nothing wrong with that! But at conferences it’s also worth listening for the dogs which don’t bark - in this case privacy, ethics, consent, that kind of stuff. These have been growing topics at UK conferences, it surprised me they weren’t more under discussion here. At one point a guy from Facebook explained that no, they weren’t planning on opening the platform up to research too quickly because the user and her privacy came first. Given how keen Facebook are on advertisers this may not be as high-minded as it sounded, but even so the statement was a rebuke: you lot, he was implying, are not an industry that respects your participants terribly much.

Anyhow, it seems to me we have a nice little research storm brewing. First of all we have an increased pragmatism: as the industry has become more and more client-centric, we’ve moved to a point where the value of the information we sell lies in its ability to help a client make good decisions. It always has been, of course, but the information could be rated another way - how valid was it? It’s all very well saying “research was never representative”, but you don’t get off that easy - representativity might not have been achievable, but it was the ideal to strive for as best you could as you looked for validity. “Fit for purpose” as a test of validity is a big shift towards, essentially, deregulation.

Now, I love social media, I think it’s stuffed with valuable information, and in a sense deregulation suits me just fine. But the industry isn’t actually embracing ‘deregulation’ as such - quite the opposite: it’s bristling with new definitions, taxonomies, codes. And this is the second part of the storm: codification - because the landscape of research is in flux, industry bodies are trying to update their rules and principles, often for the very good reason that they want to stay out of any potential legislative trouble. After all, marketers are increasingly a target for government, why wouldn’t researchers be too?

And the third part of the storm is the sheer abundance of information that now exists about people. People are sharing more information but they’re also leaking more information all the time - not to research companies, but it’s out there and a lot of the time it’s public. This element WAS talked about at the conference but generally in the sense of it being a happy opportunity.

So what we have is a collision of pragmatism and codification in a context of abundance. And I’m saying what that’s going to lead to - it is already - is black hat research. It won’t be called that of course, it probably won’t even refer to itself as research and it won’t be at many of our conferences.

What is it? Well, you have the information gathering which happens within industry codes and conforms to industry quality standards. And you have the law which lays out what information you can actually legally gather but which is full of loopholes and grey areas and stuff. And black hat research is what happens between the codes and the law. Scraping and mashing up personal data, rogue PR surveying, push polling, communities that mix research with marketing… you can add your own, I’m sure. Black hat research is going to get bigger and bigger and more important, a shadowland of information and intelligence provision which passes the “fit for purpose” test but maybe not other ones we might set it.

abbyjean: how you guys doin’ today? Interesting in that...

Tue, 20 Oct 2009 11:43:52 -0400

abbyjean:

how you guys doin’ today?

Interesting in that it’s a 6 point definition-based scale which has been turned into an 11 point scale to allow respondents to faff about: “well, I wouldn’t say it was mild, but it’s not interfering with tasks I guess….” - this is good practise I reckon.

Obviously I personally reach “bedrest required” at about pain factor 2.

Call For Opinions

Tue, 20 Oct 2009 11:28:07 -0400

When Alison Macleod of The Human Element blog posted her plea for market researchers to be more opinionated, I took it as something of a challenge. So here, quite unsupported by anything other than grumpiness and prejudice, are some of my research- (and social media) related opinions.

1. “Insights” aren’t zen koans. If you can express something that briefly, it’s probably banal.

2. Between “data” and “recommendation” comes a little thing called “argument” which we neglect rather too readily.

3. Making two different pieces of information talk to one another is the most important skill a researcher (or almost anyone) can have.

4. We may not be as good as we ought to be at interpreting information, but by god we’re better at it than 90% of the people who end up blogging that information, so we have to get the message right from the start.

5. We are really bad at making celebrities out of our great practitioners: where’s the research Rory Sutherland? Why isn’t she or he at TED?

6. An online community is a factory for unintended consequences, and most of the people using them don’t understand how they work, let alone how to analyse them well. (I am not saying I understand how they work either.)

7. I know some very bright people who do semiotic work but on stage or in case study format it almost always ends up looking like a qual black box, one set up to produce undergraduate cultural studies essays.

8. I am always scrupulously polite about neuromarketing but if you were to WIRE UP MY BRAIN at a convention you’d be able to tell my true feelings. Or I might just be thinking of how nasty the canapes were.

9. The engine powering social media isn’t “influence”, it’s favouritism. Most talk of ‘trust’ is a way of justifying cronyism. This is not always bad, still less is it avoidable, but it’s not a brave new social arrangement either.

10. Hypocritical this in the light of much of the above, but the very worst thing about market research is its unending tendency to flagellate itself and envy people who are a great deal less informed than it is.

I’m not saying these are controversial, or consistent, and certainly I bet they’re not all correct, but there you are. Opine away on your own blogs!

Goodhart's Law and Market Research

Mon, 19 Oct 2009 08:05:45 -0400

This weekend I came across Goodhart’s Law. Like a lot of “laws” it’s more Goodhart’s Observed Tendency: it basically says that when you start basing policy around an economic indicator, the information value of that indicator falls to zero. It’s a kind of decision-making equivalent of the observer effect, as seen in physics. Or, if you like, a fancy way of saying “a watched pot never boils”.

Now in its original form - talking about government policy - it’s a sly way of asserting that government intervention is useless and so economic agents should be left to their own devices. But I suspect Goodhart’s Law - or something like it - also applies at every other level of measurement activity, including the firms the majority of said agents operate in.

How might this actually work? We design metrics to simplify complex systems. But when a value designed to describe a system becomes a way of assessing success within that system, two things can happen:

- the system adapts to reflect (and therefore game) the metric

- the stuff not reflected in the metric goes unnoticed, becoming a big breeding ground for potential unintended consequences.

And the more important the metric is, the more it gets gamed.

Anyone who’s spent any time working with social media - looking at “buzz”, reputation systems, measuring “influence” etc. - won’t find these ideas particularly foreign: I wasn’t surprised to find good blog posts from a couple of years ago talking about it. You can see it happen on Twitter, say, with “follower counts”.

Over in the world of market research, though, these ideas aren’t quite as commonly expressed and I suspect many wouldn’t agree with them. In this business an aphorism I’ve heard quite a lot is “if you can’t measure it, you can’t manage it”. What Goodhart’s Law is implying is the inverse: “once you measure it, it manages you”

Let’s assume Goodhart’s Law is at least poking at a greater truth about the unreliability of metrics to assess complex systems. What can decision-makers do about it?

1. Abandon the metrics.

2. Double down on Goodhart’s Law and increase the importance of a metric until removal of the metric would crash the system.

3. Increase the opacity of the metric so gaming it becomes significantly more difficult.

All of these, of course, have their risks :)

Snark and Stability

Tue, 13 Oct 2009 19:13:50 -0400

Nitsuh Abebe is one of the smartest guys on the Internet (my patch thereof). In this post he offers a defense - or at least the best explanation I’ve seen - of how snark became a default mode of online discourse.

This bit is really crucial I think (emphasis mine):

Flippancy is more fun. The work of reaching out and explaining things is potentially dull and time-wasting; it’s just plain funnier and more exciting and more gratifying to be on the inside of shared assumptions. (We like talking to friends, not strangers.) The histories of a lot of message boards and comments boxes can be traced out along these lines: they begin with a few people earnestly explaining themselves to one another, finding common assumptions and common ground and welcoming newcomers; then they grow, and their shared assumptions solidify, and they get flip and concise and referential and giggle at newcomers who stumble in and Have to Ask.

This is exactly right, and is what I was getting at in my conference paper a couple of years ago when I talked about the shift from “content motivation” to “social motivation” within communities (of course, Nitsuh and I are thinking primarily of the same community, so beware!).

At an individual level community is all about having interesting conversations and meeting new people and suchlike. But at a group level all those conversations and interactions create a complex system out of which emerges social convention and shared knowledge and the perhaps unpleasant glue of snark. And this happens pretty much beyond the level of one or two individuals to change (well, OK, I’m not sure I believe that, for reasons I’ll come on to).

Anyway, occasionally on social media blogs you get someone saying “LOL Twitter is all about what people had for lunch” and someone else then says “No LOL @YOU because we really need this phatic stuff and it’s good for us.” And snark is kind of like next level phatic communication - we’ve invented something which combines the two crucial primate group activities of picking shit out of each others fur AND chasing other apes off the territory. Good for us! (Seriously!)

You can tell I’ve been reading Herd recently, I’m sure.

Which book also reminded me of Philip Ball’s terrific Critical Mass, and the stuff he writes there about metastable states and their applicability to social behaviour. An example of a metastable state - forgive my rubbish layman’s explanation! - is when water stays liquid well below freezing point, until it gets disturbed, and collapses into its stable state (ice) all at once. What I took from Critical Mass is an appreciation not necessarily for the science of metastability but for the concept as a wonderful metaphor for fragile equilibria.

So what I wonder in relation to communities is - and here’s the research-relevant bit - what if generosity is a metastable state of online discourse? With snark being the stable state - the stronger equilibrium that generosity (by which I specifically mean - welcoming to newcomers) is likely to tip into as a community grows and creates its social glue.

If so this would have some interesting implications. One largely unexamined baseline assumption about communities among marketers is that they stabilise around generosity: with a certain amount of light moderation they are basically self-sustaining. But if generosity is metastable we’d expect its maintenance to require more and more effort as the community grows - with consequences for the cost and energy of running one.