The Bulworth Effect And The Graveyards of Representativeness
(This is an expansion of a response to a post on the excellent Zebra Bites blog.)
In the film Bulworth, Warren Beatty plays a senator whose perspective is radically changed when he falls in love with a poor young black activist. One of the points of the film is that this is a utopian fantasy of social contact: the Bulworths of this world aren’t likely to form the ties that would sweep them into a world of social activism and shift their perspective. (I think it’s interesting, incidentally, that the film was released around the zenith of Web 1.0 hype - the idea that here was a cyberverse where we could be free from our embodied offline identities)
Now let’s think about market research and online communities. A key part of quantitative research practise is that the samples you research have to be representative of the population you’re researching. So, if you want to know about teenagers and pop music and you do your fieldwork in the junior common room at Eton, you may have a beautifully constructed survey but its results will tell you nothing useful.
When I get up at conferences to speak about online communities the question of representativeness rears its head pretty quickly. How can we be sure these things are representative? And the answer is “sorry, you can’t”. Or at least not using the standard ways of measuring it. Quantitative measure of representativeness assume that participants are discrete data points - i.e. that respondent B doesn’t get to see respondent A’s answer before they have a go.
But in an online community, the whole POINT is for respondent B to see respondent A’s answer and act accordingly. You’re not researching a sample, you’re researching a network.
In this context, representativeness isn’t just irrelevant, it can be actively harmful. If you recruit a representative sample to form your community, you may well be specifically provoking contact between individuals whose likelihood of coming into contact outside the community is very low. This is the Bulworth Effect - engineering contacts which produce radically unlikely outcomes.
And I’m now going to propose an actual hypothesis*: the representativeness of the links within an artificially constructed network is inversely proportional to the representativeness of the nodes within that network.
By notes I mean participants, and by links I mean relationships between the participants. “Artificially constructed” is there simply to indicate that if your research universe is already a network (like schoolboys at Eton, for instance), you won’t run into the Bulworth Effect.
I’m not aware of any way to measure the representativeness of links, so I’m pretty sure this is currently untestable. What it implies to me is that researchers wanting to mess with online communities have a number of unpleasant choices to make.
*smarter people than me have been thinking about this stuff, so let me know if this is an unintentional crib from someone else!
