Random Sampling

Here is an excellent post from Marginal Revolution on the perils of random sampling. Even when you think you've done it, you might have missed something.

The Wisdom of Whores

As a teenager Elizabeth Pisani discovered that she liked sneaking into the "girlie bars" of Hong Kong and talking to the hookers and the johns. Ordinarily one wouldn't expect such peculiar hobbies to pay off in a dazzling career. But beginning in the mid 1980s, people who were comfortable talking to hookers, rent boys, trannies, warias and junkies (all Pisani's terms) especially in the hot spots of Asia and Africa became scouts and spies in the war against AIDS. Information provided by these epidemiologists in the streets about where AIDS hid and how it spread has been critical to the war.

The Wisdom of Whores is Pisani's vivid account of what she has learned and why AIDS is still being fought ineffectively. I'll say more on this in the future. Here, however, is one bit relevant to the title:

We'd just finished our first survey of HIV, syphilis and risky sex among waria in Jakarta...I had the impression from the qualitative research...that waria were turning dozens of tricks a week, but the study showed they averaged only three. And since that figure came from 250 waria selected at random as the manual requires, it was certainly more accurate than the qualitative research...

"Three a week? Your insane!," snorted [sex worker and waria] Ines....Ines left school at fourteen but she is not stupid, so I explained how, through our probability sampling, we are asking a representative sample of Jakarta's waria. Ines dismissed by lecture with a wave of her manicure and a flick of her locks. "So clever, but so stupid." she sighed.

Ok, so what was the wisdom of the whore? (n.b. Pisani's term!) Bonus points if you can also describe a solution. I am going to have to get this into my econometric notes! Answer in the extension.

She explained, as if to an enthusiastic but slightly dim child, that a waria who is hanging around on a street corner to be interviewed by a research team is a waria who is not with a client. 'You are talking to all the dogs, obviously'.

Not something I learned in the lecture halls of London...but Ines is quite right. Our sample is biased towards the 'dogs,' who get picked up less than the cuter girls. So the study results underestimate the true number of clients per seller...

Ines's comments...prodded us into changing the sampling strategy...now we work with the powers-that-be (the mami, the pimps, the brothel owners) to arrange off-hours time for data collection. The principle....is that you are not cutting into people's work time, so there is less chance of talking only to the remnant sex workers who can't get a client.