Why aren’t more development indicators measured through rigorous, quantitative market research? Allegedly, because it’s very difficult. The Afrobarometer research project shows that it can be done: over 51,000 in-depth interviews across 34 countries since June 2011. (Of course, Afrobarometer can only work in countries which are not too scared of independent evaluation. You will look in vain for any data on Angola, Equatorial Guinea or Eritrea, for example)
The real reason for not using rigorous MR is that it is difficult to fix the results. Much better, an agreeable group of consultants using their own (fee-influenced) judgement or, better still, a group of PhDs with impenetrable algorithms tied to esoteric anthropological theories. Their reports can be massaged, edited, made unreadable or “clarified”; Afrobarometer shows what happens when you use quantitative surveys — people sometimes tell you that things are getting worse.
The survey’s latest findings are reported in The Guardian’s usual credulous style but were a real journalist to ask tough questions, the Afrobarometer methodology would stand up to scrutiny: a good sampling plan, sound interviewing methods, clear analysis methodology. (Actually, the main sections of The Guardian do ask tough questions — it’s just the heavily-subsidised development reporting that is so naïve and press-release driven)
The results of the Afrobarometer survey are clear: “lived” poverty remains pervasive across the continent”. That is to say, that when you ask citizens, they don’t think that their lives are getting any better. Official figures demonstrate progress for donors but a survey with a big sample and robust methods shows that most Africans see it differently. (The link is to a pdf of the Afrobarometer briefing on round 5 of their research — 12 pages of text but well worth your time)
It’s time to start using proper MR — not some confused academic derivative — to measure health performance, child survival and access to education.