Tuesday 23 July 2013

Headlines in (Social) Science: Gender, Politics, and Unreviewed Findings

Often when I see a science story in the news the first thing I do is look up the related research article. That way I can see what was actually done, and evaluate, if not the detailed methods, at least the overall scientific logic of the article. If it's particularly new or controversial I sometimes bookmark the article so I can come back later and see what else has been published in response.

Of course, this requires that there be an article to look at. Since I'm at a university, paywalls aren't a problem, but even my university library subscription doesn't get me access to articles that haven't been published yet. Or even accepted. Which brings us to the story at the centre of today's post: a study funded by the Economic and Social Research Council, a British government funding agency, authored primarily by James Curran, director of the Goldsmiths Leverhulme Media Research Centre at the University of London.

According to various headlines, this study showed that:

Across the world, women know less about politics than men 

Women know less about politics than men, study finds (that goes for Canada, too) 

Women, especially in Canada, are more ignorant of politics and current affairs than men, says UK research

Study: Women Know Less About Politics Than Men

Did the study actually show this? To answer that we need to take a close look at the details of the research. As part of a CBC radio interview here, University of Calgary prof Melanee Thomas points out that there are a lot of ways these types of studies can be misleading. She brings up the important point that there can be biases in what is defined as political knowledge. Often "politics" is restricted in definition to so-called hard news: trade, military issues and conflicts, economic and budget issues, etc. Which these are certainly important, the list often leaves out other issues that are inarguably political: health, education, and all manner of issues of local governance. It's hard to argue that health policy isn't a "political" issue, but it is often labelled "soft" news and stuck with the other "human interest" stories. (As an aside, "human interest" is an interesting term: exactly why should I care about anything that's not of interest to humans?)

The point is that the study may be biasing its result by asking questions in the domains that men, on average, know more about, and ignoring domains that are, by any objective definition, equally political and which women know more about.

Does this study fall into those pitfalls? We don't know, and neither does Melanee Thomas, because the study hasn't been released yet. It's not listed on James Curran's research website, and the ESRC site lists the work as "submitted". As anyone in academia knows, a lot can happen between "submitted" and "published".

There's a couple of things to address here. First, we have the headline magnification I've talked about before. Curran holds a press conference in which he summarizes his work; a journalist takes that work and turns it into an article; an editor takes that article and turns it into a headline. Based on other cases of this headline magnification, these levels may have resulted in headlines that bear little resemblance to what was actually shown.

But, and this is the second point, we don't know what was actually shown, because the research is unpublished. And this is where James Curran has, in my opinion, acted reprehensibly. He has used his position as an expert to promote a conclusion without allowing the underlying work to be scrutinized. It will, obviously, be scrutinized eventually, but by the time that happens the original news stories will months in the past--an eternity in the online news world. No one is going to prominently display a story that adds context and corrections to a relatively minor headline from a few months ago. So prof Curran is putting out a conclusion that can't be verified, but that adds to a narrative that has women as intrinsically less able than men in certain key areas.

I don't think that research should be subordinated to social mores or political considerations. What I'm arguing is that researchers have a responsibility to ensure that, within the best of their ability, their research is reported correctly and in context. Especially when it has the potential to further harm already marginalized groups. We've seen the harm that can be done with headlines such as "Vaccines cause Autism." The harm from "Women know less than men about politics" may not be as obvious, but that doesn't mean it's not there. For that reason, Curran had a responsibility to give as nuanced and context-filled report as he could, and to allow other researchers the chance to dispute his findings and provide their own insights. He did none of that.

No comments:

Post a Comment