When Vigilance Doesn’t Overcome Bias

Last week, my friend Dan posted a provocative article from The Atlantic. Adrienne LaFrance, a technology writer for the magazine, was worried that she was not representing enough women in her reporting. She first got worried in 2013, so she counted up how many of the people mentioned in her articles were women. It was only 25 percent! LaFrance resolved to do better, but when she audited her articles last year, women were only 22 percent of the mentions. After going through the data, LaFrance argues

Some people would argue that I’m simply reflecting reality in my work. That’s an overly generous interpretation. Another popular reaction is that my job as a journalist isn’t to actively seek out diverse sources, but to find the most qualified people to help me tell the best possible story. I only agree with that in part: Yes, my job is to serve readers by finding the best sources for my stories, but why assume that the best source isn’t a woman? By substantially underrepresenting an entire gender, I’m missing out on all kinds of viewpoints, ideas, and experiences that might otherwise sharpen and enhance my reporting.

Two things struck me from this article. First, LaFrance acknowledges a reality about day-to-day journalism that most readers overlook. When you keep going from one story to the next, it is awfully difficult to take a step back and analyze any kind of pattern in news. This is one of the main reasons I left journalism to go in to academia. Second, people love to overestimate the power that journalists have to shape their stories, even LaFrance herself:

I’m not excluding women on purpose, but I can’t say it’s an accident, either. Reporters choose whom to interview. We carefully parcel out our time as we work toward deadlines. I spent several weeks working on this story about self-driving cars, for instance, and it occurred to me as I was reporting that I hadn’t interviewed any women. In the end, deadline pressure and decisions about what to leave on the cutting-room floor trumped diversity.

LaFrance’s essay is depressing, because it gives the impression that none of her vigilance had an impact on the gender equity in her stories. My friend Dan, who has never been in a newsroom, thought it was shocking. As part of my dissertation, I examined whether phrases dealing with gender were used more or less often online than phrases on other topics from August 2008 – January 2009. I found mainstream reporters’ interest in gender as a topic was mainly a side effect of their interest in everything the presidential candidates had to say. (They had more of an inherent interest in phrases dealing with race.)

Reporters may not have that much control over the gender balance of the sources they quote. Most potential stories have a limited amount of sources who can give enough information to start the story. Someone who wanted to cover the current Republican presidential primaries would have a much larger gender imbalance than someone covering the Democratic primaries. It doesn’t matter how well-intentioned or vigilant the reporter is. The leading Republican candidates are all males. A reporter who gave Carly Fiorina 50 percent of the Republican coverage to fix a gender imbalance would have stood out as highly unusual.

When I covered politics, I chose my sources based on whether they knew things about new policies that could affect my readers. This meant I didn’t have many opportunities to choose sources based on their gender or race. The voters elected their representatives, and those officials chose which political operatives to hire. People who went in to activism made that decision before they shared their activism with reporters. Any reporter who chooses sources this way – maximizing access to factual information – will reflect the inequalities of the beat they cover. When I covered college admissions and affirmative action, I often needed non-officials to provide balance in my stories. Ward Connerly would step away from his day job to talk to any reporter about his crusade to end affirmative action. His job was to be my #1 most quoted source, and he had the financial resources to make his goal viable.

LaFrance’s argument would have been much clearer if she would have thought about the differences between selection bias and presentation bias. It is much easier to understand the problems LaFrance has with her reporting and think about solutions if we separate these issues. Let’s assume a science reporter rarely pays attention female scientists, but any story that actually gets written makes sure to make the scientist look brilliant and emphasizes the relevance of her research. In this hypothetical, the problem isn’t poor representation making female scientists look bad. Instead, we have a hypothetical where female scientists are less likely to be selected for news coverage, yet any female scientists who are selected will be represented with a positive bias.

People often get confused about the difference between selection bias and representation bias, so they talk about staying vigilant and diversity training as a solution to both problems. This type of training could be very useful in local television news, where Black criminal suspects are consistently portrayed differently than White suspects. But selection bias is a question of choosing from available sources. It is very difficult for reporters to go back in time and think of all their options for any particular story. It is almost impossible for readers to know who got left out.

I became an academic to study selection bias. While most of my peers focused on the final news story, I wanted to examine how many choices journalists have, and whether different journalists will tend to congregate on the same choices. I found that mainstream political journalists often do make similar choices, but it’s because they have relatively few good options to choose from. Selection bias is tied to broader issues of power, inequality and communications strategy. If you are selecting stories based on wanting to produce a certain kind of representation, that’s a different kind of media. Unfortunately, the audience that wants to read certain kinds of political representations overwhelmingly favors negativity and outrage. It’s an environment that makes Donald Trump’s campaign predictable for an analyst who can clearly separate selection bias from a crisis of representation.

Fixing a gender imbalance in mainstream reporting will require more than well intentioned reporters.


Postscript for academic readers: Academic journal articles are much longer than news articles. There is a wider range of sources who can contribute something – no one scholar holds all the cards like a politician can. Deadline scarcity isn’t really a factor. Put all these factors together and selection bias should be much less of an issue in academia. Therefore, vigilance against representation bias should be a more successful strategy in academia. This may also be why many academics have unrealistic expectations of journalists’ control over their narratives…


About Noah Grand

PhD in Sociology. I use statistics to predict news coverage. And home runs. View all posts by Noah Grand

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: