Category Archives: Media Bias

Media Bias, Trump Voters, and Apples vs. Oranges

When I was in graduate school, I think the most difficult concept for any scholar to measure was media bias. How do we know one story is biased while another one is not? Bias is a great and easy to use concept if we are talking about me shooting a bow and arrow as part of summer camp as a kid. There’s an objective target out there. If my shot keeps missing two inches to the left of where I aim, I can try to correct the physical errors in my shot by aiming two inches to the right of the bull’s-eye.

Objective journalists strive for the bull’s-eye every time too. However, there is no stable, fixed target for journalists to shoot at. Writing news stories involves a lot of judgment calls. Is this source credible? How do I portray them? Reasonable people can disagree about where the target should be. This means bias is an inherently relative concept when we are talking about the media. It’s based on our frame of reference. As I wrote years ago, audiences may want something different than news organizations.

Measuring media bias is awfully difficult because the target isn’t fixed. We have to choose a reasonable target – some standard of objective journalism – before we could measure how far off actual news content is. Choosing the target is almost inevitably a value-laden judgment. Even “objective” targets lead to value-laden interpretations of the results. For example, let’s say we want to compare news organizations to members of Congress. It takes a lot of work to going through the Congressional Record to build the target. All the work may not be worth it. Congress isn’t a truly “objective” target because it has a majority party. A centrist newspaper would be to the left of Congress today, even if it is closer to the average ideology of Americans.

Since looking for “objective” targets to measure media bias can be a lot of work for little reward, that’s not how most scholars have tried to study journalistic bias. It’s much easier to look at two different media organizations and measure their content relative to each other. Another favorite strategy is to choose two similar news events, then study how the apple received different news coverage than the orange. These strategies are easy to do. They don’t require special math skills for the audience to follow the argument.

Unfortunately, the “compare apples and oranges” style of study distorts our view of the world. Imagine comparing how Vox Media and Breitbart covered the recent scandals facing the Trump administration. One media organization is on the left and the other is on the right, so obviously they will cover the scandal differently. That doesn’t mean either organization makes for a good bull’s-eye. Both media organizations built their brand on ideological takes; neither is trying to be in the center!

Scholars are human, so we will probably have more sympathy towards either the apple or the orange in an “apples and oranges” study of media bias. If you read one of these studies, you will notice that they look for cases where the differences are relatively large. If the apple and the orange receive similar news coverage, no one cares. Apples and oranges studies are good at describing differences in coverage in relative detail. However, these studies lack an anchor. The only way to say that the apple is biased while the orange isn’t is if we really, really like oranges. (I like trolling with the apples and oranges metaphor because I prefer kiwi!)

***

Whenever I read a pundit or an academic study comparing the attitudes of Trump voters to the attitudes of Clinton voters, I can’t help but see the same problem. Of course there will be some differences between Trump and Clinton voters. For example, a lot of people have focused on the large difference between Trump and Clinton voters on race. Trump announced his candidacy by disparaging Mexican immigrants as rapists and undesirables. It’s easy to see Trump, see a gap in racial attitudes between voters, and assume this is a story of increasingly racist Republican voters. On the other hand, Trump’s explicit racism and the Black Lives Matter movement may have combined to make White Democrats more progressive on race than they were four years ago.

When we separate everyone in to two categories, all we can do is see the difference between those categories. We can see race polarized voters. However, it’s hard to know just how much racism pushed voters in to Trump’s camp, as opposed to anti-racism pushing other voters to Clinton, just by making a comparison.

The easiest way to write about statistical models actually makes this problem harder. To try and explain this, let’s create a quick and dirty regression model from the American National Election Study. I used age, race (white/nonwhite), gender and having a college degree as control variables. (The publicly available version of the ANES doesn’t have income yet, so take all the results here with a grain of salt. I’m not looking for an optimal model, just a teaching tool.)

Social scientists mainly look at racism via a scale of “symbolic racism.” These are four questions that social scientists started using in the 1990s after it seemed clear that people would not truthfully answer more direct questions about whether Blacks should be discriminated against. The symbolic racism scale comes from a time when social scientists almost exclusively focused on Black-White differences. I will get to Trump’s explicit racism against Mexican-Americans in a separate post. Let’s start with the well-established symbolic racism scale. All these questions are on a 5 point agree to disagree scale:

  • Irish, Italians, Jewish and many other minorities overcame prejudice and worked their way up. Blacks should do the same without any special favors.
  • Generations of slavery and discrimination have created conditions that make it difficult for blacks to work their way out of the lower class.
  • Over the past few years, blacks have gotten less than they deserve.
  • It’s really a matter of some people not trying hard enough, if blacks would only try harder they could be just as well off as whites.

We put these items in a scale, since the average response to four related questions is probably more reliable than a single question. The scale still ranges 1-5. I re-ordered responses so low values are respondents who feel additional steps are necessary to end the systematic disadvantages that African Americans face. High values represent racial resentment. To try and illustrate the strength of symbolic racism, I added a respondent’s political ideology on a 1-7 scale (1=strong liberal, 7=strong conservative). I am using a logistic regression model, where positive coefficients represent groups who voted Trump and negative coefficients represent voting for someone else (not necessarily Clinton). Non-voters are dropped from the sample.

Screen Shot 2017-05-21 at 12.27.32 AM

Screen Shot 2017-05-21 at 12.27.41 AM

If we were interpreting this model quickly, we’d look and see that the coefficient for symbolic racism is relatively large and statistically significant at the .001 level. As a voter’s symbolic racism goes up, the likelihood they voted for Trump goes up by a fairly considerable margin. Since the symbolic racism scale goes from 1 to 5, the difference isn’t as big as the difference between strong liberals and strong conservatives. Nonwhites were considerably less likely to vote Trump, and we could talk about other control variables.

At this point it’s very easy to point a finger at Trump voters and accuse them of being racists. The average Trump voter scored 3.92 on the 1-5 scale for symbolic racism. It’s an open and shut case, right? More racism causes more voting for Trump.

When I ran my regression model this quick and dirty way, I set a very unusual bull’s-eye for comparison. Strong liberals are the baseline for political ideology, because they have the lowest numerical value. However, only 3.6 percent of the respondents identified as strong liberals. Because symbolic racism is coded 1 to 5, the minimum value on the symbolic racism scale is the bull’s-eye for this regression model. It looks like symbolic racism causes an increase in voting for Trump because the value can only go up. Let’s see what happens if we reverse the symbolic racism scale to create an anti-racism index, then use it in the regression:

Screen Shot 2017-05-21 at 12.42.29 AM

All the values in this regression are identical, but the coefficient for the anti-racism index is negative. In other words, as people disagree with the statements in the symbolic racism scale, they are less likely to vote for Trump. The constant changed considerably too. You may have noticed the very low constant in the first regression model. Since logit is a multiplicative model, that low constant actually matters a great deal. However, it’s still very easy to overlook if we are skimming results. I often do!

The bull’s-eye in this model is weird. I’m comparing everyone to a hypothetical strong liberal with the maximum score on the symbolic racism index. That combination probably doesn’t exist in the real world, but that’s not the point. This model exists to show that we could just as easily make a regression model showing opposition to racist assumptions made someone less likely to vote for Trump.

Regression coefficients tell us that there’s a large difference in vote preference between people who scored high and people who scored low on the symbolic racism index. However, regression coefficients can’t tell us which end of the scale is more meaningful. When we compare groups of people, regression models don’t tell us what the bull’s eye should be. We build assumption in to the model, and these assumptions shade how we initially interpret results.

Thankfully, the American National Election Study has asked these symbolic racism questions on a fairly regular basis since 1992. We don’t need to make an assumption about whether symbolic racism surged upwards among Republicans in 2016. We can just check the data. In the interests of space, I will present changes in the index instead of going through each of the four questions. Note that this graph measures party identification not who someone voted for. It includes non-voters. “Independents” who lean towards a party tend to act like partisans, so they are included as partisans here.

sracism-timeline

Every group reported somewhat lower symbolic racism in 2016 as opposed to 2012’s high water mark. This doesn’t really seem to fit with Trump, his base, or the regression results from earlier. One possibility is that racists are learning “politically correct” ways to answer these questions. This happened with more explicit questions about racism. However, there is little evidence that Trump’s base has tried to hide or give safer socially desirable answers in public opinion polls.

The biggest change in symbolic racism scores for 2016 is White Democrats’ large decrease, moving to nearly the same level as non-white Democrats. The symbolic racism score for White Democrats was relatively unchanged from 1992 to 2012. Trump’s campaign may have polarized voters more than it pulled them towards racism. Explicit racists got more of a mainstream voice in the Republican nominee. At the same time, whites who did not want to endorse racial resentment may have moved to more explicit anti-racist positions.

We see such a large difference in symbolic racism scores between Republicans and Democrats in 2016 largely because White Democrats scores’ dropped so dramatically. The regression model that says an increase in symbolic racism made it more likely someone would vote for Trump is technically correct. But that’s not the best way to interpret the data, since the average Republican actually had a slight decrease in their symbolic racism score too. People with the most racial resentment were probably voting for any Republican. The real story in the ANES appears to be a group of white voters moving away from racial resentment, and then voting against Trump because of his explicit racism.

The “secret sauce” to quantitative research is being able to make good arguments and inferences from the data we have. It’s not that hard to build a quick and dirty stats model just by relying on our assumptions of how variables “should” work. Imagine that model fits our assumptions perfectly, like the symbolic racism-Trump voter model. It’s awfully tempting not to dig deeper. Apples to oranges comparisons play in to some of our worst cognitive biases, because it’s so easy to treat one group as the default and the other as the weird thing that must be explained. The tribalism and hyper-partisanship of today’s politics only makes this bias worse.

I spent so much time thinking about apples to oranges comparisons of media content when people were searching for evidence of “bias” that I’ve always been a little more skeptical of the logic behind these comparisons. It’s not a skill that I developed specifically to look at partisan politics. It’s not a skill that I developed from years of stats classes either. The ability not to rubber stamp apples to oranges comparisons that fit our preconceptions is something different. In some ways its harder. It takes courage to acknowledge our preconceptions may be wrong.

In other ways, the ability to question our biases is easier to learn. We don’t need special classes in stats to develop this skill. Every year I taught stats there were several students who walked in to the class having mastered it. When I was a teaching assistant, I think the class that actually focused the most on this skill was a freshman level introduction to linguistic anthropology course. We were going over differences between how men and women communicate. We didn’t just cover differences and debate nature vs. nurture. We spent more time going over whether there are certain stereotypical differences that we expect to see, so those are the only differences we remember. Naive assumptions can become self-fulfilling prophecies unless we consciously try to check them. The good news is we can all learn to check them.


Glad I’m Not Covering Clinton

In the last day or so I have seen a number of contacts retweet the following:

It’s a provocative claim! Since I spent years working on how to count news coverage in different ways, I thought it would be interesting to take a look at the methodology used here. Boehlert and his colleagues at Media Matters didn’t do the counting themselves. They picked up on “recent tabulations from Tyndall Report, which for decades has tracked the flagship nightly news programs.” I’ve never heard of Tyndall Report before. (The about page is extremely cryptic.) That not a good sign, but it could also be a good way to learn new things.

Boehlert wrote up this report as something groundbreaking, but he didn’t notice obvious red flags. He copies the Tyndall finding that 2008 was a high water mark for “issues” coverage with 220 minutes. Does 220 minutes for a year’s worth of news seem odd to you? Let’s do a little math. If we are looking at nightly news programs that only broadcast 5 days a week, that is approximately 220 weekdays from Jan 1, 2008 through election day. Which is more plausible: networks haven’t combined for more than one minute per day of “issues” coverage since 1980, or someone is using an awfully narrow definition of issues coverage.

Media Matters only offered the analysis, but I was able to quickly trace the link back to see the author(s) define their methodology:

Issues coverage is differentiated from candidate coverage thus: it takes a public policy, outlines the societal problem that needs to be addressed, describes the candidates’ platform positions and proposed solutions, and evaluates their efficacy.

The amount of time spent on issues coverage represents the attempt by television news to establish a political agenda that is driven by the perceived problems that the country faces rather than those talking points that the candidates select to promote their own causes.

This looks like an incredibly strict definition of “issues” coverage. The author(s) only include stories where the anchors sat down and said “we want to cover issue X” in detail. Bernie Sanders spent a lot of his campaign going from city to city, giving his stump speech. He talked about a lot of issues in his stump speech, like health care and the minimum wage. Certainly the progressives at Media Matters and their core audience remember this. But any media coverage of Bernie Sanders speeches or interviews with him talking about issues would be excluded from this count. Anything candidates say in a debate would be excluded.

The Tyndall Report isn’t counting how much time TV network news spends on issues. It is counting how much time TV networks spend on a very specific “where do candidates stand on the issues” type of feature. I’m not surprised TV networks have moved away from this feature. It works much better online. People can point and click to compare candidates or take interactive quizzes to see how their views line up with candidates.

I think people buy in to this report because it feels true. A study with more methodological rigor would probably find a decrease in time spent on issues. However, one of the main reasons for this is Donald Trump doesn’t spend as much time talking about issues. Hillary Clinton has responded by campaigning about Trump’s negative personality traits. As Boehlert notes, Clinton has 38 issues on her website and 112,735 words of policy fact sheets (he gives AP credit for these facts, which were published on Aug 29). This doesn’t mean Clinton has talked about each issue in detail or emphasized it on the campaign trail.

I could keep blasting “biased” media coverage if I wanted to – it’s an easy bell to ring. But I want to end on a different question. Let’s assume that out of the things coming from the Clinton camp, reporters currently consider her email server to be the most newsworthy. Could Clinton have done something different to change the narrative? Now I’ll explain why I think the answer is yes. When I studied the 2008 general election I found TV networks interest in a topic was somewhat contingent on candidates bringing up that topic. The national elite media organizations that had access to candidates followed the candidates’ agenda, while sites that lacked access were more independent.

Paradoxically, all the coverage of Clinton’s e-mails and the Wikileaks is an unintended consequence of how she chose to present herself for the general election. Clinton’s policies come off as technical, well-polished versions of fairly standard Democratic ideas for the most part. There isn’t much in terms of new thinking to capture people’s imagination. Instead, the way that Clinton has tried to capture hearts and minds is to emphasize personality traits: her experience versus Trump’s poor temperament. If both candidates want the race to primarily be about judgment, any “scandal” about Clinton will stick more.

When I was a reporter I liked writing the kind of stories that the author(s) of the Tyndall Report crave. But I would be bored out of my mind trying to write these stories in 2016. It’s not because Trump is vague on policy. I loved the challenge of trying to show when officials didn’t understand what they were talking about while still conforming to the norms of objectivity. What would bore me is writing about Hillary Clinton’s policy preferences. It feels like a long list of ideas I have heard before. Don’t get me wrong – most of these policy proposals need to be repeated because they never got a fair hearing in Congress. I’m just saying as a former newsman I recognize the lack of new. I’d rather write in detail about a new Clinton policy proposal that shakes up the Democratic status quo, but it isn’t there. New policy ideas and policy differences would be a better topic for Clinton, they would benefit the audience, and they would give reporters a more well rounded diet of things to write about. It would have been to everyone’s advantage. But contrary to the assumptions of the Tyndall report, the only way to get coverage of new policy ideas is if a candidate emphasizes their new policy ideas.

As a national reporter I might be forced to write more about Clinton’s email as the least bad option for Clinton coverage. Then I’d cry and ask for a new beat. I’d rather do features on the voters or ballot propositions than be on the Clinton plane. Ideally I’d get to use Simpsons quotes! For all the attention being paid to the Presidential race, there isn’t a whole lot of actual news, particularly from Clinton.


Can We Fact Check This?

During the Vice Presidential debate on Tuesday I saw a professor tweet that they wanted a fact check on whether the United States was the “greatest nation” in the world. It felt like an odd thing to complain about. Mike Pence used this once as a throwaway line. This statement was more forgettable than most of the debate, which is really saying something! On the other hand, I’ve been trying to write a post on what journalist can and can’t fact check for weeks. Pence provides a great example. Can we fact check a claim like the United States is the greatest nation in the world?

To start off I want you to ask yourself, how do you feel about the United States?

I love statistics, but that’s not the first thing that came to mind when I thought about the US. This is what makes fact checking a statement like Pence’s deceptively tricky. We can fact check specific outcomes. For example, if Pence said the United States has the best health care system in the world, we could look at the disproportionately high cost of health care in the US without a corresponding increase in life expectancy. There’s a lot of room for improvement. If Pence made a specific reference to America’s wealth, we could check both per capita income and inequality levels (I like this particular illustration from Pew). Whenever a politician says the United States is the greatest nation in the world at a particular thing, we can check whether that is factually true.

When US politicians say America is the greatest nation in the world, it’s rarely providing facts that America is great at X, Y and Z. It’s about giving people a feeling. If someone comes along and tells you that your feelings about the United States are wrong, that’s not a fact check. It’s a value judgment. We may have different feelings about the United States. After all, it’s a big country. That’s not the point. Pence’s claim that the United States is the “greatest nation” in the world is a great way to separate journalists’ role as literal fact checkers from how they handle moral judgment.

My guess is the professor who wanted a fact check of Pence disagrees with his broader moral values. Since Pence’s long record was barely discussed during the debate, it would be valuable for journalists to explain how Pence’s distinct moral values influence his public policy. You may recall how a wide range of Americans – including large corporations – disagreed with Pence’s support of a state law that discriminated against gays. Pence has also pushed for the strongest anti-abortion measures since Roe v. Wade and tried to block Syrian refugees from resettling in Indiana. The first law was repealed after fierce protest from the business community; the other two have been ruled unconstitutional. Pence’s moral values led him to the Republican ticket. People will disagree on whether Pence’s policy is good or bad, but I expect both sides would largely agree with this description of Pence’s social policies.

Critical academics love to attack the caricature of journalistic objectivity, arguing that everyone has their biases. But journalistic objectivity has never meant being unbiased. Objectivity is a way for journalists to distance themselves from making moral claims in their writing (Tuchman 1972). Let’s say a reporter at an objective publication felt Pence is a religious extremist. They couldn’t put those feelings in print. However, they could keep calling Pence’s opponents until someone says Pence is an extremist. This form of “he said, she said” journalism is unlikely to satisfy critical academics. Publishing a back-and-forth on Pence’s claim that the United States is the “greatest nation” in the world isn’t the same as a reporter using their own moral authority to check Pence.

Since more and more of my academic contacts have called for journalists to impose some kind of moral sanctions in their campaign journalism, I wanted to tell you a story from my reporting days. I heard Robert Byrd was putting a hold on the DREAM Act. This was back in 2002, when the bill was initially up for Senate consideration. (Got to love how quickly Congress can move!) Anyway, I called Byrd’s office and a spokesman confirmed that the Senator was responsible for the hold. I asked why, and he said Byrd needed “more time to study the bill.” I was calling in August. I knew the Senate was on recess. So I asked the spokesman whether his boss would have had enough time to read the bill by the time the Senate reconvened. He said he wasn’t sure, which is awfully suspicious.

I was mainly concerned with whether or not Byrd’s spokesman was telling me the true reason why his boss put a hold on the bill. I didn’t consider the ethics of whether a temporary hold was legitimate public policy. I didn’t try to hold Byrd accountable on some moral level for blocking a “good” bill. To be honest, I had no idea whether the DREAM Act as it was written in 2002 was a good piece of legislation by the time I filed my story. I knew there were arguments for it and against it. I knew both sides made assumptions about what was morally correct and the long-term implications of the bill.

I did not want to give a moral stance on the DREAM Act when I covered it. It’s not that I was working for an objective news organization so I “had to” avoid taking a side. I sought out objective journalism so I could share a wide range of important information and viewpoints with my readers instead of sharing my own opinions. I grew up in a household where everything was portrayed in black and white. Being able to empathize with a wide range of sources – even politicians I vehemently disagreed with – was incredibly liberating. Not having to take a moral stance while I was covering political campaigns made it easier to advocate moral stances in my private life outside of the newsroom. This also made teaching more enjoyable for me and better for my students.

When I was a political journalist, I got lied to on a regular basis. A big part of my job was sorting through the lies. You’d figure political journalists and former journalists would be the strongest voice saying readers need a strong hand to guide them through the minefield of lies. However, I found journalists always showed the most confidence in the audience. They believed that if they laid out the facts, audience members could reach the appropriate moral conclusions. Of course, some audience members will back the politician and attack even the most neutral, fact-based journalist. If that journalist jumped up and down and said the politician is an awful human being, would that moral condemnation do anything to make their article more persuasive?


Why Shoot the Objective Messenger?

Donald Trump continues to frustrate progressives. His latest offense is tweeting out a meme over the weekend calling Hillary Clinton the “Most Corrupt Candidate Ever!” Trump’s graphic appears to have been copied from a white supremacist message board. Mic.com has consistently pursued the links between Donald Trump and white supremacist groups since November, when Trump tweeted out a graphic from the “Crime Statistics Bureau” in San Francisco claiming that most whites and most blacks are killed by blacks. A wide range of news organizations criticized Trump’s November tweet, pointing out that there is no “Crime Statistics Bureau” and that most homicides with a white victim also have a white killer. Many pundits on the left expect a more vigorous media response to Trump’s latest tweets:

Carlos Meza is a research fellow at Media Matters for America, the largest think tank specializing in progressive media criticism. This type of argument dates back to the Vietnam War. Anti-war protesters wanted to get in the news in the first place (initial coverage excluded them). Then they wanted the media to avoid “taking both sides” and take an explicit anti-war stance. For 50 years there have been progressives who want media organizations to take an explicit moral stand with the left. I read a lot of these critiques in graduate school, dating back to Todd Gitlin’s relatively sympathetic take on Students for a Democratic Society. The argument hasn’t really changed over the years. Progressive critics always say news organizations care too much about money when they should be using their platforms to take moral stands.

Critics like Meza don’t spend as much time thinking about what would happen if mainstream media organizations all embraced progressive moral stances when reporting the news. What would happen if the media emphasized “combatting open bigotry” instead of “impartiality and balance”? How much could media organizations actually do here? Meza argued any media organization that takes both sides instead of taking a moral stance against Trump is “aiding and abetting evil.” Ironically, I was already working on a post explaining why it doesn’t make sense for most mainstream media organizations to take these stances before this weekend. The short answer is people don’t know why media organizations make particular choices, so they apply their political bias to interpret any news report. A media organization’s claiming that “TRUMP IS A BIGOT” won’t convince people who don’t already believe Trump is a bigot. It will only label the media organization as left-wing and make the accusation of bigotry more ideological. The full answer is much longer and explains why Trump can get so much media attention in the first place.

Connecting the Dots

Before getting in to the weeds of how media organizations work, it helps to review the research of former UCLA sociologist Harold Garfinkel. He wanted to see how people made sense and maintained order in everyday life. He intentionally provoked unsuspecting people in order to see how people made sense of things and what they took for granted. He would erase your circle and draw an X through it in tic-tac-toe. He had students try to barter for the price of their groceries and interrogate bus drivers about how we could be absolutely sure the bus was going where the driver said it was going. Tic-tac-toe makes for a funny game in the classroom, but most of these experiments outraged people. (I’m obligated to warn you not to try this at home…unless you want to mess with a telemarketer.)

My favorite of Garfinkel’s “breaching” experiments actually took place in a laboratory. Participants were allowed to ask 10 yes or no questions and get replies from a social psychology professor (presumably Garfinkel) in the next room. All the participant had to do is write down what they thought about each response before asking the next question. One after another, people asked deeply personal questions and explained how the answers made sense. However, the “expert” in the next room was just someone flipping a coin and then saying “yes” or “no” in to the microphone. The answer was completely random, but the research subjects accepted the validity of the answers. More importantly, they wrote elaborate stories to justify the response.

Sports fans know that people tend to tell bad stories when they start with an end result and then work back to discuss process, despite all the data available in sports. Let’s imagine every time Derek Jeter bats is like rolling a 20 sided die, and he gets a hit on 1-6. Random coincidence could lead him to keep rolling low numbers in big at bats – and a 30 percent chance of success isn’t that rare. However, luck and coincidence make for terribly unsatisfying stories. Sports fans are like the people in Garfinkel’s experiment. Did this team win? It’s a yes or no question. Most fans start with the final answer, then have to come up with a story of why this happened. That’s how some athletes get labeled as “clutch” and others as “chokers.” The label is usually based on winning and losing, not direct knowledge of the athlete’s decision-making and whether he made good decisions.

Why Can’t We See How Journalists Make Decisions?

I started with sports because we can objectively measure athletes’ performance, look at how many championships they won, and still tell awful stories connecting the process to the result. Imagine what it would be like if we didn’t see every play that went in to the progress of the playoffs or the regular season. That’s what it is usually like when we try to explain news coverage. Most social scientists don’t even try to compare the starting point of news events to the final story (or lack of a story). You can get published in the top sociology journal just by looking at a collection of final news stories and then proposing a story about why those news stories would have one set of themes instead of another. Why is it so hard to study the process of writing news more directly and systematically? Once we understand this, it will be much easier to understand why news organizations may not want to take explicitly moral stances against Trump or anyone else.

1) What counts as “news”?

When I taught undergraduates about the newsgathering process, I always started by asking them what counts as news. You may not be a journalist, but you can easily play along. What kinds of things would you post on social media? What do you want to read? Even when I taught a small class of under 20 people this fall, we quickly realized that no one would have identical rankings of potential news stories. Every newsroom I have worked at could get in to the same arguments. When Gaye Tuchman wrote one of the classic ethnographic studies of newsrooms in the 1970s, she found newsworthiness was the hardest thing for journalists to define. Even professionals use a “know it when I see it” philosophy to assess newsworthiness.

2) Content producers need heuristics

Let’s imagine you were working at a newspaper. Maybe it would be more appropriate to imagine you are writing your own blog like I am right now. You want to make sure you keep producing the best stories for your audience, so you keep searching for new stories and eliminating possibilities. How much do you write? The answer is nothing! The rate of new things happening in the world is faster than the rate at which we can eliminate possibilities. Even if we are writing our own blogs and don’t have to argue with anyone, deliberation takes time. The only way to get anything written is to limit your search and say “I’m going to write about the most interesting thing I can find by a certain time.”

Journalists use a wide number of heuristics to manage the influx of potential stories – and most partisan sites seem to use similar heuristics. One of the main heuristics is to sit down and write a list of who is most likely to provide news stories, then assign reporters to check in with these sources on a regular basis. Planning events in advance lets journalists know “if I show up at a particular place and time, I can produce a story.” Reporters may not be able to verify if a source is telling the truth, so they default to trusting officials because they should have legitimate knowledge to information (Fishman 1980 holds up surprisingly well).

3) Everyone has incentives to hide the process

When we read a final news story, we don’t know if the reporter spoke to any press flacks to arrange an interview. We don’t know if anything was said off the record to explain the goals of the story, limit the scope of questions, or ensure a story is published at a particular time. When reporters selectively quote from interviews, we don’t know what was left on the cutting room floor. If reporters summarize what someone said and what they meant, we don’t know if the reporter understood correctly. (Most complaints about misquoting are really complaints about misunderstanding the intent of a statement.)

It’s probably easiest to understand why journalists don’t want to show the full process. They appear to have a lot of freedom. When reporters really have a lot of options, they don’t want to be second-guessed. Remember that Meza works full time at Media Matters, a think tank founded to second-guess journalists. When reporters have limited ability to search for stories before deadline or can’t get access to desirable stories, they want to hide their failure. When in doubt, reporters rely on official sources, but they want to hide an overly cozy relationship. The news organization that opens itself up to second-guessing may lose its audience to a tight-lipped competitor.

Everyone else involved in the news process also has incentives to hide the process and just focus on the final result. Audience members don’t want to sift through the reject pile every day – we have other things to do with our lives! If a source spoke off the record to attract news coverage or coordinate an interview, they would want to hide the friendly relationship. It looks better if the politician’s event is really the most important news of the day instead of merely being the easiest story to find. (Donald Trump’s media deluge is going to be much easier to understand by the time we’re done.) Sources who don’t know how to catch journalists’ attention or who say really embarrassing things in their interview may not want to explain how they were stupid.

News Stories Aren’t Enough

We know that news stories do not emerge from some alternative dimension. When we only see the final story in the newspaper, on television or online, it may not be completely satisfying. We want to be able to tell our own story of how reporters did their job. Trump coverage is a great example. If a news organization doesn’t call Trump a bigot, many progressives will be outraged. Meza and the thousands who liked or retweeted him need a story of why the media covered Trump neutrally.

Since reporters don’t want to tell us how they make decisions, we need to construct a myth from somewhere. We know people have likes and dislikes. We know news coverage emphasizes certain things more than others. Therefore, it is very simple to infer that reporters emphasize the things they like and ignore the things they dislike. Meza engaged in a very simple form of this, asserting that journalists inherently like objectivity and neutrality. Tuchman argued objectivity is more strategic. It’s a way to minimize the number of people who are outraged like Meza is.

When I studied press conferences I found the president got three out of every four quoted words in the New York Times the next day. Shorter stories about press conferences only had one source: the president. When journalists told longer stories they added more sources to give reactions. You probably didn’t realize that. I didn’t realize it until I did the study. A certain kind of news story can be mathematically unbalanced for decades and people won’t notice. When I presented these findings most audiences were stunned. Then they got angry. It helped me realize that people aren’t used to thinking about balance or imbalance in the news as some formal mathematical property. My audience mainly thought about how news gets produced when they see a story they don’t like! This is when people need to create some sort of myth to explain why the media isn’t giving what they want.

Let’s Trust the…Politicians?

The only people with a real incentive to talk about how reporters search for and write up news stories are the people who feel screwed over by the process. Let’s say someone was left out of the news. They could complain about censorship and say the press was playing favorites. Reporters don’t want to share “here are the sources who are in my top tier, second tier, etc.” Now let’s say someone is getting negative coverage. Pick any presidential candidate from 2016. They have all made mistakes and gotten negative coverage as a result. Why is the press focusing on this one negative story instead of other positive stories? Candidates have complained about an overly negative press for 50 years. It’s a claim that news organizations have never really been able to deflect. We know reporters choose a few stories from a wider set of potential stories. Reporters want to hide the process and their own political preferences. Now that I am no longer a news reporter I can say there was little correlation between whether someone was a good source and whether I personally agreed with their politics.

There’s something very unusual in how we think about political bias in the news. Politicians sell a myth that the media is biased and there are always better stories (about them) for reporters to focus on. We believe the politicians! After all, what possible reason could someone have to lie when asking for our vote? It’s absurd to think about how many people buy in to politicians’ myth of how news gets created by biased reporters. This shows how unsatisfying reporters’ claims of expert news judgment are. We want to tell some myth or origin story of how those news stories are created and why they favor some people or topics instead of others. Politicians may be extraordinarily self-serving, but they are the main group offering an explanation. Activists like Meza – who want to fundamentally change media organizations – are the other group offering explanations.

How Does This Help Us Explain the Trump Show?

Donald Trump announced he was running for president a year ago. He has dominated the airwaves since. We probably haven’t seen such a dramatic imbalance in how much coverage one candidate got relative to others in his party’s primary. A few months before Trump announced, I predicted someone who ran a campaign based on social identity would get considerable media attention. However, Trump’s media strategy has gone beyond what I predicted. He emphasizes getting a large volume of coverage like no candidate before him. He speaks almost entirely in feelings, often subverting the entire premise of a “fact check.” I think Trump’s strategy takes full advantage of an audience that needs to manufacture coherent stories to explain patterns in news coverage and a press that refuses to tell this story themselves.

Conventional Wisdom: Avoid Gaffes

Before going in to Trump’s strategy in more detail, it is important to know how over the last 35 years political communication has gotten more professional and strategic. One of the main lessons for these strategists is to minimize gaffes. Politicians hold a lot of choreographed media events and photo ops. They may try to answer the question they wish they were asked instead of the question they were actually asked. Strategists probably don’t think they can completely bowl reporters over with these strategies. Any veteran reporter will recognize the manipulation at work. But they can’t stop it. The goal of these strategies isn’t to make a big impression. It is to limit the number of things a reporter has to choose from.

A good metaphor is a college football team that schedules Directional State University and Small School Tech to try and inflate its won-loss record. Large college football teams can essentially buy wins. Some teams also play in much more challenging leagues than others. A good team with a weak schedule may only have 2 or 3 games out of 12 that they have any real risk of losing. When most fans saw 13-0 Florida State in the first year of the college football playoff, they saw a great champion instead of a team that barely beat a number of teams in on a weak schedule. Florida State lost badly once they faced high quality competition in the playoff. Last year people were fooled again, this time by Big 10 champion Michigan State and runner up Iowa in the Rose Bowl. People saw college football teams that won a lot of games and told themselves a story that these teams were among the best in the country. They didn’t bother to check how these teams won and whether the process was solid.

Most politicians try to win our confidence with the same principle. They want to accumulate positive story after positive story mainly by limiting the pool of information that reporters have to pick from so there are only positive stories left. They try to avoid unscripted events that may give genuine moments that could inspire the audience, because these events could go badly. College football fans look the other way when a team from their conference benefits from an easy schedule. Partisan audiences do the same thing in politics. We want to see the other candidates go through the ringer and get beat up, but we assume our favorite candidate went through a much tougher schedule than they actually did. Clinton and Sanders were actually fairly nice to each other, compared to Obama v. Clinton in 2008.

Trump’s Casino Approach

Donald Trump doesn’t exactly come across as someone following the “avoid gaffes” conventional wisdom. He’s using a different strategy: try to be the lead story every day and accept some negative headlines rather than go a news cycle without being the top story. Flooding the airwaves is also a kind of gaffe deflection strategy. Most candidates try to avoid gaffes because they think one critical mistake could fester and sink a campaign. Trump tries to keep one gaffe from defining him by creating a new headline. Changing the story is an old public relations strategy. Most politicians try to use it to prevent negative stories, while businesses tend to use it after a scandal has happened. Trump is more than willing to give a new story about his campaign to distract from a gaffe, even if the new story is also negative.

If news organizations give media coverage based on whether a politicians has said something that will help the readers decide whether or not to vote for that candidate, then Trump has certainly earned the media attention he is getting. We expect certain things out of a generic Republican or generic Democrat. If a Republican says “Obama did a bad job” is that news? Republicans have been saying that non-stop for eight years. Regulation bad, free markets good is another very generic Republican position. Trump attracted attention by saying things that were well outside the Republican mainstream. He was more strident in disparaging Mexican immigrants. Trump also attacked mainstream Republicans, claiming they were too dependent on the Koch brothers and other big money Republican donors. Any member of a political party who criticizes their own party gets considerably more media attention (Groeling 2010). Sanders also got attention for attacking the Democratic National Committee. What makes Trump unique is his near exclusive reliance on a large number of news media appearances and social media messages instead of a balanced strategy including on-the-ground campaign staff or advertising.

Casinos tilt the odds so after enough games the house always wins. Trump has stacked the deck too. Whether it is intuition or calculation, Trump has a tremendous sense of what people will consider newsworthy. This doesn’t mean people agree with what Trump has to say. All it means is Trump can monopolize media attention. If people like Trump’s message, all the free media brings voters. If people don’t like Trump’s message, they may act like Meza and blame the media for giving Trump so much attention. People who don’t want to be subjected to the Trump show need some explanation for why so many media organizations pursue it. Maybe the media is politically biased. Maybe they emphasize Trump because Trump brings ratings and pageviews. The more voters question news organizations’ judgment, the more it helps Trump. Either way, Trump can get an advantage from dominating the headlines every day.

The other unusual thing about Trump’s campaign is his willingness to invent or ignore facts. Most politicians are afraid of saying something factually untrue. Reporters could run a “fact check” and embarrass the politician. Trump makes so many things up that the Associated Press used 10 fact checkers to review a speech on June 22. I don’t think Trump cares about fact checks because he is not trying to combine rational facts with emotional appeals. Trump’s campaign is entirely emotional. To borrow from Stephen Colbert, truth simply doesn’t appear to be a part of Trump’s message; Trump’s message is all about truthiness.

Truthiness Meets Hidden Journalistic Practices

Because journalists do not share every step of how they choose one news story and one angle over another, they have always required the audience to fill in gaps about who is credible. Everything a reporter shares about how they establish priorities is a gap people could use to undermine a reporter’s credibility. That’s why reporters use facts as supporting evidence as much as possible. When they don’t have facts, they use quotes and put someone’s claims in quotation marks. Most politicians and interest groups give some evidence to back their claims. We take it for granted that people will use some evidence. Trump doesn’t use facts as supporting evidence. There’s nothing to “fact check.” Audiences have to decide for themselves if Trump’s claims about America and policy proscriptions feel true, racist or fascist. Meza is the latest in a long line of progressive activists who do not trust the audience to reach the “right” conclusion without a massive shove from the media. Bigotry must be actively combatted. Journalists cannot be neutral. In his view, presenting both sides helps a bigot so much that it is “aiding and abetting evil.” (Has Meza linked to supporting evidence himself since I wrote this?)

Veteran reporters know it doesn’t matter how much you try to shove the audience to a particular position, they are still going to create myths to explain why a journalist took a particular stance or covered a particular story. When reporters actually offer an explanation it is often fuel for these hostile myths. That’s why reporters don’t accuse someone of lying without documented evidence. The accusation doesn’t make someone any less credible without proof. Let’s say CNN’s Jake Tapper stood up and screamed “You are a racist!” when interviewing Trump a few weeks ago about the judge in the Trump University case. (In case you forgot, Trump cited the judge’s “Mexican heritage” as the reason for not dismissing the lawsuit.) I assume this would make a critic like Meza happy. I think this is what he wants Tapper and other journalists to do. But Meza already agrees with the assertion that Trump is a racist.

If people weren’t convinced Trump is a racist after that interview, there’s nothing that Tapper can say to change this. People tell myths about reporters’ ideologies and biases even when they have nothing to go on besides the final news story. These myths are mainly ways to discount news stories that don’t fit our political preferences. If Tapper took an explicit anti-Trump stance, he would just make it much easier for people to discount him as an ideologue. Just think about partisan media organizations for a moment. How much are they trying to actively persuade neutrals or the other side? Most of the content is giving true believers hot takes that make them feel good. If you really want to persuade people, you have to give them enough rope so then they can decide for themselves that they want to change their mind.

Ironically, the fear that media organizations are “legitimizing” Trump appear to be unfounded. Every public opinion poll has found many more Americans disapprove of Trump than approve of him. Trump’s favorability hit a record low of 29 percent in a recent ABC/Washington Post poll and has hovered in the low 30s in other polls. More than half of those polled in both the ABC/WaPo poll and a separate Bloomberg poll have a strongly unfavorable view of Trump. People can decide for themselves how they feel about Donald Trump without objective media organizations explicitly calling Trump a bigot or a fascist or something else. A wise progressive may conclude it’s better to present Trump in his own words and let the audience conclude he is a bigot than to add explicit condemnation, which could make it seem like a biased media is out to get Trump.

If you ask people “is the news media doing a good job?” most of them will say no. They read news stories, but most news stories just don’t give readers what they want to feel. Objective media is defined by stoic attempts to distance reporters’ feelings from their narratives. Partisan media is defined by outrage. If you want something else from the media, you are probably out of luck. We need some narrative to explain why the media doesn’t give us what we want. These narratives always ascribe motivations and preferences to the media while leaving out sources. People blame the news organizations for covering Trump so much. It doesn’t matter that objective, “both sides” media coverage is leading a majority of Americans to have a strongly negative view of Trump.

What Is News?

I went to grad school to research what counts as news. How do people make these decisions? I never set out to do a breaching experiment. But when I proposed that sources may have power over journalists – and this could be quantified – I broke several of sociology’s norms. I didn’t automatically buy in to any of the common myths that ascribed motivations or bias to journalists. I wanted to empirically test these theories, to the degree that these theories were testable and a regression analysis can test any theory.

After my fourth year of graduate school I was mocked for being the only person at the media sociology panel who wanted panelists to say something about their methods instead of skipping to the data. Each panelist described a different role for sources in their study. I asked them to speak more directly about the role of sources during the Q&A. I wasn’t trying to embarrass anyone – I only ask questions at talks that I think the speaker can reasonable answer. Three of the panelists froze and the fourth tried to hide under the table! I made the mistake of emphasizing sources and strategy at a time when my peers turned to critics like Meza to fuel their echo chamber.

Like I said earlier, veteran reporters know at a certain point they have to trust their audience because there is nothing more they can say to sway their audience. I think I’ve reached that point with my blog. Unfortunately, offering factual information to support claims of how journalists make decisions is harder than finding factual information on nearly any other topic. One of the things I taught my students when teaching research methods is that people can always find a way to discount research if they want to – and some people are highly motivated to discount research about news.

I’m not going to delete this blog. I could always bring it back at some point. But after a decade of banging my head against the wall I need a break.


False Equivalence and Censorship

I avoided social media for most of last week. I knew my feeds would be full of hot takes about Trump’s rise. How could the media possibly play such a big role helping him to the nomination? I was one of the few people who wasn’t surprised. I predicted this media cycle over a year ago at the end of my dissertation. I posted my prediction and what led me to it on this blog in December. The one time I logged on last week to read non-sports tweets, I was surprised to see someone else’s predictions for the general election continue to pop up in my Twitter feed:

Among all these classes of professionals, all these institutions, that whole superstructure of US politics built around two balanced sides, there will be a tidal pull to normalize this election, to make it Coca-Cola versus Pepsi instead of Coca-Cola versus sewer water.

The US political system knows how to play the former script; it doesn’t know how to play the latter. There’s a whole skein of practices, relationships, and money flows developed around the former. The latter would occasion a reappraisal of, well, everything. Scary.

So there will be a push to lift Donald Trump up and bring Hillary Clinton down, until they are at least something approximating two equivalent choices.

This comes from a longer post from David Roberts of Vox. Roberts starts his essay with the position that “Almost irrespective of what you think of Clinton’s politics or her policies, she is manifestly more prepared to run the federal government than Donald Trump.” Regardless of the qualities of the candidates, research has shown a bias to favor the underdog and make campaigns look more competitive. Roberts argues the imbalance between candidates is the “elemental fact about the election,” but mainstream media organizations will be unable to address it. Roberts’ essay draws from a common complaint on the left: as Republicans have drifted further right, they are no longer equivalents of the Democratic Party. Roberts tries to argue it is media norms, not reality, that puts the two parties on equal footing. The “false equivalence” complaint is most common in Roberts’ beat of climate change, where there is only one acceptable side within the scientific community.

What would it look like if a major media company actually followed Roberts’ suggestion? Facebook may give us some insight. Back in 2014, Facebook implemented a “trending topics” window to try and compete with Twitter’s specialty. The company never fully specified how decisions were made. Over the last week Gizmodo has published several stories about the former journalists hired to pick out Facebook’s trending topics. I recommend both. Here is a key quote from the first story:

“I got the sense that they wanted to keep the magic about how trending topics work a secret,” said another former news curator. “We had to write in the most passive tense possible. That’s why you’d see headlines that appear in an alien-esque, passive language.”

A former employee came forward today alleging political bias. A few directives allegedly came from the corporate office. However, those directives mainly listed a set of preferred media outlets (New York Times, Time, Variety) to verify a topic is trending and less prestigious sites to avoid (World Star Hip Hop, The Blaze). Exclusion of conservative political topics was up to the whims of who was working a particular shift:

“I’d come on shift and I’d discover that CPAC or Mitt Romney or Glenn Beck or popular conservative topics wouldn’t be trending because either the curator didn’t recognize the news topic or it was like they had a bias against Ted Cruz.”

“It was absolutely bias. We were doing it subjectively. It just depends on who the curator is and what time of day it is,” said a different curator. “Every once in awhile a Red State or conservative news source would have a story. But we would have to go and find the same story from a more neutral outlet that wasn’t as biased.”

Facebook goes to great lengths to present itself as one of the few neutral pipelines for media content. The company has aggravated progressive critics in the past by claiming their goal is to give the audience what it wants. One of the few times Facebook publicly admitted failure was in the early days of the #BlackLivesMatter movement. Ferguson protests trended on Twitter while the Ice Bucket Challenge trended on Facebook. Some of these differences come from different user bases with different interests. However, Facebook has been relatively open about needing to have their thumb on the scale to make breaking news trend faster on the site to avoid future embarrassment.

Putting Facebook’s conundrum side-by-side with Roberts’ essay illustrates the fundamental problem of providing news content. True neutrality makes media companies look like they don’t care about anything but eyeballs and the least common denominator. People rely on journalists to do the hard work of putting a thumb on the scale, separating important stories from everything else. However, many people in the audience distrust that thumb on the scale. Audiences that care deeply about political news are like Roberts – they always imagine a better way to balance the news whether or not it is viable.

Most people assume objectivity is mainly a normative commitment to treat multiple sides equally. Roberts and the Gizmodo stories naively repeat this assumption. In practice, journalistic objectivity is more about deflecting accusations of bias (Tuchman 1972, link to JSTOR gated). When the president gives a press conference, the next day’s news stories do not try to give an equal balance of the president and the opposing party. In my study, the most common situation was only quoting the president. Readers don’t protest when a minor press conference leads to completely one-sided coverage. They don’t even notice! I found that the balance between quoting the president and other sources was largely a function of how much news coverage a conference received. Mainstream journalists care more about providing balance and objectivity when people are more likely to pay attention and complain about imbalance.

Now lets think about a presidential election. Large sections of the American population pay attention. News organizations do not want one side claiming they are biased. Partisans get very angry if you tell them their candidate had no shot. Republican activists argued the polls were “skewed” and tried to “unskew” them to show a competitive race. Sanders supporters have tried to cherry pick poll results all year to try and make him look like he was doing better than Clinton. Partisans are the most likely to feel that the media is biased against them. Imagine the next 100 polls said Clinton is favored by 20 points. If a reporter writes this story, Trump’s core supporters are more likely to believe the story or polls are biased than to believe their candidate is deeply unpopular.

All the hand-wringing about “false equivalency” and Facebook reminds me of the weirdest thing I ever saw in journalism. When I interned at a local television news station, the correspondents started most interviews with the camera off. Then they turned the camera on and asked the exact same question. I was shocked. How could this lead to good journalism? The most serious, grizzled correspondent at the station explained “if you get a bullshit answer, then they are stuck on camera giving a bullshit answer.” People react so badly to the image of a journalist with their thumb on the scale. Even people like Roberts who want more activist journalism seem convinced that reporters would never get it right. One of the best ways for a journalist to show a politician is manipulative and self-serving is to take their thumb off the scale. At a certain point you have to trust the audience’s judgment.


Using Baseball to Help Understand the Complexity of Political Bias

I’ve been thinking more about the differences between political and sports media since I finished my dissertation. Academics usually study political media bias. They ask questions like is Hillary Clinton getting more negative news coverage than Donald Trump or “is the media biased against Bernie Sanders?” We assume negative coverage of one politician usually means positive coverage for another. In a series of blog posts this week, I will try to explain why this is one of the worst ways to think about “bias” in media content.

Baseball reporting is a useful parallel to think about what people may have in mind when they say the media is biased. Every team has important weaknesses. Championship teams tend to lose 40 percent of their regular season games. When a team has its inevitable struggles, some fans irrationally panic. Other fans expect uniformly positive news about their favorite team. Anyone who reports honestly about a team’s weaknesses or struggles will face accusations of “hating that team.” The result is high profile baseball writers are accused of simultaneously hating every single team in the sport!

Sports examples show us that angry readers do not need to make comparisons before concluding that someone is biased against their side. All it takes for someone to say that the media is biased is to imagine some alternative form of news. When people complain that the media is too negative, they are imagining a more positive or uplifting set of stories that news organizations could publish but don’t. One of the most challenging things about the media is we don’t know the full list of stories that reporters have to choose from, given their deadline constraints. Media organizations ask readers to put faith in their story selection process. When a reader accuses a media organization of bias, they say “I don’t have faith in your ability to choose and/or present stories.”

Because claims of bias are about a lack of faith, people can make these claims without any journalistic expertise. They don’t need insight. People who claim bias don’t even need tangible evidence of a non-biased alternative. When I taught about media bias, I knew my students would already have strong opinions on whether or not the media is biased before they ever read scientific research on the subject. They had lived their entire lives with at least one party accusing the media of systematic bias. Parents, other teachers and peers could also socialize students to make certain assumptions about media organizations’ strengths and weaknesses before they entered my classroom.

Social media has made it much easier to spread claims of bias that feel right, even if the evidence is incredibly thin. Two weeks many of the Bernie Sanders supporters I know passed around the following meme:

Is this really enough information to assess a media organization?

Is this really enough information to assess a media organization?

Both headlines were published by The Washington Post within a day of each other. Sanders supporters held this up as yet another example of how the Post is biased against their side. (I rolled my eyes at this whole back and forth.) Unfortunately, Sanders supporters didn’t use weeks worth of news coverage to produce the “WaPo is biased” meme. They didn’t even invite people to read both stories and compare for themselves. They made their comparison based on the smallest bit of information: the headline. By this point, Bernie Sanders supporters had already lost faith in the Post.

One day of news coverage doesn’t make a convincing argument. Every presidential candidate has bad news cycles, just like every pro baseball team loses games. In prior postings on this blog, I advocated for thinking of political media like a conveyor belt. Certain inputs lead to certain outputs. This is how I predicted a Trump style campaign in April of 2015. I knew if a high profile candidate emphasized social identity in their campaign, each statement would get major attention from legacy and new media organizations. The media levers existed in 2008 and were just waiting for the right candidate to pull them.

Most people don’t think of news content like something that comes out the end of an assembly line. They focus on the final story – particularly when they don’t like the final outcome. When people don’t have any special knowledge of the process that leads to the outcome, they make a guess based on their other beliefs. This is what made it so depressing to study news media content while working in a sociology department. Most sociologists never read empirical studies of media processes. Sociologists aren’t all that different from any other group of Americans when it comes to media literacy. And media literacy isn’t that different from any other aspect of social life. People see outcomes, create some explanation for those outcomes, and assume they know more than they actually do.

Twenty years ago, most baseball fans were this ignorant about the game they love. They focused on the easiest to count statistics, like runs batted in and pitcher wins. Statistically inclined baseball fans have always pushed for a greater emphasis on the process that makes a certain result more or less likely. Instead of focusing on a player who got a key hit in a playoff game and inferring they were a clutch performer, baseball fans started looking at whether players with this reputation performed better over a long period of time. They didn’t. Pitchers winning a lot of games say as much if not more about being on a good team than being good at their job.

People made terrible assumptions about baseball when they looked at results alone, and then tried to infer the process that produced their results. Today there is a much larger group of baseball fans who focus on process and try not to get too emotional about what happens in a small number of regular season games. There is no analogy for people brushing off a few negative stories about their favorite politician or political cause, even though everyone in politics will have their bad days. If you distrust a media organization and think it is biased, the only way to know for sure and potentially fix the problem is to push past the outrage and understand the process. That’s what political veterans do.

Up next: Why are the most politically knowledgeable some of the worst at evaluating media bias?


When Vigilance Doesn’t Overcome Bias

Last week, my friend Dan posted a provocative article from The Atlantic. Adrienne LaFrance, a technology writer for the magazine, was worried that she was not representing enough women in her reporting. She first got worried in 2013, so she counted up how many of the people mentioned in her articles were women. It was only 25 percent! LaFrance resolved to do better, but when she audited her articles last year, women were only 22 percent of the mentions. After going through the data, LaFrance argues

Some people would argue that I’m simply reflecting reality in my work. That’s an overly generous interpretation. Another popular reaction is that my job as a journalist isn’t to actively seek out diverse sources, but to find the most qualified people to help me tell the best possible story. I only agree with that in part: Yes, my job is to serve readers by finding the best sources for my stories, but why assume that the best source isn’t a woman? By substantially underrepresenting an entire gender, I’m missing out on all kinds of viewpoints, ideas, and experiences that might otherwise sharpen and enhance my reporting.

Two things struck me from this article. First, LaFrance acknowledges a reality about day-to-day journalism that most readers overlook. When you keep going from one story to the next, it is awfully difficult to take a step back and analyze any kind of pattern in news. This is one of the main reasons I left journalism to go in to academia. Second, people love to overestimate the power that journalists have to shape their stories, even LaFrance herself:

I’m not excluding women on purpose, but I can’t say it’s an accident, either. Reporters choose whom to interview. We carefully parcel out our time as we work toward deadlines. I spent several weeks working on this story about self-driving cars, for instance, and it occurred to me as I was reporting that I hadn’t interviewed any women. In the end, deadline pressure and decisions about what to leave on the cutting-room floor trumped diversity.

LaFrance’s essay is depressing, because it gives the impression that none of her vigilance had an impact on the gender equity in her stories. My friend Dan, who has never been in a newsroom, thought it was shocking. As part of my dissertation, I examined whether phrases dealing with gender were used more or less often online than phrases on other topics from August 2008 – January 2009. I found mainstream reporters’ interest in gender as a topic was mainly a side effect of their interest in everything the presidential candidates had to say. (They had more of an inherent interest in phrases dealing with race.)

Reporters may not have that much control over the gender balance of the sources they quote. Most potential stories have a limited amount of sources who can give enough information to start the story. Someone who wanted to cover the current Republican presidential primaries would have a much larger gender imbalance than someone covering the Democratic primaries. It doesn’t matter how well-intentioned or vigilant the reporter is. The leading Republican candidates are all males. A reporter who gave Carly Fiorina 50 percent of the Republican coverage to fix a gender imbalance would have stood out as highly unusual.

When I covered politics, I chose my sources based on whether they knew things about new policies that could affect my readers. This meant I didn’t have many opportunities to choose sources based on their gender or race. The voters elected their representatives, and those officials chose which political operatives to hire. People who went in to activism made that decision before they shared their activism with reporters. Any reporter who chooses sources this way – maximizing access to factual information – will reflect the inequalities of the beat they cover. When I covered college admissions and affirmative action, I often needed non-officials to provide balance in my stories. Ward Connerly would step away from his day job to talk to any reporter about his crusade to end affirmative action. His job was to be my #1 most quoted source, and he had the financial resources to make his goal viable.

LaFrance’s argument would have been much clearer if she would have thought about the differences between selection bias and presentation bias. It is much easier to understand the problems LaFrance has with her reporting and think about solutions if we separate these issues. Let’s assume a science reporter rarely pays attention female scientists, but any story that actually gets written makes sure to make the scientist look brilliant and emphasizes the relevance of her research. In this hypothetical, the problem isn’t poor representation making female scientists look bad. Instead, we have a hypothetical where female scientists are less likely to be selected for news coverage, yet any female scientists who are selected will be represented with a positive bias.

People often get confused about the difference between selection bias and representation bias, so they talk about staying vigilant and diversity training as a solution to both problems. This type of training could be very useful in local television news, where Black criminal suspects are consistently portrayed differently than White suspects. But selection bias is a question of choosing from available sources. It is very difficult for reporters to go back in time and think of all their options for any particular story. It is almost impossible for readers to know who got left out.

I became an academic to study selection bias. While most of my peers focused on the final news story, I wanted to examine how many choices journalists have, and whether different journalists will tend to congregate on the same choices. I found that mainstream political journalists often do make similar choices, but it’s because they have relatively few good options to choose from. Selection bias is tied to broader issues of power, inequality and communications strategy. If you are selecting stories based on wanting to produce a certain kind of representation, that’s a different kind of media. Unfortunately, the audience that wants to read certain kinds of political representations overwhelmingly favors negativity and outrage. It’s an environment that makes Donald Trump’s campaign predictable for an analyst who can clearly separate selection bias from a crisis of representation.

Fixing a gender imbalance in mainstream reporting will require more than well intentioned reporters.

 

Postscript for academic readers: Academic journal articles are much longer than news articles. There is a wider range of sources who can contribute something – no one scholar holds all the cards like a politician can. Deadline scarcity isn’t really a factor. Put all these factors together and selection bias should be much less of an issue in academia. Therefore, vigilance against representation bias should be a more successful strategy in academia. This may also be why many academics have unrealistic expectations of journalists’ control over their narratives…