Scammers, Camelot, and the demand for fake news

Last week I walked past the single worst Black Friday ad I ever saw. A local clothing store advertised “openning” 10-6 on Thanksgiving. Yikes! Was this the worst Black Friday behavior I have ever seen in my neighborhood? It’s close, but I saw worse at the local Best buy a few years ago. While I was shopping for a laptop bag, a woman was complaining to customer service that she was locked out of her accounts. Apparently someone texted this woman claiming that she had won an award. She just needed to send her banking information to get paid. Of course, it’s a scam. I turned around to look at the low level best Buy employee and saw the quick look of terror on his face. How does he explain that this potential customer just got scammed and there’s nothing he can do to help? How can he be sympathetic to this confused woman walking in to Best Buy with her child instead of wanting to scold her for falling for such an obvious deception?

If you’re an old enough Internet user, you probably remember scams involving “Nigerian princes.” In case you forgot, this was a scam where someone sent bulk spam email claiming to be a Nigerian prince who has to move money offshore due to political unrest. If you give your bank account info, they would wire $10,000 to your account. Most people realized this was too good to be true, even before the scam became publicized and tech firms dedicated resources to blocking these spam e-mails. However, there were some people who desperately wanted to believe there was a Nigerian prince who would make them rich. Selfishness and laziness beat suspicion and careful research. The selfish and lazy might be pretty easy to exploit.

 

If you follow tech news over the last week, you saw headlines that the top 20 most shared stories on Facebook had more fake news stories than real ones. Google and Facebook both blocked fake news sites from their advertising sales networks this week – now that the US Presidential election is over. Facebook has always had an unusual set of “community standards” for regulating content. Visual depictions of violence and sexuality are generally banned. The company frequently claims it has tweaked its “News Feed” algorithm to show “higher quality content” as opposed to clickbait. However, Facebook has always strenuously objected to the idea that it is a media company with a profound influence on journalism. Here’s Mark Zuckerberg, trying to answer questions about whether his company helped Donald Trump win the 2016 election:

“Personally I think the idea that fake news on Facebook, which is a very small amount of the content, influenced the election in any way — I think is a pretty crazy idea. Voters make decisions based on their lived experience.”

I’m not sure anyone really believes this, even Zuckerberg. Facebook acted against fake news sites four days after Zuckerberg’s quote (and after Google’s ban). It’s a field day for people who want to blame Facebook’s lack of transparency – or social media more broadly – for Americans’ declining interest in facts and evidence. As much as people have a right to be frustrated that Facebook didn’t do anything about fake news until after the election, it’s not like Facebook was the birth of online scams. People have tried to use the Internet to try and exploit selfish and lazy users for decades. They used other technologies before the Internet. Instead of blaming Facebook, we should ask why would people want to spread misinformation with their friends and family?

My take here is probably different than most people because I actually covered a secession campaign. In 2002, the San Fernando Valley wanted to secede from the rest of Los Angeles. The secession movement started as part policy oriented and part symbolic. Every public school in Los Angeles County is in one school district. Valley voters wanted to break away from the unwieldy behemoth. They also believed their tax dollars were being used to subsidize the poorest neighborhoods of Los Angeles. On a symbolic level, Valley residents felt like they were taken for granted by a remote city hall. It wasn’t a rural / urban divide like presidential elections. If Valley voters formed their own city of Camelot, it would have been the seventh largest in the country! (Sidenote: Camelot won the vote for what to call the new city.)

By the time I started covering the story, Valley secession leaders had already conceded on their largest policy grievance. Camelot would still be part of the LA Unified School District. I talked to voters who said “what’s the point of seceding if we’re still tied to LAUSD?” Other reporters thought there was no way secessionists could get enough votes. They needed to run up enough votes from within the Valley to get a majority citywide. Secessionists didn’t fully want to campaign on keeping Valley tax revenue in the Valley either. The county ruled that Valley residents would have to financially compensate the rest of Los Angeles if secession passed, to make up for lost tax revenue.

With only a weak policy case, potholes became a major campaign issue! Some Valley residents saw every pothole as a reminder that their area didn’t get a “fair share” of city services. Mayor Hahn dispatched construction crews to smooth over problems, both literally and figuratively. (At this point I am obligated to say I don’t live in the Valley and my street gets enough flooding to become one lane only during moderate rain.) Valley secession leaders wanted voters to feel like City Hall was remote. They also reminded people that most local media organizations were located in the older area of the city and not the Valley, so they were biased against secession. The Los Angeles Daily News – which was based in the Valley – was decidedly pro-secession.

There are several things that make San Fernando Valley secession different than the 2016 presidential election. While the Valley has more Republicans than the rest of Los Angeles, it is still a majority Democratic area. Every voter is urban. The two sides had relatively similar arguments about what would happen if the Valley seceded. Valley secession leaders did minimize the potential disruptions. LA’s black neighborhoods emphasized how they would lose out financially if the Valley seceded, but there were few accusations of racism. In the end, the pro-secession movement was even more based on emotion than Donald Trump’s campaign. Trump promised to make American great again. He made incredibly vague policy promises. Trump’s promises may not be credible. But Valley secession leaders openly said they couldn’t deliver on their initial promises of divorcing LAUSD and keeping all the tax revenue.

A slight majority of San Fernando Valley voters still believed secession was a good idea and voted to leave the city! Why would they believe the promises of the secession campaign, even if all the evidence said secession wouldn’t provide tangible benefits for Valley residents?

• Disrespect: Various activists had discussed secession for decades. They felt they were not given a full share of public services, even though they paid a disproportionately high bill compared to the rest of Los Angeles. I don’t remember anyone counting what percent of potholes were unfilled in the Valley as compared to downtown. “We aren’t getting a full share” may be one of those things that is entirely symbolic and not based on rational calculation.
• If the moral claim rings true, people may turn a blind eye to empirical fact: The facts were not on Valley secession’s side. The county set the terms for secession after certifying that the Valley would be a viable independent city. The terms were not ideal for secession leaders. One balked and abandoned the campaign. The rest held on to their cause, despite mounting empirical facts about how any new city could not do what they wanted to.
• People emphasized moral claims when policy issues were murky: In 2002 the Los Angeles Unified School Board was so dysfunctional that creating a new Valley School Board with new criteria and policies seemed a lot easier than fixing LAUSD. School board governance played a major role in the 2005 mayoral race and Antonio Villaragosa’s first two years as mayor. No one really knew how to solve the giant mess. No one had a good policy idea. Voters may have been quicker to embrace the symbolic politics of secession because no one offered a policy solution to the tangible problem of underperforming schools.

Now let’s think about what we know of Trump voters:

• Disrespect: Definitely. Trump voters tend to say the federal government has forgotten them. The political class may not focus on rural areas. Remember, disrespect is a feeling that may or may not have a basis in fact. So is neglect. It’s entirely possible for multiple groups to feel disrespected by the power structure, and for those groups to hate each other.
• If the moral claim rings true, people may turn a blind eye to empirical fact: Trump offered a wide range of campaign promises, and voters didn’t seem to mind when some of these contradicted each other. In particular, Trump’s tax policies clearly favored the rich. When I think of moral claims ringing true, I think back in the first primary debate. Trump was asked if he would back the Republican nominee no matter what. He refused to say yes. Later on he attacked other Republicans for being in lobbyists’ pockets and bragged about buying influence. The moral claim was very clear: “every politician is a self-serving asshole, but I’m the only one who is honest about being an asshole.” That’s when I thought Trump had staying power.
• People emphasized moral claims when policy issues were murky: A lot of industrial workers are facing downward mobility. There’s a tendency to focus on Trump voters not being the absolute poorest, then discounting any kind of economic argument. But Trump voters are disproportionately older. Regardless of partisanship, most of the parents I have met care deeply about their children’s opportunity to have a good life. I spent my teenage years in an area that transformed drastically from farmland to suburbia. I left for college before the transformation was finished, so I’d go home and my parents were going to new malls that didn’t exist when I was growing up there. All of this was positive economic growth, but it’s still alienating. This affects how I think about declining industrial towns? Has any politician really offered a solution for declining industrial towns over the last 20-30 years? There isn’t a good plan for how to help workers whose skills are less valuable today, or the potential alienation of economic change. We got nostalgia and moralizing about trade instead of real solutions.

I can see reasons why the people who supported Trump would also be likely to buy in to fake news. I can see why people would believe the feelings contained in these stories and want to share them widely.

 

At this point, I think other left-leaning writers would just look down at Trump voters and stop writing. Buzzfeed’s story about fake news getting more Facebook engagement than real news feels plausible. It’s very plausible if you don’t know many Trump supporters and you’re looking for some explanation of how they got “fooled.” I retweeted the story without thinking twice. I didn’t even read the study! A friend of mine who voted for Trump posted a critique of the Buzzfeed study design over the weekend. I read the critique, then read the Buzzfeed methodology. Hate to say it, but Buzzfeed fooled me.

Here’s the short version of what Buzzfeed did wrong. They looked up the top 20 fake news stories and the top 20 real news stories. Top 20 lists are highly unequal. The #1 hit is far more popular than #2, but the gap between #2 and #3 is smaller, etc. One huge hit skews the entire set. Remember how Buzzfeed generated massive traffic by posting a dress where people disagreed on what color it was? To make things worse, fake news should have a natural advantage in this metric. If someone is creating fictional news, it is by definition a unique story. Legitimate news outlets don’t get many exclusives. Let’s say 10 people share a fake news story about a Trump-Clinton debate, 5 share the Washington Post’s lead story, 4 share their B story, and 3 more share their third story. More people shared information from the Washington Post than a fake news site, but the fake news site has the biggest single hit. In reality, the one fake news site is competing against dozens of high profile real media organizations and getting swamped in the total volume of Facebook engagement.

So why would people believe the Facebook fake news story?

• Disrespect: Yes. Democrats’ general election campaign was mainly an argument that Donald Trump doesn’t represent the characteristics we want in a leader. Hillary Clinton won the popular vote but lost the election. Disrespect for progressive values may be an understatement for how progressives feel today, particularly if they are focused on identity politics.
• If the moral claim rings true, people may turn a blind eye to empirical fact: The main people who bought in to Buzzfeed’s fake news story are Democrats who feel big media organizations didn’t do enough to clamp down on Trumpism. Remember how people bought the myth that network TV news was avoiding “issue” coverage – another study based on terrible methods. There that many sophisticated methodologists in the world. I can’t really fault people for not understanding the weird statistical distributions that biased the Buzzfeed study when they were barely mentioned in my years of graduate level statistics classes. (It’s going to take a separate full length post to explain in detail.)
• People emphasized moral claims when policy issues were murky: In this case we need to delete “policy details” and replace it with “research methodology.” It’s easy to do a simple study of media content that compares apples to oranges. It’s much harder to compile a list of potential stories or sources and then analyze how different media organizations treat them. But this is the only way to see how rare breakout hits are and how much more attention they get than any other story or source. The statistical skills needed to analyze these data sets are also extremely rare. When I gave talks at academic conferences, most of the other panelists didn’t know what a negative binomial regression was, let alone the audience. That’s why the burden is on me to explain.

We might all be vulnerable to news that is factually incorrect but feels true. I bought in to a flawed study in an area where I spent a decade becoming an expert on that methodology, because I didn’t bother to check the facts! How can we avoid falling victim to our own weaknesses?

I think the key is to look for potentially disconfirming evidence. I wrote about this at more length yesterday, but here’s the short version. As a person sitting in my apartment writing a blog, I don’t think I have the power to affect the supply of fake, misleading and manipulative political posts. Sure, I could jump up and down blaming Facebook. But here’s the good news. Each of us has tremendous power over our own demand for fake and misleading news. I know exerting this power isn’t easy. It’s tempting to just look for evidence that supports our claims. If you are a debater or a lawyer, you want to make an argument based on the evidence that is the best possible interpretation for your side. Evidence that we might be wrong stinks. Most of the time we don’t want to look for it, and we feel like fools if we actually find it. So why bother looking for evidence that would hurt our side?

In the end, it comes down to a question of whether we want to focus on our own feelings or convincing other people. If you just want to make yourself feel good, there isn’t much of a reason to look for disconfirming evidence. Just remember that someone out there is going to recognize your laziness. If you’re lucky, they’ll just want to embarrass you for buying in to a myth. If you’re unlucky, they’ll see you as a mark, willing to hand over political power in exchange for the right set of feelings. The best way we can empower ourselves is by looking for disconfirming evidence. We can keep ourselves from being fooled. We can’t rely on others to do it for us. At the same time, knowing other belief systems is normally a pre-condition for persuading other people.

Forget “fake” news. Focus on “news” that lacks fact.

In the last week many of my academic friends who don’t study news have gotten a lot more interested in “fake news” as a potential social problem. First there was the Buzzfeed story claiming that the top 20 fake news stories during the last few months of the election got more Facebook engagement that the top 20 “real stories.” (I’ll talk more about that study itself separately.) Then there was the NPR story on a psychology experiment showing that students couldn’t adequately separate more and less credible sources of information. Since most people don’t have direct experience teaching about news media, let alone this new issue of fake news, there’s a lot of things I could share. It’s hard to know where to start. I’m going to write a few separate posts.

Let’s start with the most basic idea of how to stop fake news. Why not just hand students a list of websites and say “these websites are fake!” Google and Facebook blacklisted a set of websites after the election, trying to keep fake news from benefitting from their advertising networks. Giving people a list of websites makes me think of giving someone a fish versus teaching someone how to fish. Is fake political news written by teenagers in Macedonia more important than politicians selectively giving information to manipulate prestigious reporters? What about the growth in political memes that have no facts, only feelings? I think this New Yorker cartoon sums it up well:

 

Caption: “I’m sorry, Jeannie, your answer was correct, but Kevin shouted his incorrect answer over yours, so he gets the points.”

I don’t know if the cartoonist is aware of the irony here. There’s a big banner saying “FACTS DON’T MATTER.” There’s a caption that captures part of the reason progressives resent Donald Trump and his electoral victory. But there are no actual facts in this political cartoon. We don’t know what Jeannie or Kevin said so we can judge the answers for ourselves. Of course, giving facts is not what political cartoons do. They try to present clever mockery. I understand why the cartoon feels true. But the key to distinguishing fake news and other kinds of emotional manipulation is being able to separate literal fact from arguments that feel true or symbolize truth without containing verifiable facts.

Just to give a preview of where I’m going, every style of telling stories has strengths and weaknesses. For my final in Sociology of Mass Communication a year ago I asked students how they would write about the Trump campaign. Why did they choose that style? What are the strengths and weaknesses of that approach? If a student couldn’t talk about the negative ramifications of their decision, they couldn’t get an A. I knew that students could think about the cons of each approach, they could walk out of my class knowing how people may try to use that style of story telling to manipulate them. Sometimes it’s politicians using the media. That’s been one of the main themes of this blog in the past. But since people are mainly worried about fake news today, I’m going to focus more on media producers trying to manipulate our emotions.

I didn’t teach about “fake news” per se when I taught sociology of mass communication a year ago. I’d only change one thing if I taught again this Winter or Spring, and it’s something much broader than fake vs. real news. Fake news is just one kind of manipulation. The New Yorker cartoon is another. Donald Trump dragging out the hot takes over Mike Pence getting booed while watching the play Hamilton is a third kind of manipulation. I’m less concerned about the differences between flat-out lies, tactical half-truths and people relying on emotional arguments because they don’t have facts or evidence to back up their claims. Each form of manipulation can be resisted by asking the same set of questions instead of taking content at face value.

1) How much does the article rely on factual claims or evidence? Some articles are entirely descriptive, trying to relay a set of facts for an audience who didn’t directly see what happened. Other articles rely heavily on moral claims and interpretation rather than empirical evidence. That’s a different type of argument and we’ll get to it in a bit. Traditional interview-based journalism offers quotes as a kind of evidence. Direct evidence from documents is rare.

2) If someone relies on factual claims, where does the evidence come from? Can you clearly trace how evidence traveled from the original source to intermediaries to your brain? For example, reporters interview people and quote them. We don’t know what the reporter decided to quote and what was left out. But we can be fairly certain that the person being quoted said those literal words. Then we can evaluate the reputation of the person being quoted. We can also evaluate the reporter’s reputation. If we can’t clearly trace the flow of information from one person to the next, they may want to hide something. Whistleblowers need anonymity for protection. However, a wide range of political operatives seek anonymity to promote half-truths and misinformation.

3) What emotions is someone trying to convey? Are they trying to make you feel a certain way about things? Some writers and meme creators’ goal is to convey a certain set of feelings. Think of the New Yorker cartoon: it wants to convey outrage. Most newspaper and network TV reporters work very hard not to convey any of their own emotions about the stories they are reporting on. This kind of stoic emotional restraint is pretty rare in other kinds of storytelling. My goal is make sure we all pause and consciously understand where a writer is coming from and what they want us to feel. I don’t want to get my emotions pushed around by anybody, even people I tend to agree with. Any time someone is giving us a set of feelings that we want to believe, we are at risk of not checking their facts and evidence as closely as we should.

4) How honest and up front is the writer about why they are doing what they are doing? I actually haven’t taught this before, but I think it’s an important follow-up to question 3. Every argument has assumptions. If someone is making an argument, how much are they willing to clearly state “here are my goals and my assumptions.” When someone is giving an interpretation of evidence, do they explain why they gave this interpretation? Do they acknowledge other potential explanations and make a case for why their interpretation is better? If someone can’t acknowledge that other interpretations exist, it’s probably because they can’t make a good case for why their interpretation is the best.

My own sense from spending my entire adult life working in or studying journalism is that it’s hard for a writer to excel at giving both factual evidence and feelings. I think it’s particularly hard to combine the two when writing about politics, since the main feelings people convey are moral outrage and judgment. It’s much easier for me to try and combine facts and feelings when recapping a baseball game than in writing this post. That’s why there are tradeoffs. No writer can be consistently good at everything. No one is perfect.

When I teach about journalism, my main goal is to get students to acknowledge these tradeoffs, then ground them in specific examples. Elite media organizations that have access and avoid reporters’ personal judgments tend to defer to sources in order to protect access. When Trump lied on Twitter about the popular vote, many leading news organizations copied his claim in the headline without any critical skepticism. Large news organizations tend to fear an inability to prove an elite source is lying more than they fear publishing an elite’s statement that is probably false. Nixon campaign aides first took advantage of this in 1968 (see Crause’s Boys on the Bus), and it’s been a staple in political operatives’ playbook ever since.

Writers who emphasize moralistic takes and emotion have more incentive to hide, selectively misinterpret or fabricate factual evidence. Someone who really wants to convince me that Trump voters are all racist isn’t all that likely to bring up other reasons why they support Trump. On the other hand, Trump voters who are not explicit supporters of the KKK are trying to emphasize all the non-racial reasons why someone would vote for Trump. Three weeks after the election and I still see both messages from friends, like it was the day after the election. That’s why things like the New Yorker cartoon stick out to me. They encapsulate outrage and victimization, but are not going to persuade anyone who doesn’t already have those feelings deep in their heart before reading the cartoon.

Being self-conscious about what we want from news is hard. I think it has always been hard. We don’t want to acknowledge that every genre of media is imperfect. People who want all-facts news may not want to acknowledge how reporters can be manipulated by powerful sources. People who want a certain set of emotions, moral stance or political ideology may not want to acknowledge there are times they put feelings before facts. Trump exploited this his entire campaign, skewing this election almost entirely towards emotion. Remember that Democrats’ main campaign theme was that Trump was emotionally unstable and personally unqualified for office. We took the bait instead of focusing on a positive message. It’s easy to see something on social media, get agitated, and react right away. This is a more pervasive and bipartisan problem than “fake news.” For another example of what makes this so hard, let’s take the following passage from the end of an article Dara Lind wrote for Vox:

Journalists have long been sensitive to the prevalence of misogyny on social media. In 2016, they’ve become alarmed by anti-Semitism on social media as well. Journalists know and work among women; they know and work among Jews.

Many of them don’t know and work among many people of color. The amount of attention paid to racism on social media (or in real life) among journalists is, accordingly, often disproportionately small — or delayed.

Let’s think about how to factually evaluate the claim that journalists pay more attention to misogyny and anti-Semitism than racism. We could try to construct a database with a list of misogynist incidents, religious bigotry and racism, and then construct a second list of writers and what they wrote about. How often did a particular group of writers tackle a particular topic? This is incredibly difficult, painstaking work. I spent years working on a project like this dealing with media and blog posts from the 2008 election as part of my dissertation. In the end, I could produce criteria for defining statements on race, gender, religion and a large number of other topics. I could produce data on how much a particular set of news organizations preferred or dispreferred phrases on each topic, relative to any other topic.

Even if I provide facts, there is no factual basis for saying whether enough attention is being paid to racism or misogyny or any other issue. The current level of attention is a fact. The ideal level of attention is a feeling. It’s a moral stance. I chose this passage from Lind’s somewhat unrelated article because it does a great job of showing how 100% fact based story telling only goes so far. A description of how much attention is currently being paid to race with no moral claim about how much attention should be paid to race will be unsatisfying for many audiences. Trust me, I have the negative reviews to prove it. On the other hand, moralistic claims that lack evidence are unsatisfying for a different audience. Lind claimed that journalists pay a disproportionately small amount of attention to racism. The rest of the article doesn’t give any additional evidence to support this claim since it is largely focused elsewhere. Unfortunately, that means I have no idea if Lind is trying to manipulate me or not.

If you want to teach critical thinking about media content and not just give students a list of fake news sites, you have to empower students to offer different moral priorities than you. I know some students in my last class liked outrage news a lot more than I did. I assume some of my readers will like it more than I do. That’s fine. My students also liked Buzzfeed’s day-to-day content a lot more than I did, while I wrote lectures while listening to three hours of college football podcasts. I made a point to illustrate a few of my eccentric non-political preferences to set a tone that I didn’t expect them to copy my political, moral preferences either. My goal is to help people make better informed choices about what media we read so we can be aware of what we are not getting. I want us to be able to protect ourselves from all different kinds of deception in the news – particularly from writers we tend to trust the most.

Leaks, Scandals and Gossip

When I was seriously considering a career in journalism, I got a surprising amount of pushback from my dad’s side of the family. They weren’t worried about the bad hours or the low pay. Instead, my dad and grandmother kept telling me stories about my great grandmother. She worked for the Democratic Party in New York City. For a while she also had a high ranking position in city government. Her position came with perks. Every Thanksgiving my dad told the story about how his grandmother was able to snag some of the best tickets for the Macy’s Thanksgiving Day Parade for him and the rest of the family!

My great-grandmother also had some influence over who the city hired for civil service positions. Grandma said “she was just helping good people from the neighborhood who were out of work.” The New York Times saw it differently. A reporter finished a story on corruption at City Hall. When my great-grandmother found out about the story, she had a small heart attack! She lived, but those dastardly investigative reporters from the Times ended her career. I never met my great-grandmother, but the rest of my family still celebrates her legacy. Her first name is the only name in the family to be passed down from one generation to another. When I started doing investigative journalism, my grandmother gave me a guilt trip like taking a side against the rest of my family.

During my time as an investigative reporter, I once received documents showing that Los Alamos Nuclear Laboratory was missing “power generators” along with a wide list of other equipment. Los Alamos is the birthplace of the atomic bomb. Two whistleblowers leaked documents showing a combination of unaccounted for equipment and personal purchases made with lab credit cards. In other words, some employees were using our tax dollars to buy themselves expensive grilling equipment. The whistleblowers blacked out any classified information then sent the documents to a government watchdog group, who sent them to any reporter who asked.

These whistleblowers show why journalists give anonymity in certain cases. Los Alamos found out who was leaking the documents and fired them. But investigative stories continued after I pointed other reporters to a suspiciously “coincidental” retirement of the University of California Vice President in charge of running the national laboratories. Once the scandal grew the whistleblowers were re-hired to help clean up the mess.

Reporters prefer having on the record sources, so a source that can only be anonymous should have access to more newsworthy information than on-the-record sources. If two events are equally newsworthy, the one that has on-the-record sources is far more valuable. However, a wide range of sources have realized they have unique access to information. They have the leverage to demand anonymity, even if they won’t get fired for speaking publicly. Sports may be even worse than politics in this regard. Art Briles’ agent is reportedly trying to plant rumors that high-profile schools are looking to hire Briles as football coach within a year of him being fired at Baylor for his part in a massive coverup of sexual assaults. Many reporters are perfectly willing to put Briles name in to a list of possible coaching hires, writing stories based completely on rumors and innuendo.

I’ve always had a pretty high standard for anonymous sources. I wanted them to provide documents, not just claims. During California’s 2002 statewide election I was put on a team to report live from the Democrats’ reception in a Los Angeles hotel. Democratic incumbent Grey Davis was deeply unpopular but many Californians saw him as the least bad option. Crowd reactions were incredibly dull and muted. What else could we expect from the lowest turnout election in state history? The night dragged on as Davis couldn’t build enough of a lead to go on stage and declare victory. (Davis won with 47.3 percent of the vote as over 10 percent of the voters went with a third party candidate. He was recalled a year later.)

I was incredibly bored covering this event, so after a while I started talking with two young men holding Corona bottles. They were right at the front of the room and actually hid the empty bottles under the stage. I didn’t see anyone else with bottled beer, so I asked where they got it. They said they got it from lieutenant governor Cruz Bustamante. He was allegedly sharing beers with supporters and didn’t bother to check whether said supporters were of legal drinking age. I rolled my eyes a bit but kept talking to them. There was no way for me to verify whether these college kids were telling the truth. Even if they were, giving a beer to consenting 20 year old isn’t much of a scandal. It’s salacious. People are interested when I tell this story years later. But Bustamante’s behavior was not malicious and would not affect the public. It’s gossip, not news.

When I see headlines about hacked DNC e-mails, I usually just shrug. Here’s one headline currently on the Washington Post: “Hacked e-mails show anxiety about Clinton Candidacy.” The lead quote is “Right now I am petrified that Hillary is almost totally dependent on Republicans nominating Trump.” I can see why people would think this is some sort of juicy tidbit. But campaign operatives wouldn’t be doing their job unless they tried to calculate the best general election opponent. If you have enough deeply committed operatives, one of them is going to send a panicked e-mail eventually. The New York Times ran a story a few days ago with the headline “Hillary Clinton’s Campaign Strained to Hone Her Message, Leaked Emails Show.” The actual e-mails were surprisingly generic. Most campaigns have to work hard to calibrate their message to get the maximum number of voters.

Reading these stories reminded me of students who would do a lazy cut and paste from lecture slides instead of synthesizing the material. Take the Times‘ story. Is there any evidence that the Clinton camp’s struggles are somehow different than other campaigns? Are these leaks just a rare chance to see what political operatives do on a regular basis? It’s easy to copy and paste the salacious bits without giving much context. People will click on the story because it seems juicy. But the salacious bits are just empty calories. The meaty story is in explaining whether or not the Clinton camp is unusual.

A friend asked me whether journalists should treat these leaked e-mail stories differently because there is evidence these e-mails come from Russian hacking. I was with my dad when the big wave of e-mails about the Democratic National Committee were leaked days before the start of the Democratic National Convention. Which is more important: the head of the DNC tried to help Hillary Clinton or Russian hackers were trying to sew discord among Democrats? My dad canvassed for Bernie Sanders so he was outraged over the content of the e-mails. I recognized that a government hacking the e-mails of a foreign political party in order to embarrass them represents a new frontier in international conflict. Every election has endorsements and professional operatives favoring certain outcomes. We already knew Clinton had an unprecedented landslide of insider endorsements. The specter of foreign hacking is new.

My father and I agreed to disagree on which story angle was more important. I was pretty sure he was being manipulated by foreign hackers. He didn’t really care if he was being manipulated, because he was already so convinced that Clinton was a poor choice for the Democrats. The bar for publishing these emails was already far too low. Most of these stories are salacious empty calories, regardless of how reporters got the information. There’s little corruption or malice, just embarrassing gossip. But people like reading gossip. Smart manipulators know this, whether they are foreign hackers or unscrupulous football executives. I’d treat the Wikileaks email stories differently because hacking, spying and cybersecurity are much more interesting and important stories.

Debate Postgame Follies

After a night of reflection, most people who watched the second presidential debate seem to agree on which moment stood out the most. Moderator Martha Raddatz asked both candidates how the campaign has changed them, and specifically asked Donald Trump if he is a changed man since the 2005 tape where he talked about sexually assaulting women. In case you missed the debate and don’t want to watch all 90 minutes, here is the relevant eight minute segment courtesy of C-Span. Trump dismissed the tape as “locker room talk,” shocking a wide range of male athletes. Hillary Clinton criticized Trump’s broader record of denigrating groups of people. Then Trump tried to pivot to Clinton’s e-mails:

Trump: “If I win, I am going to instruct my attorney general to get a special prosecutor to look into your situation, because there has never been so many lies, so much deception. There has never been anything like it, and we’re going to have a special prosecutor.

[snip]

So we’re going to get a special prosecutor, and we’re going to look into it, because you know what? People have been — their lives have been destroyed for doing one-fifth of what you’ve done. And it’s a disgrace. And honestly, you ought to be ashamed of yourself.”

And then Clinton’s reply:

Clinton: It’s just awfully good that someone with the temperament of Donald Trump is not in charge of the law in our country.

Trump: [interrupts] Because you’d be in jail.

American televised presidential debates began in 1960. We have seen candidates who hate each other before. But there is no precedent in the United States for one candidate pledging criminal charges against their opponent if they happen to win the election. We’ve seen it in Ukraine, Congo and other nations conducting some of their first votes, but never in the United States. Some commentators immediately argued this was the most important part of the debate. On the other hand, CNN didn’t even mention this exchange in their first 20 minutes of post-debate coverage. What’s going on with CNN?

I happened to watch CNN’s post-debate coverage last night. They started with an A team of three correspondents giving their first impressions of the debate from the event floor. Then they went to a B team of pundits in a studio for their first impressions. After around 25 minutes they went back to the A team. Dana Bash was now focused on Trump’s desire to put Clinton in jail. The other commentators agreed that this was unprecedented in the United States.

Before the first presidential debate, Nate Silver said on his podcast that everyone should have to wait at least 30 minutes before giving any kind of on camera post-debate analysis. He has lived up to his word on FiveThirtyEight’s post-debate podcasts, even though he and his fellow podcasters know they will be losing members of the East Coast audience who can’t stay up after midnight for their podcast to finish. If CNN had waited 30 minutes before giving any post-debate analysis, they probably would have led with this threat or Trump rejecting his running mate’s views on Syria. (As of writing this, Trump rejecting Pence is one slot higher than threatening to jail Clinton on CNN’s website.)

Summarizing debates is hard enough as it is. A lot of things happen in those 90 minutes. Even if a debate is unlikely to change someone’s vote, we all have to sit down and prioritize what was the most important thing that happened, second most important, etc. I’m glad I never had to sit down and immediately crank out a story where I had to make those calls as a professional journalist. Among other things, I’d be very cranky about “style points” since I lost a lot of high school debates my freshman year strictly on “style points.”

Over the last year I’ve gotten re-acquainted with having to write on deadline, since I was recapping baseball games. Every sports game has a clear winner and loser. However, there are still some games where it is difficult to prioritize which specific play or strategy led a team to victory. Debates are much harder to summarize because different people may legitimately have different things as their top priority. People who care about Trump as a Republican standard-bearer may focus on any Trump statement suggesting internal dissension. Women who have survived sexual assault may place the greatest emphasis on Anderson Cooper saying “You described kissing women without consent, grabbing their genitals. That is sexual assault. You bragged that you have sexually assaulted women. Do you understand that?” and Trump needing considerable prompting before he could claim he didn’t assault anyone. People worried about Trump violating the norms of American democracy may focus on his desire to jail Clinton.

It’s perfectly reasonable to pick out any of these moments as the most important thing to happen during last night’s debate. This isn’t an exhaustive list either. I just have to start somewhere. We all do. There’s a reason why the Washington Post put six stories next to each other at the top of their website a few hours after the debate. No single story can explain all the important things that happen during a debate. Of course there’s no way CNN commentators could do a good job going live right after the debate ends.

Sports Officials Get Outside Help. What if we Gave it to Debate Moderators?

A few weeks ago I posted about how moderating a debate or candidate forum is a losing proposition. It’s almost impossible to end with over half the audience happy with your performance. Moderators who challenge a candidate will be scorned by that candidate’s supporters. Moderators who don’t challenge candidates may come off as meek and lose face with partisans on both sides. Challenging a candidate tends to hurt the moderator’s future career, so I wouldn’t expect a lot of fireworks coming from Lester Holt later today.

Since I wrote that post, both campaigns have argued over whether the moderator should act as a fact checker. Trump laid some of the groundwork last week by claiming that Holt is a Democrat, even though he is registered as a Republican. Here’s Robby Mook, Clinton campaign manager, appearing on ABC’s This Week:

“All that we’re asking is that if Donald Trump lies, that it’s pointed out. It’s unfair to ask that Hillary Clinton both play traffic cop with Trump, make sure that his lies are corrected, and also to present her vision for what she wants to do for the American people.”

As we might expect, Trump campaign manager Kellyanne Conway disagreed:

“I really don’t appreciate campaigns thinking it is the job of the media to go and be these virtual fact-checkers and that these debate moderators should somehow do their bidding,”

Historically, the question of whether debate moderators should be aggressive fact checkers was not a partisan issue. Candidates and journalists favored staying out of the way and letting the candidates be the story. Janet Brown, head of the committee that organized the presidential debates, endorsed this view on CNN yesterday. For this election cycle, many journalists and pundits have argued Trump requires special rules (see this Slate interview with the executive editor of the New York Times, then see Jay Rosen here for a longer version of that argument). As a moral issue, I have favored moral aggressive fact checking since I first got the right to vote in a presidential election. However, I also felt confident in my ability to evaluate candidates’ ability to tell the truth without relying on the moderator.

The more time I spend studying journalism and then watching sports in my free time, the more I doubt whether any moderator could meet my fact-checking expectations. In the last week I saw a pitch right in the middle of the strike zone get called a ball. That umpire faced an easy, objective, technical call and got it wrong. I went to UCLA, which means I have seen a lot of Pac-12 sports. If you’re a college sports fan, you won’t be surprised that when I typed “pac 12 refs” in to Google the first auto-complete is “are the worst.” The conference’s officials are notorious for baffling and inconsistent interpretation of the rules for football and basketball. Then again, even the best officials in the sports world make mistakes. Why do we expect debate moderators to be perfect?

When I was a reporter, I made a bunch of mistakes in interviews. Sometimes I caught people lying to me right away. Sometimes I had to look things up afterwards. There were a lot of times when I looked back in my notes and didn’t have as much material as I thought I did, and I really wish I could have followed up on things. On a national stage, with less cooperative sources in Donald Trump and Hillary Clinton, catching all the lies is much harder.

As much as I care about sports, I think the risks of a “blown call” in live fact-checking of a presidential debate is much more serious. Sports leagues have universally adopted the use of supplemental off-field officials as a way to get calls right. Professional reviews from the league office help insulate the on-field officials from hostile crowds. It seems absurd to expect Lester Holt or any other moderator to do the entire job by their lonesome, with no help. Marvel’s latest superhero couldn’t achieve that feat, let alone a real person.

Simulating How a Jury of Fact Checkers Would Work

Since I am also a stats person, I decided to run a few very simple simulations to try and explain why moderator error is a bigger risk than people realize, and how a large jury of outside fact checkers could solve the problem. Let’s assume that if Clinton and Trump talked forever, they would each say 10,000 things that are fact-checkable and false. It’s probably best to call them the Clinton lying bot and the Trump lying bot, because this isn’t a simulation of how often candidates lie. This is a simulation of how well moderators could catch lying and what could happen when moderators are imperfect. (We could call them the ice bot and the fire bot instead of naming them after candidates; it makes no difference to Stata.)

I started by creating an aggressive, skilled, courageous moderator. This moderator will roll a six-sided die every time one of the candidate bots lies to them. On a 1, they don’t notice the lie right away. On a 2 through 6, they challenge the lie the next time they get to speak. No, I don’t expect a debate moderator to do any better than this while they also have to think about how to fit a large range of topics in to a small time frame.

(Sidenote: The limited time frame is another strong and largely unmentioned issue in the current fact checking debate. Political journalists will stop following up on a particular topic once it becomes clear that the president will not give a direct answer on topic X, because they might give good answers on topics Y and Z.)

Anyone who plays tabletop games or knows basic probability can guess what happens in the simulation. The aggressive moderator caught nine of the Trump bot’s first ten lies, and eight of the Clinton bot’s ten. Over the first 30 statements, this moderator is catching 83.33% of Clinton bot’s lies – the predicted mean. However, they caught 87% of Trump bot’s lies in this period. Over the full dataset the aggressive moderator would stay just as aggressive (it’s a simulation, not real life), catching 83.19% of Clinton bot lies and 83.27% of Trump bot lies.

Next I created a moderator who is really bad at fact checking. Maybe they have a very high threshold for challenging a politician. Maybe they really want to fact check but can’t focus on what a candidate is saying right now and the next question all at the same time. Either way, this moderator still gets to roll a six-sided die for every lie, but they only challenge the lie on a 6. This moderator challenged three of the Clinton bot’s first ten lies, while only challenging one of the Trump bot’s first ten. By thirty observations the poor moderator is catching one out of every six Clinton bot lies, but is still stuck at catching only one of ten from Trump bot.

Let’s imagine Lester Holt misses a lie during the real debate. Maybe he catches some but not others. There is a limit to how many lies a candidate can tell in 90 minutes – they are long winded and repetitive speakers. I wouldn’t expect a large enough sample of lies for a moderator’s forgetfulness to balance out. What are the chances that partisan audiences will tweet “Oh Lester Holt just made an innocent mistake. Things happen. Nobody is perfect.” I’m going to pan over to Holt’s colleague Matt Lauer and say the chance of Holt getting a pass is zero. We have no idea what’s going on in a moderator’s head. We don’t know if the failure to challenge a presidential candidate is an innocent mistake or a more serious attempt to influence voters. And I’m not sure we care, because even an innocent mistake can have real consequences.

What would happen if we had a room of 100 good fact checkers? I ran several simulations creating 100 fact checkers for each of Trump bot’s lies and Clinton bot’s lies. To start with, I rolled a six-sided die to set each fact checker’s evaluation for each political bot’s 10,000 lies. What I want to do here is show how different juries would make sense of those impressions and whether they would buzz the moderator saying “this response is a lie, you MUST follow up!”

Let’s assume we had a jury full of good moderators who catch a lie with a 2-6 on their die roll. With this large a group none of the 20,000 total lies in the database was red flagged by my entire group of fact-checkers. However, the crowd can pick up an individual’s mistake. Every statement was red-flagged by at least 66 fact checkers. If we could find 100 great fact checkers, they would be far superior to any individual moderator trying to fact check in real time. The converse is also true. If we got 100 of the bad fact checkers together, each needing to roll a six to catch the lie, they would never agree on whether to buzz the moderator.

In the real world, a lot of fact checking watchdogs are politically motivated. So let’s assume we have a fact checking jury of 25% Clinton supporters, 25% Trump supporters, 35% good fact checkers, 15% bad fact checkers. For this simulation the partisans will call out the opposing bot if they roll a 2-6. I also decided they would call out their own bot on a 6: partisans may hope a second question pushes their bot to a more acceptable answer. In this scenario the median is 57% of the jury detecting a lie. If it only took a simple majority to buzz the moderator and demand a follow up, this jury would be effective 95 percent of the time. If the jury acted like they had to break through a filibuster in the Senate, random error would be a much bigger issue. This partisan jury would start by buzzing in for three Clinton bot lies but only one Trump bot lie. After 300 statements the odds even out, but that’s a lot to ask.

I thought an ideal situation would be having a range of debate jurors. I made one last room with 15% dedicated Clinton supporters and 15% dedicated Trump supporters. Then I made another 20% who leaned to each candidate. They buzz in for an opposing bot’s lie on a 3 through 6 and their own candidate on a 5 or 6. The jury also has 15 percent good moderators and 15 percent bad moderators. It turns out the balanced debate jury was also the most unstable in simulations. The median result was a 50-50 deadlock. At this point it becomes rather philosophical. For debate juries to work better than a sole moderator, the key appears to be packing the jury with people willing (if not eager) to challenge both candidates if and when they distort the truth.

Why Journalists Fail

The first time I ever supervised workers could have been a disaster. I was hired to train and supervise a group of undergraduates who were reading news articles and then answering various questions about the content. It’s a common research method in communications, with well-established training protocols. You go over expectations, then give everyone a few articles to review individually. The supervisor (me) looks over initial work to see how well the new employees understand what they’ve been asked to do. I got a group of undergraduates who couldn’t even agree whether the article was printed on the top half of the page!

How could half the students get this question wrong? A lot of the questions in content analysis are subjective. For example, is this story portraying someone negatively? That’s why communication scholars require multiple coders to review the same article – they want to see how much coders agree. This story was printed at the top of the page. Every coder should be able to say so! Was I stuck with a group where half the students were lazy? Stupid? Unable to follow simple directions? Was I going to have to tell the professor supervising the project that he needed to fire some people?

The professor wanted me to meet with the undergrads as a group before I gave any additional individual feedback. I didn’t know the team, so I decided not to start the meeting by yelling and screaming. Instead, I asked them what their experience was like doing the work and answering questions in the online form we provided. The undergrads immediately brought up several problems with the form. Almost every question was on one long page, so they had to keep scrolling down. No one knew how to answer “is this coverage positive or negative?” for the story that contained both positive and negative portrayals of the featured politician. After 20-30 minutes I asked about the “is this story at the top of the page?” debacle. Everyone said yeah its at the top of the page. Then one student pointed out this question was one item in a series of checkboxes. It wasn’t a mandatory yes-no question, so they completely missed it.

By the end of the meeting I was convinced that these undergraduates all worked hard, but they were put in a position where dedication was unlikely to pay off with good work. I told the professor that the problem was with the form, and we came up with a long list of changes. The team’s attitude was a little worse for the second round of training – it’s hard to have as much enthusiasm the second time around – but they did much better work since now we put them in a position to succeed. There’s an important story here about how to lead and inspire employees to get the most out of them, particularly when working with a new team. (Of course, there are also times when it’s important to crack the whip, but I don’t want to throw anyone under the bus here.)

I think this story is also a powerful metaphor for how we tend to think of journalism. The easiest thing to do is to look at journalists’ final work product and then pick apart the failings. It’s easy to get mad because most journalism will have imperfections. But does this mean journalists themselves are lazy, stupid or immoral? Are journalists put in a position where it is hard to succeed? To see a columnist wrestling with both possibilities, let’s turn to Jim Newell of Slate discussing Trump’s press conference a week ago at the opening of Trump’s new hotel. Reporters expected direct Q&A about whether Trump was going to renounce his claim that Obama was born outside the United States, but got something different:

“It was about 10 minutes in, after two or three introductory speakers and an enthusiastic plug from Donald Trump for his new downtown property, that the cry of Admiral Ackbar began sounding in the core of my being. It’s a trap. I’m an extra in a bad commercial.

[snip]

Things went very differently. The press conference proved instead to be Trump’s troll of the media, a rick-roll—as everyone called it later—on the grand scale. It was effortlessly brought off and all it required was a manipulation of media incentives and cable news control-room politics, plus a carefully arranged use of space and taxpayer-funded security detail. You can have all your earnest thinkpieces about false balance and the like; Trump’s event on Friday was enacted media criticism.”

It’s the last line that gets me. Most of Newell’s piece is about the Trump campaign’s strategy and how they took advantage of a predictable opponent. Media criticism generally implies moral arguments about how journalists are doing a bad job and should have done something else. My friends who shared Newell’s story portrayed the event as a journalistic moral failure. But Newell’s main point is that journalists were set up to fail. He, along with every other journalist covering the event, eventually realized it. Then they got angry and tried “small” measures of revenge. In my experience this revenge isn’t always “small,” – if Newell is like I was then he’d want to do more to Trump’s team than just write this column.

As I’ve implied in previous posts, its very hard to get people to think strategically about journalism. Audiences want to see bad journalism as a moral failure, not a strategic failure based on journalists’ limited access to information. If reporters knew what Trump was going to do, they’d exclude him. But since there was a chance that Trump would give a major story, every national reporter had to take a chance on him. For all we know, if most media outlets anticipated the Trump trap and stayed away, Trump may have gone ahead with a straight-forward news conference to shame the media. It’s easy to put journalists in a bad position, particularly when audiences will just read the news story and then blame the media.

Maybe it’s easier for me to accept when journalists lose a competition with politicians because I’ve lost many times. Maybe it’s because I’m a baseball fan, and I know even the best major league baseball team loses at least 1/3 of its games. The other side has pros too, and sometimes they win. That’s why I taught students to look under the hood and think about how people work, not just the final product. If you want better journalism, don’t just yell and scream about it. Think of ways to put journalists in a better position. I think one of the most effective options would be to pay more for higher quality content – essentially paying journalists to be selective and not to write up every minor event to fulfill quotas and generate hits.

Of Course Matt Lauer Failed

Once I got more experience as a teaching assistant, I stopped asking professors to give me examples of an A paper. I started asking “what gets a B?” instead. In other words, what are the pros and cons of an average student’s paper? I knew most professors had similar ideals for A work but many of them couldn’t articulate the pros and cons found in the average student’s work. If we talked about the standards for average work in advance, it was much easier to evaluate students throughout the spectrum.

I haven’t been a teaching assistant for years, but I thought about these different ways of evaluating people’s work when I started reading reviews of Matt Lauer’s performance as moderator of the Commander-in-Chief forum on Wednesday. If you missed it, James Hohmann of the Washington Post has a good summary of all the reactions. People have an ideal of what debate moderators should do. Lauer is the inverse of professors agreeing on an A. Everyone has different standards for an ideal moderator but they agree that Lauer wasn’t it. Maybe it’s because I spent so much time grading papers, but I have a hard time grading Lauer on a simple pass/fail scale. So if you had to give Lauer a more specific grade, how many points would you give him out of 100?

It’s a weird question, I know. Unless you review media podcasts you probably don’t think about giving journalists a numerical score. People normally think of journalists and media organizations in binary terms (trust / distrust, like / dislike) or ordinal terms (favorite, second favorite, etc.) I’m asking what score you would give Lauer because there are several presidential debates coming up, so it would probably be good to define what counts as a “passing” grade for moderating a debate. If Lauer didn’t press Trump when he lied about always opposing the Iraq War, but he didn’t make any other mistakes, would you have given him a passing grade?

 

Moderators are graded very differently than other forms of journalism. Part of this is because debates are live. I used the delete key several times while writing this sentence, but live television has no delete key. I once worked at a local television station that declared someone died in a fire, went to commercial break, and then announced that person was alive. Accidentally announcing someone’s death is an embarrassing mistake, but still fixable. If Trump lies during a debate, and the moderator doesn’t challenge him immediately, it’s much harder to apologize to the audience and then go back to challenge Trump after the commercial. It’s impossible to grill Hillary Clinton over her e-mails then ask a softball question about them. Major news interviews, press conferences and debates are created to be a performance. We judge people on whether they can perform in the moment.

I read hundreds of examples of journalistic theatre for my dissertation research. Journalists have been evaluating performances in these presidential events for as long as they have been on the record. Both the theatre and the literal words people said are potential news. Writers can separate the two in to different stories. Television pundits do not separate the literal words from the performance in post-event coverage. (This has always been controversial.) It’s not unusual for a moderator like Matt Lauer to become one of the many stories after a high profile news performance.

Remember that when we read a news story, the interaction that took place between journalists and their sources is usually hidden, so we can’t really form an opinion of it. My favorite example of these negotiations at a high profile level is Bill Clinton answering questions about Monica Lewinsky, a fundraising scandal, and other domestic topics while flying over the Amazon on Air Force One. He wanted to “avoid being hammered by domestic questions” in his press conference with Brazil’s Prime Minister later that day (ABC’s Good Morning America, 10/14/1997, 7 AM broadcast).

Would you be critical of this kind of backroom deal leading to one on-the-record soundbite in a short story? These agreements are the engine that makes day-to-day reporting work. We don’t know much about these negotiations because even if a researcher gives everyone anonymity observing the interaction changes it. When Matt Lauer is interviewing the two main presidential candidates live, it’s a rare theatrical event that draws our attention. We expect journalists to do a more active job moderating the discussion because it’s the rare chance we actually get to see how they ask questions as they ask them.

 

It’s easy to look back on Matt Lauer’s performance asking questions and point out mistakes. Most pundits used this as a jumping off point to talk about what they thought the ideal moderator should do, what they should avoid doing, and how poor moderators could hurt the public. It’s important to discuss ideals. However, we know people’s definition of an “ideal” debate moderator is extremely subjective. I doubt I could sit down with my neighbors and reach a consensus on the appropriate number of questions to ask about particular topics. It’s even harder to figure out how hard to push a candidate who is giving a deceptive or non-responsive answer. One person’s “holding candidates accountable” is another’s “biased moderator hijacks debate.” Candy Crowley became a partisan lightning rod after challenging Mitt Romney in 2012.

As someone who taught sociology, I’m far less concerned about Matt Lauer’s individual performance than pundits. I don’t care if you want to give him a 55 or a 35 or a 0. They are all failing grades. What I am concerned about is the institution of journalists moderating candidate forums and debates. Can any journalist do well enough to get a passing grade moderating a debate in our current system? It’s easy to assume that Lauer is an individual incompetent and we just need to replace him with a better moderator. However, debates are such rare events that it’s easy to forget what the baseline is for an “average” moderator as we hold out for perfection. We forget that most debate moderators are closer to Lauer than Crowley, because the post-debate scrutiny makes it very hard for the Crowleys to keep their job (Crowley left TV news in 2014.) Hohmann argued “[moderators] are bound to be heavily criticized no matter what they do.” Fox News’ Chris Wallace has already explained why he will choose the more passive facilitator role:

WALLACE: I see myself as a conduit to ask the questions and basically to get the two candidates … to engage. I view it as kind of being a referee in a heavyweight championship fight. If it succeeds when it’s over, people will say, you did a great job. I don’t even remember you ever even being on the stage.

Q: What do you do if they make assertions that you know to be untrue?

WALLACE: That’s not my job. I do not believe it is my job to be a truth squad. It’s up to the other person to catch them on that. … If one of them is filibustering, I’m going to try to break in respectfully and give the other person a chance to talk. But I want it to be about them. I want it to be as much of a debate, people often talk that it’s simultaneous news conferences. I want it to be as much of a debate as possible. Frankly, with these two and the way — as Keith Jackson used to say about football rivals, these two just plain don’t like each other. I suspect I’m not going to have any problem getting them to engage with each other, but I don’t view my role as truth squading and I think that is a step too far. If people want to do it after the debate, fine, it’s not my role. “

Whatever you think of Wallace’s politics, he is offering a coherent theory of what debate moderators should do. Some progressive bloggers have already given Wallace an F, over a month before he takes his turn as debate moderator. On the other hand, staying out of the way as much as possible may be the only viable strategy for any kind of moderator to try and pass journalism’s hardest final exam. When I taught summer school, I only wanted to spend one day’s worth of class on an exam. That meant the final covered everything. I knew it would be a hard final, so I added an extra credit question to pre-emptively weight the grade. If challenging presidential candidates when they lie is so hard that almost every moderator assumes they will fail, we need to change the structure of the debate to let them succeed.

Here’s what I would do. Get a range of voters from across the political spectrum to sit in the debate room, like a jury. They have a green button when they can press when they want the moderator to dig deeper and challenge a candidate. They have a red button to press when they want the moderator to move on. If a majority (or maybe 60 percent) of the panel rings in, everyone gets notified. Moderators get the feedback, the viewing audience gets the feedback, even the candidates get an alarm. I would experiment with letting the panel pick the question topics too. The only way to become a debate moderator is to be a veteran DC journalist. These journalists are relatively experienced in interacting with politicians (it is a skill), but probably have different priorities than the audience.

I think the main advantage of this new debate institution is to diffuse responsibility away from the moderator as auteur. Wallace acknowledged he is responsible for everything: the questions, the follow-ups, when to cut someone off. He and other moderators bear the full responsibility for the performance. If the “jury of voters” decides when to challenge a candidate’s assertion, people will still get mad at the jury when it goes against them. This is actually the point – it would be easier for moderators to pursue the truth if it looks like an independent group is giving them permission to do so. I suspect letting people buzz in is also a more effective way to contain a candidate like Trump, who tend to ignore or attack journalists trying to corral him but is deeply unpopular with the general public.

People live tweet debates already. We are increasingly ready for a debate institution where people buzz in to give live reviews of candidate and moderator performances. Lots of people had feedback for Matt Lauer. Why not create a way to help him and help the country instead of just sitting on the couch saying “I could do better than that?”