Is persuasion still possible? Political scientist Joshua Kalla answers our questions
Joshua Kalla is a Ph.D. candidate in Political Science at the University of California, Berkeley whose research focuses on American political behavior and opinion. Along with Stanford’s David Broockman, Joshua Kalla co-authored a study on voter persuasion that made a splash in the media and in progressive circles in September 2017. In this study, Kalla and Broockman argue that traditional campaign communication methods (such as direct mail, canvassing, and digital advertising) have essentially no persuasive effect on voters during general elections. We asked Kalla about his recent work and its implications for campaigns.
1. Does persuasion happen at all during a campaign?
Between elections, some voters certainly change their minds. As an example, the Democracy Fund Voter Study Group found that 9.2 percent of Obama voters supported Trump in 2016. The question then becomes how much of this change occurs between elections (e.g., Obama voters didn’t like his performance and voted for change in Trump rather than continuity in Clinton) and how much occurs during the campaign (e.g., the Trump campaign, through their communication methods, caused the persuasion). Our paper tries to speak to this much narrower second question.
2. Based on your research, do you suggest that U.S. political campaigns stop spending money on persuasion?
No, our current research cannot answer this question. We study whether an additional campaign contact, such as a face-to-face canvass conversation, can persuade voters above and beyond all the other political activity and media coverage that is taking place. We find that in most cases (with some important exceptions I highlight below), these additional campaign contacts have no effect. But this does not imply campaigns should stop all persuasive efforts.
3. What’s the biggest misconception about your latest study?
The biggest misconception, as put by Vox’s Dylan Matthews, is that our results mean: “Campaigns’ attempts to win swing voters appear to not work at all.” This headline does not tell the full story. There are three important exceptions that I want to highlight.
First, we find that campaigns are able to use experiments to identify persuadable targets. Just like polls can be divided into subsets of voters (e.g., polls teach us more than just how Americans feel on average but can be divided into groups such as African American women and college students), experiments can be divided into similar subgroups. On average we might find that campaigns’ general election persuasive contacts have little effect. But within this average of zero, we can also identify voters who were persuaded and voters who reacted negatively to the persuasive messages. Using experiments to identify and then target the most persuadable voters can lead to drastic gains in persuasion.
Second, most of the misconceptions stem from focusing only on partisan general elections. In these settings, voters rely heavily on whether there is a “D” or an “R” next to a candidate’s name and rarely budge from that partisan position. But American politics is more than just partisan general elections. Campaign persuasion appears to work in primaries and in ballot measure elections, even competitive ones.
Finally, our study is about the kind of persuasion campaigns do today. We hope that practitioners, rather than abandoning persuasion as a goal, will instead attempt new and creative ideas for how to persuade voters. Many of these ideas will likely fail, but by experimenting and investing in basic research and development, it is possible that the persuasion of tomorrow can be more effective.
4. Do you think advocacy and legislative accountability campaigns can effectively engage and persuade voters on issues before a general election?
Certainly. One study we conducted attempted to reduce prejudice against transgender people and increase support for a transgender-inclusive nondiscrimination law outside of an electoral context. In this study, we found that a single approximately 10-minute conversation was able to reduce prejudice for at least 3 months and increase support for a nondiscrimination law, even after exposing voters to counterarguments.
And even in elections, we find that campaigns might be able to persuade voters on issues. For example, we find that campaign contact can be persuasive in ballot measure campaigns that focus more on issues and less on partisanship.
But not every advocacy campaign can persuade voters. In another study, we attempted to use door-to-door canvassing to reduce stigma toward women who have had abortions, but found no persuasive effect. We hope to conduct more research to get a better understanding of the issues and contexts of when we should expect advocacy and legislative accountability campaigns to be persuasive and when they might fail.
5. How much of the difficulties you’ve observed in persuasion efforts have to do with the noise that characterizes any presidential election year with so many voter outreach programs happening at the same time?
This is a really important question for understanding our results. Our experiments take voters in the real-world where they might read the newspaper, talk to friends and family, watch TV ads, and absorb other types of political information. In this setting, we then randomly assign whether additional campaign contacts – such as a door-to-door conversation or series of political mailers – change their intended vote choice. The background noise makes it less likely that any one additional campaign contact will have a persuasive effect, yet this is exactly the kind of environment campaigns and interest groups operate in. What this suggests is that persuasive efforts might be more successful if they avoid this background noise or find a way to break through it. For example, persuasive efforts appear to be more successful in ballot measure campaigns and primary elections where there is less noise from other campaigns and less media coverage. Groups might also try experimenting with persuasive efforts that are unique by targeting voters who might be receiving fewer contacts from other groups and by providing new information and messages that are not being featured by other campaigns or the media.
6. Are we limited by our current methods of measuring persuasion in evaluating and showing movement among the public?
In addition to conducting research on whether voters are persuadable, David Broockman and I (along with Jasjeet Sekhon) have produced methodological advances to make it faster, easier, and cheaper for campaigns to conduct research on persuasion. By using online surveys recruited via mail from the voter file with multiple surveys conducted before and after persuasion, we are able to reduce the costs of mail persuasion experiments by 50% and door-to-door canvass and phone persuasion experiments by 98%. These savings have enabled local campaigns such as a mayoral primary and a state legislative race to conduct cutting-edge research on persuasion.
7. What kinds of tests would you like to see campaigns do in general?
I think there are two areas that are ripe for more testing. First, we find that campaigns typically fail to persuade voters, but the math of many elections dictates that campaigns cannot abandon persuasion altogether. Instead, we need to try new ideas. I have ideas that I would like to explore, but I also want to learn from the creativity of political practitioners.
Second, most of the research on political campaigns has focused on the outputs: how many votes can a campaign generate using particular GOTV or persuasion efforts? I think we also need to explore more of the inputs, particularly around volunteers. How can campaigns recruit and retain long-term committed volunteers?
If campaigns are interested in collaborating on these research questions, they should certainly get in touch!
Bonus: Any books you’ve read recently that you think shed light on the American electorate?
Two recent books that should be broadly read by political practitioners to understand the American electorate are Paul Sniderman’s The Democratic Faith and Chris Achens and Larry Bartels’s Democracy for Realists. At points they come to opposite conclusions, but I think this complexity is good to understand.