Why do election polls seem to have such mixed results?

Political polls underestimated support for Donald Trump and overestimated support for Hillary Clinton in the 2016 presidential election. Four years later, polls correctly predicted Joe Biden’s victory over Trump, but both national and statewide polls showed a much larger lead than he ultimately gained

A report by an American Association for Public Opinion Research task force called the 2020 race the industry’s biggest dud since 1980, when polls predicted a close race and instead Ronald Reagan a landslide victory over incumbent Jimmy Carter.

The Gazette spoke this fall with John Anzalone, Biden’s chief 2020 pollster and resident fellow at the Politics Institute, about what happened in the last two elections and how the field has tried to make adjustments amid changing dynamics. political situation of the country.

Anzalone, co-founder of The Wall Street Journal poll, also worked on the presidential campaigns of Hillary Clinton (2016) and Barack Obama (2008 and 2012). His firm, Impact Research, conducted polling for Vice President Kamala Harris’ campaign, but he is not personally involved in the work. The interview has been edited for clarity and length.

What went wrong in the last two elections and has the industry made any course corrections?

It’s really important that we distinguish between what professional pollsters working for campaigns do and what public media pollsters do, because it’s very, very different. You don’t see campaign surveys.

I’m not saying there wasn’t a mistake. It was, especially in ’16. However, there are few media pollsters that spend as much money as we do to actually conduct daily interviews, using multimodal methodologies, using parity, etc. We are branded by the fact that there are dozens of cheap media pollsters today, which I think is a problem.

In 2016, there were many legitimate concerns about polling errors. Internally, among the pollsters, we discovered that we did not get the percentage of voters who did not have a college degree. We won too many service-oriented voters without a college degree. We didn’t get enough people who work with their hands, in the factory or in agriculture, drivers and the like. We also noticed that many of the interviews conducted in small towns in rural areas took place in the district office, not in rural areas. So we’ve changed a lot in terms of how we interview people and the number of voters who don’t have a college degree.

It must be admitted that Trump has changed the political dynamics in America so much. And there was no way to determine who would come out in 2016. It just wasn’t there. We saw a bit of this in 2020 as well.

I think the challenges have a lot to do with modeling who comes out. This has been an absolute mystery in the Trump era. I can’t say who will perform now.

What metrics do pollsters consider the best measure of who will turn out to vote?

We do it on a level of enthusiasm and verisimilitude, but most of our work is message development and strategy. Most media polls are large/small, direct comparison, features, job evaluations, etc. Pollsters who help with campaigns are message development strategists. Everything we do goes into a TV ad, digital ad, or speech. Yes, face-to-face conversation is important and we want to fix that, but media polling has turned every pollster into a forecaster, and that is a misreading of what we do.

Many people who were hesitant to vote when it was Biden-Trump now say they plan to vote. How do polls capture this new, ever-changing electorate?

All you can do is try to guess what percentage of your sample should be “new voters.” You have voter history from 2022, 2020, 2018, and 2016, and you have new registrants. This is not a perfect science. Who said this cycle will be the same as previous cycles where you have to increase a certain percentage nationally to win battleground states?

It’s a tough industry on a good day. During the Trump years, the more difficult challenge was finding a way to win over hard-to-reach voters. Now we know there’s a universe out there that doesn’t want to take live calls or doesn’t trust live calls, so we’ve improved a lot of that. We will need to constantly evolve, constantly improve and do better because of all the challenges we face.

But I’m proud of our industry and I’m proud of the fact that professionally, what we do – what can’t be seen – we do really well. Polling is really expensive, and most media outlets don’t spend the money necessary to conduct it properly.

What things might the Trump and Harris campaigns want to know from internal polling at this point in the race?

Presidential campaigns, regardless of whether you are a Democrat or Republican, will test both positive messages based on the strength of issues and character traits, as well as all contrasts. There is nothing that each side has not tested in terms of positive frames in both cases and negative frames in each case. It’s September and they’ve been voting for eight months.

What every campaign does is it numerically tests its convention speeches and tests its debates. (That is, monitor the reactions of sample audiences to get their immediate reaction to words, phrases and ideas in real time). Thanks to this, they can see what appeals to swing voters. You can guarantee that both of them’s convention speeches were tested, which helped refine some of the things they would say in the debate. And then they’ll test the debates, because they have two months of rallies, two months of speeches, and they have TV ads, so the more data, the better. They have the basics of their research on message, development and contrast, and now it’s about refinement.

You say that most media polls are not very reliable. Which are better?

The Wall Street Journal is undoubtedly the gold standard because it is multimodal. I think Pew Research Center is the gold standard for online surveys because they have built their own online database. And then I think the NBC poll is really good because it’s run by a Democratic and Republican company, just like The Wall Street Journal’s poll.

This story is published courtesy of the Harvard Gazette, the official newspaper of Harvard University. Additional university news can be found at Harvard.edu.