7.5: Measuring Public Opinion

Public opinion is most commonly measured by polling. A poll or survey is a process of soliciting opinions from people about a particular topic. Polls purport to speak about a group’s views, but they almost never include every member of the group. Doing so would be impractical or impossible unless the group in question were very small. Rather, pollsters ask questions of a sample (a small subset) of a population (the group whose opinions they want to know and understand).

Outside of polling, the term population usually refers to every person living in a given territory. In a polling context, the population can be (and usually is) much narrower. A typical election poll might draw a sample of 1000 respondents from all likely voters or all registered voters. People who can’t or probably won’t vote aren’t part of the population, because the poll’s purpose is to predict the result of the election. Including kindergartners or anyone else unlikely to vote wouldn’t help the poll fulfill its purpose.

Though polls typically sample less than 1% of the population they want to understand, they can still capture public opinion quite accurately. If 50% of Americans approve of the president’s job performance and a pollster interviews a random sample of 100 Americans, it is likely that the number of respondents in the sample who approve will be close to 50. The pollster could be unlucky and draw a sample in which only ten people approve, but such instances are rare if the sampling is done properly. These likelihoods and unlikelihoods can be calculated mathematically, so we can look at a well-drawn sample and be reasonably confident that its opinions closely match those of the entire population.

The trustworthiness of a poll depends largely on its representativeness. A representative sample resembles the population from which it was drawn. If the population is 50% women, for example, a representative sample would also be 50% women or very close to that. If the sample were 80% women, we might wonder whether it could accurately reflect the population’s views, given how severely it overrepresents women and underrepresents men. The same concerns apply to other potentially relevant demographic characteristics, including age, race and ethnicity, education, income, ideology, and partisanship.

Although public opinion polls have been conducted in the United States since the 1820s, scientific polling based on representative samples only became common near the middle of the 20th century. (This is why the presidential approval chart on the previous page begins with Harry S. Truman and not George Washington.) Prior to this period, many polls relied on convenience samples which overrepresented the types of people who were easiest for pollsters to reach, often to the detriment of their accuracy. Modern polling techniques achieve a high degree of representativeness by using census and election data to draw samples that closely match the population in terms of demographics. Some pollsters also weight their results to correct for overrepresentation or underrepresentation.

A jubilant Harry S. Truman holds up a newspaper erroneously proclaiming his defeat in the 1948 presidential election, based on early polls that failed to capture Truman’s eleventhhour comeback.

Even a well-executed poll based on a representative sample will rarely match the population’s opinions exactly. To acknowledge this, responsible pollsters report a margin of error alongside their poll results. This margin is determined statistically and describes a range within which a pollster is reasonably sure—usually 95% sure, to be precise—the true value of public opinion is contained. If a poll indicates that 48% of Americans approve of the president’s job performance with a ±3% margin of error, the pollster is 95% certain that the president’s true approval rating is between 45% and 51%. (This still leaves a 5% chance that the president’s true approval rating is less than 45% or greater than 51%.) The larger the sample, the smaller the margin of error (as shown in Figure 7.3 below).

Figure 7.2: Margin of error by sample size (Note: Margins shown are based on evenly split public opinion. When public opinion is lopsided, margins of error are smaller.)