7.4: Measuring Public Opinion

Public opinion is most commonly measured by polling. A poll or survey is a process of soliciting opinions from people about a particular topic. Polls purport to speak about a group’s views, but they almost never include every member of the group, as this would be impractical or impossible unless the group is very small. Rather, pollsters ask questions of a sample (a small subset) of a population (the group whose opinions they want to know and understand).

Outside of polling, the term population usually refers to every person living in a given territory. The population from which a sample is drawn for a poll, however, can be (and usually is) much narrower. A typical election poll might draw a sample of 1000 or so respondents from a population defined as all likely voters or all registered voters. People who can’t or probably won’t vote aren’t part of the population, because the poll’s purpose is to predict the result of the election. Including kindergartners (or anyone else unlikely to vote) in the population from which the sample is drawn wouldn’t help the poll fulfill its purpose.

Although polls typically use samples which are less than 1% of the population they want to learn about, they can still capture public opinion quite accurately thanks to statistics. If 50% of Americans approve of the president’s job performance and a pollster interviews a random sample of 100 Americans, it is likely that the number of respondents in the sample who approve will be close to 50. The pollster could be unlucky and draw a sample in which only ten people approve, but such instances are rare if the sampling is done properly. These likelihoods and unlikelihoods can be calculated mathematically, so we can look at the opinions expressed by a well-drawn sample and be reasonably confident that they closely resemble the opinions which would be expressed by the entire population.

The trustworthiness of a poll depends in large part on its representativeness. A representative sample is one that resembles the population from which it was drawn. If the population is 50% women, for example, a representative sample would also be 50% women (or very close to that). If the sample were 80% women instead, we might wonder whether it could accurately reflect the population’s views, given how severely it overrepresents women and underrepresents men. The same concerns apply to other potentially relevant demographic characteristics, including age, race and ethnicity, education, income, ideology, and partisanship.

Although public opinion polls have been conducted in the United States since the 1820s, scientific polling based on representative samples only became common in the middle of the 20th century (which is why the presidential approval chart on the previous page begins with Harry S. Truman and not George Washington). Prior to this period, many polls relied on “convenience samples” which overrepresented the types of people who were easiest for pollsters to reach, often to the detriment of their accuracy. Modern polling techniques achieve a high degree of representativeness by using census and election data to draw samples that closely match the population in terms of demographics and “weighting” their results to correct for over- or underrepresentation.

Even a well-executed poll based on a representative sample will rarely match the population’s opinions exactly. To acknowledge this, responsible pollsters report a margin of error alongside their poll results. This margin of error is calculated statistically and describes a range within which a pollster is reasonably sure — usually 95% sure, to be precise — the true value of public opinion is contained. If a poll indicates that 48% of Americans approve of the president’s job performance with a ±3% margin of error, that means the pollster is 95% certain that the president’s true approval rating is somewhere between 45% and 51%. (This still leaves a 5% chance that the president’s true approval rating is less than 45% or greater than 51%.) The larger the sample, the smaller the margin of error (as shown in Figure 7.2 below).

Figure 7.2: Margin of error by sample size (Note: Margins shown are based on evenly split public opinion. When public opinion is lopsided, margins of error are smaller.)