7.5: Survey Research (Summary)
- Page ID
- 309660
\( \newcommand{\vecs}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \)
\( \newcommand{\vecd}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash {#1}}} \)
\( \newcommand{\dsum}{\displaystyle\sum\limits} \)
\( \newcommand{\dint}{\displaystyle\int\limits} \)
\( \newcommand{\dlim}{\displaystyle\lim\limits} \)
\( \newcommand{\id}{\mathrm{id}}\) \( \newcommand{\Span}{\mathrm{span}}\)
( \newcommand{\kernel}{\mathrm{null}\,}\) \( \newcommand{\range}{\mathrm{range}\,}\)
\( \newcommand{\RealPart}{\mathrm{Re}}\) \( \newcommand{\ImaginaryPart}{\mathrm{Im}}\)
\( \newcommand{\Argument}{\mathrm{Arg}}\) \( \newcommand{\norm}[1]{\| #1 \|}\)
\( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\)
\( \newcommand{\Span}{\mathrm{span}}\)
\( \newcommand{\id}{\mathrm{id}}\)
\( \newcommand{\Span}{\mathrm{span}}\)
\( \newcommand{\kernel}{\mathrm{null}\,}\)
\( \newcommand{\range}{\mathrm{range}\,}\)
\( \newcommand{\RealPart}{\mathrm{Re}}\)
\( \newcommand{\ImaginaryPart}{\mathrm{Im}}\)
\( \newcommand{\Argument}{\mathrm{Arg}}\)
\( \newcommand{\norm}[1]{\| #1 \|}\)
\( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\)
\( \newcommand{\Span}{\mathrm{span}}\) \( \newcommand{\AA}{\unicode[.8,0]{x212B}}\)
\( \newcommand{\vectorA}[1]{\vec{#1}} % arrow\)
\( \newcommand{\vectorAt}[1]{\vec{\text{#1}}} % arrow\)
\( \newcommand{\vectorB}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \)
\( \newcommand{\vectorC}[1]{\textbf{#1}} \)
\( \newcommand{\vectorD}[1]{\overrightarrow{#1}} \)
\( \newcommand{\vectorDt}[1]{\overrightarrow{\text{#1}}} \)
\( \newcommand{\vectE}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash{\mathbf {#1}}}} \)
\( \newcommand{\vecs}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \)
\(\newcommand{\longvect}{\overrightarrow}\)
\( \newcommand{\vecd}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash {#1}}} \)
\(\newcommand{\avec}{\mathbf a}\) \(\newcommand{\bvec}{\mathbf b}\) \(\newcommand{\cvec}{\mathbf c}\) \(\newcommand{\dvec}{\mathbf d}\) \(\newcommand{\dtil}{\widetilde{\mathbf d}}\) \(\newcommand{\evec}{\mathbf e}\) \(\newcommand{\fvec}{\mathbf f}\) \(\newcommand{\nvec}{\mathbf n}\) \(\newcommand{\pvec}{\mathbf p}\) \(\newcommand{\qvec}{\mathbf q}\) \(\newcommand{\svec}{\mathbf s}\) \(\newcommand{\tvec}{\mathbf t}\) \(\newcommand{\uvec}{\mathbf u}\) \(\newcommand{\vvec}{\mathbf v}\) \(\newcommand{\wvec}{\mathbf w}\) \(\newcommand{\xvec}{\mathbf x}\) \(\newcommand{\yvec}{\mathbf y}\) \(\newcommand{\zvec}{\mathbf z}\) \(\newcommand{\rvec}{\mathbf r}\) \(\newcommand{\mvec}{\mathbf m}\) \(\newcommand{\zerovec}{\mathbf 0}\) \(\newcommand{\onevec}{\mathbf 1}\) \(\newcommand{\real}{\mathbb R}\) \(\newcommand{\twovec}[2]{\left[\begin{array}{r}#1 \\ #2 \end{array}\right]}\) \(\newcommand{\ctwovec}[2]{\left[\begin{array}{c}#1 \\ #2 \end{array}\right]}\) \(\newcommand{\threevec}[3]{\left[\begin{array}{r}#1 \\ #2 \\ #3 \end{array}\right]}\) \(\newcommand{\cthreevec}[3]{\left[\begin{array}{c}#1 \\ #2 \\ #3 \end{array}\right]}\) \(\newcommand{\fourvec}[4]{\left[\begin{array}{r}#1 \\ #2 \\ #3 \\ #4 \end{array}\right]}\) \(\newcommand{\cfourvec}[4]{\left[\begin{array}{c}#1 \\ #2 \\ #3 \\ #4 \end{array}\right]}\) \(\newcommand{\fivevec}[5]{\left[\begin{array}{r}#1 \\ #2 \\ #3 \\ #4 \\ #5 \\ \end{array}\right]}\) \(\newcommand{\cfivevec}[5]{\left[\begin{array}{c}#1 \\ #2 \\ #3 \\ #4 \\ #5 \\ \end{array}\right]}\) \(\newcommand{\mattwo}[4]{\left[\begin{array}{rr}#1 \amp #2 \\ #3 \amp #4 \\ \end{array}\right]}\) \(\newcommand{\laspan}[1]{\text{Span}\{#1\}}\) \(\newcommand{\bcal}{\cal B}\) \(\newcommand{\ccal}{\cal C}\) \(\newcommand{\scal}{\cal S}\) \(\newcommand{\wcal}{\cal W}\) \(\newcommand{\ecal}{\cal E}\) \(\newcommand{\coords}[2]{\left\{#1\right\}_{#2}}\) \(\newcommand{\gray}[1]{\color{gray}{#1}}\) \(\newcommand{\lgray}[1]{\color{lightgray}{#1}}\) \(\newcommand{\rank}{\operatorname{rank}}\) \(\newcommand{\row}{\text{Row}}\) \(\newcommand{\col}{\text{Col}}\) \(\renewcommand{\row}{\text{Row}}\) \(\newcommand{\nul}{\text{Nul}}\) \(\newcommand{\var}{\text{Var}}\) \(\newcommand{\corr}{\text{corr}}\) \(\newcommand{\len}[1]{\left|#1\right|}\) \(\newcommand{\bbar}{\overline{\bvec}}\) \(\newcommand{\bhat}{\widehat{\bvec}}\) \(\newcommand{\bperp}{\bvec^\perp}\) \(\newcommand{\xhat}{\widehat{\xvec}}\) \(\newcommand{\vhat}{\widehat{\vvec}}\) \(\newcommand{\uhat}{\widehat{\uvec}}\) \(\newcommand{\what}{\widehat{\wvec}}\) \(\newcommand{\Sighat}{\widehat{\Sigma}}\) \(\newcommand{\lt}{<}\) \(\newcommand{\gt}{>}\) \(\newcommand{\amp}{&}\) \(\definecolor{fillinmathshade}{gray}{0.9}\)Key Takeaways
Key Terms and Concepts
SURVEY RESEARCH
Research using questionnaires or interviews to collect data from participants.
RESPONDENTS
Individuals who complete surveys or questionnaires.
CONTEXT EFFECTS
The influence of surrounding questions or items on responses.
ITEM-ORDER EFFECT
Changes in responses depending on the order in which items are presented.
OPEN-ENDED ITEMS
Survey questions that allow participants to respond in their own words.
CLOSED-ENDED ITEMS
Survey questions with a fixed set of response options.
RATING SCALE
A response format where participants rate items on a continuum (e.g., 1 to 5).
BRUSO METHOD
An acronym for principles of good survey items: Brief, Relevant, Unambiguous, Specific, and Objective.
PROBABILITY SAMPLING
Sampling where every member of the population has a known probability of selection.
NON-PROBABILITY SAMPLING
Sampling where selection probabilities are unknown or unequal.
CONVENIENCE SAMPLING
Selecting participants based on easy availability.
SNOWBALL SAMPLING
Participants recruit other participants from their networks.
QUOTA SAMPLING
Non-random sampling to match population proportions on certain characteristics.
SELF-SELECTION SAMPLING
Participants volunteer themselves for the study.
SAMPLING FRAME
The list from which a sample is actually drawn.
SIMPLE RANDOM SAMPLING
Every member of the population has an equal chance of selection.
STRATIFIED RANDOM SAMPLING
Random sampling within each of several subgroups.
PROPORTIONATE STRATIFIED RANDOM SAMPLING
Stratified sampling where subgroup sample sizes match population proportions.
DISPROPORTIONATE STRATIFIED RANDOM SAMPLING
Stratified sampling with deliberate oversampling of some subgroups.
CLUSTER SAMPLING
Randomly selecting groups (clusters) and sampling within them.
SAMPLING BIAS
Occurs when sampling is conducted in a way such that it is not representative of the population.
NON-RESPONSE BIAS
When the respondents differ in a systematic way from those who choose not to respond.
Test Your Knowledge (answers at end of section)
1. What is the primary purpose of survey research?
A) To manipulate variables and establish causation
B) To measure variables and describe populations by studying representative samples
C) To conduct experiments in natural settings
D) To observe behavior without asking questions
2. In the Lerner et al. (2003) study on reactions to the 9/11 terrorist attacks, some participants were primed to feel anger while others were primed to feel fear. The researchers found that those primed to feel anger perceived less risk than those primed to feel fear. This study demonstrates that:
A) Survey research cannot be used in experimental designs
B) Self-report measures are always unreliable for measuring emotions
C) Surveys can be incorporated into experimental research by manipulating an independent variable and measuring its effect on survey responses
D) Only non-experimental survey research produces valid results
3. Which of the following is an example of a double-barreled question that should be avoided in survey construction?
A) How satisfied are you with your job?
B) Do you support increased funding for education and healthcare?
C) What is your age?
D) How often do you exercise?
4. A researcher asks college students 'How many alcoholic drinks do you consume in a typical day?' with response options ranging from 'a lot more than average' to 'a lot fewer than average.' According to the cognitive model of survey responding, what multiple problems does this question pose for respondents?
A) The question is too brief and uses technical language
B) Respondents must interpret ambiguous terms (e.g., 'alcoholic drinks,' 'typical day'), retrieve relevant information from memory, make calculations, and then map their answer onto vague response options like 'average'
C) The question violates the BRUSO model by being too objective
D) The response options are mutually exclusive and exhaustive, which confuses respondents
5. What is a key advantage of online/internet surveys compared to mail surveys?
A) Higher response rates
B) Lower cost and faster data collection with features like skip logic and randomization
C) Better for reaching elderly populations
D) More personal connection with respondents
6. Why did the 1936 Literary Digest poll incorrectly predict that Alf Landon would defeat Franklin Roosevelt in a landslide, while George Gallup correctly predicted Roosevelt's victory using a much smaller sample?
A) Literary Digest used probability sampling while Gallup used convenience sampling
B) Literary Digest's sample suffered from sampling bias because their mailing lists over-represented wealthier people who were more likely to vote for Landon
C) Gallup's sample was larger and therefore more accurate
D) Literary Digest experienced non-response bias while Gallup did not
Answer Key
1. B - To measure variables and describe populations by studying representative samples
Survey research aims to describe the characteristics, attitudes, or behaviors of populations by asking questions of representative samples. Surveys measure variables as they naturally occur (no manipulation), making them a form of non-experimental research. The goal is to describe populations accurately through systematic data collection from samples.
2. C - Surveys can be incorporated into experimental research by manipulating an independent variable and measuring its effect on survey responses
This study exemplifies how surveys can be used within experimental research. The researchers manipulated an independent variable (anger priming vs. fear priming) and measured its effect on a dependent variable (risk judgments) using self-report survey measures. While most survey research is non-experimental, the use of self-report measures and large samples can be combined with experimental manipulation, as this study demonstrates. This challenges the misconception that surveys are only for non-experimental research.
3. B - Do you support increased funding for education and healthcare?
Option B is double-barreled because it asks about TWO different issues (education and healthcare) in a single question. Respondents might support one but not the other, making it impossible to answer accurately. Good survey questions should ask about only one thing at a time. Each issue should be addressed in separate questions to get clear, interpretable responses.
4. B - Respondents must interpret ambiguous terms (e.g., 'alcoholic drinks,' 'typical day'), retrieve relevant information from memory, make calculations, and then map their answer onto vague response options like 'average'
This question exemplifies the complexity of the cognitive processes involved in survey responding. Respondents must: (1) interpret what counts as 'alcoholic drinks' (beer? wine?) and what constitutes a 'typical day' (weekday? weekend?), (2) retrieve relevant information from memory, (3) perform mental calculations (perhaps dividing weekly drinks by seven), (4) decide what 'average' means, and (5) map their answer onto vague response categories. A seemingly simple question can involve complex interpretation, retrieval, judgment, formatting, and editing processes. The response options are also problematic because they're not based on actual quantities and require respondents to know what 'average' is.
5. B - Lower cost and faster data collection with features like skip logic and randomization
Online surveys offer significant advantages: very low cost, rapid data collection, easy distribution, and advanced features like skip logic (branching), automatic randomization of question order, and immediate data entry. While response rates can vary and some populations (elderly, those without internet access) may be harder to reach, the cost-effectiveness and technological capabilities make online surveys increasingly popular.
6. B - Literary Digest's sample suffered from sampling bias because their mailing lists over-represented wealthier people who were more likely to vote for Landon
The Literary Digest sent ballots to millions of Americans using mailing lists from telephone directories and automobile registration lists. This created severe sampling bias because in 1936, only wealthier people owned phones and cars, and wealthier people were more likely to vote Republican (for Landon). Their huge sample size didn't overcome the fundamental problem that the sample wasn't representative of the voting population. Gallup used scientific sampling methods with a much smaller sample to ensure his sample was representative of all voters, not just wealthy ones. This watershed event demonstrated that careful survey methodology with representative samples is far more important than large sample sizes alone. It shows how sampling bias occurs when a sample is selected in a way that makes it unrepresentative of the population.