6.7: Non-Experimental Research (Summary)

Key Takeaways

• Non-experimental research is research that lacks the manipulation of an independent variable.
• There are two broad types of non-experimental research. Correlational research that focuses on statistical relationships between variables that are measured but not manipulated; and observational research in which participants are observed and their behavior is recorded without the researcher interfering or manipulating any variables.
• In general, experimental research is high in internal validity, correlational research is low in internal validity, and quasi-experimental research is in between.
• Correlational research involves measuring two variables and assessing the relationship between them, with no manipulation of an independent variable.
• Correlation does not imply causation. A statistical relationship between two variables, X and Y, does not necessarily mean that X causes Y. It is also possible that Y causes X, or that a third variable, Z, causes both X and Y.
• While correlational research cannot be used to establish causal relationships between variables, correlational research does allow researchers to achieve many other important objectives (establishing reliability and validity, providing converging evidence, describing relationships, and making predictions)
• Correlation coefficients can range from -1 to +1. The sign indicates the direction of the relationship between the variables and the numerical value indicates the strength of the relationship.
• Researchers often use complex correlational research to explore relationships among several variables in the same study.
• Complex correlational research can be used to explore possible causal relationships among variables using techniques such as partial correlation and multiple regression. Such designs can show patterns of relationships that are consistent with some causal interpretations and inconsistent with others, but they cannot unambiguously establish that one variable causes another.
• Qualitative research is an important alternative to quantitative research in psychology. It generally involves asking broader research questions, collecting more detailed data (e.g., interviews), and using non-statistical analyses.
• Many researchers conceptualize quantitative and qualitative research as complementary and advocate combining them. For example, qualitative research can be used to generate hypotheses and quantitative research to test them.
• There are several different approaches to observational research including naturalistic observation, participant observation, structured observation, case studies, and archival research.
• Naturalistic observation is used to observe people in their natural setting; participant observation involves becoming an active member of the group being observed; structured observation involves coding a small number of behaviors in a quantitative manner; case studies are typically used to collect in-depth information on a single individual; and archival research involves analyzing existing data.

Key Terms and Concepts

NON-EXPERIMENTAL RESEARCH

Research that does not involve manipulation of an independent variable.

CORRELATIONAL RESEARCH

Research examining statistical relationships between variables without manipulation.

OBSERVATIONAL RESEARCH

Systematic observation and recording of behavior.

CROSS-SECTIONAL STUDIES

Studies comparing different groups at a single point in time.

COHORT EFFECT

Differences between age groups due to generational experiences rather than age itself.

LONGITUDINAL STUDIES

Studies following the same participants over an extended period of time.

CROSS-SEQUENTIAL STUDIES

Studies combining cross-sectional and longitudinal approaches.

POSITIVE RELATIONSHIP

A relationship where both variables increase or decrease together.

NEGATIVE RELATIONSHIP

A relationship where one variable increases as the other decreases.

RESTRICTION

Limitation in the range of a variable that can affect correlation strength.

RANGE

The span of values for a variable.

DIRECTIONALITY PROBLEM

The inability to determine which variable causes changes in the other.

THIRD-VARIABLE

A variable that causes changes in both measured variables, creating a spurious correlation.

CORRELATION MATRIX

A table showing correlations among multiple variables.

FACTOR ANALYSIS

A statistical method for identifying underlying dimensions among multiple variables.

STATISTICAL CONTROL

Removing the influence of extraneous variables mathematically.

PARTIAL CORRELATION

The correlation between two variables after removing the influence of a third variable.

REGRESSION

A statistical technique for predicting one variable from one or more other variables.

PREDICTOR VARIABLE

A variable used to predict an outcome in regression.

OUTCOME VARIABLE OR CRITERION VARIABLE

The variable being predicted in regression.

SIMPLE REGRESSION

Involving one predictor variable.

MULTIPLE REGRESSION

Involving two or more predictor variables.

QUANTITATIVE RESEARCH

Research using numerical data and statistical analysis.

QUALITATIVE RESEARCH

Research focusing on in-depth exploration of meaning and experience using non-numerical data.

INTERVIEWS

Data collection through in-depth conversations with participants.

FOCUS GROUPS

Group discussions on specific topics moderated by a researcher.

GROUNDED THEORY

An approach to developing theory inductively from qualitative data.

THEORETICAL NARRATIVE

A coherent explanation developed from qualitative analysis.

MIXED-METHODS RESEARCH

Research combining both quantitative and qualitative approaches.

TRIANGULATION

Using multiple methods to study the same phenomenon for convergence.

NATURALISTIC OBSERVATION

Observing behavior in natural settings without intervention.

DISGUISED NATURALISTIC OBSERVATION

Observation where participants are unaware they are being observed.

UNDISGUISED NATURALISTIC OBSERVATION

Observation where participants know they are being observed.

REACTIVITY

Changes in behavior that occur because participants know they are being observed.

HAWTHORNE EFFECT

Improvement in performance due to awareness of being observed.

PARTICIPANT OBSERVATION

Observation where the researcher becomes part of the group being studied.

STRUCTURED OBSERVATION

Observation using predetermined categories and coding schemes.

CODING

Categorizing observations or data systematically.

CASE STUDY

An in-depth examination of a single individual, group, or event.

ARCHIVAL RESEARCH

Research involving the analysis of archival data already collected for another purpose.

CONTENT ANALYSIS

A family of approaches for analyzing archival data.

Test Your Knowledge (answers at end of section)

1. What is the primary limitation of non-experimental research compared to experimental research?

A. Non-experimental research is more expensive

B. Non-experimental research is less valid

C. Non-experimental research requires larger sample sizes

D. Non-experimental research cannot establish cause-and-effect relationships because variables are not manipulated

2. A researcher compares American and Japanese college students' self-esteem scores by administering the Rosenberg Self-Esteem Scale to both groups. This is a correlational study (not an experiment) because:

A. It involves quantitative data

B. It compares two groups

C. The researcher did not manipulate nationality; students' nationality existed before the study

D. It uses a standardized measure

3. A researcher finds a correlation of r = -0.65 between hours of sleep and anxiety levels. What does this indicate?

A. More sleep causes lower anxiety

B. As hours of sleep increase, anxiety levels tend to decrease

C. Sleep and anxiety are unrelated

D. Less sleep causes higher anxiety

4. A study finds a strong positive correlation between per capita chocolate consumption and Nobel prizes won by a nation. The most likely explanation is:

A. Eating chocolate causes increased intelligence leading to Nobel prizes

B. The correlation is due to measurement error

C. Winning Nobel prizes causes countries to consume more chocolate in celebration

D. This is a spurious correlation driven by a third variable such as wealth or education investment

5. In multiple regression analysis, what advantage does this technique offer over simple bivariate correlation?

A. It is easier to calculate

B. It allows researchers to examine the relationship between one outcome variable and multiple predictor variables simultaneously

C. It proves causation

D. It requires smaller sample sizes

6. A researcher finds that violent TV viewing and aggression are correlated at r = +.35. After statistically controlling for SES using partial correlation, the correlation drops to +.03. This suggests:

A. SES is a third variable largely driving the relationship between TV violence and aggression

B. Violent TV viewing causes aggression

C. The original correlation was due to measurement error

D. Aggression causes people to watch more violent TV

7. What is the main focus of qualitative research?

A. Testing hypotheses with statistical analysis

B. Understanding the meaning, experiences, and perspectives of participants through in-depth, non-numerical data

C. Establishing cause-and-effect relationships

D. Measuring variables with standardized instruments

8. Lindqvist and colleagues studied families of teenage suicide victims using unstructured interviews. They used grounded theory, which means they:

A. Started with a specific hypothesis and tested it statistically

B. Started with the data and developed themes and theoretical interpretations grounded in those data

C. Used only quantitative measures

D. Manipulated variables to test causal relationships

9. The case study of HM, who lost the ability to form new long-term memories after hippocampus removal, illustrates both a strength and limitation of case studies because:

A. It provided crucial insights about memory consolidation but cannot determine causation or be generalized

B. It proves that the hippocampus causes memory formation

C. It has both high internal and external validity

D. It can be replicated with other participants

Answer Key with Explanations

1. D - Non-experimental research cannot establish cause-and-effect relationships because variables are not manipulated

The key limitation of non-experimental research is that researchers observe or measure variables as they naturally occur without manipulation. Without manipulating an independent variable and using random assignment, researchers cannot establish causal relationships - they can only identify associations or describe phenomena.

2. C - The researcher did not manipulate nationality; students' nationality existed before the study

The key point is that manipulation by the researcher is what defines an experiment. Since nationality existed before the study, it cannot be manipulated, making this correlational research.

3. B - As hours of sleep increase, anxiety levels tend to decrease

A correlation of -0.65 indicates a moderate to strong negative (inverse) relationship - as one variable increases, the other tends to decrease. However, correlation does NOT imply causation. While the pattern shows sleep and anxiety move in opposite directions, we cannot conclude that sleep causes changes in anxiety from correlational data alone.

4. D - This is a spurious correlation driven by a third variable such as wealth or education investment

The fact that nations that have won more Nobel prizes tend to have higher chocolate consumption probably reflects geography in that European countries tend to have higher rates of per capita chocolate consumption and invest more in education and technology (once again, per capita) than many other countries. This is a third-variable problem where geography/wealth explains both variables.

5. B - It allows researchers to examine the relationship between one outcome variable and multiple predictor variables simultaneously

Multiple regression allows researchers to examine how multiple predictor (independent) variables relate to a single outcome (dependent) variable simultaneously. This helps identify which predictors are most important and can control for confounding by examining each predictor's unique contribution while holding other predictors constant.

6. A - SES is a third variable largely driving the relationship between TV violence and aggression

After statistically controlling for SES the correlation between violent television viewing and aggression dropped to +.03 suggests that SES is indeed a third variable that is driving the relationship.

7. B - Understanding the meaning, experiences, and perspectives of participants through in-depth, non-numerical data

Qualitative research focuses on understanding phenomena in depth through non-numerical data like interviews, observations, and documents. The goal is to capture the richness, complexity, and meaning of human experience rather than to quantify variables or test hypotheses statistically. It asks 'why' and 'how' questions rather than 'how much' or 'how many.'

8. B - Started with the data and developed themes and theoretical interpretations grounded in those data

In qualitative research using grounded theory, researchers start with the data and develop a theory or an interpretation that is "grounded in" those data. They do this analysis in stages. First, they identify ideas that are repeated throughout the data. Then they organize these ideas into a smaller number of broader themes. Finally, they write a theoretical narrative. The Lindqvist study exemplifies this approach.

9. A - It provided crucial insights about memory consolidation but cannot determine causation or be generalized

HM's case provides important insights into the role of the hippocampus in memory consolidation. However, it also emphasizes limitations. Case studies lack the proper controls that true experiments contain. As such, they suffer from problems with internal validity, so they cannot be used to determine causation. Additionally, because case studies are often of a single individual, researchers cannot generalize their conclusions to other individuals.

References

Abrams, L. S., & Curran, L. (2009). “And you’re telling me not to stress?” A grounded theory study of postpartum depression symptoms among low-income mothers. Psychology of Women Quarterly, 33, 351–362.

Bryman, A. (2012). Social Research Methods (4th ed.). Oxford: Oxford University Press.

Bushman, B. J., & Huesmann, L. R. (2001). Effects of televised violence on aggression. In D. Singer & J. Singer (Eds.), Handbook of children and the media(pp. 223–254). Thousand Oaks, CA: Sage.

Cacioppo, J. T., & Petty, R. E. (1982). The need for cognition. Journal of Personality and Social Psychology, 42, 116–131.

Cohen, D., Nisbett, R. E., Bowdle, B. F., & Schwarz, N. (1996). Insult, aggression, and the southern culture of honor: An “experimental ethnography.” Journal of Personality and Social Psychology, 70(5), 945-960.

Diener, E. (2000). Subjective well-being: The science of happiness, and a proposal for a national index. American Psychologist, 55, 34–43.

Festinger, L., Riecken, H., & Schachter, S. (1956). When prophecy fails: A social and psychological study of a modern group that predicted the destruction of the world. University of Minnesota Press.

Freud, S. (1961). Five lectures on psycho-analysis. New York, NY: Norton.

Geertz, C. (1973). The interpretation of cultures. New York, NY: Basic Books.

Glaser, B. G., & Strauss, A. L. (1967). The discovery of grounded theory: Strategies for qualitative research. Chicago, IL: Aldine.

Jouriles, E. N., Garrido, E., Rosenfield, D., & McDonald, R. (2009). Experiences of psychological and physical aggression in adolescent romantic relationships: Links to psychological distress. Child Abuse & Neglect, 33(7), 451–460.

Kraut, R. E., & Johnston, R. E. (1979). Social and emotional messages of smiling: An ethological approach. Journal of Personality and Social Psychology, 37, 1539–1553.

Levine, R. V., & Norenzayan, A. (1999). The pace of life in 31 countries. Journal of Cross-Cultural Psychology, 30, 178–205.

Lindqvist, P., Johansson, L., & Karlsson, U. (2008). In the aftermath of teenage suicide: A qualitative study of the psychosocial consequences for the surviving family members. BMC Psychiatry, 8:26. Retrieved from http://www.biomedcentral.com/1471-244X/8/26

Loftus, E. F., & Pickrell, J. E. (1995). The formation of false memories. Psychiatric Annals, 25, 720–725.

Messerli, F. H. (2012). Chocolate consumption, cognitive function, and Nobel laureates. New England Journal of Medicine, 367, 1562-1564.

Milgram, S. (1963). Behavioral study of obedience. Journal of Abnormal and Social Psychology, 67, 371–378.

Milgram, S. (1974). Obedience to authority: An experimental view. New York, NY: Harper & Row.

Pelham, B. W., Carvallo, M., & Jones, J. T. (2005). Implicit egotism. Current Directions in Psychological Science, 14, 106–110.

Peterson, C., Seligman, M. E. P., & Vaillant, G. E. (1988). Pessimistic explanatory style is a risk factor for physical illness: A thirty-five year longitudinal study. Journal of Personality and Social Psychology, 55, 23–27.

Plomin, R., DeFries, J. C., McClearn, G. E., & McGuffin, P. (2008). Behavioral genetics (5th ed.). New York, NY: Worth.

Radcliffe, N. M., & Klein, W. M. P. (2002). Dispositional, unrealistic, and comparative optimism: Differential relations with knowledge and processing of risk information and beliefs about personal risk. Personality and Social Psychology Bulletin, 28, 836–846.

Rentfrow, P. J., & Gosling, S. D. (2008). The do re mi’s of everyday life: The structure and personality correlates of music preferences. Journal of Personality and Social Psychology, 84, 1236–1256.

Rosenhan, D. L. (1973). On being sane in insane places. Science, 179, 250–258.

Todd, Z., Nerlich, B., McKeown, S., & Clarke, D. D. (2004) Mixing methods in psychology: The integration of qualitative and quantitative methods in theory and practice. London, UK: Psychology Press.

Trenor, J.M., Yu, S.L., Waight, C.L., Zerda. K.S & Sha T.-L. (2008). The relations of ethnicity to female engineering students’ educational experiences and college and career plans in an ethnically diverse learning environment. Journal of Engineering Education, 97(4), 449-465.

Watson, J. B., & Rayner, R. (1920). Conditioned emotional reactions. Journal of Experimental Psychology, 3, 1–14.

Wilkins, A. (2008). “Happier than Non-Christians”: Collective emotions and symbolic boundaries among evangelical Christians. Social Psychology Quarterly, 71, 281–301.

Exercises

• Discussion: For each of the following studies, decide which type of research design it is and explain why.
  • A researcher conducts detailed interviews with unmarried teenage fathers to learn about how they feel and what they think about their role as fathers and summarizes their feelings in a written narrative.
  • A researcher measures the impulsivity of a large sample of drivers and looks at the statistical relationship between this variable and the number of traffic tickets the drivers have received.
  • A researcher randomly assigns patients with low back pain either to a treatment involving hypnosis or to a treatment involving exercise. She then measures their level of low back pain after 3 months.
• Discussion: For each of the following, decide whether it is most likely that the study described is experimental or non-experimental and explain why.
  • A cognitive psychologist compares the ability of people to recall words that they were instructed to “read” with their ability to recall words that they were instructed to “imagine.”
  • A manager studies the correlation between new employees’ college grade point averages and their first-year performance reports.
  • An automotive engineer installs different stick shifts in a new car prototype, each time asking several people to rate how comfortable the stick shift feels.
  • A food scientist studies the relationship between the temperature inside people’s refrigerators and the amount of bacteria on their food.
  • A social psychologist tells some research participants that they need to hurry over to the next building to complete a study. She tells others that they can take their time. Then she observes whether they stop to help a research assistant who is pretending to be hurt.
• Practice: For each of the following statistical relationships, decide whether the directionality problem is present and think of at least one plausible third variable.
  • People who eat more lobster tend to live longer.
  • People who exercise more tend to weigh less.
  • College students who drink more alcohol tend to have poorer grades.
• Practice: Construct a correlation matrix for a hypothetical study including the variables of depression, anxiety, self-esteem, and happiness. Include the Pearson’s r values that you would expect.
• Discussion: Imagine a correlational study that looks at intelligence, the need for cognition, and high school students’ performance in a critical thinking course. A multiple regression analysis shows that intelligence is not related to performance in the class but that the need for cognition is. Explain what this study has shown in terms of what is related to good performance in the critical thinking course.
• Discussion: What are some ways in which a qualitative study of girls who play youth baseball would likely differ from a quantitative study on the same topic? How would the data differ by interviewing girls one-on-one rather than conducting focus groups or surveys?
• Practice: Find and read a published case study in psychology. (Use case study as a key term in a PsycINFO search.) Then do the following:
  • Describe one problem related to internal validity.
  • Describe one problem related to external validity.
  • Generate one hypothesis suggested by the case study that might be interesting to test in a subsequent study.