2.9: Overview of the Scientific Method (Summary)

Key Takeaways

• Research in psychology can be described by a simple cyclical model. A research question based on the research literature leads to an empirical study, the results of which are published and become part of the research literature.
• The research literature in psychology is all the published research in psychology, consisting primarily of articles in professional journals and scholarly books.
• Early in the research process, it is important to conduct a review of the research literature on your topic to refine your research question, identify appropriate research methods, place your question in the context of other research, and prepare to write an effective research report.
• There are several strategies for finding previous research on your topic. Among the best is using PsycINFO, a computer database that catalogs millions of articles, books, and book chapters in psychology and related fields.
• Research questions expressed in terms of variables and relationships between variables can be suggested by other researchers or generated by asking a series of more general questions about the behavior or psychological characteristic of interest.
• It is important to evaluate how interesting a research question is before designing a study and collecting data to answer it. Factors that affect interestingness are the extent to which the answer is in doubt, whether it fills a gap in the research literature, and whether it has important practical implications.
• It is also important to evaluate how feasible a research question will be to answer. Factors that affect feasibility include time, money, technical knowledge and skill, and access to special equipment and research participants.
• A theory is broad in nature and explains larger bodies of data. A hypothesis is more specific and makes a prediction about the outcome of a particular study.
• Working with theories is not “icing on the cake.” It is a basic ingredient of psychological research.
• Like other scientists, psychologists use the hypothetico-deductive method. They construct theories to explain or interpret phenomena (or work with existing theories), derive hypotheses from their theories, test the hypotheses, and then reevaluate the theories in light of the new results.
• Variables vary across people or situations and may be quantitative (e.g., age) or categorical (e.g., course subject).
• A sample is a small subset of a larger population that is selected to participate in the research study. There are many different ways of sampling participants including convenience sampling and simple random sampling.
• Experimental research involves manipulating an independent variable to observe the effects on a measured dependent variable while non-experimental research involves measuring variables as they naturally occur (i.e., without manipulating anything).
• Research can be conducted in the field or the lab. Laboratory experiments tend to have high internal validity (allowing us to make strong causal conclusions), while field studies often have more external validity (allowing us to generalize to the real world).
• The mean, median, and mode are measures of central tendency used to describe the typical, average, or center scores in a distribution. The range, standard deviation, and variance are measures of how dispersed or spread apart the scores are. Measures of central tendency and dispersion are important descriptive statistics.
• Inferential statistics allow researchers to determine whether their findings are statistically significant, that is, whether they are unlikely to be due to chance alone and therefore are likely to represent a real effect in the population.
• Since statistics are probabilistic in nature we never know if our conclusions are correct. We can make type I errors (concluding an effect is real when it is not) or type II errors (concluding there is no effect when there actually is a real effect in the population).
• Theories can be supported by not proved. Similarly, disconfirming a hypothesis does not necessarily mean that theory has been disproved.
• The final step of the research process involves reporting results at scientific conferences, in journal articles, and/or in books.

Key Terms and Concepts

MODEL OF SCIENTIFIC RESEARCH

The typical steps to conduct research including formulating a question, conducting an empirical study, analyzing results, forming conclusions, adding to the research literature.

PROFESSIONAL JOURNALS

Periodicals that publish original research articles after peer review.

DOUBLE-BLIND PEER REVIEW

A review process where neither the author nor reviewer knows the other's identity.

EMPIRICAL RESEARCH REPORTS

Articles describing new studies conducted by the authors, including methods and results.

REVIEW ARTICLES

Articles that summarize and synthesize previously published research on a topic.

THEORETICAL ARTICLE

An article presenting a new theory or evaluating existing theories.

META-ANALYSIS

A statistical technique combining results from multiple studies to estimate overall effect sizes.

SCHOLARLY BOOKS

Books written by researchers for other researchers and practitioners in a field.

MONOGRAPH

A scholarly book on a single topic, typically written by one author or a small team.

EDITED VOLUMES

Books containing chapters written by different authors, compiled by one or more editors.

PSYCINFO

The primary electronic database for searching psychological research literature.

INTERESTINGNESS

The degree to which a research question's answer is in doubt, fills a gap, or has practical implications.

FEASIBILITY

Whether a research question can be successfully answered with available resources and methods.

THEORY

A coherent explanation or interpretation of one or more phenomena.

HYPOTHESIS

A specific, testable prediction derived from a theory.

HYPOTHETICO-DEDUCTIVE METHOD

The process of using theories to generate specific hypotheses that can be tested.

LOGICAL

A characteristic of hypotheses that follow logically from theory or previous observations.

DEDUCTIVE REASONING

Moving from general theories or principles to specific predictions or hypotheses.

INDUCTIVE REASONING

Using specific observations to develop general theories or principles.

POSITIVE

A statement form that specifies what will happen, rather than what will not happen.

VARIABLE

Any quantity or quality that varies across people, situations, or times.

QUANTITATIVE VARIABLE

A variable measured numerically on a scale.

CATEGORICAL VARIABLE

A variable consisting of distinct categories or groups.

OPERATIONAL DEFINITION

A precise description of how a variable will be measured or manipulated.

POPULATION

The entire group of individuals that researchers are interested in studying.

SAMPLE

A subset of the population that is actually studied.

SIMPLE RANDOM SAMPLING

A sampling method where every member of the population has an equal chance of selection.

CONVENIENCE SAMPLING

Selecting participants based on their availability and accessibility.

INDEPENDENT VARIABLE

The variable that the researcher manipulates in an experiment.

DEPENDENT VARIABLE

The variable that is measured as the outcome in a study.

EXTRANEOUS VARIABLES

Variables other than the independent variable that might affect the dependent variable.

CONFOUNDS

Extraneous variables that vary systematically with the independent variable, making causal conclusions impossible.

LABORATORY STUDY

Research conducted in a controlled laboratory setting.

FIELD STUDY

Research conducted in natural, real-world settings.

INTERNAL VALIDITY

The degree to which a study establishes a causal relationship between variables.

EXTERNAL VALIDITY

The degree to which research findings generalize beyond the specific study.

FIELD EXPERIMENTS

The independent variable is manipulated in a natural setting.

MODE

The most frequently occurring score in a distribution.

MEDIAN

The middle score when all scores are arranged in order.

MEAN

The arithmetic average of all scores.

RANGE

The difference between the highest and lowest scores.

STANDARD DEVIATION

A measure of the average distance of scores from the mean.

VARIANCE

The average of squared deviations from the mean; the square of the standard deviation.

CORRELATION COEFFICIENT

A statistic describing the strength and direction of a relationship between two variables.

INFERENTIAL STATISTICS

Statistical methods for drawing conclusions about populations based on sample data.

STATISTICALLY SIGNIFICANT

A result unlikely to have occurred by chance alone, typically p < .05.

TYPE I ERROR

Researcher incorrectly rejects the null hypothesis.

TYPE II ERROR

Researcher fails to reject a false null hypothesis.

DRAWING CONCLUSIONS

Interpreting results in light of hypotheses and evidence.

REPORTING THE RESULTS

Presenting findings following scientific conventions.

INTERPRETATION

Making meaning from statistical outcomes.

LIMITATIONS

Factors that restrict the strength or generality of conclusions.

GENERALIZABLE

Extent to which findings apply beyond the study sample.

Test Your Knowledge (answers at end of section)

1. 1. According to the simple model of scientific research in psychology, what step comes immediately after formulating a research question?

   A. Analyze the resulting data and draw conclusions

   B. Conduct an empirical study designed to answer the question

   C. Consider whether the theory supports the hypothesis

   D. Publish the findings in a peer-reviewed journal

   2. Research on cell phone use and driving demonstrates that the scientific research process is cyclical because:

   A. Each study must be repeated multiple times before publication

   B. Published research leads to new questions, which lead to new research

   C. Researchers must return to formulating questions after analyzing data

   D. The same methodology must be used in all subsequent studies

   3. Which of the following is described in the chapter as a key source of inspiration for research ideas?

   A. Personal experience and informal observations

   B. Only reviewing published journal articles

   C. Waiting for completely novel ideas to emerge

   D. Avoiding topics that have been previously studied

   4. Why is Wikipedia considered unsuitable as a basis for sound scientific research according to the chapter?

   A. It is too difficult to access and navigate

   B. Authors are anonymous, may lack formal training, and content continually changes

   C. It only contains information from pop psychology books

   D. It does not include any peer-reviewed sources

   5. An empirically testable research question must be expressed in terms of:

   A. Abstract philosophical concepts that cannot be measured

   B. A single variable or relationship between variables that can be measured

   C. Only correlational relationships between variables

   D. Broad generalizations about human behavior

   6. If you find that your research question has already been studied scientifically, the chapter suggests you should:

   A. Immediately abandon it and find a completely different topic

   B. Refine the question by considering other ways to measure variables or different populations

   C. Only replicate the study exactly as originally conducted

   D. Wait until the previous research becomes outdated

   7. How does a scientific theory differ from a hypothesis?

   A. A theory is untested while a hypothesis has been proven true

   B. A hypothesis is a coherent explanation of phenomena while a theory is a specific prediction

   C. A theory is a broader explanation while a hypothesis is a specific prediction about new phenomena

   D. There is no meaningful difference between theories and hypotheses

   8. In the study by Schwarz and colleagues on assertiveness judgments, participants who recalled 6 examples rated themselves as MORE assertive than those who recalled 12 examples. This finding supported which theory?

   A. The number-of-examples theory

   B. The ease-of-retrieval theory

   C. Both theories equally

   D. Neither theory

   9. What is an operational definition?

   A. A theoretical explanation of a psychological concept

   B. A definition of a variable in terms of precisely how it will be measured

   C. The statistical procedure used to analyze data

   D. A hypothesis stated in measurable terms

   10. A researcher measures whether drivers using cell phones are more likely to be male or female without manipulating any variables. This study would be classified as:

   A. Experimental research because it involves observation

   B. Non-experimental research because nothing is manipulated

   C. A laboratory study because it uses careful measurement

   D. Invalid research because there is no independent variable

   11. What is the primary difference between descriptive and inferential statistics?

   A. Descriptive statistics summarize data while inferential statistics draw conclusions about populations from samples

   B. Descriptive statistics are used for experiments while inferential statistics are used for correlational studies

   C. Descriptive statistics require larger samples than inferential statistics

   D. Descriptive statistics are more accurate than inferential statistics

   12. A researcher concludes there is no effect of a treatment when there actually is a real effect in the population. This error is called:

   A. A Type I error (false positive)

   B. A Type II error (missed opportunity)

   C. A sampling error

   D. A confounding variable

   13. According to the chapter, why can scientists never 'prove' a theory to be true?

   A. Because statistical tests are not accurate enough

   B. Because there is always the possibility of errors

   C. Because statistics are probabilistic and there is always the possibility that results occurred by chance

   D. Because theories must be replaced every decade

   14. The 'problem of induction' means that even observing many confirming cases (e.g., white swans) cannot definitively prove a general principle because:

   A. The observations might reflect experimenter bias

   B. A disconfirming case (e.g., black swan) could always be discovered later

   C. Statistical significance is required for all theories

   D. Theories must be published before they can be proven

   Answer Key with Explanations

   1. B - Conduct an empirical study designed to answer the question

   The model shows that after formulating a research question, researchers conduct an empirical study designed to answer that question. Only after conducting the study do they analyze data and draw conclusions.

   2. B - Published research leads to new questions, which lead to new research

   Cell phone and driving research demonstrates this cycle as follows: early studies on verbal tasks and perceptual/motor tasks led to questions about cell phone use specifically, which led to studies comparing driving with and without cell phones, which then led to new questions about passengers versus cell phone conversations. Because the research literature is one of the primary sources of new research questions, this process can be thought of as a cycle. New research leads to new questions, which lead to new research, and so on.

   3. A - Personal experience and informal observations

   Personal experience and informal observations are important sources of research ideas, along with professional literature and previous research.

   4. B - Authors are anonymous, may lack formal training, and content continually changes

   Wikipedia may contain some valuable information, because its authors are anonymous and may not have any formal training or expertise in that subject area and its content continually changes, it is unsuitable as a basis of sound scientific research. This distinguishes it from peer-reviewed sources where authors are identified, have credentials, and content is stable once published.

   5. B - A single variable or relationship between variables that can be measured

   Empirically testable research questions must be expressed in terms of a single variable or relationship between variables that can be measured. This allows the question to be answered through systematic observation.

   6. B - Refine the question by considering other ways to measure variables or different populations

   If research has been conducted on your research question, you can further explore the topic by refining the question, for example: 'Are there other ways to define and measure the variables? Are there types of people for whom the relationship might be stronger or weaker?' The example given in the chapter is that while research showed women and men speak the same number of words, researchers could still ask about elderly people, other cultures, or different measures of talkativeness.

   7. C - A theory is a broader explanation while a hypothesis is a specific prediction about new phenomena

   A theory is a coherent explanation or interpretation of phenomena, while a hypothesis is a specific prediction about new phenomena that should be observed if a particular theory is accurate.

   8. B - The ease-of-retrieval theory

   Both theories held that such judgments are based on relevant examples that people bring to mind. However, one theory was that people base their judgments on the number of examples they bring to mind and the other was that people base their judgments on how easily they bring those examples to mind. The number-of-examples theory predicted those recalling 12 examples would judge themselves MORE assertive, but the ease theory predicted those recalling 6 examples (easier) would judge themselves MORE assertive. The finding that 6-example participants rated themselves as more assertive provided particularly convincing evidence in favor of the ease-of-retrieval theory over the number-of-examples theory.

   9. B - A definition of a variable in terms of precisely how it will be measured

   An operational definition defines a variable in terms of precisely how it will be measured in a particular study. This allows researchers to translate conceptual variables into concrete measurements.

   10. B - Non-experimental research because nothing is manipulated

   This question tests the critical distinction between experimental and non-experimental research. The defining feature in this example is that nothing is manipulated - variables are simply measured as they naturally occur. It is important to point out that non-experimental does not mean nonscientific. Non-experimental research is scientific in nature.

   11. A - Descriptive statistics summarize data while inferential statistics draw conclusions about populations from samples

   Descriptive statistics are used to organize or summarize a set of data, while inferential statistics are used to draw conclusions about populations based on sample data and assess the probability that results reflect real effects.

   12. B - A Type II error (missed opportunity)

   A Type II error occurs when a researcher concludes that their results are not statistically significant when in reality there is a real effect in the population and they just missed detecting it. In contrast, a Type I error is when a researcher concludes that their results are statistically significant (so they say there is an effect in the population) when in reality there is no real effect in the population.

   13. C - Because statistics are probabilistic and there is always the possibility that results occurred by chance

   Scientists avoid the word 'prove' because statistics are probabilistic in nature and findings can reflect errors. There is always the possibility that results occurred by chance, so scientists can only support theories, not prove them with absolute certainty.

   14. B - A disconfirming case (e.g., black swan) could always be discovered later

   This is a version of the "problem of induction." One cannot definitively prove a general principle (e.g., "All swans are white.") just by observing confirming cases (e.g., white swans)—no matter how many. It is always possible that a disconfirming case (e.g., a black swan) will eventually come along. For these reasons, scientists tend to think of theories—even highly successful ones—as subject to revision based on new and unexpected observations.

   References

Adair, J. G., & Vohra, N. (2003). The explosion of knowledge, references, and citations: Psychology’s unique response to a crisis. American Psychologist, 58, 15–23.

Collet, C., Guillot, A., & Petit, C. (2010). Phoning while driving I: A review of epidemiological, psychological, behavioral and physiological studies. Ergonomics, 53, 589–601.

Drews, F. A., Pasupathi, M., & Strayer, D. L. (2004). Passenger and cell-phone conversations in simulated driving. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 48, 2210–2212.

Milgram, S. (1963). Behavioral study of obedience. Journal of Abnormal and Social Psychology, 67, 371–378.

Mueller, P. A., & Oppenheimer, D. M. (2014). The pen is mightier than the keyboard: Advantages of longhand over laptop note taking. Psychological Science, 25(6), 1159-1168.

Schwarz, N., Bless, H., Strack, F., Klumpp, G., Rittenauer-Schatka, H., & Simons, A. (1991). Ease of retrieval as information: Another look at the availability heuristic. Journal of Personality and Social Psychology, 61, 195–202.

Weisberg, R. W. (1993). Creativity: Beyond the myth of genius. New York, NY: Freeman.

Zajonc, R. B., Heingartner, A., & Herman, E. M. (1969). Social enhancement and impairment of performance in the cockroach. Journal of Personality and Social Psychology, 13, 83–92.

Zajonc, R. B. (1965). Social facilitation. Science, 149, 269–274

Zajonc, R.B. & Sales, S.M. (1966). Social facilitation of dominant and subordinate responses. Journal of Experimental Social Psychology, 2, 160-168.

Exercises

• Practice: Find a description of an empirical study in a professional journal or in one of the scientific psychology blogs. Then write a brief description of the research in terms of the cyclical model presented here. One or two sentences for each part of the cycle should suffice.
• Watch the following TED Ed video, in which David H. Schwartz provides an introduction to two types of empirical studies along with some methods that scientists use to increase the reliability of their results:
  

  A YouTube element has been excluded from this version of the text. You can view it online here: https://kpu.pressbooks.pub/psychmethods4e/?p=271

  • Reading in print? Go to https://youtu.be/GUpd2HJHUt8 or scan this QR code with your phone:

• Practice: Use the techniques discussed in this section to find 10 journal articles and book chapters on one of the following research ideas: memory for smells, aggressive driving, the causes of narcissistic personality disorder, the functions of the intraparietal sulcus, or prejudice against the physically handicapped.
• Watch the following video clip produced by UBCiSchool about how to read an academic paper (without losing your mind):
  

  A YouTube element has been excluded from this version of the text. You can view it online here: https://kpu.pressbooks.pub/psychmethods4e/?p=271

  • Reading in print? Go to https://youtu.be/SKxm2HF_-k0 or scan this QR code with your phone:

• Practice: Generate three research ideas based on each of the following: informal observations, practical problems, and topics discussed in recent issues of professional journals.
• Practice: Generate an empirical research question about each of the following behaviors or psychological characteristics: long-distance running, getting tattooed, social anxiety, bullying, and memory for early childhood events.
• Practice: Evaluate each of the research questions you generated in Exercise 2 in terms of its interestingness based on the criteria discussed in this section.
• Practice: Find an issue of a journal that publishes short empirical research reports (e.g., Psychological Science, Psychonomic Bulletin and Review, Personality and Social Psychology Bulletin). Pick three studies, and rate each one in terms of how feasible it would be for you to replicate it with the resources available to you right now. Use the following rating scale: (1) You could replicate it essentially as reported. (2) You could replicate it with some simplifications. (3) You could not replicate it. Explain each rating.
• Practice: Find a recent empirical research report in a professional journal. Read the introduction and highlight in different colors descriptions of theories and hypotheses.
• Practice: Using the research article you found in a professional journal identify whether the study was experimental or non-experimental. If it was experimental identify the independent and dependent variables.
• Practice: Using the research article you found in a professional journal identify which descriptive statistics were reported.
• Practice: Describe why theories can be supported but not proved.