2.3.2: Research Questions

Learning Objectives

1. Describe some techniques for turning research ideas into empirical research questions and use those techniques to generate questions.
2. Explain what makes a research question interesting and evaluate research questions in terms of their interestingness.

Research Questions

We started reviewing the scientific method with reviewing research literature, the research process could easily start with research questions. As the model suggestions, informal observations or practical problems can lead to a research question, which would lead to a review of the research literature on those topics.

Generating Empirically Testable Research Questions

Once you have a research idea, you need to use it to generate one or more empirically testable research questions, that is, questions expressed in terms of a single variable or relationship between variables. One way to do this is to look closely at the discussion section in a recent research article on the topic. This is the last major section of the article, in which the researchers summarize their results, interpret them in the context of past research, and suggest directions for future research. These suggestions often take the form of specific research questions, which you can then try to answer with additional research. This can be a good strategy because it is likely that the suggested questions have already been identified as interesting and important by experienced researchers.

But you may also want to generate your own research questions. How can you do this? First, if you have a particular behavior or psychological characteristic in mind, you can simply conceptualize it as a variable and ask how frequent or intense it is. How many words on average do people speak per day? How accurate are our memories of traumatic events? What percentage of people have sought professional help for depression? If the question has never been studied scientifically—which is something that you will learn when you conduct your literature review—then it might be interesting and worth pursuing.

If scientific research has already answered the question of how frequent or intense the behavior or characteristic is, then you should consider turning it into a question about a relationship between that behavior or characteristic and some other variable. One way to do this is to ask yourself the following series of more general questions and write down all the answers you can think of.

• What are some possible causes of the behavior or characteristic?
• What are some possible effects of the behavior or characteristic?
• What types of people might exhibit more or less of the behavior or characteristic?
• What types of situations might elicit more or less of the behavior or characteristic?

In general, each answer you write down can be conceptualized as a second variable, suggesting a question about a relationship. If you were interested in talkativeness, for example, it might occur to you that a possible cause of this psychological characteristic is family size. Is there a relationship between family size and talkativeness? Or it might occur to you that people seem to be more talkative in same-sex groups than mixed-sex groups. Is there a difference in the average level of talkativeness of people in same-sex groups and people in mixed-sex groups? This approach should allow you to generate many different empirically testable questions about almost any behavior or psychological characteristic.

If through this process you generate a question that has never been studied scientifically—which again is something that you will learn in your literature review—then it might be interesting and worth pursuing. But what if you find that it has been studied scientifically? Although novice researchers often want to give up and move on to a new question at this point, this is not necessarily a good strategy. For one thing, the fact that the question has been studied scientifically and the research published suggests that it is of interest to the scientific community. For another, the question can almost certainly be refined so that its answer will still contribute something new to the research literature. Again, asking yourself a series of more general questions about the relationship is a good strategy.

• Are there other ways to define and measure the variables?
• Are there types of people for whom the relationship might be stronger or weaker?
• Are there situations in which the relationship might be stronger or weaker—including situations with practical importance?

For example, research has shown that women and men speak about the same number of words per day—but this was when talkativeness was measured in terms of the number of words spoken per day among university students in the United States and Mexico. We can still ask whether other ways of measuring talkativeness—perhaps the number of different people spoken to each day—produce the same result. Or we can ask whether studying elderly people or people from other cultures produces the same result. Again, this approach should help you generate many different research questions about almost any relationship.

Another way to derive research questions is from theories. A theory is a coherent explanation or interpretation of one or more phenomena. Although theories can take a variety of forms, one thing they have in common is that they go beyond the phenomena they explain by including variables, structures, processes, functions, or organizing principles that have not been observed directly. Asking if a particular theory is accurate in specific situations is a research question.

Evaluating Research Questions

Researchers usually generate many more research questions than they ever attempt to answer. This means they must have some way of evaluating the research questions they generate so that they can choose which ones to pursue. In this section, we consider two criteria for evaluating research questions: the interestingness of the question and the feasibility of answering it.

Interestingness

How often do people tie their shoes? Do people feel pain when you punch them in the jaw? Are women more likely to wear makeup than men? Do people prefer vanilla or chocolate ice cream? Although it would be a fairly simple matter to design a study and collect data to answer these questions, you probably would not want to because they are not interesting. We are not talking here about whether a research question is interesting to us personally but whether it is interesting to people more generally and, especially, to the scientific community. But what makes a research question interesting in this sense? Here we look at three factors that affect the interestingness of a research question: the answer is in doubt, the answer fills a gap in the research literature, and the answer has important practical implications.

First, a research question is interesting to the extent that its answer is in doubt. Obviously, questions that have been answered by scientific research are no longer interesting as the subject of new empirical research. But the fact that a question has not been answered by scientific research does not necessarily make it interesting. There has to be some reasonable chance that the answer to the question will be something that we did not already know. But how can you assess this before actually collecting data? One approach is to try to think of reasons to expect different answers to the question—especially ones that seem to conflict with common sense. If you can think of reasons to expect at least two different answers, then the question might be interesting. If you can think of reasons to expect only one answer, then it probably is not. The question of whether women are more talkative than men is interesting because there are reasons to expect both answers. The existence of the stereotype itself suggests the answer could be yes, but the fact that women’s and men’s verbal abilities are fairly similar suggests the answer could be no. The question of whether people feel pain when you punch them in the jaw is not interesting because there is absolutely no reason to think that the answer could be anything other than a resounding yes.

A second important factor to consider when deciding if a research question is interesting is whether answering it will fill a gap in the research literature. According to Emerald Publishing Group's guide to the peer review process, an important criteria for your research question should be "Does the article or case study say something original? Does it add to the body of knowledge?" So you could start with a question about how to help community members, and start reading academic literature to understand the issue better. You can use this literature review to determine whether your original research question has already been answered. This means that the question has not already been fully answered by scientific research.

A final factor to consider when deciding whether a research question is interesting is whether its answer has important practical implications. Again, the question of whether taking notes by hand improves learning has important implications for education, including classroom policies concerning technology use. The question of whether cell phone use impairs driving is interesting because it is relevant to the personal safety of everyone who travels by car and to the debate over whether cell phone use should be restricted by law.

Feasibility

A second important criterion for evaluating research questions is the feasibility of successfully answering them. There are many factors that affect feasibility, including time, money, equipment and materials, technical knowledge and skill, and access to research participants. Clearly, researchers need to take these factors into account so that they do not waste time and effort pursuing research that they cannot complete successfully. However, as a newcoming to research methodology, you are encouraged to not drop a research question when you can't immediately think of an easy way to answer the question. It takes years of practice plus creativity to design high-quality research studies!

Looking through a sample of professional journals during your literature review will help you see how other researchers have tested similar questions. Keep in mind, too, that research tends to be carried out by teams of highly trained researchers whose work is often supported in part by government and private grants. Also, keep in mind that research does not have to be complicated or difficult to produce interesting and important results. Looking through a sample of professional journals will also reveal studies that are relatively simple and easy to carry out—perhaps involving a convenience sample of university students and a paper-and-pencil task.

A final point here is that it is generally good practice to use methods that have already been used successfully by other researchers. For example, if you want to manipulate people’s moods to make some of them happy, it would be a good idea to use one of the many approaches that have been used successfully by other researchers (e.g., paying them a compliment). This is good not only for the sake of feasibility—the approach is “tried and true”—but also because it provides greater continuity with previous research. This makes it easier to compare your results with those of other researchers and to understand the implications of their research for yours, and vice versa.

Research Hypotheses

Once you have a general idea for a research question, you can start identifying the variables that you'd like to focus on and how they might related. This becomes your research hypothesis. A research hypothesis is a specific prediction about how variables interact with each other that can be tested. Research hypotheses are developed by considering existing evidence and using reasoning to infer what will happen in the specific context of interest. Hypotheses are usually expressed as statements, but these predictions can be rephrased from research questions. You could start with a broad research question, such as "What study skills will help me pass my class?" or a more specific research question, such as "Will organizing my notes help me pass my class?". To turn this into a research hypothesis, you could answer this question (with a comparison group) by stating: "Students who organize their notes regularly will earn higher grades than students who do not organize their notes."

Good Research Hypotheses

There are three general characteristics of a good hypothesis. First, a good hypothesis must be testable and falsifiable. We must be able to test the hypothesis using the methods of science and it must be possible to gather evidence that will disconfirm the hypothesis if it is indeed false (Popper’s falsifiability criterion). Second, a good hypothesis must be logical. As described above, hypotheses are more than just a random guess. Hypotheses should be informed by theories or observations and logical reasoning. Finally, the hypothesis should be positive. That is, the hypothesis should make a positive statement about the existence of a relationship or effect, rather than a statement that a relationship or effect does not exist. As scientists, we don’t set out to show that relationships do not exist or that effects do not occur so our hypotheses should not be worded in a way to suggest that an effect or relationship does not exist. The nature of science is to assume that something does not exist and then seek to find evidence to prove this wrong, to show that it really does exist. That may seem backward to you but that is the nature of the scientific method. The underlying reason for this is beyond the scope of this chapter, but it has to do with statistical theory. You may remember that a null hypothesis always says that nothing is happening, and that is what inferential statistical analyses are testing. Then, if our data align, we can reject the null hypothesis that nothing is happening and hopefully support the research hypothesis (that something is happening, and it's happening in the direction that we predicted). What null hypothesis significance testing can't have is to retain the null hypothesis (that nothing is happening) and support the research hypothesis, so all research hypotheses must predict that the variables do interact in some way.

Research Hypotheses and the Goals of Science

Exercise \(\PageIndex{1}\)

What were the three Goals of Science (see section 2.2)?

Answer

The three goals of science listed in section 2.2 were:

1. To describe,
2. To predict, or
3. To intervene.

We can create research hypotheses based on the three goals of science. Starting with the easiest (and least interesting) goal, a research hypothesis that predicts a description would predict the state of the topic. For Imada et al.'s (2024) study of ingroup favoritism, this could be to show that ingroup favoritism exists, or that each of the options to explain ingroup favoritism (reputation management or expectation of cooperation) do happen. For example, "We hypothesize that people treat ingroup members better than outgroup members." The second goal of science, to predict, is often a research hypothesis that compares to groups or two variables to show that they are related. An example related to ingroup favoritism could be: "We hypothesize that higher expectations of cooperation from ingroup members are related to higher ingroup favoritism." The third goal of science, intervention, may have a similar research hypothesis as the second goal, but the research design would differ. To make predictions, we could ask all sorts of people questions about their reputation management, expectations of ingroup cooperation, and ingroup favoritism. But to intervene, we would have to somehow nudge participants to either focus on reputation management or to focus on expectations of ingroup cooperation, then measure which foci was most impactful on measures of ingroup favoritism. Students usually recognize the simplicity of the first goal of science (description), but often get stuck on the second goal (prediction). To actually improve outcomes for your clients, customers, students, or patients, professionals need to focus on the third goal of science (intervention). It is not enough to know that medical marijuana use is related to pain reduction; we want to ensure that the treatment that we prescribe causes pain reduction. If medical marijuana works because of its anti-inflammation properties, then pain researchers could focus on the best anti-inflammatory treatments.