1.4: Identifying the research question (and an aside about thoery)

The model presents the research process as circular, but identifying the research question is a good starting point. In this step, we specify what it is that we want to learn more about. Usually, but not always, this takes the form of a question. It could also be a statement of research purpose, though. When doing empirical research, it’s important to develop a question that can be answered—or that one can attempt to answer—based on observations. A simple research question would be How many candidates for public office use negative campaign advertisements to detract from their opponents? We could come up with a defensible answer (we rarely come up with absolutely conclusive answers in social research) to this question based on observations.

There are other types of questions that empirical social research cannot answer. Empirical social research methods do not answer normative questions. Normative questions are questions that are answered based on opinions, values, and subjective preferences. Normative questions often have the word should in them: Should candidates for public office use negative campaign advertisements? Should donations to churches be tax deductible? Should corporations be required to disclose lobbying expenses? Should universities consider race in making admissions decisions? In these examples, no amount of systematic observation can provide a defensible answer to the question; ultimately, answering these questions is a matter of subjective values. However—and this is a very important however—empirical research can help us develop better informed opinions about these normative questions. To help develop a better informed opinion about whether or not candidates should use negative campaign ads, a researcher might investigate related empirical questions, such as How do negative campaign ads affect voter behavior? and How do negative campaign ads affect voters’ opinions about the endorsing candidate? Social researchers, then, don’t run away from normative questions—most interesting questions are normative—but, instead, look for opportunities for empirical research to shed light on normative questions.

Even this, though, is oversimplifying a bit too much. It’s naïve to think that doing empirical research is value-free. Our values influence our decisions throughout the entire research process, from what to study, to how to make observations, to how we make sense of what we observe. Objectivity is a worthy goal when doing empirical social research, but it is an elusive goal, and we should always try to be aware of and transparent about how our own biases affect our research.

Still other interesting questions are the domain of legal analysis, philosophy, or history, not empirical social science research. Legal analysis is required to tackle questions like Can state governments constitutionally cede authority to local governments to allow or ban carrying handguns in public parks? Questions about events from the distant past (an admittedly ambiguous standard) are generally left to historians, though some questions reside in a gray area where empirical research methods could be used to learn about historical events.

The distinction between the domains of social research and history raises an important point: When conducting social research, our goal is usually to build knowledge that is generalizable; that is, we usually want to be able to apply what we learned from our observations to other cases, settings, or times. We may make observations of one local election, but with the goal of generating knowledge that could be applied to local elections in other jurisdictions, to future or past local elections, or to citizen participation in administrative rulemaking at the local level. While historians may be more likely to do research to build in-depth knowledge about a single case, we rarely undertake a social research project with the goal of generating knowledge that would be applied only to understanding what we’ve directly observed. (A partial exception to this would be when we conduct case studies, discussed later—but this is only a partial exception.)

Empirical research questions can have different purposes. Some empirical social science research questions seek to describe social phenomena. Sometimes, you’ll see the phrase mere description used, and some research methods textbook authors will say that description doesn’t even count as research. This is nonsense. Describing social phenomena based on systematic observations is certainly a legitimate purpose of social science research.

When these textbook authors diminish the importance of description, what they have in mind as more suitable research purposes are explanation and prediction. By pursuing these research purposes, we are now exploring questions of causality. If we’re explaining something, we’ve observed something occur, and then we’re looking back in time, in a sense, to figure out what caused it to occur: Why were high- and middle-income independent voters less likely to vote for the Democratic candidate than low-income independent voters in the last gubernatorial election? There, we’ve observed something interesting about the last gubernatorial election, and we want to figure out what happened before to explain it. If we’re predicting something, we observe past trends or the state of things now and use those observations to predict what will happen in the future: How will low-income voters vote in the upcoming state senate election? We’ll come back to the notion of causality shortly.

Research questions with the purposes of description, explanation, and prediction are all pursued using a broad range of social research methods. A fourth research purpose, understanding, though, is more tightly coupled with a narrower range of research methods—those methods that center around collecting and analyzing qualitative data. Qualitative data are usually words, but they can also be pictures or sounds—basically, any data that are not numeric. Transcripts of interviews with campaign managers, the text of administrative agencies’ requests for proposals, the text of Supreme Court opinions, survey respondents’ answers to open-ended questions, and pictures of people in a political protest are all examples of qualitative data. (Quantitative data, on the other hand, are numeric. More on different types of data later.) With the research purpose of understanding, we are not using the term “understanding” in its colloquial sense; instead, we mean “understanding” with the connotation of verstehen, a German word that doesn’t translate into English very well but carries the idea of understanding someone else’s subjective experiences. When conducting research with the goal of verstehen, we want to achieve an in-depth understanding of others’ opinions, attitudes, motivations, beliefs, conceptual maps, and so on. Typically, this would involve talking with them, listening to their words, or reading what they’ve written—thus the association of qualitative data collection with research questions that have the goal of achieving understanding-qua-verstehen.

To be clear: Research projects with the purposes of description, explanation, and prediction use the full range of research methods, including the collection of both quantitative and qualitative data; research projects with the purpose of understanding generally use methods focused on collecting qualitative data.

Research questions, then, can pursue one or more of these four purposes—description, explanation, prediction, and understanding—but where do research questions come from? At some point in their studies, most students will know the fear of the blank page: Where do I start? What is my research question? Research questions might occasionally arrive in a flash of inspiration, but, usually, their origins are more mundane and require more work. I think most social researchers would agree that their research questions come from some combination of four starting points: deduction, induction, previous research, and what I’ll just describe for now as one of the research profession’s dirty little secrets.

The classic “correct” textbook answer to the question of where research questions come from is deduction from theory. By employing deductive thinking, we start with a theory and deduce the research questions that it suggests.

Before going any further into deducing research questions, though, we should pause for a moment on that other term, theory. A theory is simply a set of concepts and relationships among those concepts that helps us understand or explain some phenomenon—for us, a social phenomenon. Sometimes, theories are very formal; they’re written down in a concise statement in a definitive form by a specific author or group of authors, and they include a wholly specified set of concepts; everybody knows what’s in the theory and what’s out. Maslow’s Hierarchy of Needs—that model of human motivation that seems to crop up in every other undergraduate course—comes to mind as an example of a formal theory. In this theory, a specific set of concepts (the need for socialization, the need for security, and so on) are related in a specific way to explain why people do what they do. Other theories, though, are relatively loose; they’re evolving, they’re gleaned from across a wide range of writings and assembled in different ways by different people, and there might be disagreement over precisely which concepts are included and which are not. I once used something called “crowding out theory” as it applies to charitable giving to nonprofit organizations, and I had to piece together my own version of this theory by reading what a lot of other people had written about it. My version would have looked somewhat like others’, but not identical. My formulation of the theory linked concepts like charitable giving, government funding, donors’ perceptions of government funding, and nonprofit managerial capacity to predict how charitable donors would react to nonprofit organizations receiving different types of government subsidy.

(A quick aside to students interested in studying public policies, programs, and organizations. You are my people. When we conduct research about a particular program, public policy, or organization, a model of the program, policy, or organization often plays the role of theory in the research process. A logic model, for example, depicts a program in terms of its inputs, activities, outputs, and outcomes—not unlike a set of concepts and relationships among those concepts. I’ve provided an example of a logic model and how it can generate a lot of applied research questions in Appendix A.)

... Everyone else—just in case you skipped that last paragraph: You should read Appendix A, too—you’ll find the examples of empirical research questions helpful.

A theory (or program, policy, or organization model), needn’t be such a complicated thing, but I think many students are like I was as an undergraduate student (and even into my graduate study years): intimidated by theory. I didn’t totally understand what theory was, and I thought handling theory was best left to the professionals. Like most students, I thought of theory as an antique car—the kind of antique car that is kept in pristine condition, all shiny and perfect, in its climate-controlled garage, rolled out only to show off, and then rolled back in for safe keeping. It turns out, though, that most researchers don’t view theories this way at all. Instead, they view their theories as beat-up pickup trucks. They’re good insofar as they’re useful for doing their job. It’s OK if they get dinged up in the process. They’re not just rolled out for showing off; they’re used to help understand the world, driven as far as they’ll go. (I stole this analogy from one of my professors, Gordon Kingsley, but, like a good theory, I’ve modified it a bit to suit our purposes here.)

As suggested by our model of the research process, theory is at the center of the entire process (not just at the beginning like in some other models); it’s the touchstone for every step along the way, including the step at hand: identifying a research question. To develop a research question, we can start with a theory and all its concepts and relationships among those concepts to deduce research questions—questions that, essentially, ask whether the theory matches observations in the real world. Maslow’s Hierarchy of Needs, for example, might suggest the question, Are voters whose basic needs are not being met more likely than others to support candidates who promise to alleviate citizens’ security and safety needs? Here, we have developed a question that uses a theory as a starting point, at least, for explaining a political phenomenon. How did we deduce this research question from our theory? The theory helped us identify relevant concepts, like voters’ security and safety needs and candidates’ promises to alleviate them, and a potential relationship between these concepts and what we’re interested in explaining, voters’ choice of candidate. (And like most empirical research based on Maslow’s theory, we might have difficulty finding much empirical support for it.)

Research questions may also be developed inductively by observing social phenomena and then developing research questions based on what has been observed. Perhaps you observe more men than women in your political science courses but not in your other courses. You can make this casual observation the basis of a research question: Are men more likely to take political science courses than women? or How does students’ sex relate to their course selection? or How does gender socialization affect students’ selection of majors? Researchers with an application orientation may simply experience a problem and develop a research question to figure out how to overcome it: Why did unemployment benefit claim processing time increase by 50% last year? You may find that your casual observations reflect regularities confirmed through systematic observations, and, ultimately, you may even develop a theory or modify an existing theory based on what you learned. So, whereas a deductive research process begins with theory and generalizations that lead to observation, an inductive research process begins with observations that lead to generalizations and theory.

Our model of the research process points to another source of research questions: previous research. Previous research usually refers to all of the publications that report the results of research that has already been conducted on a given topic. We use previous research to develop research questions in a couple of ways. If there’s a social phenomenon we’d like to learn more about, a good starting point is to read all of the previous research on that topic. Once we have a command of that body of knowledge, we can start to identify gaps, internal inconsistencies, unresolved questions, and emerging research directions in the literature. It’s one small step further to develop research questions that build on the existing body of research. Sometimes, using previous research is more literal; often, an article, chapter, or book will include a section titled something like “Recommendations for future research,” and, voilà, you have a research question. (As portrayed in the model, generating research questions isn’t the only use of previous research; it’s used throughout the entire research process, as we’ll see.)

And then there’s the dirty little secret of the social research professions. Sometimes we begin, not nobly with a theory, not astutely with our own observations, not studiously with previous research, but shamelessly with available data. An aspiring researcher can simply comb through data in hand in search of a research question that can be asked of it. Have access to data collected through the General Social Survey, a public opinion survey conducted every two years?

Read through the table of contents, find some questions that might go together, and try it out. Let the availability of the data—not theoretic or practical import or even your own casual observations—make you interested in a research question. This approach is roundly criticized because it smacks of data fishing; it’s almost always possible to find some patterns in your data, even if it’s just a fluke. Data fishing is exploiting these fluky patterns by making them seem important even when they’re not. Baseless dataset dredging is not a good starting point for conducting research. It happens, though. Untenured assistant professors and dissertation- writing doctoral students are under tremendous pressure to publish research, and the unfortunate truth is that papers reporting “null findings” don’t get published very often. Safer to start with a pattern you’ve stumbled upon in your data and then figure out how to make it sound important, like something you went looking for, so the thinking goes. This approach isn’t entirely bad; there are legitimate ways to conduct data mining (the more acceptable term). Data are collected because someone thought they were important, so it’s not inconceivable that you could uncover important, unanticipated patterns in your data. Thinly disguised data fishing, though, is quickly identified and disregarded by other researchers.

Before we wrap up our consideration of research questions, we should spend a moment unpacking the notion of causality. Three concepts will help us understand how social research approaches questions of cause-and-effect: probabilistic causality, multiple causation, and underlying causal mechanisms. When we seek causal explanations in social research, we rarely talk in absolutes. The type of causality often studied in the physical sciences is deterministic causality, meaning definite cause-and-effect relationships: Flipping the switch causes the light to come on. In the social sciences (though not exclusively in the social sciences), we are almost always studying questions of probabilistic causality, meaning cause-and-effect relationships that are more or less likely to occur: People are less likely to vote for incumbents when the unemployment rate is high. We are also almost always explaining and predicting phenomena that have multiple, interacting causes—multiple causation. Why do some people have higher incomes than others? This surely has many causes—education, age, ability, parents’ wealth, motivation, discrimination, opportunity, job choice, attitudes toward work and money, and so on. And these causes, themselves, affect each other. Most advanced social research attempts to figure out these complex, interacting cause-and-effect relationships. When we make causal claims like age affects income, we are really masking a more complex web of cause-and-effect relationships. Does our age really, inherently, affect our income? Not really. Age affects income in the sense that this ostensible relationship is the manifestation of a more complex underlying causal mechanism. This underlying causal mechanism explains why age seems to affect income—a cause-and-effect story about biological development, the accumulation of education and experience, and the demands of different stages of life. We’ll revisit underlying causal mechanisms in the next section when we learn about independent and dependent variables.