4.7: Unobtrusive Research

Learning Objectives

• Identify some general strengths and weakness of self-report measures.
• List types of unobtrusive measures of behavior.
• Identify some general strengths and weakness of unobtrusive research.
• Apply how the Hawthorne Effect could affect self-report measures.

Alternatives to Self-Report Measures

We've spent a lot of time talking about gathering self-report information. Participants often try to be honest and accurate in their self-reports, but sometimes that's not possible: when people do not have the ability to report on their behavior and when people do not have the willingness to report on their own behavior.

When might participants not have the ability to report on themselves? The common reasons are:

• Fatigue
• Vocabulary
• Knowledge (of self, motivations)

Let's unpack those ideas. First, fatigue. While it is true that sometimes participants may be too sleep deprived to answers questions about themselves accurately, fatigue tends to affect self-report measures when the measures are really long. Many careers require applicants to take a personality test. The Minnesota Multiphasic Personality Inventory-2 (MMPI-2, Butcher et al., 1989) is a common assessment. This personality survey has 567 questions, and takes respondents about an hour to an hour and a half to complete. Imagine how mentally and emotionally tired you would be after having answers 560 questions about yourself over the course of an hour or more. I bet at that point, your self-report may not be as accurate as it was it was when you first started the survey.

Second, vocabulary. Many times, researchers ask questions of respondents using the technical language of the field of study. Respondents will try to answer honestly and accurately, but they might not know the answers because they do not know what some of the words in the questions mean. Third, self-knowledge. Imagine that a researcher asked how often you had worked out in the last week. You may think about your goal of working out three days a week, so you assume that you worked out three times in the past week. However, if the researcher checked your activity on a wearable device like a smart watch, they might notice that your heart rate and movement indicated working out only two times that week. It might not be that you intentionally provided false information, but that you didn't take the time to provide accurate information. Additionally, people explain why they behave certain ways even when they do not actually know why that do something. This TED Talk (Johansson, 2016) shows how people explain why they made a choice, even when that was not actually the choice that they made.

Finally, some people do not have the willingness to report accurately. This usually occurs when you are motivated to look good (social desirability). To avoid some of this pitfalls in self-report, we will consider ways unobtrusively measure behavior. Unobtrusive research refers to methods of collecting data that don’t interfere with the participants under study.

Unobtrusive Research

Researchers who seek evidence of what people actually do, as opposed to what they say they do (as in survey and interview research), might wish to consider using unobtrusive methods. The unobtrusive methods that will be discussed in the follow sub-sections are:

• Physical Trace
• Archival Data

Some researchers treat observation as unobtrusive measures, and sometimes it is. But observational research can manipulate variables (think structured observations), and sometimes are undisguised. Because of the variety of observational methodologies, this type of measurement was described separately from these other unobtrusive methods.

Benefits of Unobtrusive Research

Research participants, either intentionally or unintentionally, altering their behaviors because they know they are being studied, known as the Hawthorne effect, is not a concern for unobtrusive researchers because the participants do not know that they are being studied. While unobtrusive research projects, like all research projects, face the risk of introducing researcher bias into the work, researchers employing unobtrusive methods do not need to be concerned about the effect of the research on their participants. In fact, this is one of the major strengths of unobtrusive research.

Another benefit of unobtrusive research is that it can be relatively low-cost compared to some of the other methods we’ve discussed. Researchers may be able to access data without having to worry about paying participants for their time (though certainly travel to or access to some documents and archives can be costly, as well as the time it takes to make observations).

A benefit of archival data is that it is well suited to studies that focus on processes that occur over time. Archival data can provide accurate information about examine processes that occurred decades before data collection began, and are a cost-effective way to examine long-ranging processes. Unobtrusive methods do not rely on retrospective accounts, which may be subject to errors in memory, as some longitudinal surveys do.

Weaknesses of Unobtrusive Research

While there are many benefits to unobtrusive research, this method also comes with a unique set of drawbacks. Because unobtrusive researchers analyze data that may have been created or gathered for purposes entirely different from the researcher’s aim, problems of validity sometimes arise in such projects. It may also be the case that data sources measuring whatever a researcher wishes to examine simply do not exist. This means that unobtrusive researchers may be forced to tweak their original research interests or questions to better suit the data that are available to them.