3.4: Methods of Data Collection

Let’s look more closely at some actual techniques, or research methods, used to measure, assess or test variables. Depending partly on what type of method is used, the design would be considered experimental, correlational or quasi experimental, and as discussed above, each type of design has different advantages and disadvantages, particularly in terms of control and generalizability.

Observational Studies

Observational studies involve watching and recording the actions of participants. This may take place in the natural setting, such as observing children at play at a park (naturalistic observation), or behind a one-way glass while children are at play in a laboratory playroom (structured observation). The researcher may follow a checklist and record the frequency and duration of events (perhaps how many conflicts occur among 2-year-olds) or may observe and record as much as possible about an event (such as observing children in a classroom and capturing the details about the room design and what the children and teachers are doing and saying).

In general, observational studies have the strength of allowing the researcher to see how people behave rather than relying on self-report. What people do and what they say they do or would do are often very different. A major weakness of observational studies is that they do not allow the researcher to explain causal relationships. Yet, observational studies are useful and widely used when studying children. Another disadvantage of observation methods is that children, and in fact people in general, tend to change their behavior when they know they are being watched (known as the Hawthorne effect).

Naturalistic and Laboratory Experiments

As discussed in the section on research design, Experiments are designed to test hypotheses (or specific statements about the relationship between variables) in a controlled setting in efforts to explain how certain factors or events produce outcomes. A variable is anything that changes in value. Concepts are operationalized or transformed into variables in research, which means that the researcher must specify exactly what is going to be measured in the study.

Three conditions must be met in order to establish cause and effect. Experimental designs are useful in meeting these conditions.

1. The independent and dependent variables must be related. In other words, when one is altered, the other changes in response. (The independent variable is something altered or introduced by the researcher. The dependent variable is the outcome or the factor affected by the introduction of the independent variable. For example, if we are looking at the impact of exercise on stress levels, the independent variable would be exercise; the dependent variable would be stress.)
2. The cause must come before the effect. Experiments involve measuring subjects on the dependent variable before exposing them to the independent variable (establishing a baseline). So we would measure the subjects’ level of stress before introducing exercise and then again after the exercise to see if there has been a change in stress levels. (Observational and survey research does not always allow us to look at the timing of these events, which makes understanding causality problematic with these designs.)
3. The cause must be isolated. The researcher must ensure that no outside, perhaps unknown variables are actually causing the effect we see. The experimental design helps make this possible. In an experiment, we would make sure that our subjects’ diets were held constant throughout the exercise program. Otherwise, diet might really be creating the change in stress level rather than exercise.

One basic experimental design involves beginning with a sample (or subset of a population) and randomly assigning subjects to one of two groups: the experimental group or the control group. The experimental group is the group that is going to be exposed to an independent variable or condition the researcher is introducing as a potential cause of an event. The control group is going to be used for comparison and is going to have the same experience as the experimental group but will not be exposed to the independent variable. After exposing the experimental group to the independent variable, the two groups are measured again to see if a change has occurred in the experimental group, and not in the control group. If so, we are in a better position to suggest that the independent variable caused the change in the dependent variable.

The major advantage of the experimental design is that of helping to establish cause and effect relationships. A disadvantage of this design is the difficulty of translating much of what happens in a laboratory setting into real life.

That is why naturalistic experiments where the researcher enters into a natural situation and changes some variable of interest can increase the generalizability of experimental research. However, while the independent variable is still being manipulated by the researcher, there is some loss of control over other variables that are likely to influence the dependent variable.

Case Studies

Case studies involve exploring a single case or situation in great detail. Information may be gathered with the use of observation, interviews, testing, or other methods to uncover as much as possible about a person or situation. Case studies are helpful when investigating unusual situations such as brain trauma or children reared in isolation. And they are often used by clinicians who conduct case studies as part of their normal practice when gathering information about a client or patient coming in for treatment. Case studies can be used to explore areas about which little is known and can provide rich detail about situations or conditions. However, the findings from case studies are very difficult to generalize or apply to larger populations; this is because cases are not randomly selected and no control group is used for comparison.

For example, case studies are used when studying children born as conjoined twins. Clearly this is not a situation commonly encountered or seen. There are also famous examples of case studies where situations occurred accidentally, but then the person was studied extensively in order to learn about the variables involved. One famous example of this in psychology and neuroscience is Henry Molaison who went through surgery to treat epilepsy when he was in his 20s, which resulted in anterograde amnesia (an inability to form new memories after the removal of his hippocampus). Researchers studied Mr Molaison for many years until his death in 2008 and his brain has been preserved as well currently at UC Davis.

Figure \(\PageIndex{1}\): A case study is a collection of a lot of data about one specific participants or a very small group of participants.[1]

Surveys

Surveys are familiar to most people because they are so widely used. Surveys enhance accessibility to subjects because they can be conducted in person, over the phone, through the mail, or online. A survey involves asking a standard set of questions to a group of subjects. In a highly structured survey, subjects are forced to choose from a response set such as “strongly disagree, disagree, undecided, agree, strongly agree”; or “0, 1-5, 6-10, etc.” This is known as Likert Scale. Surveys are commonly used by sociologists, marketing researchers, political scientists, therapists, and others to gather information on many outcome and predictor variables in a relatively short period of time. Surveys typically yield surface information on a wide variety of factors, but may not allow for in-depth understanding of human behavior.

Of course, surveys can be designed in a number of ways. They may include forced choice questions and semi-structured questions in which the researcher allows the respondent to describe or give details about certain events. Some of the most difficult aspects of designing a good survey is wording questions in an unbiased way and asking the right questions so that respondents can give a clear response rather than choosing “undecided” each time. Knowing that 30% of respondents are undecided is of little use! Another issue is that each question in a survey needs to be answerable individually and uniquely. For example, if a survey question was worded – “I see myself as an honest and hardworking individual” it is unclear whether I should answer yes or no if I think of myself as honest, but somewhat lazy! So, a lot of time and effort should be placed on the construction of survey items. One of the benefits of having forced choice items is that each response is coded so that the results can be quickly entered and analyzed using statistical software. Analysis takes much longer when respondents give lengthy responses that must be analyzed based on some different criteria.

Once you decide what questions to ask, surveys are easy to use, but not always the right method. Survey use assumes that research participants are able to understand the questions, know the answers to the questions, are motivated to answer, and capable of honest answers.

Surveys are useful in examining stated values, attitudes, opinions, and reporting on practices. However, they are based on self-report or what people say they do rather than on observation and this can limit accuracy.

Attributions:

Child Growth and Development by Jennifer Paris, Antoinette Ricardo, and Dawn Rymond, 2019, is licensed under CC BY 4.0

Research Methods in Developmental Psychologyby Angela Lukowski and Helen Milojevich is licensed under a CC BY-NC-SA 4.0

[1] Image by MaryGeorge is licensed under CC BY-SA 4.0