The goal of a survey is to collect data from a representative sample of a population to draw conclusions about that larger population.
Learning Objectives
Assess the various types of surveys and sampling methods used in sociological research, appealing to the concepts of reliability and validity
Key Points
The sample of people surveyed is chosen from the entire population of interest. The goal of a survey is to describe not the smaller sample but the larger population.
To be able to generalize about a population from a smaller sample, that sample must be representative; proportionally the same in all relevant aspects (e.g., percent of women vs. men).
Surveys can be distributed by mail, email, telephone, or in-person interview.
Surveys can be used in cross-sectional, successive-independent-samples, and longitudinal study designs.
Effective surveys are both reliable and valid. A reliable instrument produces consistent results every time it is administered; a valid instrument does in fact measure what it intends to measure.
Key Terms
cross-sectional study: A research method that involves observation of a representative sample of a population at one specific point in time.
successive-independent-samples design: A research method that involves observation of multiple random samples from a population over multiple time points.
longitudinal design: A research method that involves observation of the same representative sample of a population over multiple time points, generally over a period of years or decades.
sample: A subset of a population selected for measurement, observation or questioning, to provide statistical information about the population.
Selecting a Sample to Survey
The sample of people surveyed is chosen from the entire population of interest. The goal of a survey is to describe not the smaller sample but the larger population. This generalizing ability is dependent on the representativeness of the sample.
Nuclear Energy Support in the U.S.: This pie chart shows the results of a survey of people in the United States (February 2005, Bisconti Research Inc.). According to the poll, 67 percent of Americans favor nuclear energy (blue), while 26 percent oppose it (yellow).
There are frequent difficulties one encounters while choosing a representative sample. One common error that results is selection bias —when the procedures used to select a sample result in over- or under-representation of some significant aspect of the population. For instance, if the population of interest consists of 75% females, and 25% males, and the sample consists of 40% females and 60% males, females are under represented while males are overrepresented. In order to minimize selection biases, stratified random sampling is often used. This is when the population is divided into sub-populations called strata, and random samples are drawn from each of the strata, or elements are drawn for the sample on a proportional basis.
For instance, a Gallup Poll, if conducted as a truly representative nationwide random sampling, should be able to provide an accurate estimate of public opinion whether it contacts 2,000 or 10,000 people.
Modes of Administering a Survey
There are several ways of administering a survey. The choice between administration modes is influenced by several factors, including
costs,
coverage of the target population,
flexibility of asking questions,
respondents’ willingness to participate and
response accuracy
Different methods create mode effects that change how respondents answer, and different methods have different advantages. The most common modes of administration can be summarized as:
Telephone
Mail (post)
Online surveys
Personal in-home surveys
Personal mall or street intercept survey
Hybrids of the above
Participants willing to take the time to respond will convey personal information about religious beliefs, political views, and morals. Some topics that reflect internal thought are impossible to observe directly and are difficult to discuss honestly in a public forum. People are more likely to share honest answers if they can respond to questions anonymously.
Questionnaire: Questionnaires are a common research method; the U.S. Census is a well-known example.
Types of Studies
Cross-Sectional Design
In a cross-sectional study, a sample (or samples) is drawn from the relevant population and studied once. A cross-sectional study describes characteristics of that population at one time, but cannot give any insight as to causes of population characteristics.
Successive-Independent-Samples Design
A successive-independent-samples design draws multiple random samples from a population at one or more times. This design can study changes within a population, but not changes within individuals because the same individuals are not surveyed more than once. Such studies cannot, therefore, identify the causes of change over time necessarily.
For successive independent samples designs to be effective, the samples must be drawn from the same population, and must be equally representative of it. If the samples are not comparable, the changes between samples may be due to demographic characteristics rather than time. In addition, the questions must be asked in the same way so that responses can be compared directly.
Longitudinal Design
A study following a longitudinal design takes measure of the same random sample at multiple time points. Unlike with a successive independent samples design, this design measures the differences in individual participants’ responses over time. This means that a researcher can potentially assess the reasons for response changes by assessing the differences in respondents’ experiences. However, longitudinal studies are both expensive and difficult to do. It’s harder to find a sample that will commit to a months- or years-long study than a 15-minute interview, and participants frequently leave the study before the final assessment. This attrition of participants is not random, so samples can become less representative with successive assessments.
A link to the YouTube element can be found in Contributors & Attributions section.
Writing survey questions: Researchers must carefully design survey questions to ensure they receive accurate and unbiased results.
Reliability and Validity
Reliable measures of self-report are defined by their consistency. Thus, a reliable self-report measure produces consistent results every time it is executed.
A test’s reliability can be measured a few ways. First, one can calculate a test-retest reliability. A test-retest reliability entails conducting the same questionnaire to a large sample at two different times. For the questionnaire to be considered reliable, people in the sample do not have to score identically on each test, but rather their position in the score distribution should be similar for both the test and the retest.
Self-report measures will generally be more reliable when they have many items measuring a construct. Furthermore, measurements will be more reliable when the factor being measured has greater variability among the individuals in the sample that are being tested. Finally, there will be greater reliability when instructions for the completion of the questionnaire are clear and when there are limited distractions in the testing environment.
Contrastingly, a questionnaire is valid if what it measures is what it had originally planned to measure. Construct validity of a measure is the degree to which it measures the theoretical construct that it was originally supposed to measure.
Fieldwork and Observation
Ethnography is a research process that uses fieldwork and observation to learn about a particular community or culture.
Learning Objectives
Explain the goals and methods of ethnography
Key Points
Ethnographic work requires intensive and often immersive long-term participation in the community that is the subject of research, typically involving physical relocation (hence the term fieldwork).
In participant observation, the researcher immerses himself in a cultural environment, usually over an extended period of time, in order to gain a close and intimate familiarity with a given group of individuals and their practices.
Such research involves a range of well-defined, though variable methods: interviews, direct observation, participation in the life of the group, collective discussions, analyses of personal documents produced within the group, self-analysis, and life-histories, among others.
The advantage of ethnography as a technique is that it maximizes the researcher’s understanding of the social and cultural context in which human behavior occurs.
The advantage of ethnography as a technique is that it maximizes the researcher’s understanding of the social and cultural context in which human behavior occurs. The ethnographer seeks out and develops relationships with cultural insiders, or informants, who are willing to explain aspects of their community from a native viewpoint. A particularly knowledgeable informant who can connect the ethnographer with other such informants is known as a key informant.
Key Terms
ethnography: The branch of anthropology that scientifically describes specific human cultures and societies.
qualitative: Of descriptions or distinctions based on some quality rather than on some quantity.
Fieldwork and Observation
Ethnography is a qualitative research strategy, involving a combination of fieldwork and observation, which seeks to understand cultural phenomena that reflect the knowledge and system of meanings guiding the life of a cultural group. It was pioneered in the field of socio-cultural anthropology, but has also become a popular method in various other fields of social sciences, particularly in sociology.
Ethnographic work requires intensive and often immersive long-term participation in the community that is the subject of research, typically involving physical relocation (hence the term fieldwork). Although it often involves studying ethnic or cultural minority groups, this is not always the case. Ideally, the researcher should strive to have very little effect on the subjects of the study, being as invisible and enmeshed in the community as possible.
Participant Observation
One of the most common methods for collecting data in an ethnographic study is first-hand engagement, known as participant observation. In participant observation, the researcher immerses himself in a cultural environment, usually over an extended period of time, in order to gain a close and intimate familiarity with a given group of individuals (such as a religious, occupational, or sub-cultural group, or a particular community) and their practices.
Fieldwork and Observation: One of the most common methods for collecting data in an ethnographic study is first-hand engagement, known as participant observation.
Methods
Such research involves a range of well-defined, though variable methods: interviews, direct observation, participation in the life of the group, collective discussions, analyses of personal documents produced within the group, self-analysis, and life-histories, among others.
Interviews can be either informal or formal and can range from brief conversations to extended sessions. One way of transcribing interview data is the genealogical method. This is a set of procedures by which ethnographers discover and record connections of kinship, descent, and marriage using diagrams and symbols. Questionnaires can also be used to aid the discovery of local beliefs and perceptions and, in the case of longitudinal research where there is continuous long-term study of an area or site, they can act as valid instruments for measuring changes in the individuals or groups studied.
Advantages
The advantage of ethnography as a technique is that it maximizes the researcher’s understanding of the social and cultural context in which human behavior occurs. The ethnographer seeks out and develops relationships with cultural insiders, or informants, who are willing to explain aspects of their community from a native viewpoint. The process of seeking out new contacts through their personal relationships with current informants is often effective in revealing common cultural common denominators connected to the topic being studied.
Experiments
Experiments are tests designed to prove or disprove a hypothesis by controlling for pertinent variables.
Learning Objectives
Compare and contrast how hypotheses are being tested in sociology and in the hard sciences
Key Points
Experiments are controlled tests designed to prove or disprove a hypothesis.
A hypothesis is a prediction or an idea that has not yet been tested.
Researchers must attempt to identify everything that might influence the results of an experiment, and do their best to neutralize the effects of everything except the topic of study.
Since social scientists do not seek to isolate variables in the same way that the hard sciences do, sociologists create the equivalent of an experimental control via statistical techniques that are applied after data is gathered.
A control is when two identical experiments are conducted and the factor being tested is varied in only one of these experiments.
Key Terms
control: A separate group or subject in an experiment against which the results are compared where the primary variable is low or nonexistent.
experiment: A test under controlled conditions made to either demonstrate a known truth, examine the validity of a hypothesis, or determine the efficacy of something previously untried.
hypothesis: Used loosely, a tentative conjecture explaining an observation, phenomenon, or scientific problem that can be tested by further observation, investigation, or experimentation.
Scientists form a hypothesis, which is a prediction or an idea that has not yet been tested. In order to prove or disprove the hypothesis, scientists must perform experiments. The experiment is a controlled test designed specifically to prove or disprove the hypothesis. Before undertaking the experiment, researchers must attempt to identify everything that might influence the results of an experiment and do their best to neutralize the effects of everything except the topic of study. This is done through the introduction of an experimental control: two virtually identical experiments are run, in only one of which the factor being tested is varied. This serves to further isolate any causal phenomena.
An Experiment: An experiment is a controlled test designed specifically to prove or disprove a hypothesis.
Of course, an experiment is not an absolute requirement. In observation based fields of science, actual experiments must be designed differently than for the classical laboratory based sciences. Due to ethical concerns and the sheer cost of manipulating large segments of society, sociologists often turn to other methods for testing hypotheses.
Since sociologists do not seek to isolate variables in the same way that hard sciences do, this kind of control is often done via statistical techniques, such as regressions, applied after data is gathered. Direct experimentation is thus fairly rare in sociology.
Scientists must assume an attitude of openness and accountability on the part of those conducting an experiment. It is essential to keep detailed records in order to facilitate reporting on the experimental results and provide evidence of the effectiveness and integrity of the procedure.
Documents
Documentary research involves examining texts and documents as evidence of human behavior.
Learning Objectives
Describe different kinds of documents used in sociological research
Key Points
This kind of sociological research is generally considered a part of media studies.
Unobtrusive research involves ways of studying human behavior without affecting it in the process.
Documents can either be primary sources, which are original materials that are not created after the fact with the benefit of hindsight, or secondary sources that cite, comment, or build upon primary sources.
Typically, sociological research involving documents falls under the cross-disciplinary purview of media studies, which encompasses all research dealing with television, books, magazines, pamphlets, or any other human-recorded data. The specific media being studied are often referred to as texts.
Sociological research involving documents, or, more specifically, media studies, is one of the less interactive research options available to sociologists. It can provide a significant insight into the norms, values, and beliefs of people belonging to a particular historical and cultural context.
Content analysis is the study of recorded human communications.
Key Terms
content analysis: Content analysis or textual analysis is a methodology in the social sciences for studying the content of communication.
documentary research: Documentary research involves the use of texts and documents as source materials. Source materials include: government publications, newspapers, certificates, census publications, novels, film and video, paintings, personal photographs, diaries and innumerable other written, visual, and pictorial sources in paper, electronic, or other “hard copy” form.
media studies: Academic discipline that deals with the content, history, meaning, and effects of various media, and in particular mass media.
Documentary Research
It is possible to do sociological research without directly involving humans at all. One such method is documentary research. In documentary research, all information is collected from texts and documents. The texts and documents can be either written, pictorial, or visual in form.
The material used can be categorized as primary sources, which are original materials that are not created after the fact with the benefit of hindsight, and secondary sources that cite, comment, or build upon primary sources.
Media Studies and Content Analysis
Typically, sociological research on documents falls under the cross-disciplinary purview of media studies, which encompasses all research dealing with television, books, magazines, pamphlets, or any other human-recorded data. Regardless of the specific media being studied, they are referred to as texts.
Media studies may draw on traditions from both the social sciences and the humanities, but mostly from its core disciplines of mass communication, communication, communication sciences, and communication studies.
Researchers may also develop and employ theories and methods from disciplines including cultural studies, rhetoric, philosophy, literary theory, psychology, political economy, economics, sociology, anthropology, social theory, art history and criticism, film theory, feminist theory, information theory, and political science.
Government Documentary Research: Sociologists may use government documents to research the ways in which policies are made.
Content analysis refers to the study of recorded human communications, such as paintings, written texts, and photos. It falls under the category of unobtrusive research, which can be defined as ways for studying human behavior without affecting it in the process. While sociological research involving documents is one of the less interactive research options available to sociologists, it can reveal a great deal about the norms, values, and beliefs of people belonging to a particular temporal and cultural context.
Use of Existing Sources
Studying existing sources collected by other researchers is an essential part of research in the social sciences.
Learning Objectives
Explain how the use of existing sources can benefit researchers
Key Points
Archival research is the study of existing sources. Without archival research, any research project is necessarily incomplete.
The study of sources collected by someone other than the researcher is known as archival research or secondary data research.
The importance of archival or secondary data research is two-fold. By studying texts related to their topics, researchers gain a strong foundation on which to base their work. Secondly, this kind of study is necessary in the development of their central research question.
Key Terms
secondary data: Secondary data is data collected by someone other than the user. Common sources of secondary data for social science include censuses, organizational records, and data collected through qualitative methodologies or qualitative research.
Archival research: An archive is a way of sorting and organizing older documents, whether it be digitally (photographs online, e-mails, etc.) or manually (putting it in folders, photo albums, etc.). Archiving is one part of the curating process which is typically carried out by a curator.
primary data: Data that has been compiled for a specific purpose, and has not been collated or merged with others.
Using Existing Sources
The study of sources collected by someone other than the researcher, also known as archival research or secondary data research, is an essential part of sociology. In archival research or secondary research, the focus is not on collecting new data but on studying existing texts.
Existing Sources: While some sociologists spend time in the field conducting surveys or observing participants, others spend most of their research time in libraries, using existing sources for their research.
By studying texts related to their topics, researchers gain a strong foundation on which to base their work. Furthermore, this kind of study is necessary for the development of their central research question. Without a thorough understanding of the research that has already been done, it is impossible to know what a meaningful and relevant research question is, much less how to position and frame research within the context of the field as a whole.
Types of Existing Sources
Common sources of secondary data for social science include censuses, organizational records, field notes, semi-structured and structured interviews, and other forms of data collected through quantitative methods or qualitative research. These methods are considered non-reactive, because the people do not know they are involved in a study. Common sources differ from primary data. Primary data, by contrast, are collected by the investigator conducting the research.
Researchers use secondary analysis for several reasons. The primary reason is that secondary data analysis saves time that would otherwise be spent collecting data. In the case of quantitative data, secondary analysis provides larger and higher-quality databases that would be unfeasible for any individual researcher to collect on his own. In addition, analysts of social and economic change consider secondary data essential, since it is impossible to conduct a new survey that can adequately capture past change and developments.
Contributors and Attributions
Curation and Revision. by: Boundless.com. CC BY-SA
Nuclear energy poll usa. (Public Domain; Wikimedia via commons.wikimedia.org/wiki/File:Nuclear_energy_poll_usa.png)
Ethnography. (CC BY-SA; Wikipedia via en.Wikipedia.org/wiki/Ethnography)
Cultural Anthropology/History of Anthropological Theory. (CC BY-SA; Wikibooks via en.wikibooks.org/wiki/Cultural_Anthropology/History_of_Anthropological_Theory)
Ethnography of Fiddle/Ethnography. (CC BY-SA; Wikibooks via en.wikibooks.org/wiki/Ethnography_of_Fiddle/Ethnography)
Participant observation. (CC BY-SA; Wikipedia via en.Wikipedia.org/wiki/Participant_observation)
ethnography. (CC BY-SA; Wiktionary via en.wiktionary.org/wiki/ethnography)
qualitative. (CC BY-SA; Wiktionary via en.wiktionary.org/wiki/qualitative)
Introduction to Sociology/Sociological Methods. (CC BY-SA; Wikibooks via en.wikibooks.org/wiki/Introduction_to_Sociology/Sociological_Methods%23Experiment)
control. (CC BY-SA; Wiktionary via en.wiktionary.org/wiki/control)
experiment. (CC BY-SA; Wiktionary via en.wiktionary.org/wiki/experiment)
hypothesis. (CC BY-SA; Wiktionary via en.wiktionary.org/wiki/hypothesis)
