5.3: Measurement

Learning Objectives

By the end of this section, you will be able to:

• Analyze different types of measurement
• Evaluate the quality of measures
• Explore existing measures of regime type

Types of measurement

When operationalizing a concept, one important consideration is the kind of measure that will be used. Measurement is “the assignment of numbers or labels to units of analysis to represent variable categories.”⁵ In other words, measurement is putting values on variables. Measurement is highly concrete insofar as it entails translating observations of the world into standard units. Those units can still be very abstract, but measurement is a crucial step for creating the data that can then be analyzed. For example, the research and advocacy organization Freedom House uses a scale that ranges from 0 to 100 to measure the levels of freedom, political and civil, in countries around the world.⁶ The number 100 on the Freedom House scale does not equate to 100 units of something tangible, the way we would measure, say, pounds of flour, yet it is a more precise way of thinking about differing levels of freedom around the world. Based on the Freedom House scale, it is possible to compare levels of freedom across countries and over time and analyze trends more systematically.

There are four types of measures: nominal, ordinal, interval, and ratio. Table 5.3 below summarizes them briefly.

Table \(\PageIndex{1}\): Types of measures

Type of Measure	Description
Nominal	Observations are classified into two or more categories, with numerical values assigned to each category Example: Racial and ethnic categories maintained by the US Census Bureau
Ordinal	Observations are rank ordered, with numbers assigned to indicate the rank ordering on some dimension Example: Attitude question on surveys ranging from 1 = “Strongly Disagree” to 5 = “Strongly Agree”
Interval	Observations fall along a scale with standard units Example: Timeline that ranges from 1945 to 2000 with 5-year periods of time demarcated
Ratio	Interval ratio with an absolute zero Example: Age, weight

Nominal measures are focused on classification. One example of a nominal measure are the racial and ethnic categories used in the US Census, such as “Black or African American” (a racial category) or “Hispanic” (an ethnic category).⁷ Aristotle offered a nominal measure of regime type when he listed six different types of regime, including “democracy” and “tyranny” and so forth.⁸ Numbers can be assigned to each category, but these assignments are arbitrary and not useful for rank-ordering categories. Good nominal measures are those which are exhaustive and mutually exclusive. A nominal measure is exhaustive when every observation falls within the given categories. A well-constructed nominal measure should also have mutually exclusive categories, meaning that there is no overlap between categories. On these criteria, it would appear that the racial categories used by the US federal government are problematic. First, they are not exhaustive as they do not include the possibility of classifying individuals who identify as two or more races. Second, the categories are not mutually exclusive, as the “White” and “Black or African American” racial categories both include individuals who may trace their geographic origins to the African continent.

Ordinal measures classify and rank-order observations. Observations fall along some ranking system, with numbers assigned to different ranks. One example of an ordinal measure is a survey question which asks respondents whether they “strongly agree,” “somewhat agree,” “[are] neutral,” “somewhat disagree,” and “strongly disagree” with a statement, and there are numerical values in descending or ascending order assigned to each response category. Another example of an ordinal measure are socioeconomic categories which may range from “lower class” to “lower middle class” to “middle class” to “upper class”. Note that the categories in ordinal measures provide some information about relative rankings. For example, someone in the upper class probably has higher household income than someone in the lower class. However, ordinal measures are not designed for mathematical manipulation. One should not take the average of all the responses to a survey question noted above to arrive at the “average” level of agreement to a statement.

An interval measure contains numerical values which are assumed to have equal distances between each unit. Taking the Freedom House scale mentioned previously, which ranges from 0 to 100, countries fall on this scale based on observations of levels of freedom in each country. Another example of interval measurement is the numerical score you might receive for each exam in your class, which typically ranges from 0 to 100. Mathematical manipulation can be conducted on these measures. For example, if you received an 80 and a 70 on your two exams, they could be averaged to yield an average exam score of 75 (assuming the exams were worth the same percentage of your final grade).

Ratio measures are interval measures that have a true zero. An example of this is weight or age. What is the significance of a measure having a true zero? This allows for statements comparing observations on the ratio. For example, if two people are 20 and 40, it is possible to state that the 40-year-old is twice as old as the 20-year-old. Taking the example of interval measure noted previously, there is no true zero on the Freedom House measure. It could be the case, for example, that countries fall below zero but are just not captured by the criteria used for the scale. And for an interval measure such as Freedom House’s, it is not possible to state that a country ranked 60 on the Freedom House scale is twice as free as a country ranked 30 on that scale.

Comparing across these four types of measures, each yields information that builds upon the contributions of the previous kind of measure. Nominal measures help with classification. It follows from this that nominal measures allow for counting the total number or frequency of some category within the classification system. Ordinal measures classify as well, but they also allow for ordering observations. Interval measures classify and rank order observations, but they also present equal intervals for measuring observations. Finally, for those variables where there is a true zero, ratio measures allow for classification, rank ordering, and measuring intervals. They also allow for assessing the relative value of observations.

Quality of measures

An important consideration when determining a measure for a concept is whether that measure is of high quality. Some criteria for evaluating this are the precision and accuracy of the proposed measure. A precise measure is one that is exact. For example, consider how to measure education levels. Doing so by tracking the schools from which an individual has graduated is one measure, and it is passably precise. (For example, an individual may graduate from elementary, then middle and high school.) Counting the years that an individual has attended school is perhaps a more precise measure, since not all education systems may be divided into elementary, middle, and high school levels. This second approach allows for more fine-grained data collection – i.e., more precise data – for analysis.

Accuracy presents additional challenges. An accurate measure is one which measures the underlying concept that it was intended to measure. This relates to two characteristics, reliability and validity. A reliable measure is where there is a low possibility of measurement error. One way to assess this is to see whether different researchers still arrive at the same findings when applying the same measure. Reliable measures are those which have the potential for replicability, one of the standards for evaluating the robustness of a research finding. A valid measure is more difficult to evaluate, but it basically reduces to whether a measure is meaningful. For example, is an IQ test a valid way to measure a person’s intelligence? Validity is difficult to assess and therefore hotly debated among researchers.

One way to think about precision, reliability, and validity is to imagine a dart board with concentric circles and a bull’s eye in the center. The bull’s eye in the center of the dart board is the concept that a researcher is trying to measure. A precise measure would be a dart that has a fine needle rather than a fat needle. A reliable measure would be one where repeated darts thrown at the dart board all land on the same spot on the target. That doesn’t mean the darts have landed on the bull’s eye, but at least they are landing on the same spot again and again. A valid measure would be one where repeated darts thrown at the dart board sometimes hit the bull’s eye, but the darts may be scattered all over the target. But a reliable and valid measure would be one where darts thrown at the target consistently strike the bull’s eye. (Note that measures may be reliable but not valid. Measures may also be valid but not reliable. And they may be neither, which means the darts are not striking the target at all but instead landing all over the adjacent wall.)

Figure \(\PageIndex{1}\): Dart board as metaphor for precision, reliability, and validity of measure by Christina B. Castro, “Dart board,” 2008, Flickr creative commons, CC BY-NC 2.0

Applying concepts and measures: Some measures of regime type

To circle back to the discussion raised at the beginning of this chapter, the concept of regime has been a perennial focus of political science since antiquity. Regime, or the collection of rules by which political authority is organized in a society, is a locus of political power. Scholars also believe that variation in regime types over time and space can help with understanding outcomes such as individual well-being and societal prosperity.⁹ This chapter began by examining how Aristotle sought to conceptualize his observations of political authority, settling on the concept of “constitution” which we today refer to as “regime”. Early attempts to operationalize and conceive of measures for regime focused on the number of leaders in power and in whose interest they ruled. Political scientists at present have conceived of myriad measures for regime type. This section will examine two different measures which present examples of ordinal and interval measures.

Professor Barbara Geddes of the Political Science Department at the University of California, Los Angeles, offers one ordinal measure for understanding the diverse group of countries in the world which are commonly referred to as authoritarian regimes, or nondemocracies. (We can think of a democracy most simply as a country where there are free and fair elections; a nondemocracy is where these are absent.) For Geddes, nondemocracies include everything from North Korea under the Kim family to Brazil under military dictatorship. Looking at the sheer diversity of nondemocracies in the world, and narrowing her focus to the twentieth century, Geddes devised several categories for dictatorships of the world. The categories she devised were personalist, military, single party, and hybrids of these three categories.¹⁰

Table \(\PageIndex{2}\): Geddes types of nondemocracy (Example of a nominal measure)

Type of dictatorship	Description
Personalist	Rule by a single person Example: Zimbabwe under Robert Mugabe, 1980-2017
Military	Rule by military leaders Example: Turkey, 1960-1965; military coup in 1960 and general in power through 1965
Single party	Rule by a single political party Example: People’s Republic of China under the Chinese Communist Party, 1949-present
Hybrid	May be combinations of two or three of the above categories Example: North Korea under the rule of the Kim family, Workers’ Party of Korea, and North Korean military since 1953

This ordinal measure for dictatorship offers a first cut at classifying a very diverse universe of cases. There are qualitative differences between the categories constructed by Geddes, for example whether political leadership is concentrated in a single person, the military, a political party, or some combination of these three. Note that there isn’t any rank ordering of these types of nondemocracies on any dimension. Because of this, it is not possible to consider whether, for example, a greater concentration of leadership in fewer individuals correlates with greater wealth concentration in the country. Geddes’ measure strives to be exhaustive, as she argues that every nondemocracy in the world during the twentieth century might fit into one of these four categories. There may be questions, however, about the reliability of this measure. Might another researcher, starting from scratch, categorize countries in the same way as Geddes? China, for example, might be categorized as a personalist regime under Mao Zedong’s rule (1949-1976) rather than a single party regime.

A second example of a widely used interval measure of regime type is known as Polity IV. This measure considers the entire range of regime types, from highly undemocratic to so-called consolidated democracies of the world.¹¹ It places observations on a scale that ranges from -10 (for highly undemocratic) to +10 (for highly democratic). As the Polity Project webpage notes,

“[Polity IV] envisions a spectrum of governing authority that spans from fully institutionalized autocracies through mixed, or incoherent, authority regimes (termed "anocracies") to fully institutionalized democracies.

“The ‘Polity Score’ captures this regime authority spectrum on a 21-pont scale ranging from -10 (hereditary monarchy) to +10 (consolidated democracy). The Polity scores can also be converted into regime categories in a suggested threepart categorization of ‘autocracies’ (-10 to -6), ‘anocracies’ (-5 to +5 and three special values: -66, -77 and -88), and ‘democracies’ (+6 to +10).”¹²

The Polity datasets are publicly available and downloadable from the internet. Scores are available for 151 countries ranging over the period 1800-2017, with annual observations for each country. Countries are placed each year on this -10 to +10 scale depending on the degree of political competition observed, citizen participation, and constraints on the executive. The higher a country scores on these dimensions, the higher its Polity Score. Canada, for example, has a Polity Score of +10 over the period 1946-2017.

Note that this measure rank-orders countries along some underlying dimension of “authoritarianism,” where those countries which are deeply authoritarian are closer to -10 while those which are further from authoritarianism, or more democratic, are closer to +10. While the Polity Score is an interval measure of regime type, the excerpt above also suggests that this can be an ordinal measure with the following categories: autocracy, anocracy, and democracy.

Polity Score today is considered one of the most precise and reliable measures for regime type. Its validity, like the validity of most every measure for regime type, is debated. By one scholar’s count, there exist today at least nine interval measures of democracy alone.¹³ The endeavor continues. Projects which culminate in measures such as Polity Score are valuable for putting words and measures to concepts which we know are deeply consequential.

5 Raymo, James M. 2009. “Methods of Sociological Inquiry.” Course slides, University of Wisconsin-Madison.

6 See Freedom House online at https://freedomhouse.org/

7 A summary of US Census Bureau racial and ethnic categories is available at https://www.census.gov/quickfacts/fa...e/US/RHI625218

8 One of Aristotle’s criteria for these categories was the number of rulers, so there was also some attempt at interval measurement in his classification of regime type. However, this was mixed with his focus on whether rulers ruled in the common or private interest.

9 See, for example, Przeworski, Adam et al. 2000. Democracy and Development: Political Institutions and WellBeing in the World, 1950-1990. Cambridge: Cambridge University Press.

10 See Geddes, Barbara. 2003. Paradigms and Sand Castles: Theory Building and Resaerch Design in Comparative Politics. Ann Arbor, MI: University of Michigan Press.

11 Consolidated democracy refers to those democracies where democratic institutions such as elections, checksand-balances within government, and civil society, are robust and democracy is widely accepted as the ideal kind of political authority.

12 The Polity Project is available online at http://www.systemicpeace.org/polityproject.html. This website contains downloadable datasets and codebooks.

13 Pemstein, Daniel, Meserve, Stephen A., and Melton, James. 2010. “Democratic Compromise: A Latent Variable Analysis of Ten Measures of Regime Type,” Political Analysis 18: 426-449.