15.7: Selected response items

Common formal assessment formats used by teachers are multiple choice, matching, and true/false items. In selected response items students must select a response provided by the teacher or test developer rather than constructing a response in their own words or actions. Selected response items do not require that students recall the information but rather recognize the correct answer.

Tests with these items are called objective because the results are not influenced by scorers’ judgments or interpretations and so are often machine scored. Eliminating potential errors in scoring increases the reliability of tests but teachers who only use objective tests are liable to reduce the validity of their assessment because objective tests are not appropriate for all learning goals (Linn & Miller, 2005). Effective assessment for learning as well as assessment of learning must be based on aligning the assessment technique to the learning goals and outcomes.

For example, if the goal is for students to conduct an experiment then they should be asked to do that rather that than being asked about conducting an experiment.

Common problems

Selected response items are easy to score but are hard to devise. Teachers often do not spend enough time constructing items and common problems include:

1. Unclear wording in the items

• True or False: Although George Washington was born into a wealthy family, his father died when he was only 11, he worked as a youth as a surveyor of rural lands, and later stood on the balcony of Federal Hall in New York when he took his oath of office in 1789.

2. Cues that are not related the content being examined.

• A common clue is that all the true statements on a true/false test or the corrective alternatives on a multiple-choice test are longer than the untrue statements or the incorrect alternatives.

3. Using negatives (or double negatives) the items.

• A poor item. “True or False: None of the steps made by the student was unnecessary.”
• A better item. True or False: “All of the steps were necessary.”

Students often do not notice the negative terms or find them confusing so avoiding them is generally recommended (Linn & Miller 2005). However, since standardized tests often use negative items, teachers sometimes deliberately include some negative items to give students practice in responding to that format.

4. Taking sentences directly from a textbook or lecture notes.

Removing the words from their context often makes them ambiguous or can change the meaning. For example, a statement from Chapter 3 taken out of context suggests all children are clumsy. “Similarly, with jumping, throwing and catching: the large majority of children can do these things, though often a bit clumsily.” A fuller quotation makes it clearer that this sentence refers to 5-year-olds: For some fives, running still looks a bit like a hurried walk, but usually it becomes more coordinated within a year or two. Similarly, with jumping, throwing and catching: the large majority of children can do these things, though often a bit clumsily, by the time they start school, and most improve their skills noticeably during the early elementary years.” If the abbreviated form was used as the stem in a true/false item it would obviously be misleading.

5. Avoid trivial questions

e.g. Jean Piaget was born in what year?

1. a) 1896
2. b) 1900
3. c) 1880
4. d) 1903

While it important to know approximately when Piaget made his seminal contributions to the understanding of child development, the exact year of his birth (1880) is not important.

Strengths and weaknesses

All types of selected response items have a number of strengths and weaknesses.

• True/False items are appropriate for measuring factual knowledge such as vocabulary, formulae, dates, proper names, and technical terms. They are very efficient as they use a simple structure that students can easily understand, and take little time to complete. They are also easier to construct than multiple choice and matching items. However, students have a 50 percent probability of getting the answer correct through guessing so it can be difficult to interpret how much students know from their test scores. Examples of common problems that arise when devising true/false items are in Table 37.

Table \(\PageIndex{3}\): : Common errors in selected response items (type of assessment item/ common errors/ example)

Type of item	Common errors	Example
True/False	The statement is not absolutely true—typically because it contains a broad generalization.	T/F: The President of the United States is elected to that office. This is usually true but the US Vice President can succeed the President.
True/False	The item is opinion not fact	T/F: Education for K-12 students is improved though policies that support charter schools. Some people believe this, some do not.
True/False	Two ideas are included in item	T/F: George H Bush the 40th president of the US was defeated by William Jefferson Clinton in 1992. The 1st idea is false; the 2nd is true making it difficult for students to decide whether to circle T or F.
True/False	Irrelevant cues	T/F: The President of the United States is usually elected to that office. True items contain the words such as usually generally; whereas false items contain the terms such as always, all, never.
Matching	Columns do not contain homogeneous information	Directions: On the line to the US Civil War Battle write the year or confederate general in Column B. Column A Ft Sumter 2nd Battle of Bull Run Ft Henry Column B General Stonewall Jackson General Johnson 1861 1862 Column B is a mixture of generals and dates.
Matching	Too many items in each list	Lists should be relatively short (4–7) in each column. More than 10 are too confusing.
Matching	Responses are not in logical order	In the example with Spanish and English words (Exhibit 1) should be in a logical order (they are alphabetical). If the order is not logical, student spend too much time searching for the correct answer.
Multiple Choice	Problem (i.e. the stem) is not clearly stated problem	New Zealand Is the worlds’ smallest continent Is home to the kangaroo Was settled mainly by colonists from Great Britain Is a dictatorship This is really a series of true-false items. Because the correct answer is 3, a better version with the problem in the stem is Much of New Zealand was settled by colonists from Great Britain Spain France Holland
Multiple Choice	Some of the alternatives are not plausible	Who is best known for their work on the development of the morality of justice? Gerald Ford Vygotsky Maslow Kohlberg Obviously Gerald Ford is not a plausible alternative.
Multiple Choice	Irrelevant cues	Correct alternative is longer Incorrect alternatives are not grammatically correct with the stem Too many correct alternatives are in position “b” or “c” making it easier for students to guess. All the options (e.g. a, b, c, d) should be used in approximately equal frequently (not exact as that also provides clues).
Multiple Choice	Use of “All of above”	If all of the “above is used” then the other items must be correct. This means that a student may read the 1st response, mark it correct and move on. Alternatively, a student may read the 1st two items and seeing they are true does nor need to read the other alternatives to know to circle “all of the above.” The teacher probably does not want either of these options.

---

In matching items, two parallel columns containing terms, phrases, symbols, or numbers are presented and the student is asked to match the items in the first column with those in the second column. Typically, there are more items in the second column to make the task more difficult and to ensure that if a student makes one error they do not have to make another.

Matching items most often are used to measure lower level knowledge, such as persons and their achievements, dates and historical events, terms and definitions, symbols and concepts, plants or animals and classifications (Linn & Miller, 2005). An example with Spanish language words and their English equivalents is below:

Directions: On the line to the left of the Spanish word in Column A, write the letter of the English word in Column B that has the same meaning.In matching items, two parallel columns containing terms, phrases, symbols, or numbers are presented and the student is asked to match the items in the first column with those in the second column. Typically there are more items in the second column to make the task more difficult and to ensure that if a student makes one error they do not have to make another. Matching items most often are used to measure lower level knowledge such as persons and their achievements, dates and historical events, terms and definitions, symbols and concepts, plants or animals and classifications (Linn & Miller, 2005). An example with Spanish language words and their English equivalents is below:

Exhibit \(\PageIndex{1}\): : Spanish and English translation

Column A	Column B
___ 1. Casa	A. Aunt
___ 2. Bebé	B. Baby
___ 3. Gata	C. Brother
___ 4. Perro	D. Cat
___ 5. Hermano	E. Dog

1. Father
2. House

---

While matching items may seem easy to devise it is hard to create homogenous lists. Other problems with matching items and suggested remedies are in Table 37.

---

Multiple Choice items are the most commonly used type of objective test items because they have a number of advantages over other objective test items.

• Most importantly, they can be adapted to assess higher levels thinking such as application as well as lower level factual knowledge. The first example below assesses knowledge of a specific fact, whereas the second example assesses application of knowledge.

Exhibit 2 \(\PageIndex{2}\)

Who is best known for their work on the development of the morality of justice?

1. a) Erikson
2. b) Vygotsky
3. c) Maslow
4. d) Kohlberg

(Adapted from Linn and Miller 2005, p, 193).

Exhibit \(\PageIndex{3}\)

Which one of the following best illustrates the law of diminishing returns

1. a) A factory doubled its labor force and increased production by 50 per cent
2. b) The demand for an electronic product increased faster than the supply of the product
3. c) The population of a country increased faster than agricultural self sufficiency
4. d) A machine decreased in efficacy as its parts became worn out

(Adapted from Linn and Miller 2005, p, 193).

Table \(\PageIndex{4}\): Common errors in constructed response items

Type of item	Common errors	Examples
Completion and short answer	There is more than one possible answer.	Where was US President Lincoln born? The answer could be in a log cabin, in Kentucky, etc.
Completion and short answer	Too many blanks are in the completion item so it is too difficult or doesn’t make sense.	In ________ theory, the first stage, ________ is when infants process through their ________ and________ ________.
Completion and short answer	Clues are given by length of blanks in completion items.	Three states are contiguous to New Hampshire: ________ is to the West, ________ is to the East, and ________ is to the South.
Extended response	Ambiguous questions	Was the US Civil War avoidable? Students could interpret this question in a wide variety of ways, perhaps even stating “yes” or “no.” One student may discuss only political causes another moral, political and economic causes. There is no guidance in the question for students.
Extended response	Poor reliability in grading	The teacher does not use a scoring rubric and so is inconsistent in how he scores answers especially unexpected responses, irrelevant information, and grammatical errors.
Extended response	Perception of student influences grading	By spring semester the teacher has developed expectations of each student’s performance and this influences the grading (numbers can be used instead of names). The test consists of three constructed responses and the teacher grades the three answers on each students’ paper before moving to the next paper. This means that the grading of questions 2 and 3 are influenced by the answers to question 1 (teachers should grade all the 1st question then the 2nd etc).
Extended response	Choices are given on the test and some answers are easier than others	Testing experts recommend not giving choices in tests because then students are not really taking the same test creating equity problems.

---

Constructed response items

Formal assessment also includes constructed response items, in which students are asked to recall information and create an answer—not just recognize if the answer is correct—so guessing is reduced.

• Constructed response items can be used to assess a wide variety of kinds of knowledge and two major kinds are discussed: completion or short answer (also called short response) and extended response.

---

Completion and short answer

Completion and short answer items can be answered in a word, phrase, number, or symbol. These types of items are essentially the same only varying in whether the problem is presented as a statement or a question (Linn & Miller 2005). For example:

Completion: The first traffic light in the US was invented by…………….

Short Answer: Who invented the first traffic light in the US?

These items are often used in mathematics tests, e.g.

3 + 10 = …………..?

If x = 6, what does x(x-1) =……….

Draw the line of symmetry on the following shape

A major advantage of these items is they that they are easy to construct. However, apart from their use in mathematics they are unsuitable for measuring complex learning outcomes and are often difficult to score. Completion and short answer tests are sometimes called objective tests as the intent is that there is only one correct answer and so there is no variability in scoring but unless the question is phrased very carefully, there are frequently a variety of correct answers. For example, consider the item

Where was President Lincoln born?………………..

The teacher may expect the answer “in a log cabin” but other correct answers are also “on Sinking Spring Farm”, “in Hardin County” or “in Kentucky”. Common errors in these items are summarized in Table 38.

---

Extended response

Extended response items are used in many content areas and answers may vary in length from a paragraph to several pages. Questions that require longer responses are often called essay questions. Extended response items have several advantages and the most important is their adaptability for measuring complex learning outcomes— particularly integration and application. These items also require that students write and therefore provide teachers a way to assess writing skills. A commonly cited advantage to these items is their ease in construction; however, carefully worded items that are related to learning outcomes and assess complex learning are hard to devise (Linn & Miller, 2005).

Well-constructed items phrase the question so the task of the student is clear. Often this involves providing hints or planning notes. In the first example below the actual question is clear not only because of the wording, but because of the format (i.e. it is placed in a box). In the second and third examples planning notes are provided:

Example 1: Third grade mathematics:

The owner of a bookstore gave 14 books to the school. The principal will give an equal number of books to each of three classrooms and the remaining books to the school library. How many books could the principal give to each student and the school?

Show all your work on the space below and on the next page. Explain in words how you found the answer. Tell why you took the steps you did to solve the problem.

From Illinois Standards Achievement Test, 2006;

Example 2: Fifth grade science: The grass is always greener

Jose and Maria noticed three different types of soil, black soil, sand, and clay, were found in their neighborhood. They decided to investigate the question, “How does the type of soil (black soil, sand, and clay) under grass sod affect the height of grass?”

Plan an investigation that could answer their new question.

In your plan, be sure to include

• Prediction of the outcome of the investigation
• Materials needed to do the investigation
• Procedure that includes:
• logical steps to do the investigation
• one variable kept the same (controlled)
• one variable changed (manipulated)
• any variables being measure and recorded
• how often measurements are taken and recorded (From Washington State 2004 assessment of student learning)

Writing prompt

Some people think that schools should teach students how to cook. Other people think that cooking is something that ought to be taught in the home. What do you think? Explain why you think as you do. Planning notes

Choose One:

• i think schools should teach students how to cook
• I think cooking should l be taught in the home
• I think cooking should be taught in ……………………………..because………(school) or (the home)

(From Illinois Measure of Annual Growth in English)

A major disadvantage of extended response items is the difficulty in reliable scoring. Not only do various teachers score the same response differently but also the same teacher may score the identical response differently on various occasions (Linn & Miller 2005). A variety of steps can be taken to improve the reliability and validity of scoring.

First, teachers should begin by writing an outline of a model answer. This helps make it clear what students are expected to include. Second, a sample of the answers should be read. This assists in determining what the students can do and if there are any common misconceptions arising from the question. Third, teachers have to decide what to do about irrelevant information that is included (e.g. is it ignored or are students penalized) and how to evaluate mechanical errors such as grammar and spelling. Then, a point scoring or a scoring rubric should be used.

In point scoring components of the answer are assigned points. For example, if students were asked:

What are the nature, symptoms, and risk factors of hyperthermia?

Point Scoring Guide:
Definition (natures) 2 pts
Symptoms (1 pt for each) 5 pts
Risk Factors (1 point for each) 5 pts
Writing 3 pts

This provides some guidance for evaluation and helps consistency, but point scoring systems often lead the teacher to focus on facts (e.g. naming risk factors) rather than higher level thinking that may undermine the validity of the assessment if the teachers’ purposes include higher level thinking. A better approach is to use a scoring rubric that describes the quality of the answer or performance at each level.