# 2.7.1: Evaluation Guide

$$\newcommand{\vecs}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} }$$ $$\newcommand{\vecd}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash {#1}}}$$$$\newcommand{\id}{\mathrm{id}}$$ $$\newcommand{\Span}{\mathrm{span}}$$ $$\newcommand{\kernel}{\mathrm{null}\,}$$ $$\newcommand{\range}{\mathrm{range}\,}$$ $$\newcommand{\RealPart}{\mathrm{Re}}$$ $$\newcommand{\ImaginaryPart}{\mathrm{Im}}$$ $$\newcommand{\Argument}{\mathrm{Arg}}$$ $$\newcommand{\norm}[1]{\| #1 \|}$$ $$\newcommand{\inner}[2]{\langle #1, #2 \rangle}$$ $$\newcommand{\Span}{\mathrm{span}}$$ $$\newcommand{\id}{\mathrm{id}}$$ $$\newcommand{\Span}{\mathrm{span}}$$ $$\newcommand{\kernel}{\mathrm{null}\,}$$ $$\newcommand{\range}{\mathrm{range}\,}$$ $$\newcommand{\RealPart}{\mathrm{Re}}$$ $$\newcommand{\ImaginaryPart}{\mathrm{Im}}$$ $$\newcommand{\Argument}{\mathrm{Arg}}$$ $$\newcommand{\norm}[1]{\| #1 \|}$$ $$\newcommand{\inner}[2]{\langle #1, #2 \rangle}$$ $$\newcommand{\Span}{\mathrm{span}}$$$$\newcommand{\AA}{\unicode[.8,0]{x212B}}$$

## What are concept inventories?

(Excerpted from presentation by J. Knight, UC Boulder).

• Multiple choice (usually) instruments that address fundamental concepts and contain known student misunderstandings
• Developed through an iterative process that includes gathering evidence of validity and reliability through student and faculty interviews
• Diagnostic: can identify specific misunderstandings and measure student learning over time
• Objective: not tied directly to a course, but rather to a set of concepts

## Guidelines for using concept inventories

(Dirks, Wenderoth, Withers Assessment in the College Science Classroom, 2013).

• Protect the test!
• Must be given in a proctored environment to keep questions from getting out to students.
• Use for evaluation only
• Not a learning tool.
• When used for pre-/post-testing
• Use the same testing context
• Can use same or isomorphic questions (Resource: Research Methods Knowledge Base – W. Trochim, 2013)
• Normalized learning gain
• <g> = (%post - %pre)/(100-%pre)

## Other Resources

Table below from Dirk et al., (2014) Assessment in the College Science Classroom, Ch7 Appendix A; Freeman, NYC.

##### Concept Inventories in Astronomy

Lunar Phases

Light and Spectroscopy

Hufnagel 2002

Lindell and Olsen 2002

Bardar et al., 2007

##### Concept Inventories in Biology

Genetics Concept Inventory (GCA)

Genetics Literacy Assessment Instrument 2 (GLAI-2)

Conceptual Inventory of Natural Selection (CINS)

Biology Literacty (http://bioliteracy.net/)

Diagnostic Question Clusters: Biology

Host Pathogen Interactions (HPI)

Introductory Molecular and Cell Biology Assessment (IMCA)

Smith et al., 2008

Bowling et al., 2008

Anderson et al., 2002

Klymkowsky et al., 2010

Wilson et al., 2006; D’Avanzo 2008

Shi et al., 2010

##### Concept Inventories in Chemistry

Chemistry Concept Inventory

Mulford and Rbonison 2002

Krause et al., 2003

##### Concept Inventories in Engineering

Engineering Thermodynamics Concept Inventory

Heat Transfer

Materials Concept Inventory

Signals and Systems Concept Inventory

Static Concept Inventory

Thermal and Transport Science Concept Inventory (TTCI)

Midkiff et al., 2001

Jacobie et al., 2003

Krause et al., 2003

Wage et al., 2005

Steif et al., 2005

Streveler et al., 2011

##### Concept Inventories in Geoscience

Geoscience Concept Inventory (GCI)

Libarkin and Anderson, 2005

##### Concept Inventories in Math and Statistics

Statistics Concept Inventory (SCI)

Calculus Concept Inventory (CCI)

Allen 2006

Epstein 2005

##### Concept Inventories in Physics

Force Concept Inventory (FCI)

The Force and Motion Conceptual Evaluation (FMCE)

Thermal Concept Evaluation

Brief Electricity and Magnetism Assessment (BEMA)

Conceptual Survey in Electricity and Magnetism (CSEM)

Hestenes et al., 1992

Thornton and Sokiloff 1998

Ding et al., 2006

Maloney et al., 2001

##### Measuring Students Science Process and Reasoning Skills

Rubric for Science Writing

Student-Achievement and Process Skills Instrument

Timmerman et al., 2010

Bunce et al., 2010

##### Measuring Student Attitudes about Science, Research or Study Methods

Revised Two-Factor Study Process Questionnaire

Student Assessment of Their Learning Gains (SALG) Instrument