Skip to main content
Social Sci LibreTexts

7A: Evaluation Guide

  • Page ID
    55451
  • \( \newcommand{\vecs}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \)

    \( \newcommand{\vecd}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash {#1}}} \)

    \( \newcommand{\id}{\mathrm{id}}\) \( \newcommand{\Span}{\mathrm{span}}\)

    ( \newcommand{\kernel}{\mathrm{null}\,}\) \( \newcommand{\range}{\mathrm{range}\,}\)

    \( \newcommand{\RealPart}{\mathrm{Re}}\) \( \newcommand{\ImaginaryPart}{\mathrm{Im}}\)

    \( \newcommand{\Argument}{\mathrm{Arg}}\) \( \newcommand{\norm}[1]{\| #1 \|}\)

    \( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\)

    \( \newcommand{\Span}{\mathrm{span}}\)

    \( \newcommand{\id}{\mathrm{id}}\)

    \( \newcommand{\Span}{\mathrm{span}}\)

    \( \newcommand{\kernel}{\mathrm{null}\,}\)

    \( \newcommand{\range}{\mathrm{range}\,}\)

    \( \newcommand{\RealPart}{\mathrm{Re}}\)

    \( \newcommand{\ImaginaryPart}{\mathrm{Im}}\)

    \( \newcommand{\Argument}{\mathrm{Arg}}\)

    \( \newcommand{\norm}[1]{\| #1 \|}\)

    \( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\)

    \( \newcommand{\Span}{\mathrm{span}}\) \( \newcommand{\AA}{\unicode[.8,0]{x212B}}\)

    \( \newcommand{\vectorA}[1]{\vec{#1}}      % arrow\)

    \( \newcommand{\vectorAt}[1]{\vec{\text{#1}}}      % arrow\)

    \( \newcommand{\vectorB}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \)

    \( \newcommand{\vectorC}[1]{\textbf{#1}} \)

    \( \newcommand{\vectorD}[1]{\overrightarrow{#1}} \)

    \( \newcommand{\vectorDt}[1]{\overrightarrow{\text{#1}}} \)

    \( \newcommand{\vectE}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash{\mathbf {#1}}}} \)

    \( \newcommand{\vecs}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \)

    \( \newcommand{\vecd}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash {#1}}} \)

    \(\newcommand{\avec}{\mathbf a}\) \(\newcommand{\bvec}{\mathbf b}\) \(\newcommand{\cvec}{\mathbf c}\) \(\newcommand{\dvec}{\mathbf d}\) \(\newcommand{\dtil}{\widetilde{\mathbf d}}\) \(\newcommand{\evec}{\mathbf e}\) \(\newcommand{\fvec}{\mathbf f}\) \(\newcommand{\nvec}{\mathbf n}\) \(\newcommand{\pvec}{\mathbf p}\) \(\newcommand{\qvec}{\mathbf q}\) \(\newcommand{\svec}{\mathbf s}\) \(\newcommand{\tvec}{\mathbf t}\) \(\newcommand{\uvec}{\mathbf u}\) \(\newcommand{\vvec}{\mathbf v}\) \(\newcommand{\wvec}{\mathbf w}\) \(\newcommand{\xvec}{\mathbf x}\) \(\newcommand{\yvec}{\mathbf y}\) \(\newcommand{\zvec}{\mathbf z}\) \(\newcommand{\rvec}{\mathbf r}\) \(\newcommand{\mvec}{\mathbf m}\) \(\newcommand{\zerovec}{\mathbf 0}\) \(\newcommand{\onevec}{\mathbf 1}\) \(\newcommand{\real}{\mathbb R}\) \(\newcommand{\twovec}[2]{\left[\begin{array}{r}#1 \\ #2 \end{array}\right]}\) \(\newcommand{\ctwovec}[2]{\left[\begin{array}{c}#1 \\ #2 \end{array}\right]}\) \(\newcommand{\threevec}[3]{\left[\begin{array}{r}#1 \\ #2 \\ #3 \end{array}\right]}\) \(\newcommand{\cthreevec}[3]{\left[\begin{array}{c}#1 \\ #2 \\ #3 \end{array}\right]}\) \(\newcommand{\fourvec}[4]{\left[\begin{array}{r}#1 \\ #2 \\ #3 \\ #4 \end{array}\right]}\) \(\newcommand{\cfourvec}[4]{\left[\begin{array}{c}#1 \\ #2 \\ #3 \\ #4 \end{array}\right]}\) \(\newcommand{\fivevec}[5]{\left[\begin{array}{r}#1 \\ #2 \\ #3 \\ #4 \\ #5 \\ \end{array}\right]}\) \(\newcommand{\cfivevec}[5]{\left[\begin{array}{c}#1 \\ #2 \\ #3 \\ #4 \\ #5 \\ \end{array}\right]}\) \(\newcommand{\mattwo}[4]{\left[\begin{array}{rr}#1 \amp #2 \\ #3 \amp #4 \\ \end{array}\right]}\) \(\newcommand{\laspan}[1]{\text{Span}\{#1\}}\) \(\newcommand{\bcal}{\cal B}\) \(\newcommand{\ccal}{\cal C}\) \(\newcommand{\scal}{\cal S}\) \(\newcommand{\wcal}{\cal W}\) \(\newcommand{\ecal}{\cal E}\) \(\newcommand{\coords}[2]{\left\{#1\right\}_{#2}}\) \(\newcommand{\gray}[1]{\color{gray}{#1}}\) \(\newcommand{\lgray}[1]{\color{lightgray}{#1}}\) \(\newcommand{\rank}{\operatorname{rank}}\) \(\newcommand{\row}{\text{Row}}\) \(\newcommand{\col}{\text{Col}}\) \(\renewcommand{\row}{\text{Row}}\) \(\newcommand{\nul}{\text{Nul}}\) \(\newcommand{\var}{\text{Var}}\) \(\newcommand{\corr}{\text{corr}}\) \(\newcommand{\len}[1]{\left|#1\right|}\) \(\newcommand{\bbar}{\overline{\bvec}}\) \(\newcommand{\bhat}{\widehat{\bvec}}\) \(\newcommand{\bperp}{\bvec^\perp}\) \(\newcommand{\xhat}{\widehat{\xvec}}\) \(\newcommand{\vhat}{\widehat{\vvec}}\) \(\newcommand{\uhat}{\widehat{\uvec}}\) \(\newcommand{\what}{\widehat{\wvec}}\) \(\newcommand{\Sighat}{\widehat{\Sigma}}\) \(\newcommand{\lt}{<}\) \(\newcommand{\gt}{>}\) \(\newcommand{\amp}{&}\) \(\definecolor{fillinmathshade}{gray}{0.9}\)

    What are concept inventories?

    (Excerpted from presentation by J. Knight, UC Boulder).

    • Multiple choice (usually) instruments that address fundamental concepts and contain known student misunderstandings
    • Developed through an iterative process that includes gathering evidence of validity and reliability through student and faculty interviews
    • Diagnostic: can identify specific misunderstandings and measure student learning over time
    • Objective: not tied directly to a course, but rather to a set of concepts

    Guidelines for using concept inventories

    (Dirks, Wenderoth, Withers Assessment in the College Science Classroom, 2013).

    • Protect the test!
    • Must be given in a proctored environment to keep questions from getting out to students.
    • Use for evaluation only
    • Not a learning tool.
    • When used for pre-/post-testing
    • Use the same testing context
    • Can use same or isomorphic questions (Resource: Research Methods Knowledge Base – W. Trochim, 2013)
    • Normalized learning gain
    • <g> = (%post - %pre)/(100-%pre)

    Content-independent metrics

    Other Resources

    Table below from Dirk et al., (2014) Assessment in the College Science Classroom, Ch7 Appendix A; Freeman, NYC.

    Concept Inventories in Astronomy

    Astronomy Diagnostic Test (ADT)

    Lunar Phases

    Light and Spectroscopy

    Hufnagel 2002

    Lindell and Olsen 2002

    Bardar et al., 2007

    Concept Inventories in Biology

    Genetics Concept Inventory (GCA)

    Genetics Literacy Assessment Instrument 2 (GLAI-2)

    Conceptual Inventory of Natural Selection (CINS)

    Biology Literacty (http://bioliteracy.net/)

    Diagnostic Question Clusters: Biology

    Host Pathogen Interactions (HPI)

    Introductory Molecular and Cell Biology Assessment (IMCA)

    Smith et al., 2008

    Bowling et al., 2008

    Anderson et al., 2002

    Klymkowsky et al., 2010

    Wilson et al., 2006; D’Avanzo 2008

    Marbach-Ad et al., 2009

    Shi et al., 2010

    Concept Inventories in Chemistry

    Chemistry Concept Inventory

    Mulford and Rbonison 2002

    Krause et al., 2003

    Concept Inventories in Engineering

    Engineering Thermodynamics Concept Inventory

    Heat Transfer

    Materials Concept Inventory

    Signals and Systems Concept Inventory

    Static Concept Inventory

    Thermal and Transport Science Concept Inventory (TTCI)

    Midkiff et al., 2001

    Jacobie et al., 2003

    Krause et al., 2003

    Wage et al., 2005

    Steif et al., 2005

    Streveler et al., 2011

    Concept Inventories in Geoscience

    Geoscience Concept Inventory (GCI)

    Libarkin and Anderson, 2005

    Concept Inventories in Math and Statistics

    Statistics Concept Inventory (SCI)

    Calculus Concept Inventory (CCI)

    Allen 2006

    Epstein 2005

    Concept Inventories in Physics

    Force Concept Inventory (FCI)

    The Force and Motion Conceptual Evaluation (FMCE)

    Thermal Concept Evaluation

    Brief Electricity and Magnetism Assessment (BEMA)

    Conceptual Survey in Electricity and Magnetism (CSEM)

    Hestenes et al., 1992

    Thornton and Sokiloff 1998

    Yeo and Zadnick 2001

    Ding et al., 2006

    Maloney et al., 2001

    Measuring Students Science Process and Reasoning Skills

    Rubric for Science Writing

    Student-Achievement and Process Skills Instrument

    Timmerman et al., 2010

    Bunce et al., 2010

    Measuring Student Attitudes about Science, Research or Study Methods

    Colorado Learning Attitudes about Science Survey (CLASS)

    Revised Two-Factor Study Process Questionnaire

    Student Assessment of Their Learning Gains (SALG) Instrument

    Survey of Undergraduate Research Experiences

    Views About Science Survey (VASS)

    http://www.colorado.edu/sei/class

    Biggs et al, 2001

    http://www.salgsite.org/

    Lopatto 2004

    Halloun and Hestenes 1998


    This page titled 7A: Evaluation Guide is shared under a not declared license and was authored, remixed, and/or curated by Mark Baillie.