6A: Evaluation Guide
- Page ID
- 43444
What are concept inventories?
(Excerpted from presentation by J. Knight, UC Boulder).
- Multiple choice (usually) instruments that address fundamental concepts and contain known student misunderstandings
- Developed through an iterative process that includes gathering evidence of validity and reliability through student and faculty interviews
- Diagnostic: can identify specific misunderstandings and measure student learning over time
- Objective: not tied directly to a course, but rather to a set of concepts
Guidelines for using concept inventories
(Dirks, Wenderoth, Withers Assessment in the College Science Classroom, 2013).
- Protect the test!
- Must be given in a proctored environment to keep questions from getting out to students.
- Use for evaluation only
- Not a learning tool.
- When used for pre-/post-testing
- Use the same testing context
- Can use same or isomorphic questions (Resource: Research Methods Knowledge Base – W. Trochim, 2013)
- Normalized learning gain
- <g> = (%post - %pre)/(100-%pre)
Content-independent metrics
- Typically assess skills or affective domain
- Critical thinking, views of science, enthusiasm for the discipline…
- Can be used as pre/post, but typically post only
- Resource: FLAG – Field-tested Learning Assessment Guide - http://www.flaguide.org/index.php
- Mental Measures Yearbook http://buros.org/mental-measurements-yearbook
- http://www.salgsite.org/
- https://www.tntech.edu/cat/
- http://www.criticalthinking.org/pages/critical-thinking-testing-and-assessment/594
- https://www2.viu.ca/studentsuccessse...sInventory.pdf
Other Resources
- Summer Institutes website: http://www.summerinstitutes.org/
- University of Colorado – SEI: http://www.colorado.edu/sei/
- SERC: https://serc.carleton.edu/index.html
- https://serc.carleton.edu/NAGTWorkshops/departments/degree_programs/metrics.html
- UW BERG: http://uwberg.com/teaching-resources/
Table below from Dirk et al., (2014) Assessment in the College Science Classroom, Ch7 Appendix A; Freeman, NYC.
Concept Inventories in Astronomy |
|
Astronomy Diagnostic Test (ADT) Lunar Phases Light and Spectroscopy |
Hufnagel 2002 Lindell and Olsen 2002 Bardar et al., 2007 |
Concept Inventories in Biology |
|
Genetics Concept Inventory (GCA) Genetics Literacy Assessment Instrument 2 (GLAI-2) Conceptual Inventory of Natural Selection (CINS) Biology Literacty (http://bioliteracy.net/) Diagnostic Question Clusters: Biology Host Pathogen Interactions (HPI) Introductory Molecular and Cell Biology Assessment (IMCA) |
Smith et al., 2008 Bowling et al., 2008 Anderson et al., 2002 Klymkowsky et al., 2010 Wilson et al., 2006; D’Avanzo 2008 Marbach-Ad et al., 2009 Shi et al., 2010 |
Concept Inventories in Chemistry |
|
Chemistry Concept Inventory |
Mulford and Rbonison 2002 Krause et al., 2003 |
Concept Inventories in Engineering |
|
Engineering Thermodynamics Concept Inventory Heat Transfer Materials Concept Inventory Signals and Systems Concept Inventory Static Concept Inventory Thermal and Transport Science Concept Inventory (TTCI) |
Midkiff et al., 2001 Jacobie et al., 2003 Krause et al., 2003 Wage et al., 2005 Steif et al., 2005 Streveler et al., 2011 |
Concept Inventories in Geoscience |
|
Geoscience Concept Inventory (GCI) |
Libarkin and Anderson, 2005 |
Concept Inventories in Math and Statistics |
|
Statistics Concept Inventory (SCI) Calculus Concept Inventory (CCI) |
Allen 2006 Epstein 2005 |
Concept Inventories in Physics |
|
Force Concept Inventory (FCI) The Force and Motion Conceptual Evaluation (FMCE) Thermal Concept Evaluation Brief Electricity and Magnetism Assessment (BEMA) Conceptual Survey in Electricity and Magnetism (CSEM) |
Hestenes et al., 1992 Thornton and Sokiloff 1998 Yeo and Zadnick 2001 Ding et al., 2006 Maloney et al., 2001 |
Measuring Students Science Process and Reasoning Skills |
|
Rubric for Science Writing Student-Achievement and Process Skills Instrument |
Timmerman et al., 2010 Bunce et al., 2010 |
Measuring Student Attitudes about Science, Research or Study Methods |
|
Colorado Learning Attitudes about Science Survey (CLASS) Revised Two-Factor Study Process Questionnaire Student Assessment of Their Learning Gains (SALG) Instrument Survey of Undergraduate Research Experiences Views About Science Survey (VASS) |
http://www.colorado.edu/sei/class Biggs et al, 2001 Lopatto 2004 Halloun and Hestenes 1998 |