Study test data - Schreyer Institute for Teaching Excellence

Schreyer Institute
for Teaching Excellence
301 Rider Building
227 West Beaver Avenue
University Park, PA 16802
Phone: (814) 865-8681
Fax: (814) 865-8659

104-105 Pollock Building
University Park, PA 16802
Scanning: (814) 863-2802
Testing: (814) 863-8116
Fax: (814) 863-2322

A Division of Undergraduate Education at Penn State.

Study test data to determine if questions have been compromised

It is important to use as simple a procedure as possible to secure a test against compromise. More extensive methods all require more effort and care and should only be used if it is clear that early testing students are sharing information with those testing later. The item analysis report provides information that can be used to examine the performance of questions over time within a semester and over semesters. The standard item analysis available after a test has been completed can be used to monitor the performance of both individual questions and the test as a whole. This information can be valuable in helping instructors determine if they need to use more complicated test creation procedures to ensure fair tests.

Study question difficulty and time used in answering each question over time
If a test is administered over a day or several days, it is possible to examine item analysis reports that are calculated separately for each of the time periods. If a question becomes increasingly easier over time, then that may be evidence that early testing students have found a way to help later testing students. Similarly, if the amount of time taken by students in answering a question tends to shrink over time, then that may be an additional indication that other methods must be explored to secure the test over an extended time period. It is important to consider time slices that contain a relatively large number of test takers to ensure against spurious results based on unrepresentative groups of individuals.