How to create questions that are parallel
Having different questions in each test may seem like a certain way of ensuring that students will not be able to give help to other students during an extended testing period. It is relatively easy to accomplish with modern testing software systems. As a result, many instructors will be tempted to use this technique, but it is important to recognize important potential problems that can complicate the testing process when question sampling is used. One consequence of not preparing adequately to use this method is that the many forms of the tests will not be equally difficult nor will they assess the same course objectives.
In order to discuss the potential problems several new terms must be defined. We refer to a group of questions as a sampling group if it is used to support the random selection of just one of two or more questions when a test is created. Questions are judged to be parallel if they meet certain requirements. The sampling of questions must only be used if the questions are parallel. For a group of questions to be parallel, all questions in the group must:
- be equally difficult for test takers
- work in the same way (i.e., have the same level of discrimination between high and low scoring students)
- trap the same set of misconceptions (distractor effectiveness)
- refer to the same instructional outcome at a fairly detailed level of specificity