Tools and Resources

This file contains a list of "item-writing rules," which will help you to write multiple choice questions in a way that will improve the ability of the test to focus on the content and prevent students from guessing the correct answer without knowing the material. The rules were developed by experts in the field of psychometrics, like the people who write questions for SATs or GREs.

This IDEA paper from the Kansas State IDEA Center Resources provides guidelines for creating effective multiple choice tests.

This handout was provided at the workshop "Writing High Quality Multiple Choice Questions" (10/26/2018) by Hoi K. Suen

Distinguished Professor Emeritus

Educational Psychology

The Penn State University

These PowerPoint slides accompanied a presentation by Linda Suskie delivered via Zoom on Tuesday, Apr. 25, 2017. Multiple-choice tests can have a place in many courses. If they’re well designed, they can yield useful information on student achievement of many important course objectives, including some thinking skills. An item analysis of the results can shed light on how well the questions are working as well as what students have learned. Viewers will be able to use principles of good question construction to develop tests, develop test questions that assess thinking skills as well as conceptual understanding, and use item analysis to understand and improve both test questions and student learning. Be sure to open the handouts file listed below as you view the presentation!

These handouts (minus quizzes for test security) accompanied a presentation by Linda Suskie delivered via Zoom on Tuesday, Apr. 25, 2017. Multiple-choice tests can have a place in many courses. If they’re well designed, they can yield useful information on student achievement of many important course objectives, including some thinking skills. An item analysis of the results can shed light on how well the questions are working as well as what students have learned. Viewers will be able to use principles of good question construction to develop tests, develop test questions that assess thinking skills as well as conceptual understanding, and use item analysis to understand and improve both test questions and student learning.

Form for weighting items for exams with items that have different weights. Used for scanning multiple choice tests that use bubble sheets.

Student ratings are not the only option to provide evidence in the evaluation of teaching. There is a broad range of alternatives to consider beyond student ratings in the delicate decision-making processes to improve teaching and determine the promotion and tenure of faculty. Yet, despite the constant barrage of attacks on the integrity, reliability, and validity of student ratings, their use in higher education is at an all-time high.
So what do student ratings actually contribute to decisions about teaching and faculty? Should they be abandoned? Should you focus on the other options? This article examines student ratings and 14 alternatives to guide your plans to evaluate teaching in your department.

Practical suggestions for writing exams and techniques for creating questions from Boston University School of Public Health.

Large classes are among the most important because many students enrolled are new to the college experience. The big challenges of teaching large classes include finding ways to engage students, providing timely feedback, and managing logistics. When faced with these challenges, many instructors revert to lectures and multiple-choice tests. There are alternatives. This special report describes some alternative teaching and course management techniques to get students actively involved without an inordinate amount of work on the instructor’s part. From the Teaching Professor, Magna.

Abstract: Writing multiple-choice test items to measure student learning in higher education is a challenge. Based on extensive scholarly research and experience, the author describes various item formats, offers guidelines for creating these items, and provides many examples of both good and bad test items. He also suggests some shortcuts for developing test items. All of this advice is based on extensive scholarly research and experience. Creating valid multiple-choice items is a difficult task, but it contributes greatly to the teaching and learning process for undergraduate, graduate, and professional-school courses.

Author: Thomas M. Haladyna, Arizona State University

Keywords: Multiple-choice items, selected response, test-item formats, examinations

This is a peer-reviewed article published in the journal of Studies in Educational Evaluation. Its focus is the accurate interpretation of student ratings data (including Penn State's SRTE) and appropriate use of the data to evaluate faculty. It includes recommendations for use and interpretation based on more than 80 years of student ratings research. Most colleges and universities use student ratings data to guide personnel decisions so it is critical that administrators and faculty evaluators have access to the cumulative knowledge about student ratings based on multiple studies, rather than single studies that have not been replicated, studies based on non-representative populations, or that are from a single discipline.

The article provides an overview of common views and misconceptions about student ratings, followed by clarification of what student ratings are and are not. It also includes two sets of guidelines for administrators and faculty serving on review committees.

Everywhere you turn, colleagues are talking about evidence-based teaching. But even when
the evidence is convincing, it can be tough to choose a strategy and begin using it well. This
navigational guide will help you get started.
Horii, C. V. (2018) Wise Instructional Choices in an Evidence-driven Era. NEA Higher Education Advocate, 36(3), 6-9.

Often called “peer observation of teaching ” or “peer evaluation of teaching,” peer review of teaching (PRT) involves seeking feedback from an informed colleague for the purposes of improving one’s practice (formative assessment) and/or evaluating it (summative assessment). Texas A&M University's Faculty Performance Evaluation Task Force recommended having separate review processes for formative and summative assessment using multiple sources of data from students, peers, administrators, and as well as faculty themselves for evaluating teaching. Includes institutional perspectives and supporting videos from the University of Texas.

In this rationale, Natalie Parker, Director of CETL and Distance Education, Texas Wesleyan University, advocates for replacing high stakes exams with multiple-attempt, low-stakes quizzes. The “testing effect”, in which students recall more information about a topic after testing than after re-reading the material, was first reported by Abbott in 1909. Subsequent studies have confirmed that repeated testing is an effective way for students to recall material.

Best Practices in the Evaluation of Teaching, by Stephen L. Benton, The IDEA Center and Suzanne Young, University of Wyoming
Effective instructor evaluation is complex and requires the use of multiple measures—formal and informal, traditional and authentic—as part of a balanced evaluation system. The student voice, a critical element of that balanced system, is appropriately complemented by instructor self-assessment and the reasoned judgments of other relevant parties, such as peers and supervisors. Integrating all three elements allows instructors to take a mastery approach to formative evaluation, trying out new teaching strategies and remaining open to feedback that focuses on how they might improve. Such feedback is most useful when it occurs in an environment that fosters challenge, support, and growth. Rather than being demoralized by their performance rankings, faculty can concentrate on their individual efforts and compare current progress to past performance. They can then concentrate on developing better teaching methods and skills rather than fearing or resenting comparisons to others. The evaluation of teaching thus becomes a rewarding process, not a dreaded event.
Keywords: Evaluation of teaching, summative evaluation, formative evaluation, mastery orientation

Item Analysis (a.k.a. Test Question Analysis) is an empowering process that enables you to improve mutiple-choice test score validity and reliability by analyzing item performance over time and making necessary adjustments. Knowledge of score reliability, item difficulty, item discrimination, and crafting effective distractors can help you make decisions about whether to retain items for future administrations, revise them, or eliminate them from the test item pool. Item analysis can also help you to determine whether a particular portion of course content should be revised or enhanced.

Overview of best practices for using PowerPoint as an instructional tool. Handout includes numerous tips and illustrative slides focused on: debate about the proper role of PowerPoint in education; using the assertion-evidence model in slide design in order to promote learning; making user-friendly choices about graphic design.