Multiple Choice Testing

This Q&A was adapted with permission from the book Chalk Talk: E-advice from Jonas Chalk, Legendary College Teacher, edited by Donna M. Qualters and Miriam Rosalyn Diamond –

Question

Dear Jonas,

I teach a couple of large introductory sections and use multiple-choice exams. Some of my colleagues say that is “smart teaching” and others say it’s lazy teaching. What do you think? Are there ways to increase the effectiveness of using multiple choice tests?

Signed: A) Smart B) Lazy

Answer

Dear A) Smart B) Lazy,

I say “yes” to both sides depending on HOW and WHY instructors use multiple-choice tests. Multiple choice testing is an efficient way to get feedback on learning from large sections and I’ll talk about several good reasons to use them. Effective multiple choice testing is also very difficult to do without a lot of thought and preparation. Actually, a study showed that new instructors needed almost 30 minutes PER question to construct a “good” multiple-choice exam. So there is a balance in time spent on multiple choice versus other forms of testing if you want to do it right.

When should you use multiple choice testing? Most experts will tell you that multiple choice test are good when you want to test the breadth of student learning with less depth, as in a survey course. They’re also good when you want to test different levels of learning around a concept to see if students have not only understood a concept, but can use it as well. Multiple choice testing also provides some professional development for students, since most of them will have to take some form of high stakes standardized test (licensure, grad school) in their future. Multiple choice tests can also be effective when it is not necessary for you to determine HOW they formulated their answer only that they CAN formulate the answer.

There are several benefits to using a multiple-choice format. Besides being easy to grade, you can correlate specific questions to specific learning objectives for your course and feedback on areas of your curriculum if a large percentage of students fail a specific question that is well constructed. You can also compare performance from class to class or year to year if you have a large, reliable question bank.

Although there are valid educational reasons to use multiple choice testing, remember that construction of a meaningful multiple choice test is very difficult and time consuming, so don’t wait until the day before the test to decide to use this format.

If you decide that multiple choice testing is the best format to assess the learning in your course, here are some points to consider as you construct the test items.

Is the item clear and concise? Writing good stems (i.e. the question) takes considerable thought. Be sure that you use active voice, avoid common typos, and have all essential information in the stem.
Stems can be in the form of questions or incomplete statements. Questions are clearer to understand as opposed to incomplete sentences that are open to interpretation.
State questions or statements in the positive form. Using negatives is not a way to not confuse students – Huh?? If you must use negatives, emphasize them somehow in italics or bold to alert students.
Write the correct response immediately after you write the stem and keep it brief and simple.
Add distractors (i.e. the wrong answers) AFTER you’ve written the right answer and be sure that the form of the distracters matches the form of the right answer. Long right answers and short distractors clue students into the answer.
Check the grammar in distractors. The wrong tense will alert students that this is a distractor.
Don’t make the distractors too similar to the correct answer. Your goal should be to determine student learning, comprehension and ability to apply information, not to “trick” the students into choosing the wrong answer. (Remember, they’ll be nervous and under time pressure as it is.
Be sure that you test higher levels of learning in your test. It is possible to construct multiple-choice questions that test problem solving, application, and synthesis skills. Our tendency is to keep multiple-choice tests to knowledge and comprehension levels only.
Use “none of the above” or “all of the above” with caution (some experts say never), especially with “best answer” stems. However, this format may be helpful in stems that involve mathematical operation. Sometimes, students have been able to successfully argue for the “some” or “all of the above” response, when I had intended for only one to be the correct answer.
Make sure the length of the test is appropriate to the time students have to take the test.
Check to be sure that you’ve distributed answers keys (A, B, C, D) evenly and randomly over the exam.
Lastly and most importantly, have someone else review your test. Ask your reviewer to tell you if there might be multiple interpretations of your stem, if there might be another possible right answer you didn’t consider, if your right answer is too obvious or if there are spelling/typo errors.

One of the limitations of multiple-choice tests is the inability to see how students formulate their answers. If you need to understand thought processes for a couple of questions, you can still use the multiple-choice format. For those one or two questions, add a simple addendum to that question – “Explain your choice” and give them a couple of lines to do it. (This will also help you discern which students are good guessers and which have actually mastered the material.

Remember multiple choice tests should not be designed to trick students, but rather to be rigorous assessment tools of student learning. There is much more information about writing good multiple-choice tests. If you’re interested in more depth, try How to write better tests by Lucy Jacobs, Indiana University.

Good luck,

Jonas

Quick Tip

As an assignment – or for extra credit – you can ask students to formulate possible multiple choice questions from the course material. This gives them a means to interact with the material, test their own understanding, and anticipate the kind of questions that may actually be asked. It can serve both as a way to enforce that they read assignments, as well as a way to provide a format they can use in studying. It will also give you a bank of possible questions (which you may need to further adapt, refine and polish).

______

This content was adapted with permission from the book Chalk Talk: E-advice from Jonas Chalk, Legendary College Teacher, edited by Donna M. Qualters and Miriam Rosalyn Diamond.

Browse the complete Index of Teaching Challenges – Chalk Talk