SDU Universitetspædagogik

MCQ and similar tests

There are a number of test methods in which students must give concise answers to a number of questions, either by selecting multiple-choice options or by providing a brief answer of their own. Examples of such tests are:

• Multiple choice questions (MCQ), where the student has to select one best to a questions based on predetermined choices
• Fill-in-the-blank, where the student has to insert one or more words in an empty space in a sentence
• Short-answer, where the student has to formulate a short answer, typically a few lines

MCQ and similar tests are best suited for testing knowledge, basic concepts or applied knowledge requiring a relatively short, unambiguous answer.

The quality of MCQ tests depends on how the questions are written and on the number of questions. It is recommended to write quite long questins (e.g. a description of a case or a situation) while the possible answers must be relatively short, precise and uniform in their wording – see references below. The validity and reliability of MCQ tests is a function of the number of questions in the test – 50 questions or more are not unusual.

Fill-in-the-blank could be used to test for example basic understanding of words and concepts.

Short-answer test can in most circumstances with advantage be replaced by MCQ, which is cheaper to conduct.

MCQ and similar tests are rarely used as the only assessment method for a course; they are frequently used in combination with other types of testing, as they are primarily useful for testing knowledge and some types of problem solving skills e.g. clinical reasoning. The latter requires specialist expertise on designing the questions.

Learning objective that can be addressed using this assessment method
Knowledge of theory, method and practice	X
Understanding and reflecting on theory, method and practice	(X)
Applying methods and tools
Assessing problem definitions and selecting solution models	(X)
Communicating and discussing academic issues
Dealing with complex situations – in the context of studying or work
Ability to independently initiate collaboration
Taking responsibility for personal learning and development

X indicates that this assessment method is suitable for testing the learning outcome. (X) indicates that this assessment method is of only limited use for this.

Evaluating the method of assessment

Assessment Criteria
Validity	It is possible to cover a wide range of content – wide content validity. Construct validity is ensured by carefully considering how the questions are asked, and which answers are possible.
Reliability	Objectivity in the definition of right and wrong answers results in very high reliability. In the case of MCQ, the more questions the test includes and the larger the number of possible answers (up to a maximum of 5–6) for each question, the less likely it is that students will arrive at the correct answer by guessing.
Backwash effect from testing to teaching	This type of testing inherently signals that surface-level knowledge will suffice; this can encourage students to superficial instead of deep learning. There will also be a risk of the teacher focusing on the students’ acquisition of rote knowledge and of planning the teaching around delivery of information if this is the only assessment method used.
Resources	It takes time to produce a good test. On the other hand, this is one of the least cost-intensive types of test to implement and assess.
Digitisation	Can be taken with or without Internet access. It is possible to prevent Internet access by using a lock-down browser. Enables the randomising of questions and possible answers so students cannot exchange possible answers. Enables the collection of ‘good’ questions and answers in a database whose content can be expanded, exchanged and qualified with the help of national and international teaching colleagues in the same field. Can further minimise costs of marking. The risk of students using disallowed aids is greater but this can be minimised, if the ratio between the number of questions and the time available to answer is such that it is not possible to use disallowed aids in the time available.
Acceptance	The acceptance of these testing methods differs substantially between faculties. Some teachers and students regard MCQs as an unacademic, simplified testing method. This is why it is particularly important to state what it can – and cannot – assess, and to ensure that other types of assessment are used as a supplement.

 

Additional reading

Higgins, E.& Tatham, L. (2003). Assessing by multiple choice question (MCQ) tests, Manchester Metropolitan University, http://homepages.shu.ac.uk/~edsjlc/ict/multi_choice_q's/mcq's%20(U%20Bristol).doc

Kehoe, Jerard (1995). Writing multiple-choice test items. Practical Assessment, Research & Evaluation, 4(9).
Retrieved February 22, 2011 from http://PAREonline.net/getvn.asp?v=4&n=9

Case, S. & Swanson, B. (2001). Constructing written test questions in the basic and clinical sciences.
National Board of Medical Examiners. http://www.nbme.org/publications/item-writing-manual.html

---