A question of questions
Some very wise friends and I share our top tips on effective quiz- and question-writing in learning.
Quizzes. Particularly in digital learning, they sometimes get a bad rap. And yet, their constituent part – the question – has been an integral part of learning, indeed of thinking, for aeons.
Last week, I was doing a quality audit of a series of online multi-choice / multi-select questions. Inevitably, I started reflecting on why there might be such negativity about them, whether in the guise of quizzes, or tests, or knowledge checks, or any other wrapper, given the intrinsic value of the question to learning and knowledge development.
Aside from probably the biggest trap of overuse, leading to monotony due to lack of variety, I concluded that, at the risk of stating the blinkin’ obvious:
Questions aren’t bad. Only bad questions are bad.
With that in mind, I did a bit of internal research (aka asked a few of my esteemed friends and colleagues for their opinions!). Between us, we’ve come up with our favourite top tips for good question (and question bank / quiz) design.
Know your platform
“All of the above.” “How about this one?” I’ve seen both the former as a multi choice option, and the latter as one of a series of questions about a particular mathematical concept. They were in our brilliant QA Learning Platform, which randomises both question order and answer option order. Have you spotted the issue here?! Aside from the problem of having an “All of the above” option in the first place (if I see it, I choose it!), know what your lovely platform is going to do to your carefully-crafted question that looks so great on the spreadsheet.
Oh, and if the platform doesn’t randomise options, make sure you don’t always put the correct option first! Trust me; I’ve seen it…Focus on your distractors
How will you design them so that they’re plausible, but definitely incorrect? A couple of tips: consider the length. If the correct answer is twice the length of the others, you’re giving the learner a bit of a clue there. Also the relevance. Do the distractor options still relate directly to the question, and the prior learning?Give quality feedback
If your platform allows for feedback, use it! And use it wisely. It’s where your learners can reinforce their understanding. Explain WHY the correct options ARE correct, but also explain WHY the distractors AREN’T. Considering this may well assist you in the second tip: if there are common misunderstandings on a particular topic, the combination of quality distractors and quality feedback are definitely your allies.Get the level right
There are two elements to this.
Firstly: know your audience. Is the topic new to your learners, or are they building on previous knowledge and experience? Make sure that your range of questions are pitched accordingly.
Secondly: mix it up a bit! It’s OK to have some confidence-boosters, but if everyone comes out with 100%, it’s too simple. Vary the difficulty. You should also include some stretch questions, to really get learners thinking and applying.Factual or applied?
You’ll probably move more towards the latter as learners become more proficient in the topic, but include both from the start wherever possible. Take what they’ve learned, apply it to a scenario. Can they adapt what they’ve learned? If not, they may need to deepen their understanding.K.I.S.S. (Keep it simple, sunshine!)
You’re testing their knowledge, not their ability to decrypt inordinately circuitous lexicon (see what I did there?), or whole strings of acronyms (see title). Using an unfamiliar acronym? Spell it out alongside; reinforce the learning.
Make sure the language doesn’t get in the way of the question. If unsure, ask a friend. Ask a friend anyway. Is what you’re asking clear from your wording?
Also, avoid double negatives if you can; better to reword the question. You'll be helping: assorted neurodivergent learners, learners who are still getting their heads round new topics, learners who may be not be working in their first language…Bank!
If your platform allows, build a larger bank of questions from which the system draws a set number. That way, your learners won’t all get the same questions, and a learner can, if allowed, retake the quiz and have at least some new questions. That said…Remember Goldilocks and the Three Bears
She may have had questionable morals, but Goldilocks had a point about the whole “just right” thing. Consider the situation, and try to pitch the number of questions presented accordingly. Nobody wants to click into a quick knowledge check and be presented with 25-odd questions to plough through. Better to have a bank, present maybe 5, and make clear that if they repeat the quiz, they should get at least some new questions. That way, they have the choice. Equally, who wants to dig out the one correct answer from 8 or 9 subtly different options?! And are the 50/50 odds of a True/False really a test of knowledge, rather than luck?Know your platform #2
I know, I know. Blame it on my work doing platforms strategy.
We talked about knowing what your platform WILL do to your questions. But what CAN it do? Is there the option to toggle options such as randomisation on / off at quiz or question level? Can you add tags for subject or difficulty? Can you include graphics (if so, remember your ALT text!), code, read-aloud option, links etc? Can you vary the number of options? What question types are supported? If you know this before you start, your quiz will be a lot richer at the end.Enjoy it
If you find writing your questions dull, I’d pretty much guarantee your learners are going to find answering them even more so!
Have a bit of fun with writing scenarios and examples.
Get a bit creative.
Mix things up.
But remember 1 - 9 above.
I / we could have continued for much longer! But I have to remember point 8. So since 10 is a nice round number, I’ll hand over to you. What others would you add?
My thanks to Ann, Ben, Angela, and John for their valued contributions to this list.
If you liked it, why not look them up on LinkedIn via the links on their names? Here’s my LinkedIn profile too, while you’re there. Always happy to connect; mention you found me here.