Setting and Scoring Guidelines

We've updated the materials on the website. Our newest template for questions and scoresheets for team and pairs games can be found on the information page.

Scoring 

Some scorers may be unfamiliar with the latest score sheet format with diagonal lines. We've found that teams who use this score sheet make fewer mistakes, and it helps the results secretary collate the scores correctly. 

Here's an example completed sheet:

• Above the line is blank or 2 - did that person get their own question correct or not?
• Below the line is a tally of correct answers passed to that person.
• Questions correctly answered by nobody are tallied at the bottom.
• Mistakes are clearly crossed out and corrected.

Setting

One of the strengths of the AQL is the range and diversity of quizzes set, owing to our policy of every team setting quizzes.

Most quizzes we do are good, but it takes effort to make a quiz that isn't imbalanced or just too hard. Here are some recommendations for how to develop a great set of questions:

1. Check every answer - even if you absolutely 100% know something is correct!
2. Make sure each question is clearly expressed and has one unarguable answer. If there is more than one possible answer, it should be made explicit what is and is not acceptable - for example "name one of the three mushrooms which..." or "other than the patella, which bone in the human body..." or "which two politicians...".
3. Sometimes a question has a sensible answer which is the one you're hoping for, and another answer which is more obscure (or just not the one you were thinking of, but is technically correct). Make sure you allow either answer to be given (see point 1!).
4. Make explicit what you need for the answer. In general accept surnames alone, unless an answer is genuinely ambiguous without a first name.
5. Help the question reader with guidance. For example, state what can be prompted or not. For answers where there isn't a fixed wording say something like "accept any wording that demonstrates clear understanding".
6. Have a good variety of questions. Avoid too many questions on one topic. Even in the broadest area (geography, sport, etc), there is never a need for more than eight questions in one field. Don't have more than a couple of questions on a limited subject (e.g. Olympics, Yorkshire).
7. Don’t make your questions too obscure, even if you are an expert in that particular field. Always err on the side of accessibility.
8. Make a conscious effort to make your questions accessible to younger players. This is especially needed in the area of popular culture. TV programmes on now are accessible to all; questions on TV of the 1950s, 60s and 70s are often very frustrating for anyone aged 50 and below. Always ask: would a 40-year old have any chance of answering this?
9. Please check questions are balanced - both between teams and between players! If there are multiple setters, one person needs to collate the questions and take responsibility for making sure that the set is balanced. If more than one person is setting the quiz, it might be advisable to give different setters specific areas in which to set questions. 
10. Pairing questions - where questions are written in pairs of similar, associated themes - can be useful to give a fair range, however it isn't necessary and can be dangerous. If pairing is used, you must take great care to ensure that the answering of the first question doesn't help someone answer the later question; this includes incorrect guesses of the first question, or conversation between teams after it's been answered!
11. Avoid trick questions, questions where only three or four guesses are realistically possible and avoid attempts to be funny. 
12. Don't ever ask questions on collective nouns.

Balance 

Probably, the most frequent complaint involves quizzes being imbalanced. Someone needs actively to check that the quiz being submitted is balanced. If there are four film questions, does each side have two? Does one player’s eight questions cover eight diverse areas? If the answer to these questions is no, the questions need rejigging. Similarly, if you have consciously set some hard and some easier questions, are they balanced between the two sides? 

Balancing a quiz is necessarily time-consuming, but leads to fairer, more enjoyable quizzes.


Accessibility

By all means ask some demanding questions, but more than eight totally unanswered questions is too many.

Aim to regenerate your team and encourage younger people to play and set questions. If the League is to survive, younger people need to enjoy taking part.


Checking

And, finally, have you checked your quiz with team-mates or with other teams? If you do, areas of weakness are soon likely to be mentioned and made apparent! Many teams now play friendlies in the summer to test their questions out or ask someone from another team to have a look at them.