Transcription
Hello, my name is Megan Owen, and I'm a food preparation and nutrition teacher, as well as a senior associate at AQA. This presentation intends to directly apply the general principles of assessment to our subject.
In this session, I will talk through five key areas of assessment, including the structure of a GCSE food paper, our assessment objectives, multiple choice questions, important topics to consider when constructing questions, and the creation of mark schemes. Hopefully, you'll have already seen our "What Makes Good Assessment" videos about the principles of assessment. These cover concepts such as reliability and validity, and what those terms mean. In this video, I'm going to talk about how those principles apply in food preparation and nutrition.
The question paper for GCSE food preparation and nutrition is divided into two sections: section A and section B. Section A is made up of 20 multiple choice questions. Each question presents students with four possible responses to choose from. Section B is made up of a selection of questions ranging from 2 to 12 marks, which require students to provide a written response.
Across both sections of the paper, students are tested on all five core topic areas from the specification: food, nutrition and health; food science; food safety; food choice; and food provenance. There is no set format to the question paper in terms of the order that topics are presented in. What's more, questions with different numbers of marks appear at different points throughout the paper. And when writing the paper, the positioning of questions is considered. For example, 8 and 12 mark questions are not positioned at the start or the end, and lower demand questions are presented first and last to help students get going and to allow them to finish positively.
There are four assessment objectives that are used to assess students within GCSE food preparation and nutrition. AO1 is: demonstrate knowledge and understanding of nutrition, food, cooking and preparation. AO2 is: apply knowledge and understanding of nutrition, food, cooking and preparation. AO3 is: plan, prepare, cook and present dishes, combining appropriate techniques. And AO4 is: analyze and evaluate different aspects of nutrition, food, cooking and preparation, including foods made by themselves and others.
AO1, AO2, and AO4 are all assessed through the question paper. 40 marks are allocated to both AO1 and AO2 questions, and 20 marks allocated to AO4 questions. AO3 is only assessed through the non-examination assessment task that students complete during year 11. AO3 is not assessed through the exam paper.
Questions for AO1 are typically the most straightforward questions on the question paper, requiring recall of knowledge from students. All of the 20 multiple choice questions are AO1, and some of the questions in section B are AO1 questions. AO1 questions tend to range from 1 to 4 marks. Here you can see examples of previous AO1 questions. As you can see, the two questions from section B use command words that prompt students to write clear points without any explanation. Alongside "give" and "identify," you might see typical command words such as "state" and "name," amongst others.
Questions for AO2 require students to demonstrate and apply knowledge and understanding through explanations. These questions typically range from 3 to 6 marks and can apply to any of the topic areas from the specification. However, they are particularly useful to assess students' ability to apply knowledge and understanding of food science processes, as seen in the first question here regarding the formation and function of gluten when making bread. AO2 questions are also used for questions linked to recipe problem-solving, which is a recurring feature on the question papers. This is because students are required to apply their knowledge and understanding to given scenarios.
AO4 is assessed through 8 and 12 mark questions, which involve higher order thinking skills. They are therefore likely to pose the most challenge to students. AO4 questions are split into two parts: analyze and evaluate. Students are expected to provide more detailed responses. We will explore these types of questions later on when we look at mark schemes.
We are now going to have a look at multiple choice questions, otherwise known as MCQs. Whilst MCQs may be the quickest for students to answer, writing them can be a difficult and lengthy process due to a range of considerations that need to be made, which we will discuss in due course. However, designed well, MCQs assess students on elements of all five topic areas in a short amount of time, validly testing recall and application of knowledge. They can also feel more accessible to students, which leads to fewer unanswered questions. In this section, we will explore the components of a multiple choice question and look at the range of considerations that must be made when writing them.
This is an example of a previous multiple choice question. The first part of the question is known as a stem. Stems can either be a question, or they can be written as a sentence that students must complete by choosing the correct answer. When writing stems, as in all questions, it is important that they are concise so that it is clear to the student what is being asked of them. Any unnecessary information should be removed, and the wording should remain minimal.
Underneath the stem, we have the possible answers. The correct answer is called the key, and the incorrect answers are known as distractors. In GCSE food preparation and nutrition, each multiple choice question will present four options, one of which will be the correct answer. The answer options must be carefully selected so that there is only one correct answer. This correct answer should be clear to students who know and understand the topic well, and the distractors should be plausible wrong answers. In order for the question to be a robust tool for assessment and differentiation, as seen here, distractors in GCSE food preparation and nutrition are written in alphabetical order. As a result of this, students cannot explore patterns with the responses they are choosing. There really are none.
As well as being presented in alphabetical order, it is also important that distractors do not stand out for other reasons. For example, if "stilt and cheese" had been written as a distractor rather than "cheese," this would have stood out as a two-word distractor amongst one-word distractors.
This slide shows another example of a multiple choice question. In this one, the stem is written as a question. The distractors are concise and written in alphabetical order. Capital letters are used for these distractors, as punctuation is used at the end of the stem. Typically, multiple choice questions will be worded positively and without the use of the word "not." However, use of the word "not" is appropriate here, as our students will have learned about "which one of the life stages does not follow the Eatwell Guide." The word "not" is written in bold to draw students' attention to it.
In this example, you can again see that a key word in the stem has been emboldened to make it stand out to students. This technique will sometimes be used to highlight a key part of the stem that students should be aware of. For example, with this question, 80°C could arguably be a correct answer, as food could be reheated to this temperature. However, 75°C is the minimum temperature that should be reached, as this is the temperature at which bacteria are killed. Just like worded distractors are listed in alphabetical order, numbers will always be written in numerical order.
This is an example of a poorly written multiple choice question. Study the question to identify the issues. How many issues can you spot? Pause the video here to make some notes.
Let's go through and identify the issues with this question. Firstly, the stem is quite lengthy. It is sometimes necessary for stems to run over onto two lines; however, in this one, there are definitely words that could be removed. When it comes to the distractors, there are numerous errors to identify. Most importantly, the distractors are poorly chosen, as there is more than one correct answer. Hypertension and coronary heart disease can both result from eating high levels of salt in the diet. This is unfair for students and would cause confusion, as in GCSE food preparation and nutrition, each multiple choice question only has one correct answer. It is also unfair, as many students may be unfamiliar with the term hypertension, as high blood pressure is referred to on the specification. In addition, hypertension may stand out as a correct answer, as it is the only one-word response. It is important that distractors are consistent in length. If correct answers stand out, this can be seen to be leading students, impacting the quality and validity of the assessment. Furthermore, the distractors are not listed in alphabetical order, and they have been capitalized despite forming the end of a sentence.
Pause the video again and have a good rewriting this question. This slide presents alternative wording for the question. How does it compare to what you wrote? Let's explore the changes in more detail. Firstly, the stem is far more concise to reduce reading load for students. This makes the question more accessible. Next, the distractors have been changed so that they are all three words or parts, but still all linked to diet-related health conditions that students should have covered as part of the GCSE specification. Most importantly, the distractors have been changed so that there is only one correct answer listed: high blood pressure, and the terminology now matches what is included on the specification. The distractors have also been presented in alphabetical order and are written in lowercase lettering to finish a sentence from the stem.
In summary, when writing MCQs, it is important to keep the stem concise. This is to reduce reading load for students and to reduce the risk of misinterpretation. Ensure there is only one correct answer, the key. The effectiveness, quality, and validity of MCQs in GCSE food preparation and nutrition is significantly impacted if more than one answer can be selected. Ensure suitable distractors are used. Distractors should be plausible responses to the stem rather than standing out as being incorrect. This ensures that students must demonstrate knowledge and understanding to answer multiple choice questions correctly. Avoid use of negatives where possible, for example, "which one does not." This is to reduce the risk of students misreading and misinterpreting a question. Consider the order that questions are presented in. If students are presented with a really challenging question to start them off, it may throw them. It is therefore important to ensure more challenging questions are dispersed throughout.
MCQs can be an effective assessment tool to utilize in lessons. For example, you could use MCQs at the end of a lesson for retrieval practice, recapping learning from the previous lesson. You could also interleave knowledge from previous topics to test recall. Use MCQs at the end of a lesson to test students' understanding from the lesson. This also enables misconceptions to be quickly identified, which can inform planning for future lessons. Set MCQs for homework tasks, utilizing online quizzing platforms to automate marking and provide students with immediate feedback. This enables more independent assessment and checking of knowledge. And finally, you could use MCQs as a way to introduce a new topic and assess prior knowledge. Utilizing MCQs as standard practice within lessons would also enable students to practice them on a regular basis. This will help them feel better prepared for completing section A when they sit their final exam.
We are now going to explore considerations that are made when writing other questions for the question paper. When writing questions for the question paper, it is really important that they are accessible to all students. To help with this, considerations are made about contexts that might be helpful. For example, some terms within the subject have dual meanings. In these instances, and to guide students with answering questions correctly, it may be appropriate to include an example of a response that could be given. This strategy has been used with the question on screen to avoid confusion with controls that students may have considered for NEA task one. When examples are provided, the mark scheme will stipulate that students must not be credited for repeating the example in their answer.
With other questions, it may be more appropriate to add context before the question is presented. Whilst this does not prevent students from misinterpreting questions, it does reduce the risk of them doing so. Have a look at this question. How might students have misinterpreted it without the given context? As molds can be equipment that are used to shape food, such as cheese, but also other dishes like jelly, without reference to microorganisms, students may have focused on the shaping of cheese rather than the addition of molds.
Given the nature of the question paper, a range of dishes will be named in section A and section B to provide students with context when answering questions. However, these must be carefully selected to ensure accessibility for all students. For example, this question identifies dishes that should be known to all students. Dishes may be named when linking to British and international cuisine, or for questions when students are required to analyze nutritional content. However, dishes are consistently used to enable students to apply knowledge and understanding of food science. With named dishes, prompting them to discuss key ingredients used, for example, when analyzing why problems may have occurred or offering solutions to explain how problems could be overcome. Named dishes can also help students to contextualize processes they are being asked to explain, as shown in the example. It is essential that dishes are carefully selected to ensure questions are accessible for all students. Note how, with this example, the context and the question are presented on different lines. This makes it easier for students to read and ensure focus is being left on the question being asked.
When dishes are named on the question paper, regional and international differences in terms of student locations are taken into consideration. For example, students outside of Wales may not be familiar with ingredients that are used to make Welsh cakes, so these would not be a suitable dish to reference on the question paper. Scones, on the other hand, are more likely to be widely known. Furthermore, reference to international cuisine and dishes needs to take into consideration that schools will have explored different cuisines, both through lessons and completion of NEA. It is therefore difficult to name specific dishes from other cuisines unless they are widely known.
Pause the video here to have a go at adding context to the three questions listed. How could you rephrase or extend the questions to make them more accessible to students?
For the question asking students to explain the process of dextrinization, students would be able to provide a clear explanation if they are able to link the term to a starch-based product, such as bread or pizza. Whilst dextrinization is also seen when baking a cake, this would not be a suitable dish to name, as students are likely to confuse dextrinization with caramelization, focusing on the browning of sugar rather than the browning of starch.
For the process of gelatinization, students may benefit from having a dish to talk about, such as custard or cheese sauce. Both of these sauces are referenced on the specification. Gelatinization is also seen in dishes like lemon curd and sweet and sour, however, as fewer students will be familiar with making these, they would not provide a suitable context.
For gluten formation, students will access the question better when linking it to either pasta making or bread making, both of which are included on the specification.
As well as considering context, it is also essential to consider the language used when constructing a question. The way in which questions are structured and worded is considered throughout the question paper production process to ensure that they can be read easily by students. This is to ensure that questions are accessible and to avoid misinterpretation. Throughout the question paper, efforts are also made to ensure consistency in terms of command words and phrasing. For example, for AO1 questions in section B, students may be asked to "identify," "name," or "state." For consistency, the same command words would be used where appropriate for similar questions to aid accessibility and understanding. Lastly, it is important that neutral language is used throughout the question paper. So, unless it's absolutely necessary, we avoid the use of names and genders in this specification.
To help ensure students are well tuned with reading questions correctly, it is advisable to make the answering and marking of exam questions a regular feature of your lessons. As part of this, activities can be implemented that ensure students take context on board when answering questions. For example, you could encourage students to annotate exam questions to identify key features, for example, by highlighting or underlining command words, keywords, etc. You could present questions to students with the context removed and encourage students to share how they would respond based on the information available. You could then share the context and encourage students to reflect on how they would have answered the question differently and how the context enables them to improve their response. This should encourage students to read questions carefully, recognizing that all information provided is purposeful.
Well-written question papers must be accompanied by high-quality and comprehensive mark schemes. Mark schemes are written at the same time as the question paper, with thorough research carried out during the process to ensure possible responses for each question are documented as much as possible. Changes can be made to the mark scheme ahead of publication of the final version based on the responses from students where appropriate. Detailed and accurate mark schemes aid the consistent marking of question papers by examiners, enabling the exam process to be fair, reliable, and valid. It is important that mark schemes are designed to accurately respond to each question on the question paper so that it's clear to examiners how marks should be awarded.
Some sections of the mark scheme are level-based, whereas others are points-based. We will look at examples of each now.
Each question on the mark scheme, regardless of whether they are points-based or levels of response marking, will have the following information: stem, alongside any context that was given to students, and marking guidance. As part of the marking guidance, the mark scheme will specify which assessment objective is being assessed by the question. It will also advise examiners how to award marks. For AO1 questions, like the points-based question that is visible on the slide now, it will typically say "award one mark per valid point." For AO2 questions, it may say "award one mark per point" or "two marks for each fully explained point." As part of the guidance, there may also be statements such as "do not accept repeated answers," which helps to make examiners more vigilant. The indicative content shares a range of possible responses that students may give for each question. Sometimes it will state "credit of valid responses." At the end of the indicative content section, this is to recognize that students may provide other responses that are accurate but have not been listed. This helps to ensure that students are awarded marks for demonstrating knowledge and understanding that falls outside of points recognized.
In addition to points-based marking, there is also levels of response marking. These mark schemes are used to guide marking alongside the indicative content. Levels of response are always used for AO4 questions to assess students' ability to analyze and evaluate. However, they can also support the marking of some AO2 questions to assess students' ability to apply knowledge and understanding. There are basic principles that apply when marking responses against levels of response. Reading through the whole response enables you to process what the student is saying. From this, you can then identify which mark band the response fits into. When a band is decided, a mark can be decided. Including questions which require marking by levels of response is important because this style of question assesses students' skills rather than just what they know. The levels of response are applied holistically because level descriptors combine a number of features. When writing levels of response questions, it is important that the levels on the mark schemes are clearly differentiated so they are clear and straightforward to use. It is also important that there are links to the indicative content.
For the example given, have a look at the descriptors for each mark band and identify the key differences. Then have a go at writing a response that would fall into the top mark band. What would you be looking for from students? Pause the video here to give the task a try.
This activity is something that you could do with students to assist exam preparation. When completing the question paper, students might not know if they are answering a question that will be assessed by levels of response. However, reviewing previous levels of response questions can aid discussion about how to develop responses to ensure points are clearly conveyed and explained, with relevant knowledge incorporated to demonstrate understanding. As part of this, you could also review students' responses from online resources or complete self and/or peer assessment of written responses by your students. There is also scope to write model answers live with your class.
More generally, it is advisable to encourage students to regularly interact with mark schemes when completing exam-style questions so that they learn how they will be assessed when sitting their final exam. Reviewing previous levels of response questions can aid discussion about how to develop responses to ensure points are clearly conveyed and explained, with relevant knowledge incorporated to demonstrate understanding. Some additional tasks you may choose to complete with your students are: review student responses from online resources so that students can apply mark schemes to written answers and review what students have done well and how responses could be improved; complete self and/or peer assessment of responses written in class so that students can apply mark schemes to their own work; write model answers live of your class so that they can clearly see how responses should be constructed to meet mark scheme criteria. More generally, it is advisable to encourage students to regularly interact with mark schemes when completing exam-style questions. For example, you could provide part-completed answers and ask students to improve them in line with the mark scheme; ask students to write model answers using the mark scheme; and finally, you could provide indicative content and encourage students to consider what the question may have been.
Thank you for watching. If you have any follow-up questions or feedback regarding this presentation, or anything else in relation to food, food preparation and nutrition, then please do let AQA know via one of the mediums on display.