Centre Judgements: Teaching Staff Interviews, Summer 2020

Steve Holmes, Darren Churchward, Emma Howard, Ellie Keys, Fiona Leahy, Diana Tonin and Beth Black examine how teaching staff produced centre assessment grades and student rank orders after England’s summer 2020 exams were cancelled. Their 54 interviews capture decisions before results day and the grading reversal, rather than retrospective explanations prompted by those outcomes.^[1]

What the interviews can establish

Ofqual researchers conducted semi-structured online interviews between July 28 and August 12, 2020. They lasted roughly 40 minutes to two hours, averaging about an hour. Participants came from an earlier survey’s volunteers. The researchers sought a cross-section of roles, institutions and qualifications, then invited additional people when response rates were low. They could not fully control the final sample. Five researchers used an agreed thematic coding scheme, refined through initial coding and shared-transcript consistency checks (§2.2).^[1]

The findings explain experiences and recurring ways of reasoning, rather than the national prevalence or accuracy of each practice. The interviews cover general qualifications and vocational or technical qualifications, whose evidence and assessment requirements differed. The report’s discussion notes that most interviews concerned general qualifications.^[1]

From evidence to a grade

Some centres designed a common process centrally; others delegated more to departments. Centrally designed approaches ranged from spreadsheets combining marks into provisional grades to guidance specifying how staff should exercise judgment. Data-driven procedures could still involve adjustments, and judgment-led procedures still used evidence. These were different combinations of evidence and discretion, rather than a simple division between calculation and teacher knowledge (§§3.1–3.2).^[1]

For examined qualifications, mock exams were particularly important, alongside class tests, coursework and existing predicted grades. Staff also considered students’ trajectories and circumstances. A pupil expected to improve through late revision was difficult to assess when the remaining work had never happened. Vocational qualifications often supplied more completed or partly completed assessed work, making the question less dependent on predicting a final exam (§7.4).^[1]

Ranking added another problem. Some processes ranked students from data before assigning grades; others assigned grades first and then ranked within them. Staff described particular scrutiny near grade boundaries because they expected standardization to move those students. Comparing students taught in different GCSE tiers was difficult because their mock papers differed; one department used questions common to both papers to help compare them (§3.2.6). This is a reported local practice, not a tested universal ranking method.^[1]

Anticipating the next decision

Historical results sometimes contextualized the proposed grades and sometimes acted as a firm ceiling. Some centres matched previous outcomes closely to avoid external standardization; others allowed an increase or expected the later model to adjust it. Teachers could disagree with managers who reduced grades to fit a historical distribution. The report therefore finds variation in the generosity of submitted judgments as well as in the process used to reach them (§§3.3, 7.3).^[1]

Interviewees identified a specific tension: historical exam results include people who underperform on the day, but teachers cannot reliably predict which otherwise capable students would do so. Giving several borderline students the benefit of the doubt can be reasonable case by case while producing a different overall distribution. Participants’ confidence that their judgments were fair is evidence of their views, rather than an independent validation that bias had been eliminated.^[1]

Process evidence and outcome evidence

Published May 17, 2021, this study complements the later statistical comparison. The interviews describe how judgments were produced and how anticipated standardization influenced them. The discrepancy study measures differences between outputs. Neither observes the cancelled examinations that would resolve every individual prediction. Together they show why a grading system’s inputs, institutional incentives and final selection rule all need examination.^[1]