Universities had to make radical and rapid changes to assessment this spring. We take a look at what they did, the challenges they faced and what it all means for the long-awaited move to more authentic and adaptable assessment.
“I’m hoping that the current situation will have opened the Pandora’s box in that more academics will see the advantages of using digital technology for assessment and use this as a stepping stone or as a building block for exploring what the possibilities are going forward,”
says Mariann Rand-Weaver, vice-provost (education), Brunel University.
The assessment revolution has been a long time coming. There have been many small-scale trials of digital assessment. There has been acknowledgement of its benefits, from greater effectiveness and fairness for students and teachers to efficiency and cost savings for universities.
There have been pockets of digital best practice. But when lockdown hit the UK in March, what was the predominant mode of summative assessment across the university sector? Pen and paper, in person. Suddenly, universities had weeks, not years, to innovate.
It was a stark challenge: how do you transform those long-established processes at speed and at scale? Hundreds of thousands of students, who were about to sit in ranks in exam halls to decide the outcome of three or more years’ work, faced an uncertain future. Within universities, academic and professional staff raced to work out what was possible, desirable and fair.
The result, inevitably, was a wide variety of approaches. Some universities managed to accelerate or even turbocharge planned strategies while others put them on hold.
Trailblazing
At one extreme, the University of London (UoL) took 40,000 students sitting around 500 exams in 160 countries from face-to-face, solely pen and paper to digital testing in one move, which included digitally proctored exams. Craig O’Callaghan, UoL worldwide director of operations and deputy chief executive, said at the time:
“What we were expecting to do maybe two or three years down the road we’re going to try and do this summer. We’re making an enormous step change in our assessment piece at this moment. In the UK, I don’t think anyone else is trying to do this.”
UoL rapidly produced three assessment routes using the platforms Moodle and Turnitin for most exams and digitally-proctored exams for about 10,000 students using Janison and CoSector. It also extended the timetable to enable large-scale testing and practice for students.
Fast progress
Brunel University started in a stronger position than many: about 20% of students were already experienced in bring-your-own-device exams and it was in its third year of rolling out the platform WISEflow for digital exams and all course work assignments, supported by a well-established laptop loan scheme.
It moved to open-book, take-home exams, without locked-down devices or remote proctoring, mostly sat in a strictly exam-length timeframe (ie a two-hour exam or three-hour exam) with questions revised where necessary for suitability and some types of assessment changed to include longer pieces of work.
“I would be surprised if we don’t find that we’ve got to a place where we wanted to get to much quicker as a result,”
says Mariann Rand-Weaver, vice-provost (education).
Radical thinking
The Open University (OU) took a different route – a light-touch approach to summative assessment.
Of more than 300,000 assignments scheduled between April and June, only those that were essential, due to regulatory reasons or the weighting of the course towards a final assessment, were to go ahead. The rest would be cancelled and grades assigned based on continuous TMA results, though students could request postponement.
The switch was made easier because the OU had generally moved away from final exams in recent years, with fewer than a third of modules requiring them.
A key premise of the approach was to reduce student stress at a time of already heightened anxiety. In addition, the OU is thinking radically about assessment more generally. As Klaus-Dieter Rossade, director of assessment programme, says,
“The question might be, do we still need an exam going forward? Some may be required by regulators but the argument to really question whether you need one will be ever greater.”
Challenges and trade-offs
Scaling up rapidly is complex. Universities’ quick fixes not only put theory very rapidly into practice, sometimes for the first time, but exposed a range of challenges and trade-offs.
Perhaps the most common, and most pressing, is balancing the security of locked-down and digitally proctored exams against the ability of students at home to access the necessary technology and connectivity – and sometimes just a quiet space for a long-enough time. This security/equity trade-off lies at the heart of the challenge to make assessment both more trustworthy and more adaptable.
There was also a need for balance between mitigating student stress and meeting the demands for rigour, both from students themselves and to achieve the recognition of professional, statutory and regulatory bodies that is critical to the career paths of many students.
And there were the necessary skills. In the Jisc digital experience insights survey 2019, only 34% of HE teaching staff said they were offered regular opportunities to develop their digital skills and only 13% were given time and support to innovate. This has clear implications for staff readiness to embrace new tools.
The pace demanded by lockdown compelled many universities to stay with what they knew rather than risk introducing unfamiliar platforms or tools at a time of uncertainty and anxiety.
What more might be possible?
We are now moving into a very short time for reflection. It’s likely that the present challenges will continue into 2020-21, with uncertainties about how the year is to look, concerns about future pandemic waves and the possibility that a return to ranks in exam halls may not be practical in 2021, whether or not it’s desirable.
So what more might be possible for assessment if the challenges of spring 2020 could be resolved?
A little more than a month before lockdown, Jisc had started this conversation with our report, the future of assessment, five pinciples, five targets for 2025. This suggests that, by 2025, digital technology will enable assessment that meets five key goals: more authentic, more accessible, appropriately automated, more continuous and more secure. It laid out five targets for the development of digital assessment, to be achieved in five years.
The key is that assessment in the future has to be relevant for the context of that future, not of the past.
Technology can now help assess an individual’s ability, for example, not just to retain knowledge but to apply it practically, to demonstrate the ability to acquire, weigh and use evolving knowledge and skills, and to work in a team, solve complex problems, critique, innovate, challenge assertions or collaborate at distance.
This answers the complex, many-hued needs of employers as well as providing more rounded and transferable qualifications for the students themselves.
“As a sector we need institutions to work together to innovate, to collaborate with both technology and software providers, and employers.
Making sure that we are able to share good practice and have enough training opportunities and support for staff, so that we take them with us on this journey.”
Mariann Rand-Weaver, vice-provost (education), Brunel
The next steps
Five years now feels like a luxury, not a stretch. The 2021 assessment calendar is near and the brief window of opportunity is open now.
The coming weeks and months will decide whether 2021 will be a near re-run of 2020 – or the start of a journey that goes beyond creating digital versions of analogue exams to provide new forms of assessment that so far have only been talked about.
If universities and platform providers are to build systems that move in this direction, the present period needs to generate crucial questions: how are those needs being evaluated, how should assessment evolve and what collaboration between universities, and with technology providers, is needed?
Find out more
Read more about these case studies and others in our report assessment rebooted, part of the series from fixes to foresight: Jisc and Emerge Education insights for universities and startups.
Join us for learning and teaching reimagined, where together we’ll shape what learning and teaching looks like in readiness for 2021-22 – and a roadmap to get there.
No comments:
Post a Comment