Concern at digital exams

Terry Crooks
Terry Crooks
Serious concerns have been raised about the New Zealand Qualifications Authority's plans to scrap paper-based exams and replace them with digital exams that can be sat ''online, anytime''.

NZQA announced earlier this month that it aimed to put all subject examinations - minus a few that are incompatible, such as art - online by 2020.

Chief executive Karen Poutasi said plans were advanced, and about 100 schools would this year complete a maths assessment online.

Paper assessment would also carried out to provide back-up.

She said pupils could do digital assessment twice a year, three times a year, or ''online, anytime''.

However, former University of Otago educational assessment research unit co-director Emeritus Prof Terry Crooks has raised major concerns about the initiative.

He said there was already substantial use of digital assessment within schools for internally assessed standards, and no reason why it should not increase in the future.

''But I am much less sanguine about the goal of moving most external assessments from paper-based to digital, and especially about the vision of these assessments occurring anywhere, anytime.''

New Zealand had a long history of national secondary qualifications being awarded through national exam processes, under tightly controlled conditions, he said.

''If assessment of many of the externally assessed NCEA standards were to become less controlled and less consistent because of the push for digital assessment, school and community trust in the NCEA qualifications would be seriously at risk.''

At present, each exam is held nationally at the same time, in rooms with specially employed managers and supervisors.

Exams are marked by a national panel of examiners, who are monitored by fellow examiners and NZQA personnel, so inconsistencies can be identified and corrected, and marking schedules can be revised if unexpected difficulties arise with particular exam questions or marking criteria, he said.

''It is unlikely to be possible to use similar procedures for external examinations conducted digitally at multiple times during the year.''

Another issue was many schools would not have enough computers or digital devices available in one or more supervised exam rooms.

''Those devices would need to be sufficiently similar in capability, and in the restrictions on what programmes and information could be accessed during the examination, that national fairness in the assessment conditions could be assured.

''Even if they were achieved, copying all of the resulting electronic files of student work and transferring those copies reliably to NZQA for a national marking process would pose major challenges.''

If it was not possible to conduct an external exam digitally on the same day and time slot throughout the country, or have the resulting computer files marked by a supervised national panel, then the assessments would have to be more under the control of schools, Prof Crooks said.

''In essence, these changes would remove some key characteristics of external assessment, and risk, making the assessment largely indistinguishable from internal assessment.''

Prof Crooks was also concerned that if external standards could be assessed at more than one time during the year, it would not be appropriate to use the same assessment questions or tasks for the multiple occasions.

''Otherwise, students taking the later assessments could learn details of their tasks in advance.

''Thus, more than one examination version would be required, each with its own marking guide.''

Ensuring consistency in difficulty would be a huge technical challenge, he said. If digital assessment occurred on multiple occasions during the year, it was unlikely the marking could be handled nationally, and even more unlikely that it could be done by a single marking panel.

''There would be substantially greater risk of inconsistency in the standards applied to pupils' work.''

Prof Crooks also noted few end-of-course exams or major tests in tertiary education could be taken digitally. Hand-written responses remained the norm, and there was no sign of that changing, he said.

john.lewis@odt.co.nz

Add a Comment

 

Advertisement