The report investigated why the online system failed on several occasions between 30 October and 10 November. (file image) Photo: Supplied
An independent review says the Qualifications Authority was too relaxed about its relationship with the British company providing the online NCEA exam platform.
The review into last year's online exam failures said reporting requirements should have been more rigorous and there were few penalty clauses in the authority's contract with the company, RM.
The report investigated why the online system failed on several occasions between 30 October and 10 November with the exam platform "lagging" and log-ins stopping.
It compared the failures to a Swiss cheese - multiple holes in different parts of the system lined up at the same time.
It said there were problems with the Assessment Master online platform itself, inadequate pre-testing, and a defect that multiplied each log-in which placed significant strain on the database.
The report said the problems were the last straw for tired teachers and created fears of a repeat this year.
It said a repeat was unlikely and all involved were committed to improvement.
But it warned the Qualifications Authority must improve its internal operating model and its partnership with the exam vendor, British company RM.
It said the vendor did not test the compatibility of its exam platform with the security system which NZQA used to verify students' log-ins.
That was a problem because the security system multiplied every individual log-in, contributing to a situation where the platform could not cope.
"This was a defect introduced by the vendor, but not picked up in vendor testing," the report said.
There were also problems with the vendor's Assessment Master platform and its capacity, non-optimal NZQA vendor management and contract governance, and inadequate and misaligned pre-testing of the platform.
The report stressed the importance of avoiding a repeat.
"From a system perspective, any repetition would not only damage NZQA's digital assessment programme but tarnish the qualifications themselves and public confidence in them," it said.
'We need the detail'
The Secondary Principals Council was calling for urgent action to restore confidence in the exams.
Chairperson Kate Gainsford said last year's events were very stressful for everyone involved, especially the students.
"It is reasonable to expect fast lessons will have been learned and steps taken to make sure the technology and related systems work without issue this year. Whatever it takes to restore our trust in the system, it must be done."
Gainsford told Checkpoint some of the problematic areas that contributed to the exam "fiasco" were based on budgetary considerations NZQA had around the procurement of staff.
"If that was a problem beforehand, you would anticipate that needing to be remedied with a bit of money now to make sure it doesn't happen again."
NZQA has just proposed cutting 35 staff.
Gainsford said there was a lack of clarity around the cuts but even less so around how or who would be doing the work so there was a quality product when it came to exams.
"There are very high expectations around what we should be able to be offering and putting in front of our young people who have worked all year to be ready for their assessments and we've heard from NZQA in particular that quality remains a top priority but that doesn't mean that the service that's being promised is able to be clearly followed through in terms of an excellent experience."
Her message to the education minister?
"We really need greater clarity on the plans to make sure negative impact on young people and schools if there's cuts across ministry and NZQA - we need the detail."
