A serious complication has emerged in Leaving Certificate exams run by the Irish State each year.. An exam Superindtendent accidentally distributed the wrong paper in one exam centre earlier this week. He put out the exam questions for Paper 2 of the English examination, which wasn’t the subject being examined. The paper was, it seems, only on students’ desks for a few minutes before the error was noticed. However, in this age of twitter, bebo, myspace, facebook and such things, details of the exam questions were soon being discussed in school yards the length and breadth of the country. 

To make matters worse, the Superintendent in question failed to notify the Department of Education until more than 6 hours after the paper was leaked.

As with all things governmental, an investigation is underway. Denials of responsibility have issued from various entities involved.  The Superintendent in question has been dismissed. The exam is being rescheduled, causing disruption to study timetables across the land.

But an examination (no pun intended) of the facts reveals a telling IQ Trainwreck.

• The exam scripts that got muddled by the Superintendent were packaged in similar packaging. The similarity was such that the State Examinations Commision had issued warnings to the Superintendents that the exams were wrapped in similar packaging. (source: Irish Examiner, Friday June 5th 2009)

One of the factors that determines the quality of information is the quality of information presentation. Indtroducing ambiguity into visual information invites error. Tom Redman, in his book Data Driven, describes the presentation of information as a key step in how information is used and a key part of its complexity. Redman tells us that a number of disciplines need to come together to make even the simplest information and data useful, including:

Presenting data in ways that make it easy for customers to understand and use them. Only in this last step do data and information contribute to internal operations and decisions…

Packaging two sets of highly sensitive information in highly similar packaging which is similar enough that a warning is required makes it hard for customers (Redman uses “customers” to mean the actual consumers of the information – in this case the Superintendent) invites misunderstanding and error.

Yes, the Superintendent could have and should have double checked the paper was the right paper before handing it out, but a key contributing cause was the use of overly similar packaging for both exams.

• The Superintendent didn’t report a leak of sensitive information in a timely manner

All too often this happens in business. A laptop gets stolen, a memory stick gets mislaid, sensitive information gets left on a train. A key element of the response to this kind of problem is knowing that there is actually a problem, so early reporting to authorities of the leak is imperative. Had the State Examinations Commission had the information in a timely manner perhaps the cost of fixing the gaffe would be less.

• The cost of remedying the issue is now put at approximately EUR 1 Million

The solution that the Department of Education and State Examinations Commission has come up with is to run a totally new exam paper on Saturday. That means:

• Extra costs for transport for students to the exams (where State-funded school transport is used)
• Extra salary costs for Superintendents and their assistants
• Extra salary costs for school staff who are required to be on-site during exams.
• The costs of printing a whole new batch of exam papers.

And of course, it being a Saturday:

The SEC is finalising arrangements for a deferred sitting of English papers for a small number of students from the Jewish community after getting legal advice that asking them to sit an exam on their Sabbath, when their religion prohibits it and it is against their conscience, could have been unconstitutional. All other students will be expected to attend, in line with other timetabled exams.

For more on that particular complication, see the Irish Times’ detailed story.

So, why is this an IQTrainwreck?

1. The similarity in packaging on the exam papers was a key root cause. This is (or should be) a straightforward process of ensuring that all exam subjects and levels are distinctly colour coded and ensuring that packaging is not similar. Issuing a reminder is simply trying to inspect a defect out of the process. Yes, the Superintendent has to carry responsibility as well for not double checking but avoidable similarity should have been avoided (ergo preventing the confusion)
2. The lack of rigour regarding the reporting of the accidental distribution of the wrong paper is inexcusable. 
3. The cost impact of the error is extremely significant, particularly given the current state of Irish Government finances. EUR1 Million is a challenging amount to find in your budget at short notice.
4. The disruptive impact on students during a stressful time can’t be underestimated. 
5. The further complication presented by Ireland’s multi-culturalism adds further challenges (and potentially costs) for the SEC, the Department, and the students.

(On that note of multiculturalism, one is left wondering if the ISM school in Tripoli, Libya that offers the Irish Leaving Certificate to its students will have received their replacement exam papers yet of if they are even aware of the issue.)