From today’s Irish Times comes a story which shows the real significance and impact of a common Information/Data Quality problem, transposition of letters or numbers.
A Swedish couple holidaying in Italy were looking forward to their visit to the lovely sunny island of Capri.
Unfortunately a “finger flub” on their GPS put them 650 kilometres north and inland of their intended destination in the lovely Italian industrial town of Carpi.
Oh dear.
A serious complication has emerged in Leaving Certificate exams run by the Irish State each year.. An exam Superindtendent accidentally distributed the wrong paper in one exam centre earlier this week. He put out the exam questions for Paper 2 of the English examination, which wasn’t the subject being examined. The paper was, it seems, only on students’ desks for a few minutes before the error was noticed. However, in this age of twitter, bebo, myspace, facebook and such things, details of the exam questions were soon being discussed in school yards the length and breadth of the country.
To make matters worse, the Superintendent in question failed to notify the Department of Education until more than 6 hours after the paper was leaked.
As with all things governmental, an investigation is underway. Denials of responsibility have issued from various entities involved. The Superintendent in question has been dismissed. The exam is being rescheduled, causing disruption to study timetables across the land.
But an examination (no pun intended) of the facts reveals a telling IQ Trainwreck.
One of the factors that determines the quality of information is the quality of information presentation. Indtroducing ambiguity into visual information invites error. Tom Redman, in his book Data Driven, describes the presentation of information as a key step in how information is used and a key part of its complexity. Redman tells us that a number of disciplines need to come together to make even the simplest information and data useful, including:
Presenting data in ways that make it easy for customers to understand and use them. Only in this last step do data and information contribute to internal operations and decisions…
Packaging two sets of highly sensitive information in highly similar packaging which is similar enough that a warning is required makes it hard for customers (Redman uses “customers” to mean the actual consumers of the information – in this case the Superintendent) invites misunderstanding and error.
Yes, the Superintendent could have and should have double checked the paper was the right paper before handing it out, but a key contributing cause was the use of overly similar packaging for both exams.
All too often this happens in business. A laptop gets stolen, a memory stick gets mislaid, sensitive information gets left on a train. A key element of the response to this kind of problem is knowing that there is actually a problem, so early reporting to authorities of the leak is imperative. Had the State Examinations Commission had the information in a timely manner perhaps the cost of fixing the gaffe would be less.
The solution that the Department of Education and State Examinations Commission has come up with is to run a totally new exam paper on Saturday. That means:
And of course, it being a Saturday:
The SEC is finalising arrangements for a deferred sitting of English papers for a small number of students from the Jewish community after getting legal advice that asking them to sit an exam on their Sabbath, when their religion prohibits it and it is against their conscience, could have been unconstitutional. All other students will be expected to attend, in line with other timetabled exams.
For more on that particular complication, see the Irish Times’ detailed story.
So, why is this an IQTrainwreck?
(On that note of multiculturalism, one is left wondering if the ISM school in Tripoli, Libya that offers the Irish Leaving Certificate to its students will have received their replacement exam papers yet of if they are even aware of the issue.)
Information Quality professionals (and indeed quality management professionals in general) often recite a mantra that good quality begins at the beginning of a process, that it must be designed in, and that defects need to be fixed as close to the start of the information chain as possible.
A post today on DataQualityPro.com from Dylan Jones highlights the significant truth that lies behind all these statements.
Dylan’s son was born in April. The first thing the State did for him was to slap an identifier on him. The second thing they did (to summarise Dylan’s excellent and forensic post) was to make a mess of linking the local hospital ID to a National patient record.
That error propogated and resulted in the parents of another child over 90 miles away getting an appointment for a medical checkup relating to Dylan’s son. It seems that the efforts made to correct the error Dylan spotted when his son was born haven’t propogated half as fast as the original error.
And that’s the problem. How many other processes and silo’d systems has this error propogated into? How many more times in Dylan’s son’s life will be be confused with another child 90 miles away? What other ‘life-events’ will this error impact? In future, how will he find himself trapped by his number?
Ultimately, Dylan’s son is not a number, he’s a human being.
We recently posted a long Trainwreck on the problems with Google Health due to poor quality information. It is possible that an error like the one affecting Dylan’s son could result in incorrect patient data about Dylan’s son being transferred to this type of electronic patient record. Who would be responsible for the impacts if that information was acted on in haste without Dylan (or Mrs Jones) being there to point out that the information was wrong?
Given that in an Irish hospital in 2003 the medical staff failed to act on an error in an expectant mother’s chart and delivered a baby 39 days prematurely, despite the parent’s insistence that there were errors in the chart, highlights that simple errors in medical records can have significant impacts. That the baby in question died further highlights that these impacts can be catastrophic, which means that the standard of care for quality in medical records needs to be high.
Dylan’s investigations also uncovered some other weaknesses/flaws in patient data quality which are unsettling. We’d suggest you take a look at Dylan’s post for more details on those.
For actual inconvenience and annoyance to Dylan’s family, and for the potential for catastrophic loss or injury, this counts as a definite IQTrainwreck.
Following on from this morning’s story about the New Zealand overdraft fiasco, a few further cases of Information Quality trainwrecks in Financial services have come to our attention.
This first one is from 2001 and was found on the BBC.co.uk website, with further reporting from The Telegraph
Back in 2001, David Hickey was emigrating from Ireland to Spain. He asked his bank to change IR£1500 into pesetas, but an error in the bank meant that the amount transferred was in euros, not pesetas. IR£1500 was approximately 300,000 pesetas. Mr Hickey received EUR300,000 into his account.
The bank eventually had to take legal action in Spain to freeze Mr Hickey’s accounts with a view to getting the money back.
At the time, the bank declined to comment further to the media on the matter and the Irish police were of the view that no criminal offence had taken place because of the ‘technical error’ (i.e. IQTrainwreck) involved.
We’re researching the other IQTrainwrecks that came to light this morning on this theme, not least to make sure we haven’t covered them here already. Expect further updates in the coming days.
From our ever vigilant sources over at The Register comes this story of duplicated information resulting in confusion and costs to the UK Taxpayer.
It seems that the UK DVLA has issued duplicate tax discs to concientous motorists who renewed their motor tax on-line.
A DVLA spokesman told theRegister: “As a result of an error, a number of customers, who recently purchased tax discs on line or by phone, were issued with duplicate tax discs.
“Once the problem was identified, swift action was taken to rectify it. All customers affected are being sent a letter of apology and the erroneous discs have been cancelled.”
So. Let’s sum this one up…
This is a simple example of the costs to organizations of poor quality information. A classic IQTrainwreck scenario.
Courtesy of #dataquality twitterers Steve Tuck and Stephen Bonner comes this story from the BBC about a school in Cheshsire whose parents received a truancy notice about their daughter which threatened to ban her from her end of year prom for being over 30% below the target attendance rate for students.
The young girl, Megan, had possibly the best excuse ever for playing hookey from school however. According to her mother:
“Megan doesn’t go to that school any more. She’s been dead for two months now so it’s not surprising her attendance is low.”
It appears that inconsistencies between two computer systems in the school resulted in the school’s left hand not knowing what the right hand was doing with regard to student information.
Megan’s name had been taken off the school roll when she died, and removed from the main school database,” the spokeswoman said.
“However, unknown to the school, her details had remained in a different part of the computer system and were called up when the school did a mail merge letter to the parents of all Year 11 students about their prom”.
Reading the comments from the software providers in the BBC story, it would also appear that the software lacks a “dead student” flag to enable them to exclude deceased students from administrative mailings.
This is a classic IQTrainwreck because it resulted in distress and upset to Megan’s parents, landed on the BBC News website (with video no less) , has been flashed across Twitter, and has now wound up here.
Also, this failure of the computer systems to allow the left hand of the school (the student register systems) to know what the right hand (the Capita system) was doing is not dissimilar to the circumstances of the recent court case of Ferguson v British Gas where the defences put forward by British Gas that erroneous debt collection letters were ‘computer generated’ and so they couldn’t have been harassing the plaintiff were dismissed by the Court of Appeal in England and Wales.
So we can add a potential legal risk to the list of reasons why this is an IQTrainwreck.
Yesterday’s Irish Times carried a story of how, after some extensive detective work, the Irish Police finally tracked down the most dangerous driver in Ireland, a Polish gentleman by the name of Prawo Jazdy.
This individual was given hundreds of speeding tickets and parking tickets over the past few years, and by June 2007 he had over 50 separate entries in the Irish Police computer system. So slippery was this offender that he kept giving police a different address almost every time he was stopped and ticketed.
He was finally brought to book by a quick thinking officer in the Traffic Corps of the Irish Police who tracked him down using a structured Master Data source. Surely this is an example of Information and Intelligence lead policing at its best? Continue reading
The BBC today (2008-10-23) carries a news story about the under-reporting of serious crime due to the mis-classification of certain crimes. The story is examined in more detail in an op-ed piece “How the police missed the violence”..
The problem has come about when the intended victim manages to escape serious harm when attacked. The example given concerns a pub fight in which the assailant attempts to “glass” the victim but she suffers only minor injuries not the intended major damage to her face. The police then classify the crime not as “grievous bodily harm with intent to cause serious injury.” but as merely(?) “grievous bodily harm”.
14 forces have gone back through their statistics and reclassified GBH’s as the more serious crime where intent was clearly present but no great injury was suffered. This has resulted in a 22% jump in reported violent crime over the same period last year although it has not affected the total number of offences reported.
The IQ lesson here concerns the way in which business terms are defined and then applied. The key word in the definition is “intent” but many officers on the ground concentrate on the phrase “to cause serious injury”.Mind you as an ordinary man in the street I think I would classify GBH as a violent crime but officially in falls into the “Other personal crime (with Injury)” category which is not seemed a serious category.
Sat-Nav, it’s everywhere. When my mother-in-law starts asking me about it and my wife starts to swear by it (rather than at me) on long journeys I know that the technology has reached mass market acceptance.
Spare a thought then for hauliers, tour buses and other large vehicles trying to navigate the Ring of Kerry using their trusty Sat Navs. As reported by the Irish Examiner newspaper today, it seems that satnav reliant drivers are going the wrong way around the Ring of Kerry route and are getting stuck in the many narrow tunnels and mountain passes that dot the route.
“Large vehicles should go around the spectacular road anti-clockwise, starting from Killarney and heading in towards Killorglin. However, Killarney FF Cllr Tom Doherty said numerous vehicles using GPS this summer went against the traffic flow and got stuck. ” (from Irish Examiner).
The root cause here would seem to be manifold, but ultimately boil down to the accuracy of information and how that information is presented.
The traffic flow arrangement on the Ring of Kerry has no legal standing at present, despite having been agreed a number of years ago. Therefore, satnav manufacturers may not have factored it into their route maps and underlying rules because they didn’t know about it (completeness of information). However, it also appears that the signage around the Ring of Kerry isn’t sufficiently clear to alert bus and truck drivers that there is a non-standard traffic flow in place (quality of information presentation).
The Irish Examiner reports that Killarney Town Council is to write to Satnav manufacturers to alert them to the issue.
Of course, Co. Kerry isn’t the only place where over-reliance on satnav has landed people in difficulty…
A Satnav system directs drivers to a bridge that doesn’t exist (accuracy of information)
Who’d have thought that two places might have the same name? (accuracy of information)
Conclusion
Satnav is a useful tool, but there are issues with the quality of information provided by satnavs. Of course, there are also problems with some of the knowledge workers (drivers) who blindly follow their satnav directions without applying common sense to ensure that the size of the road is consistent with the size of their vehicle. Also, satnav errors are often compounded by poor quality of signage and other information that might help drivers make better decisions (like not driving into the deep river ford)
First off – an apology for not posting a bit more regularly. It’s not that there aren’t any IQ Trainwrecks, it’s just that there have been so many recently we’ve been spoiled for choice for the ones to use and we haven’t had time to edit and compile them all.
However, one that jumped out of the headlines this morning is the news that Her Majesty’s Revenue and Customs in the UK has shelved it’s home sales statistics report until September , “adding to concerns over the quality of official data which help inform interest rate policy at a time when the economy is teetering on the edge of recession” (according the the Irish Times).
Just what you need in a time of economic crisis – inaccurate and unreliable information to support planning or even measure how good or bad things are. According to the report:
City economists and even the Bank of England have been questioning the reliability of several official date series including trade, growth and retail sales.
Apparently, inconsistencies were found between statistics to published this month (August) and last month. According to the HMRC spokesperson quoted in the article:
“All months in the statistical series are affected, with the differences showing falls in some months and increases in others. We are working to understand the reason for these differences so that a reliable set of statistics can be produced.”
Interestingly, from an Information Quality perspective, one of the main groups that ‘consume’ this information don’t seem to be surprised by the existence of errors but rather by the fact that the report has been withdrawn.
The article highlights a possible root cause of error:
“The ONS, which is also replacing its main measure of wages because it was found it did not capture the true picture, relocated from London to Newport in Wales this year in a move to cut down on costs – leading to a large number of staff changes “
Why is this an IQ trainwreck?Absence of timely and reliable information about economic performance affects the ability of the government to plan and manage economic policy to manage the impacts of an economic down turn and avoid more serious difficulties for individuals.