Author Archives: Daragh O Brien

About Daragh O Brien

Daragh O Brien is the Managing Director of Castlebridge Associates. This site has been one of his side projects for a decade. It needs some love and attention...

No child left behind (except for those that are)

Leave a reply

Steve Sarsfield shares with us this classic tale of IQ Trainwreck-ry from Atlanta Georgia.

An analysis of student enrollment and transfer data carried out by the Atlanta Journal-Constitution reveals a shocking number of students who appear to be dropping out of school and off the radar in Georgia.  This suggests that the dropout rate may be higher and the graduation rate lower than previously reported.

Last year, school staff marked more than 25,000 students as transferring to other Georgia public schools, but no school reported them as transferring in, the AJC’s analysis of enrollment data shows.

Analysis carried out by the State agency responsible was able to track down some of the missing students. But poor quality information makes any further tracking problematic if not impossible.

That search located 7,100 of the missing transfers in Georgia schools, state education spokesman Dana Tofig wrote in an e-mailed statement. The state does not know where an additional 19,500 went, but believes other coding errors occurred, he wrote. Some are dropouts but others are not, he said.

In a comment which should warm the hearts of Information Quality professionals everywhere, Cathy Henson, a Georgia State education law professor and former state board of education chairwoman says:

“Garbage in, garbage out.  We’re never going to solve our problems unless we have good data to drive our decisions.”

She might be interested in reading more on just that topic in Tom Redman’s book “Data Driven”.

Drop out rates consitute a significant IQ Trainwreck because:

  • Children who should be helped to better education aren’t. (They get left behind)
  • Schools are measured against Federal Standards, including drop out rates, which can affect funding
  • Political and business leaders often rely on these statistics for decision making, publicity,  and campaigning.
  • Companies consider the drop out rate when planning to locate in Georgia or elsewhere as it is an indicator of future skills pools in the area.

The article quotes Bob Wise on the implications of trying to fudge the data that sums up the impact of masking drop outs by miscoding (by accident or design):

“Entering rosy data won’t get you a bed of roses,” Wise said. “In a state like Georgia that is increasingly technologically oriented, it will get you a group of people that won’t be able to function meaningfully in the workforce.”

The article goes on to highlight yet more knockon impacts from the crummy data and poor quality information that the study showed:

  • Federal standard formulae for calculation of dropouts won’t give an accurate figure if there is mis-coding of students as “transfers” from one school to another.
  • A much touted unique student identifier has been found to be less than unique, with students often being given a new identifier in their new school
  • Inconsistencies exist in other data, for example students who were reported “removed for non-attendance” but had  zero absent days recorded against them.

Given the impact on students, the implications for school rankings and funding, the costs of correcting errors, and the scale and extent of problems uncovered, this counts as a classic IQTrainwreck.

The terror of the Terrorist Watch list

Leave a reply

A source who wishes to remain anoynymous sent us this link to a story on Wired.com about the state of the US Government’s Terrorist watch list.

The many and varied problems with the watch list have been covered on this blog before.

However, the reason that this most recent story constitutes an IQTrainwreck is that it seems that, despite undertakings to improve quality, the exact opposite has actually happened given:

  • The growth in the number of entries on the list
  • The failures on the part of the FBI to properly maintain and update information in a timely manner.

According to the report 15% of active terrorism suspects under investigation were not added to the Watch list. 72% of people cleared in closed investigations were not removed.

The report from the US Inspector General said that they “believe that the FBI’s failure to consistently nominate subjects of international and domestic terrorism investigations to the terrorist watchlist could pose a risk to national security.”

That quote sums up why this is an IQTrainwreck.

Continue reading →

Irish State Exam leak being studied.

Leave a reply

A serious complication has emerged in Leaving Certificate exams run by the Irish State each year.. An exam Superindtendent accidentally distributed the wrong paper in one exam centre earlier this week. He put out the exam questions for Paper 2 of the English examination, which wasn’t the subject being examined. The paper was, it seems, only on students’ desks for a few minutes before the error was noticed. However, in this age of twitter, bebo, myspace, facebook and such things, details of the exam questions were soon being discussed in school yards the length and breadth of the country. 

To make matters worse, the Superintendent in question failed to notify the Department of Education until more than 6 hours after the paper was leaked.

As with all things governmental, an investigation is underway. Denials of responsibility have issued from various entities involved.  The Superintendent in question has been dismissed. The exam is being rescheduled, causing disruption to study timetables across the land.

But an examination (no pun intended) of the facts reveals a telling IQ Trainwreck.

One of the factors that determines the quality of information is the quality of information presentation. Indtroducing ambiguity into visual information invites error. Tom Redman, in his book Data Driven, describes the presentation of information as a key step in how information is used and a key part of its complexity. Redman tells us that a number of disciplines need to come together to make even the simplest information and data useful, including:

Presenting data in ways that make it easy for customers to understand and use them. Only in this last step do data and information contribute to internal operations and decisions…

Packaging two sets of highly sensitive information in highly similar packaging which is similar enough that a warning is required makes it hard for customers (Redman uses “customers” to mean the actual consumers of the information – in this case the Superintendent) invites misunderstanding and error.

Yes, the Superintendent could have and should have double checked the paper was the right paper before handing it out, but a key contributing cause was the use of overly similar packaging for both exams.

  • The Superintendent didn’t report a leak of sensitive information in a timely manner

All too often this happens in business. A laptop gets stolen, a memory stick gets mislaid, sensitive information gets left on a train. A key element of the response to this kind of problem is knowing that there is actually a problem, so early reporting to authorities of the leak is imperative. Had the State Examinations Commission had the information in a timely manner perhaps the cost of fixing the gaffe would be less.

  • The cost of remedying the issue is now put at approximately EUR 1 Million

The solution that the Department of Education and State Examinations Commission has come up with is to run a totally new exam paper on Saturday. That means:

  • Extra costs for transport for students to the exams (where State-funded school transport is used)
  • Extra salary costs for Superintendents and their assistants
  • Extra salary costs for school staff who are required to be on-site during exams.
  • The costs of printing a whole new batch of exam papers.

And of course, it being a Saturday:

The SEC is finalising arrangements for a deferred sitting of English papers for a small number of students from the Jewish community after getting legal advice that asking them to sit an exam on their Sabbath, when their religion prohibits it and it is against their conscience, could have been unconstitutional. All other students will be expected to attend, in line with other timetabled exams.

For more on that particular complication, see the Irish Times’ detailed story.

So, why is this an IQTrainwreck?

  1. The similarity in packaging on the exam papers was a key root cause. This is (or should be) a straightforward process of ensuring that all exam subjects and levels are distinctly colour coded and ensuring that packaging is not similar. Issuing a reminder is simply trying to inspect a defect out of the process. Yes, the Superintendent has to carry responsibility as well for not double checking but avoidable similarity should have been avoided (ergo preventing the confusion)
  2. The lack of rigour regarding the reporting of the accidental distribution of the wrong paper is inexcusable. 
  3. The cost impact of the error is extremely significant, particularly given the current state of Irish Government finances. EUR1 Million is a challenging amount to find in your budget at short notice.
  4. The disruptive impact on students during a stressful time can’t be underestimated. 
  5. The further complication presented by Ireland’s multi-culturalism adds further challenges (and potentially costs) for the SEC, the Department, and the students.

(On that note of multiculturalism, one is left wondering if the ISM school in Tripoli, Libya that offers the Irish Leaving Certificate to its students will have received their replacement exam papers yet of if they are even aware of the issue.)

Information Quality Problems in Danish EU Elections

2 Replies

In a story that is bound to bring tears to the eyes of at least one regular contributor to on-line discussions about information quality, news reaches us from Jan Erik Ingvaldsen of a series of information quality disasters in the Danish EU elections which take place on Sunday.

Denmark is also holding a referendum on the same day, which some are viewing as a root cause for these problems.

Here is the story in Danish (big thank you to Jan Erik for this link via twitter)

Here is a rough translation in Google English. As google’s machine translations are never 100% accurate, we’d welcome any links to this story in English.

The problems (as we can make them out from the translation)

  • In Ikast-Brande, a municipality of 1700 people, voters received polling cards directing them to the wrong polling stations (wrong people, wrong ballot boxes)
  • In Stvens, the polling station is correct (right people going to the right ballot box), but the address given for the polling station is wrong.
  • In Vejle the wrong zip code was included in the map of the electoral district (i.e. people entitled to vote in a given polling station)
  • 2700 voters in three constituencies have been instructed to vote in two different locations
  • Some voters who are entitled to vote in both the Referendum and the EU Parliament elections have only received polling cards for one of the two ballots
  • Some postal voters report that they have  not received any information on the candidates running in the area their vote is being counted in.

Danish speaker – please correct my listing of the problem (admitting your problems is the first step on the road to recovery).

Jan Erik Ingvaldsen has a great post which summarises the problems in English.

Despite all of this, some elections commentators say that the preparations for the election are good, and they will sort out the problems when the complaints come in. Hanging Chads anyone?

While the fact that the Danes are running an EU election and a referendum on the same day, with different voter eligibility rules for each, goes some way to explaining the challenges that might have contributed to these problems, the defence is weakened by the fact that Ireland is running two ballots as well this week for EU Parliament elections and Local Government Councils and has not had reports of similar problems (yet).

Of course, the problem in Ireland is that our electoral register is wonderfully inaccurate.

For the impact that poor quality information can have on democratic processes, the view that the errors and impact can be “inspected out”, and because investigating this story made me have to figure out Google’s attempts at translation, this is being classed as a definite IQ Trainwreck.

Also, as Jan Erik points out in his blog, if this was happening in a South American or African nation there would be widespread media outcry.

Thanks to Jan Erik  (@jeric40 on twitter) for flagging this one to our attention.

I am not a number – I’m a human being!

Leave a reply

Information Quality professionals (and indeed quality management professionals in general) often recite a mantra that good quality begins at the beginning of a process, that it must be designed in, and that defects need to be fixed as close to the start of the information chain as possible.

A post today on DataQualityPro.com from Dylan Jones highlights the significant truth that lies behind all these statements.

Dylan’s son was born in April. The first thing the State did for him was to slap an identifier on him. The second thing they did (to summarise Dylan’s excellent and forensic post) was to make a mess of linking the local hospital ID to a National patient record.

That error propogated and resulted in the parents of another child over 90 miles away getting an appointment for a medical checkup relating to Dylan’s son. It seems that the efforts made to correct the error Dylan spotted when his son was born haven’t propogated half as fast as the original error.

And that’s the problem. How many other processes and silo’d systems has this error propogated into? How many more times in Dylan’s son’s life will be be confused with another child 90 miles away? What other ‘life-events’ will this error impact? In future, how will he find himself trapped by his number?

Ultimately, Dylan’s son is not a number, he’s a human being.

We recently posted a long Trainwreck on the problems with Google Health due to poor quality information. It is possible that an error like the one affecting Dylan’s son could result in incorrect patient data about Dylan’s son being transferred to this type of electronic patient record. Who would be responsible for the impacts if that information was acted on in haste without Dylan (or Mrs Jones) being there to point out that the information was wrong?

Given that in an Irish hospital in 2003 the medical staff failed to act on an error in an expectant mother’s chart and delivered a baby 39 days prematurely, despite the parent’s insistence that there were errors in the chart, highlights that simple errors in medical records can have significant impacts. That the baby in question died further highlights that these impacts can be catastrophic, which means that the standard of care for quality in medical records needs to be high.

Dylan’s investigations also uncovered some other weaknesses/flaws in patient data quality which are unsettling. We’d suggest you take a look at Dylan’s post for more details on those.

For actual inconvenience and annoyance to Dylan’s family, and for the potential for catastrophic loss or injury, this counts as a definite IQTrainwreck.

Double Debits – directly. (Another banking IQTrainwreck)

6 Replies

Courtesy of our Irish colleagues over on Tuppenceworth.ie comes yet another tale of poor quality information in financial services. Although this time it is at the lower end of the scale, at least on a per customer basis. However, the impacts on a customer are still irksome and problematic. And the solution the bank has put in place is a classic example of why inspecting defects out of a process is never an exact or value adding science.

It seems that Bank of Ireland has recently introduced some new software. Unfortunately, a bug in the software has resulted in certain transactions (deductions) being posted multiple times to accounts, resulting in cash-strapped Irish people being more strapped for cash than they’d expected.

Simon McGarr, (one of the authors over at Tuppenceworth) sums up the story and the reason why this is an IQTrainwreck:

I spotted a double charge on my account, for a pretty significant sum of money (is there any other kind?).

When I rang up to query it, I was told Bank of Ireland have changed their computer systems recently (Two weeks or so).

As a result, some transactions are being applied to accounts twice if they were processed through Laser [a debit card system in Ireland — ed.], or if they were a Pass machine [what the Irish call ATMs –ed.] withdrawal.

They say that if you spot the double charge, and ring them up to complain, they’ll send an email to their programmers to reverse the second charge.

I suggested to the polite customer services person that the bank might want to warn their clients to be alert for these double charges, as they could suffer additional charges (from appearing to breach their overdraft limits, for example) unless they spotted the bank’s mistake.

(Emphasis is added by this author)

Simon goes on to add (in a comment) that he has been without the benefit of his hard earned cash for 10 days (and counting).

Continue reading →

Dublin bank bungled foreign exchange transaction (in 2001)

Leave a reply

Following on from this morning’s story about the New Zealand overdraft fiasco, a few further cases of Information Quality trainwrecks in Financial services have come to our attention.

This first one is from 2001 and was found on the BBC.co.uk website, with further reporting from The Telegraph

Bank bungles pesetas/euros

Back in 2001, David Hickey was emigrating from Ireland to Spain. He asked his bank to change IR£1500 into pesetas, but an error in the bank meant that the  amount transferred was in euros, not pesetas. IR£1500 was approximately 300,000 pesetas. Mr Hickey received EUR300,000 into his account.

The bank eventually had to take legal action in Spain to freeze Mr Hickey’s accounts with a view to getting the money back.

At the time, the bank declined to comment further to the media on the matter and the Irish police were of the view that no criminal offence had taken place because of the ‘technical error’ (i.e. IQTrainwreck) involved.

Other Trainwrecks

We’re researching the other IQTrainwrecks that came to light this morning on this theme, not least to make sure we haven’t covered them here already. Expect further updates in the coming days.

Antipodean Bankers Sheepish over Overdraft Bungle (Again)

3 Replies

New Zealand Couple do bunk with bungled overdraft funds

Courtesy of  @Firstlink and @DataQualityPro on Twitter, and hot of the presses at BBC News comes this case of an IQTrainwreck in New Zealand.

Police are hunting for a pair of nefarious desperadoes who took advantage of a poor unsuspecting bank which deposited NZ$10 Million in their account instead of the NZ$10,000 that they had requested. That’s a difference of three zeroes, but in the wrong place.

The couple, it seems, have withdrawn an undisclosed amount of the money and appear to have left the country. They are being pursued by the authorities through Interpol, and it seems the Australian bank that gave them the money by accident are eager to have it back:

Westpac media relations manager Craig Dowling said the bank was “pursuing vigorous criminal and civil action to recover a sum of money stolen”.

Now, this is not the first time that something like this has happened in the Antipodes, and the last time it was WestPac who were over-generous with their credit.

Oops we did it again before

Back in September 2007 we carried the story of New South Wales business man Victor Ollis who, as a result of an undetected (by WestPac) error had benefited to the tune of AUD$11 Million. The quote below is from the original story on Australia’s news.com.au

Mr Ollis had an automatic transfer facility with the bank, which topped up his business account using funds from his personal account.

The transfers should have been stopped after his personal account was overdrawn in February 2004, the court heard yesterday.

But due to an error at Westpac, his account continued to be replenished – only with money “from the bank’s own pocket”.

Between June and December 2005, Westpac honoured cheques totalling about $11 million written by Mr Ollis.

WestPac sued Mr Ollis and were awarded their money back plus interest.  However, as Mr Ollis was apparently terminally ill at the time, there is a chance that they never got their money back.

A proud tradition in banking

However, WestPac are not alone in the pantheon of banking information quality trainwrecks. (We won’t talk about the current Global Financial Crisis and how some of its roots can be traced back to poor quality information… not yet anyway).

My personal favourite from our archive of Banking Information Quality Trainwrecks has to be this one though…

  • From Australia in  December 2007 – “Cat Gets Credit Card“. In this case it wasn’t WestPac but the Bank of Queensland who goofed.

But we’re only scratching the surface

Here at IQTrainwrecks.com we know we are only scratching the surface of these issues. Please contact us with your examples of Information Quality Trainwrecks (particulary in banking) so we can add them to the Roll of Honour.

These are the IQ trainwrecks in your neighbourhood

Leave a reply

Stumbled upon this lovely pictorial IQTrainwreck today on Twitter. Thanks to Angela Hall (@sasbi) for taking the time to snap the shot and tweet it and for giving us permission to use it here. As Angela says on her Twitpic tweet:

Data quality issue in the neighborhood? How many street signs (with diff names) are needed? Hmmmm

Data quality issue in the neighborhood? How many street signs... on Twitpic In the words of Bob Dylan: “How many roads must a man walk down?”

The Retail Data Nightmare

Leave a reply

Over at SmartDataCollective, Daniel Gent has shared an excellent example of a very common scenario in organizations across the globe… the troubling matter of the duplicated, fragmented and inconsistent customer details.

He shares a story with his readers about a recent trip to a retail store which used his telephone number as the search string to find his customer profile. The search returned no fewer than 7 distinct customer records, all of them variations on a theme. Daniel enumerates the records thusly:

1) One past owner of the phone from over 15 years ago;
2) Three versions of my name;
3) Two versions of my wife’s name; and,
4) One record that was a joint name account.

The implications that Daniel identifies for this are direct and immediate costs to the business:

  • Multiple duplicate direct mailings per year offering services
  • Multiple call centre contacts offering yet more services
  • Potential problems with calling on his warranty for the goods he bought because the store can’t find which of his customer records the warranty details are associated to.

Of course, this is just the tip of the ice-berg.

Daniel’s experience is all too common. And it is a Retail Data Nightmare that can quickly turn into an Information Quality trainwreck if left unchecked.