Tag Archives: economic impact of information quality

Did an Excel coding error destroy the economies of the Western world?

Leave a reply

The title of this post is taken from an article by Paul Krugman (Nobel prize winning economist) in the New York Times of the 18th of April 2013. And it really is a good question that sums up the significance of the information quality problems that have emerged in an economic model which has been used to guide the actions of governments and non-governmental organisations in response to the global financial crisis.

Krugman’s article summarises the background very succinctly but we’ll summarise it again here:

  1. In 2010 two Harvard economists, who between them had served with and advised a number of governmental and supra-governmental organisations, produced a paper that argued that there was a key threshold above which Government debt became unsustainable and had a negative effect on economic growth. That threshold was 90%.
  2. That threshold was used as a key benchmark to inform policies for dealing with government debt crises in Europe and elsewhere. It became an article of faith (despite some economists questioning the causation/correlation relationship being argued). The official line being taken with countries with sovereign debt challenges was that austerity was required to reduce debt below 90% to prevent a fall off in growth – and there was academic research to prove it.
  3. However other researchers struggled to replicate the results presented in the original paper – decline in growth was never as severe and the causal relationship was never as definitive. Eventually one researcher got access to the original spreadsheet and uncovered methodological issues and fundamental calculation errors, including a formula calculating an average that left out data points (5 countries were omitted).

The reanalysis of the spreadsheet data, correcting for methodology issues and for calculation errors found no average negative growth above the 90% threshold. According to author Mike Konzcal on economics blog NextNewDeal.net:

They find "the average real GDP growth rate for countries carrying a public debt-to-GDP ratio of over 90 percent is actually 2.2 percent, not -0.1 percent as [Reinhart-Rogoff claim]." [UPDATE: To clarify, they find 2.2 percent if they include all the years, weigh by number of years, and avoid the Excel error.] Going further into the data, they are unable to find a breakpoint where growth falls quickly and significantly.

Konzcal goes on to hope that future historians will recognise that:

one of the core empirical points providing the intellectual foundation for the global move to austerity in the early 2010s was based on someone accidentally not updating a row formula in Excel.

An alternative analysis of the data presented on NextNewDeal.net also raises questions  about the causal relationship and dynamic that the original paper proposed (that high government debt causes decline in demand).

Paul Krugman has posted further updates on his NYTimes blog today.

Impact?

As with many information quality errors, the impacts of this error are often not immediate. Among was the potential impacts of this spreadsheet error and the nature of the causal dynamic are:

  • Austerity policies in Ireland, Greece, Cyprus, Italy, Portugal, Spain and other countries
  • Business failures (due to fiscal contractions in an economy reducing supply of investment finance, weaker demand, longer payment cycles etc)
  • Reduction in public services such as health care, and increases in taxation
  • Increases in Suicide in Austerity countries (e.g. Greece)

 

Conclusion

Where data and its analysis becomes an article of faith for policy or strategy it is imperative that attention be paid to the quality of the data and its analysis. In this case, opening up the data for inspection sooner might have allowed for a more timely identification of potential issues.

It also highlights the importance of careful assessment of cause and effect when looking at the relationship between two factors. This is an important lesson that Information Quality professionals can learn when it comes to figuring out the root cause of quality problems in the organisation.

Accounting Accountability

Leave a reply

From Europe we learn of two stories with similar characteristics that tick all the boxes for classic Information Quality Trainwrecks.

 

From Germany we hear that due to errors in internal accounting in the recently nationalised Hypo Real Estate, the German National debt was overstated by €55 Billion (US$76 bn approx). This was doubly embarrassing for Germany as they had spent the last while criticising the accuracy of accounting by the Greek Government.

According to the Financial Post website:

In an era of austerity where their government has squabbled tirelessly for two years over a mooted €6-billion tax cut, Germans found it hard to fathom that their government was so suddenly and unexpectedly 55-billion euros better off.

The net effect of the error being found and fixed is that Germany’s Debt to GDP ratio will be 2.6% lower than previously thought.

The root cause appears to be a failure to standardise accounting practices between two banks who were being merged as part of a restructuring of the German banking system. This resulted in the missing billions being accounted for incorrectly on the balance sheet of the German government who owns the banks in question.

From Ireland we have a similar story of missing Billions. In this case a very simple accounting error resulted in monies that were loaned from one State agency (the National Treasury Management Agency) to another State Agency (the Housing Finance Agency) being accounted for by the Department of Finance in a way which resulted in €3.6billion being added to the Irish National Debt figures.

This (almost co-incidentally) resulted in a 2% misstatement of the Irish National debt. Also co-incidentally it is exactly the same figure as the Irish Government is seeking to reduce net expenditure by in its forthcoming budget.

The problem was first spotted by the NTMA in August of last year (2010) but, despite a number of emails and phone calls from the NTMA to the Department of Finance the error was not fixed until October 2011. For some reason there was a failure in the Department to recognise the error, understand the significance, or take action on it.

The Secretary General of the Department of Finance blames middle-management:

Secretary general of the department Kevin Cardiff said the error was made at “middle management” level and was never communicated up to a more senior level. He said the department was initiating an internal inquiry to examine the issue and would establish an external review to look at the systems and to put safeguards in place to ensure such mistakes were not repeated in the future.

Information Quality Professionals of course would consider looking at the SYSTEM, and part of that is the organisation culture which is in place in the Department which prevented a significant error in information from being acted upon.

Lessons to Learn:

There are a lot of lessons to learn from these stories. Among them:

  1. When bringing data together from different organisations, particularly when those organisations are being merged, it is important to ensure y0u review and standardise the “Information Product Specification” so that everyone knows what the standard terms, business rules, and meaning of data are in the NEW organisation and ACROSS organisational boundaries. Something as simple as knowing who has to put a value in the DEBIT column and where the corresponding CREDIT needs to be put should be clearly defined. Operational Definitions of critical concepts are essential.
  2. When errors are found, there needs to be clear and open channels of communication that allow the errors to be logged, assessed, and acted on where they have a material or significant effect. Organisational cultures where internal politics or historic arrogance lead managers to assume that the issue isn’t there or isn’t their problem ultimately result in the issue becoming bigger and more difficult to deal with.
  3. Don’t shoot the messenger. Don’t blame the knowledge worker. But ensure that there are mechanisms by which people can take accountability and responsibility. And that starts at the the top.