Practice Notes

Thought Piece

Problems with data warehouses

As a data expert, it always staggers me when companies have multiple data warehouses. There may be good reasons for this, for instance legacy systems acquired through mergers or acquisitions. But they should be consolidated as soon as possible and senior directors should try their best to have only one data warehouse in the company to ensure maximum productivity and minimum confusion.

"Companies seem to be far more willing to buy shiny new systems than they do to spend the equivalent amount of money, or perhaps even less, on fixing the data within their existing set up."

"It is hard work implementing a new data system and that effort could be better spent in correcting the problematic data in the existing data warehouse."

"living with multiple data warehouses is more expensive eventually"

Even when companies do take decisive action to fix their data problems they should take care to ensure they make the right decisions. Companies seem to be far more willing to buy shiny new systems than they do to spend the equivalent amount of money, or perhaps even less, on fixing the data within their existing set up. I have yet to come across a situation where the reason the data is wrong is entirely down to the system. It is hard work implementing a new data system and that effort could be better spent in correcting the problematic data in the existing data warehouse.

One client is talking about spending 4m on upgrading SAP but none of the problems that it is addressing will be resolved doing that because the problems are in the data, and in particular the absence of a data dictionary. Implementing a 'solution' that has no chance of solving the problem is crazy. Such a move means that you are building the new system on sand. At some point the data will have to be migrated from the old to the new system so you either have to fix the data issues: so why buy a new system? Or sweep them under a carpet: er... so why buy a new system?


Why Buy a new system?

One company I know has three active warehouses: it drives the CEO crazy and leads to near-decision making paralysis. And there is no good excuse for so many. Sales, Finance and Operations all have their own separate warehouses because of the culture of mistrust that exists between all three areas. Ironically this mistrust is driven in part by the fact that every time any data is requested, the three systems produce different answers. That happens partly because the three functions can't even agree on how to describe the various business lines so revenues and costs are not allocated on a like-for-like basis.

That is no way to run a business. There should be one warehouse where all the data is merged and validated and therefore seen to be correct. However this company is not unique in its possession of little islands of data, nor in its inability to do little more than put on a sticking plaster as a quick fix when a much more fundamental revision is required. While it is cheaper in the short term, and gentler on the capital expenditure budget, living with multiple data warehouses is more expensive eventually. Both in terms of a hefty ongoing maintenance bill and, most importantly of all, in terms of the lost opportunity cost of running a dysfunctional business.























.

John Urquhart

John Urquhart

Share by: