|
|
|
|
Quotable Quotes from Gartner GroupI can hardly have a page with quotes by Metagroup without a similar effort by Gartner Group. Gartner Group also publish some 'free' opinions on the Data Warehousing industry. I have browsed through the 'free' pieces and copied with references those that I thought were most interesting. Many of the 'free' opinions reference papers which are 'fee' opinions and I for one would encourage readers to purchase some of the more interesting 'fee' opinions. Again. The 'normal text' is directly quoted from public Gartner Group documents as referenced. My own opinion is expressed in 'italics'.
Data Warehouse – Essential Infrastructure for BI15 November 2001. From AV-14-8985 by Kevin Strange
Through to 2004, more than half of Global 2000 enterprises will fail to properly use BI, and in the process they will give market share away to those that do (0.8 probability). It never ceases to amaze me that, despite the overwhelming value in implementing data warehouses and effective BI, the majority of companies still do it poorly or not at all. One of the most consistent patterns is the ‘do it in 3 months’ demand from most businesses. It is strange for any rational person to believe that the IT system mess created over the last 20 years can somehow be magically ‘cleaned up’ by a 3 month project. Sure, you can make a start but BI that makes a difference takes a while. The other thing that never ceases to amaze me is the low skill level of people who are given BI projects to do. In so many cases I see people who clearly do not understand how to implement a BI project given the task of BI running BI projects.
The heart of a durable business intelligence strategy is a data warehouse. Amen to that!! Plenty of people still argue the point.
Implementing the data warehouse infrastructure appropriate for supporting BI can be a daunting challenge – an effort often underestimated and unrecognised. The challenges are political, cultural and technical. With so many challenges, the majority of organisations that begin implementing a data warehouse actually end up implementing a data mart. Therefore, some of these organisations believe the data warehouse was a failure because the implementation failed to meet their dynamic analytical (application neutral) requirements without major changes to the data mart infrastructure. Many companies go down this multiple separate data marts
approach. This has been well
exposed as being a rather expensive and ineffective way to go. My experience has been somewhere around the 4th
data mart it is cheaper to scrap the lot and start again to build an integrated
model than try and ‘fix’ what is there.
The usually problems are lack of reconciliation across disparate data
marts and lack of consistency in data meanings.
The numbers don’t add up and such lack of agreement between reporting
systems will rapidly undermine senior managements confidence in all the data
marts. My view? There should be one version of the truth and it should be
replicated to wherever it is needed.
Organisations need to focus on the infrastructure of a data warehouse and recognise the need for application neutrality, despite the phased implementation approach. When they do not, the results, in many cases, will be unwanted data marts in “stovepipes”. Over the last few years, decentralisation of decision making and the budgets for implementing applications has exacerbated this stovepipe issue. Again. Amen for such an accurate and forthright statement. With respect to the analytical (BI) side of CRM, at least 65% of the efforts are implemented in an unintegrated fashion, based on a function (different efforts by different departments), rather than on a more strategic initiative – the sum is greater than the parts. With a ‘functional’ approach, there are redundant costs and inconsistent efforts for integrating data and performing an analysis. Incomplete and bad-quality data can be more harmful than no information at all. Early in the meteoric rise and rise of Siebel (1997) I was invited to hear the Siebel pitch. I was quite surprised to hear that the standard pitch was to put Siebel in within 6 months for enterprise wide CRM. I asked the rep about how they proposed to provide a single integrated view of the customer upon which to implement Siebel in major companies in 6 months. The answer, of course, was that the customer data integration issues could be left until after Siebel was implemented. This sales pitch seems to have been adopted by many of the CRM vendors and plenty of companies (like 65%) have moved ahead with a CRM project with no integrated customer information. For me it’s hard to imagine anyone with any significant experience in CRM style activities inside a company would move forward in such an unintegrated environment. A warning, I guess, to all who read this who consider doing CRM with no integrate customer database. The Reasons a Data Warehouse Implementation Can FailImplementation of a data warehouse is not easy, and those
that fail do so for some of the following common reasons: No high-level manager devoted full-time to the project: Neglected or insufficient planning: Lack of expertise available internally or via a service
provider: Underestimation of required resources for
implementation: Neglecting to perform total cost of ownership (TCO)
analysis: I would comment that a TCO is actually pretty hard to do early in the process. Early in the life of the DW project I usually recommend keeping it small, focused, inexpensive until the company is comfortable with the real investment that is required. Wrong Technology: Insufficient attention to the data: A pretty good list. Readers would do well to avoid each of these. I plan to publish my own reasons for Data Warehouse project failure as well.
Research Note: COM-15-1975D. McCoy 9 January 2002
BI appears fragmented going into 2002. On one hand, we see
tactical deployments and disjointed efforts by many enterprises that have been
roiled by the economic cost cutting requirements of tighter budgets.
The reason I included this little snippet was that I was quite surprised that so many organisations are still not doing performance management across the company.
CPM: A Strategic Deployment of BI ApplicationsAV-16-3211 Nigel Rayner 9 May 2002 Corporate performance management (CPM) combines business intelligence (BI) with performance methodologies, processes and metrics. Enterprises can use CPM to leverage BI initiatives and gain insight into their business. Executives and managers have always needed to understand how their businesses are performing. However, recent events, such as the “dot-com” crash, the tough economic climate and the fallout from high-profile corporate failures like Enron have sharply increased executive focus on understanding and managing corporate performance. A shame we had to have a crash for many senior managers to start insisting on getting the right numbers provided to them. Gartner has highlighted the importance of business intelligence (BI) in providing insight into the business. A challenge for many executives is to understand how BI melds with performance management methodologies (such as the balanced scorecard) and management processes (such as budgeting, planning and forecasting). Too many enterprises have treated these as disconnected initiatives, perhaps implementing a balanced scorecard in isolation from the BI strategy. Gartner defines corporate performance management (CPM) as the combination of methodologies, metrics, processes and systems used to monitor and manager the business performance of an enterprise. Understanding the convergence between these aspects of CPM is key to enterprise success – enterprises that effectively deploy CPM solutions will outperform their industry peers. My comment is this. Too often we see people trying to define ‘the box that something fits into’ like BI or DW or CPM or CRM or BAM. Name your three letter acronym. A long time ago a very workable definition of data warehousing was proposed by Bill Inmon. It still works. A part of the definition is a ‘collection of data to support the management decision making process’. If the data warehouse is where you store your data to support your management decision making processes it is pretty obvious that significant portions of BI, CPM, CRM and BAM style data would be sent to the data warehouse and that the data warehouse is a cornerstone of the information architecture of any large company. Whenever you hear your BI, CPM, BAM, DW, CRM project manager trying to put up definitions of the differences between these three letter acronyms you should be worried. My opinion is that you project manager should be talking about how to deliver value to the business based on business projects that will be supported by ‘some technology’.
The Data Warehouse: Dead or Alive?
COM-13-5988 H. Dresner 2nd May 2001. CommentaryData warehousing has had its share of successes and failures during the past 15 years. However, it’s still an enterprise’s best chance to get valuable data for analysis quickly and reliably. Data warehousing is not new. Enterprises have engaged in the transformation and replication of data for decades. It wasn’t until the early 1990s that the name “data warehousing” was introduced. There are many reasons and benefits for using a data warehouse. How, then, does that explain the numerous data warehousing failures? Data warehousing has become a taboo term within some enterprises during the past few years. Some CIOs have forbidden their IS managers and business planners from referring to their data warehouse as a “data warehouse”. Enterprises frequently complain about design complexity, data movement and transformation issues, and development costs. I can identify with this. I was once asked to present to the CIO of the Australian branch of a very large global insurance company and told I was not allowed to say the word Data Warehouse. So everywhere I usually said Data Warehouse I said Decision Support System. The result? I was criticised for calling a decision support system more than it actually is. Some days you can’t win!! That ‘Data Warehousing’ has become a taboo term is also unjustified. For years I have only ‘sold’ consulting services to companies who have ‘failed’ at building their data warehouse at least two times. (And there were plenty of them.) The reasons they have ‘failed’ previously are almost never to do with technology. I had one customer who complained of the poor run times and high support costs of their ETL, which turned out to be written in VB. I had another customer who main production system vendor sold them a ‘data warehouse’ which turned out to be an image copy of the operational system to different disk subsystem. Failures yes, because of technology? I think not. Most of the failures I have seen are because the people doing the project simply did not understand what it was going to take to get the project done. There issues are things like the project was not tied closely enough to the business benefits to be derived. The business did not understand what they were going to do with the data warehouse once it was delivered. I have included a recent ‘Gartner Group’ list of why DW projects fail just a little further up on this page. But making the term ‘data warehouse’ taboo makes no more sense than making the term ‘cobol’ taboo. Why do it at all? Simply put, existing operational systems (and their databases) were not designed for end-user access and analysis, ie., business intelligence (BI). By cleansing, simplifying, enriching and localizing data, end users get more value faster than any (or many) operational systems can deliver. Yet, because there are so many potential obstacles and risks, many enterprises avoid the subject entirely. I was one of those people in the late 80s talking about designing systems based on a relational enterprise model and then overlaying the application logic and business logic on top of that enterprise model. We lost this argument to ‘packages’. In so many presentations I explained to companies considering buying packages that though the package may be great, they were buying an island of information that they would have difficulty integrating with any other data in the organisation. “Surely it can’t be that hard, just extract some data from here and integrate it with that data over there, we’ll be fine.” My favourite cartoon of the time was two people looking at a bunch of mainframe computers and one character says to the other. “The data integration problem won’t be too bad, all the computers are blue.” Well, having lost this argument so many times, my next career was born from the loss. A lot of companies I know are paying very large amounts of money to integrate data from multiple packages. In some cases the integration cost of getting data between applications is a significant portion of the overall project costs. There is no doubt about it, every time a company buys an operational system that is a package there is a significant amount of work to expose the data in the package to other operational systems and the data warehouse. I haven’t reached a conclusion as to how web services are going to affect this, but I have a feeling it is going to be more difficult in future to integrate data from distributed web services than it is today from distinct packages. I look forward to doing my first web services data warehouse!! When we dig a bit deeper
into data warehousing failures, we generally find that the root-cause issues are
not technology challenges, but rather lack of organizational and business
planning. One of the most consistent causes I have seen for data warehouse failure is the failure to apply something like the Management Decision Making Process as a guideline for building the Data Warehouse. Most projects have failed by the time the requirements document is signed off because the requirements are not what the business needs to measure, manage and monitor the business. Vendors Run Amuck
Vendors present
additional obstacles to organizations attempting to build a warehouse.
Often promising benefits, vendors sell their wares and services to
unsuspecting customer and then turn a deaf ear when problems arise. Vendors also promote
dubious products and technologies – some of which become discredited, only to
rise from the ashes several years later. In
T-13-5175, “Why the Virtual Data Warehouse Still Won’t Work,” we explore a
technology that sounds compelling, but should be avoided. In T-13-4734, “The
Packaged Data Warehouse Myth”, we debunk the notion that there is anything
“packaged” about it. Having been on both sides of the fence (vendor and customer) my comment would be this. Vendors are trying to sell customers products and services. This is their job and it is what they get paid for. One can hardly criticise a vendor for making promises in order to get the deal. I, for one, wish that customers who have been misled or over promised and under-delivered would be more public in their comments. However, it is very, very rare that a customer will publicly criticise a vendor. Because of this it is no small wonder vendors will promise the world to get the deal. There is almost never a down side. The most amusing example I ever saw was a company bought a $US1M systems integration project from a vendor and, naturally, they asked for references. One of the references supplied was for a project where the vendor was actually thrown out for non-performance and in the end settled quietly out of court for a substantial sum. Yet this next customer didn’t even call the reference supplied. Clearly, buying an SI project is a situation of 'buyer beware' and it is clearly the customers responsibility to call the references. The Right AttitudeSuccess with data warehousing is possible, even likely, when done properly. It must begin by setting proper and realistic expectations. In COM-13-5615, “Data Warehouse Acceptance: Cultural vs. Technical Issues”, we explore how the IS organisation and uses must work together to set practical goals for a data warehouse and the importance of user acceptance. I would go one further. Done properly it’s actually pretty hard not to be successful in building a data warehouse. Successful being defined as recovering the cost of the data warehouse investment in some reasonable time period. Such as 12 to 24 months. The fastest payback period I have ever heard of is 2 weeks. This means that the data warehouse was used to perform spare parts analysis and orders for parts in excess of the cost of the data warehouse were cancelled in the first two weeks. Real money, retained in the company. Usually it takes a bit longer to pay for the data warehouse, like 3-6 months after the first subject area is up and running, depending on how much you spend up front. But using a simple model like the Management Decision Making Process to guide your thinking makes severe failures relatively difficult to achieve. Bottom Line: Data Warehousing is alive and well. However, many enterprises will continue to damage themselves by going about it the wrong way. Enterprises should tackle data warehousing armed with an appropriate set of methodologies and technologies, if there is to be any hope of success. It is interesting to note that despite such a resounding endorsement for using an appropriate methodology most companies simply do not pay money for data warehousing methodologies. Companies seem quite happy to pay money for all sorts of other methodologies. I have even had some experiences where customers have insisted on using the OLTP build methodology to build the data warehouse with the associated problems that causes. When I was at Ardent we struggled to sell the Iterations methodology to a large number of customers. Ascential Software still markets the Iterations methodology but there are not a lot of copies being sold. The twist on this is that every customer ‘expects’ the vendor to have a data warehousing methodology but don’t want to pay money for it.
Business Intelligence Imperative
AV-13-8785 Howard. Dresner 17th July 2001. Doing business is information-intensive. Enterprises are being pushed to share information with increasingly more audiences. The business intelligence imperative insists we elevate BI to a strategic initiative now, or risk disaster! Ignorance is the greatest threat to modern business. This risk of not knowing is immense. And, incomplete information can be even more harmful than no information, because we proceed and make decisions and act with conviction, falsely believing we know the true nature of the situation. Business Intelligence (BI) strives to eliminate guessing and ignorance in enterprises by leveraging the mountains of quantitative data that enterprises collect every day in a variety of corporate applications. I would tend to agree that believing you know what you are doing when you don’t is more dangerous than knowing you don’t know what you are doing and hence not doing anything. I don’t believe any major global corporate disaster is heading our way through ignorance or mis-information. However, a few companies have paid dearly for poor monitoring systems for traders (Barings, All State to name just two). I don't expect these two to be the last companies that turn out to lose vast sums of money due to one or two rouges. And we have clearly seen that when a number of rogues get together the losses can be mind boggling. (Enron and Worldcom) How Can the BI Imperative Be Achieved?Developing complete perspective requires an intimate knowledge of a business’ many dimensions. Modest-sized businesses, with limited numbers of products, customers and suppliers, can obtain this perspective without the benefit of technology. All others must rely on technology. This requires leveraging systems that capture the many aspects of the business experience. These systems collect and manage vast amounts of data. The data is growing by as much as 100 percent per year. By extracting data from operational data sources, transforming and integrating it and placing it into data warehouses or data marts for delivery to users through BI, the same intimacy can be had by any enterprise. In the absence of BI, a “fact gap” exists: A condition where users make decision and assess risk and opportunities based upon anecdotal, incomplete or outdated information. This isn’t much better than guessing, leaving most businesses seriously exposed. In the early 90s in Australia a law was passed making company directors liable for company failures if negligence or fraud could be proven. We had intense interest in data warehousing at the time. For years I talked with customers about ‘Fact Based Decision Making’ knowing the facts of a given situation prior to making major investment decisions. We have a smaller ‘fact gap’ than we did 10 years ago, but the gap between what the business actually looks like versus what the reporting systems tell management the business looks like is still significant. And we have not yet rid ourselves of ‘middle management’ properly ‘interpreting’ the figures for their boss. For senior management, knowing the true state of the business is very rarely a bad thing. Most countries have removed the ‘ignorance is bliss’ clause. What Can I Do Today?BI can help enterprises today in at least four areas.
The Right Mindset: Enterprises must change the way they think about BI and what it can do to help drive the enterprise. The Right Architecture: Once an enterprise is committed to BI as a strategic initiative, aligning the business requirements with a solid technical architecture is crucial. A Solid Foundation: BI Demands a solid and reliable foundation in the form of a data warehouse. This is among the most difficult and resource-consuming stages of BI development and deployment. However, without this foundation, BI is all but useless. A Glimpse Toward the Future: Although most enterprises focus on near-term implementation, they should understand where the industry is headed and how BI and data warehousing can help them become simultaneously competitive and efficient. Three key trends suggest that BI in the future will be profoundly different than today. The first trend is the need for real-time insight. In “Business Activity Monitoring: The Promise and Reality” (COM-13-9992), we describe a future of zero-latency BI and identify some of the near–term obstacles to achieving it today. Second, increased collaboration, as part of the BI process is often absent today. In “What BI Needs Next is Collaboration” (COM-13-9855), we review how a collaborative approach to BI will make the analysis part of business processes more effective. And finally, mobile computing is already having some impact upon BI deployments. In Mobile and Wireless BI: Is Its time Here or Near? (T-13-8684), we delve into when mobile based BI will truly be valuable to enterprises. Now is the time for all enterprises to raise BI to the level of strategic initiative. The value to the business is proven. With the right mindset, methodology and technologies (with an eye to the future), success is likely.
The BI Competency Centre – Organising for Success
AV-14-4898 Nigel Rayner 18th September 2001 Business intelligence “enlightenment”, where BI delivers real insight to change the business and generate new revenue opportunities, can only be achieve by establishing a BI competency centre. Enterprises that can progress through the different levels of BI understanding will reach a stage of BI “enlightenment”. Here, BI becomes embedded in the systems and processes of the business to build a more-agile enterprise that can anticipate and react faster than its competitors to changing business conditions and new profit opportunities. "Enlightenment", sounds rather religious. However, there does seem to be a clear progression in the use of BI and Data Warehousing within organisations. Obstacles on the Road to EnlightenmentUnfortunately, the road to enlightenment is not an easy one. Through to 2004, more than half of the global 2000 enterprises will fail to properly use BI, and in the process they will give market share away to those that do (0.8 probability). Even in the most-advanced enterprises, information services departments are forever responding to management and user requests for customise reports in response to critical business needs. But as these demands grow, this model (which never woks well anyway) breaks down. Users get less-timely information and are forced to fend for themselves, often making decisions with incomplete, outdated or anecdotal information. This has multiple, negative effects on the IS organisation and the business:
Lack of coordination between the IS organisation and the business is one of the major challenges on the road to BI enlightenment. Even if the executive team sees the strategic importance of BI, any BI strategy is likely to fail if users and IS are not aligned toward common goals. Other challenges are:
These can conspire to block progress to BI enlightenment, leading to under-delivery of expected benefits and user disillusionment.
The Right Organisation is CrucialTo overcome these challenges, enterprises must put in place the organisation to successfully deliver a BI strategy. The key to success is a BI competency centre. In “Building a Successful BI Competency Centre” (COM-14-2516) we define the roles and responsibilities of a BI competency center, with guidelines for establishing successful reporting lines. However, just setting up a BI competency centre is not enough to guarantee success. I'm not at all sure it is necessary to have something as
'grand' as a 'BI Competency Center'. I am sure there needs to be a central team
who perform development on the information systems (BI, DW et al), who know the
data and the rules for calculating the data. Knowledge of the data, where it
came from, how it gets to the end user and the calculations performed along the
way is a very substantial piece of knowledge which takes most people a long time
to learn. The idea that all this information can just be written down and
anyone can then reference it has never worked effectively. And even though
the tools are now able to expose the code that is used to perform such work it
is still not a trivial exercise to follow the trail of numerous data elements
flowing into something like a Business Objects report. We still need people with
knowledge of the data to be available. A number of factors must be considered to ensure the BI
competency centre is able to deliver on expectations:
|
|
Send mail to peter@peternolan.net with
questions or comments about this web site.
|
