Products
One of the purposes of my web site is to sell the data warehousing products
that I am developing. The table below provides bookmarks to the headings for each of the
products. All the publicly available information for the products are on this
page.
I am very excited about these products. I have spent a lot of time thinking
about what is 'missing' from the ETL products that I use on a regular basis. To
date I
have developed two products to do things that are 'hard to do' in the ETL tools.
These products can be used with or without ETL tools. I am developing more
products and utilities to assist ETL developers. I would be very interested for
people to send me emails on what they feel is 'missing' and could be built using
utilities to complement the ETL tools.
As someone who has been building large data warehouses since 1990 and star
schema data warehouses since 1993 I know that these two products will dramatically reduce the
elapsed time and cost of developing your star schema data warehouse.
If you are interested in buying the Instant Data Warehouse or the C Data Warehouse Generator please contact
me on peter@peternolan.net or on my
mobile phone 353-87-958-1732.
Best Regards
Peter Nolan
The Instant Data Warehouse
Link
|
Description
|
| The Instant Data Warehouse |
The Instant Data Warehouse is a tool to enable you to build
your data warehouse much faster than previously possible. Even faster than
with any of the ETL tools available on the marketplace!!
Using the Instant Data Warehouse you can expect to be able to:
 | Build 5-6 dimension table processing jobs per day!! |
| Build 2-3 fact table processing jobs per day!! |
This is an order of magnitude faster than the leading ETL tools available
today. And, any additional fields you choose to add require no code
changes. Something the ETL tools are inherently unable to do today.
|
| The Instant Data Warehouse FAQs |
This is the Frequently Asked Questions page for the Instant Data Warehouse.
If you are considering evaluating the Instant Data Warehouse this is a
good page to read because you will see the common questions and answers
for the product. |
| The Instant Data Warehouse Product
Support |
This is the link to the public product support for the Instant Data Warehouse.
If you are evaluating the Instant Data Warehouse or you are a user of the
Instant Data Warehouse this is a good page to read because you will see
posts for product support. |
The C Data Warehouse Generator
Link
|
Description
|
| C Data Warehouse Generator |
The C Data Warehouse Generator is a tool to enable you to
build your data warehouse much faster than previously possible. Faster than with any of the ETL tools available on the
marketplace.
By using the C Data Warehouse Generator you can generate C code with
embedded SQL calls to the database based on a set of parameters that
define the data warehouse data model.
Using the C Data Warehouse Generator you can expect to be able to:
| Build 1-2 dimension table processing jobs per day. |
| Build 1 fact table processing job per day. |
The C Data Warehouse requires more time than the 'Instant Data
Warehouse' because it requires more set up and it generates code.
This code generation is the biggest 'feature' differentiator to the
'Instant Data Warehouse'. By having the source code generated you
can have the maximum control over the code you implement in production.
|
As an ETL developer, ask yourself "What would life be like if we could load a data warehouse without having to
write code to do it?" What would that be like? In a word, it would be
'magic'.
This 'magic' is here and available, now.
If you would like to be able to take data from your staging area and load
your star schema data warehouse without writing (or generating) a single line of
code, or specifying all those mappings/jobs to do the work. The Instant Data
Warehouse is for you.
The Instant Data Warehouse imbeds years of ETL experience into the single fastest method available
today to get a star schema data warehouse up
and running.
This is as
'Instant' as it gets.
Background
In November I completed the C Data Warehouse Generator. The C Data Warehouse
Generator is a great step forward in making available a generalised suit of C
code to load star schema data warehouses. Using C and embedded SQL, it is as
'open' as you can get for a development environment.
Having put the C Data Warehouse Generator onto the marketplace the
obvious next question was:
"Is it possible to remove the need to perform code generation and to
perform all customisation of processing at run time yet not require an extensive
repository like the ETL tools to achieve this run time
customisation?"
If all 'customisation' of processing for loading a data warehouse can be
performed at run time without requiring extensive repositories of metadata like
the ETL tools an extremely fast and efficient method of building a data
warehouse would be possible.
The question intrigued me. I decided to experiment to determine what
might be possible.
The 'experiment' was developed with the following design points:
| The customer should not have to write/generate any code to load a data
warehouse. ETL Nirvana! |
| It should run on unix and win32. |
| It should be able to attached to the widest possible set of data sources
while still using widely available interfaces to those data sources. |
| It should be as fast as possible. |
| It should require no code changes and virtually no testing to add new
columns to dimension tables or fact tables. That is, 'maintenance' should be
'free' or as close to 'free' as possible. |
| It should support nulls. Nulls are something I always recommend against in
a data warehouse but some people do put nulls into data warehouses so this
tool had to support nulls. |
The answers to these design points were:
| The language it should be written in is C++. |
| The data access mechanism should be ODBC. Nothing else is as open in that
it can be called under win32 or unix. |
| All data access should be completely encapsulated so that other client
libraries could be plugged into the code in the future if demand was there
for it. |
| It should should always gather table and column metadata from parameters
and the database catalogue rather than a repository. |
| It should never refer to field names or move data referring to field names
in the code. All movement of data should be by
field name and the matching field names should be 'discovered' by the
programs. |
In short, the experiment worked. And now I am pleased to be able to bring the
'Instant Data Warehouse' to the Data Warehousing community.
Productivity Improvements
You will gain some quite unbelievable productivity improvements when
using the Instant Data Warehouse. Once you have defined your data models,
using the Instant Data Warehouse you can expect to be able to:
| Build 5-6 dimension table processing jobs per day per experienced
developer*. |
| Build 2-3 fact table processing jobs per day per experienced developer*. |
| Make changes to the columns stored in fact or dimension tables with no
code changes and practically no testing. |
This is an 'order of magnitude' faster than writing the equivalent
processing using the leading ETL tools available
today.
*An experienced developer means a person experienced at implementing star
schema data warehouses using the Instant Data Warehouse. Instant Data
Warehouse 'developers' do not need to know any 'language' except how to create
tables and views in the source databases and the data warehouse database.
There is NO CODE required to load data warehouses using the
Instant Data Warehouse.
As one of the few people expert in the use of both Informatica PowerCenter
and Ascential DataStage I can assure you that the Instant Data Warehouse is much
faster for the initial development of jobs to load fact/dimension tables.
The productivity difference is even greater in maintenance because the Instant
Data Warehouse requires no changes to add new columns to fact/dimension tables.
All the ETL tools I am familiar with require developers to change and test
'code' in order to add new columns to fact/dimension tables.
Summary of Major Features
The most important features of the Instant Data Warehouse are:
| You write NO CODE to read data from one ODBC data
source and load it into a data warehouse accessed via another ODBC data
source. That's correct. NO CODE. I do believe this is a
'first'.
|
| The Instant Data Warehouse maintains:
 | Type 1 and Type 2 dimensions. |
| Detail level and summary level fact
tables. |
|
| The Instant Data Warehouse is aware of column names and data types.
| The Instant Data Warehouse moves data from source to target based on
column names not based on a 'mapping'/'job' that you have to define in a
'repository'. You tell the Instant Data Warehouse the source view and
target view as run time parameters and it figures out which columns to
move by matching column name. |
| Because all data movement is by column name when you add new columns
to fact or dimension tables (a very common occurrence) you need NO
CODE CHANGES to add these new columns. |
|
| All regular ODBC data types are supported. (The exception is that blobs are not currently
supported.) |
| The Instant Data Warehouse supports many levels of aggregates in dimension
and summary fact tables and enables the addition of new aggregates with no
code changes. Just like my published C code. |
| The Instant Data Warehouse is null aware.
| For example in a type 2
dimension if a field has changed from null to a value then the Instant Data
Warehouse is aware of the change to the field. |
|
| The Instant Data Warehouse includes auditability and control features so
that it is possible to track the number of rows moved between files and
tables for every run by userid used to run the job. |
| The Instant Data Warehouse only accesses views and therefore places no
restrictions on the names of the tables in your source systems or data
warehouse. |
| It is written in C++/ODBC and is written to be able to run on win32 and
unix. Currently it is tested on SQL Server 2000 and Windows 2000.
Oracle users should contact me to discuss availability of the Oracle
version. |
Who is the Instant Data Warehouse intended for?
The Instant Data Warehouse is intended for companies who fit into one or more of the
following categories:
| You are building a star schema Data Warehouse on a tight budget.
| Which is most companies at the moment. |
|
| You want to write an absolute minimum of any kind of code to build the
Data Warehouse. |
| You have decided not to purchase an ETL tool. |
| You have bought an ETL tool and want to complement the ETL tool with features
of the Instant Data Warehouse. |
| Your data warehouse loading times can be achieved using ODBC rather than
native drivers supported by embedded SQL. |
| You have an existing data warehouse and you are finding that the
maintenance cost for new columns is significant. You want to reduce your
maintenance costs. |
If you feel you fit into one ore more of these categories I have written
two presentations you might like to browse through. One presentation
for
business managers/ project sponsors of data warehouse projects and one
presentation for data
warehouse project managers. The second presentation contains more technical
details than the business manager presentation.
The Business Managers Presentation
The topics covered in this presentation are:
| Let's talk about your business |
| What do you really want? |
| What is your opinion of Data Warehousing? |
| The situation to date |
| Introducing the Instant Data Warehouse |
| What is the Instant Data Warehouse? |
| What will the Instant Data Warehouse do for me? |
| How much is the Instant Data Warehouse? |
| Why would I invest in the the Instant Data Warehouse? |
To download this presentation please click here.
The Project Managers Presentation
The topics covered in this presentation are:
| Who is Peter Nolan? |
| Let's talk about ETL |
| What is the Instant Data Warehouse? - Summary |
| Who is the Instant Data Warehouse intended for? |
| What will the Instant Data Warehouse do for me? |
| What is the Instant Data Warehouse? - Details |
| How much is the Instant Data Warehouse? |
| What do I get for my money? |
| Why would I invest in the the Instant Data Warehouse? |
To download this presentation please click here.
If you are reading this section you have probably taken a look at the published
code on my Downloads page and wondered if
the C Data Warehouse Generator might be of value to you.
One of the reasons I have published the sample code is to demonstrate the
output of the C Data Warehouse Generator. The C code on my Downloads
page is actually generated by the C Data Warehouse Generator.
The generator is intended for companies who fit into one or more of the
following categories:
| You are building a star schema DW on a tight budget.
| Which is most companies at the moment. |
|
| You want to use existing skills and a very standard language to build the
Data Warehouse. |
| You have decided not to purchase an ETL tool. |
| You have bought an ETL tool and want to augment the ETL tool with features
of the C Data Warehouse Generator. |
| You are using DTS but want to augment DTS with the features of the C Data
Warehouse Generator. |
| You have professional programming staff available to perform data
warehouse development. |
If you feel you fit into one ore more of these categories I have written a
presentation you might like to browse through.
The C Data Warehouse Generator Presentation
The topics covered in the presentation are:
| Let's talk about ETL |
| What is the C Data Warehouse Generator? - Summary |
| Who is the C Data Warehouse Generator intended for? |
| What will the C Data Warehouse Generator do for me? |
| What is the C Data Warehouse Generator? - Details |
| How much is the C Data Warehouse Generator? |
| What do I get for my money? |
| Why would I invest in the the C Data Warehouse Generator? |
To download the presentation on the Instant Data Warehouse please click here.
| 
