Lecture2 - Rabieramadan.org

Data Warehouse Fundamentals

Rabie A. Ramadan, PhD 2

Your Assignment 

For an airlines company, how can strategic information increase the number of frequent flyers? Discuss giving specific details.



You are a Senior Analyst in the IT department of a company manufacturing automobile parts. The marketing heads are complaining about the poor response by IT in providing strategic information. Draft a proposal to them explaining the reasons for the problems and why a data warehouse would be the only viable solution.

2

Lecture Objectives 

Review formal definitions of a data warehouse

 





Discuss the defining features Distinguish between data warehouses and data marts Study each component or building block that makes up a data warehouse

3

What is a Data Warehouse? (a practitioner’s viewpoint) 

“A data warehouse is simply a single, complete, and consistent store of data obtained from a variety of sources and made available to end users in a way they can understand and use it in a business context” – Barry Devlin, IBM Consultant



“A data warehouse is a database of data gathered from many systems and intended to support management reporting and decision making” – Michael Corey et al, CTO of OneWarranty.com 4

What is a Data Warehouse? (a Classical viewpoint) 

According to W. H. Inmon (Building a Data Warehouse, 1992) “A DW is a subject oriented, integrated, time varying, non-volatile collection of data that is used primarily in organizational decision making.”

5

WHAT IS DATA WAREHOUSING 

A data warehouse is typically a dedicated database system for decision making that is separate from the production database(s) used operationally. It differs from production system in that:

• it covers a much longer time horizon than transaction • •

systems it includes multiple databases that have been processed so that the warehouse’s data are defined uniformly (i.e., ‘clean’ data) it is optimized for answering complex queries from managers and analysts. 6

Standard DB v. DW

7

CHARACTERISTICS

8

CHARACTERISTICS

9

Characteristics of a Data Warehouse

10

Characteristics of a Data Warehouse

11

SUBJECT ORIENTATION 

Data is organized around major subjects of the enterprise.

12

Subject Oriented 

Data warehouses are designed to help you analyze data.

For example, to learn more about your company's sales data, you can build a warehouse that concentrates on sales. 

Using this warehouse, you can answer questions like "Who was our best customer for this item last year?" This ability to define a data warehouse by subject matter, sales in this case, makes the data warehouse subject oriented.



E.g. claims data are organized around the subject of claims and not by individual applications of Auto Insurance and Workers’ Comp 13

Class Activity 

•

A data warehouse is a subject oriented. What would be the major critical business subject for : A local community bank as a business unit

Customer Profit Loans

14

Integrated 

Integration is closely related to subject orientation.



Data warehouses must put data from disparate sources into a consistent format. They must resolve such problems as naming conflicts and inconsistencies among units of measure.





When they achieve this, they are said to be integrated. 15

Non volatile 

Non-volatile means that, once entered into the warehouse, data are not changed/updated.



This is logical because the purpose of a warehouse is to enable you to analyze what has occurred.

16

Time Variant 







In order to discover trends in business, analysts need large amounts of data. This is very much in contrast to online transaction processing (OLTP) systems, where performance requirements demand that historical data be moved to an archive. The data are kept for many years so they can be used for trends, forecasting, and comparisons over time. A data warehouse's focus on change over time is what is meant by the term time variant. 17

Data Granularity

18

DATA MARTS 

Data Mart: A scaled-down version of the data warehouse

 

A data mart is a small warehouse designed for the Small Business Unit (SBU) or department level.



It is often a way to gain entry and provide an opportunity to learn



Major problem: if they differ from department to department, they can be difficult to integrate enterprise-wide 19

Data Warehouse and Data Mart

20

Data Mart and Data Warehouse

21

Data Warehouse COST 

Data warehouses are not cheap •

Median cost to create (does not include operating

cost) = $2.2M 

Multimillion dollar costs are common



Their design and implementation is still an art and they require considerable time to create. 22

Data Warehouse SIZE 

Being designed for the enterprise so that everyone has a common data set, they are large and increase in size with time.



Typical storage sizes run from 50 Gigabytes to several Terabytes

23

APPLICATION - DATA MINING 



Also known as Knowledge Data Discovery (KDD) Mining terminology refers to finding answers about a business from the data warehouse that the executive or analyst had not thought to ask

24

Data Warehouse Architectures

25

Data Warehouse Architectures: Basic

26

Data Warehouse Architectures: with a Staging Area

27

Data Warehouse Architectures: with a Staging Area and Data Marts

28

A General Architecture for Data Warehousing

29

30

Problems and Issues

31

Data Systems Supporting DW

32

Class Activity

33

Class Activity 

What are the main components of a data warehouse for your school system?

34

Project 

Egypt Election System

•

Governorates’ database system

•

Summarization System

•

Data Warehouse Server

•

Web page with query based system

• Multiple databases on Multiple Servers • Meta data

35