Tuesday 10 March 2009

Required landscape in a DWH environment

This post is the second article with title

'Requirements and procedures for the successful deployment and maintenance of a Data Warehouse and Business Intelligence environment'
The articles describe the requirements for developing, deploying and maintaining a Data Warehouse environment including Business Intelligence solutions within the contexts:
Change/ Problem Management (the process of managing changes and problems, in other words, how does a customer and/or a user submit a change/problem request, how is it prioritized, tracked and implemented?).
Configuration management (the versioning, labeling and tracking of code and content components during the development life cycle)
Release Management (it designates and schedules software and hardware release components to migrate it to the live environment as a set).



One of the most important aspects of building Business Intelligence solutions is the requirement to build and support multiple environments, which allow developers and testers to:

• Make changes to the (etl)-scripts/reports
• Add new subject areas
• Add or update dimensions
• Verify and test the accuracy of the data
• Experiment with joining/conforming subject areas

without effecting each other and/ or the production reports/data.

At least four environments are required:

• A development environment where data analysts, ETL specialists, and report writers could add new or enhance existing components of the warehouse and test their changes.
This can be used a system test environment. In needed a separate system test (T) environment can be set up.
• An acceptance test environment which would be used to perform the final verification of data and reports prior to deploying into production.
• A production mirror environment. This environment mirrors production reports, web pages and data. This is used as fallback.
• Production environment

To maintain stability, development has only access to the development/test environment and administration (and no one else) to acceptance and production. In case of an emergency request or an ad hoc report this can differ.

Depending on the change frequency, the refresh of the acceptance environment will have the same frequency as the production environment or will have a refresh every time a change is published.

For training purposes a separate environment (R) can be set up. Usually, acceptance can not be used for this purposed (unless the performance is ok).

Below is a graphical presentation of an advisable environment (fallback machines are not presented, every server is intended to have fallback-mechanisms).
Depending on the intensity of use a less server configuration is sufficient. A single server configuration can only be advised for a development environment.

landscape

About tooling: To minimize administration problems a single-product-configuration is advised. Nowadays Business Intelligence products cover the whole range of Data Warehouse requirements. Cognos has its own ETL-tool, Informatica delivers high-end reporting products, Oracle and Microsoft are doing well in Business Intelligence and SAP BI brings together a powerful business intelligence platform and data warehousing capabilities.