'Requirements and procedures for the successful deployment and maintenance of a Data Warehouse and Business Intelligence environment'
The articles describe the requirements for developing, deploying and maintaining a Data Warehouse environment including Business Intelligence solutions within the contexts:
Change/ Problem Management (the process of managing changes and problems, in other words, how does a customer and/or a user submit a change/problem request, how is it prioritized, tracked and implemented?).
Configuration management (the versioning, labeling and tracking of code and content components during the development life cycle)
Release Management (it designates and schedules software and hardware release components to migrate it to the live environment as a set).
We distinguish different types of Business Intelligence requests:
Incidents (IN)
General Questions (GQ)
Problem requests (PR)
(Ad hoc) reports (AQ)
Change requests (CR)
Data Manipulations (DM)
Emergency requests (ER)
The requests have origin in the user. They, initially, always refer to the user layer (reports \ dashboard\ reporting tool) and not to the architecture that creates the product for the customer. A request can also arise in the Data Warehouse team but it can always be allocated to one of the above request types and must always be treated as if they come from a user.
If the problem/change management tool is not capable of storing sources, attachments of requesters must be filed in a content management store so developers can access it.
Non CR’s are raised as an issue/ticket in a problem/change management tool with a certain priority. Change requests can be raised as a project or as an issue (depending on the size of effort). Incidents, problem requests, ad hoc queries and general questions can be raised as a single ticket or can be assigned to a project (related to a CR). SLA stuff. High tickets not only need to be entered in the tool, but has to be communicated directly to the assignee to guarantee that it will be picked up immediately.
User support must be equipped with an intake form to register the call as complete as possible to avoid misunderstandings. User support must be trained to distinguish the different types of requests.
INCIDENTS
Incidents refer to an offline or error mode of the user layer. These are technical questions, not related to data.
Offline mode
In this case the product is not present:
The (web) portal is not accesible
The report is not refreshed
The report is not visible (on the web or on the network)
Error mode
In this case the product crashes:
The user can not log in (security problem)
A canned (olap) report can not be opened
A non-canned report won’t run or is killed
An error message appears, but the result seems to be fine.
An error appears when a power user creates a new report.
User Support reproduces the incident (with or without terminal services) and creates a solution if a user performed incorrect actions (or in case the user forget his authorization code). User support must possess a user configuration and power user configuration to be able of reproducing the incident. If the user acted correctly an high priority ticket must be raised. In case of error 4 a low priority and in case of error 5 a medium priority is sufficient. If the problem concerns all users (e.g. offline mode of the webportal) communication must take place to the users. In other cases user support should report the incident to the ower of the report (key user). In case of error type 5: In this case User Support is the problem owner if it concerns an error related to the tool itself, its configuration, or if it’s a database connection problem. Development issues of the power users should only be managed if the meta-data repository (i.e. Cognos Catalog, Business Objects Univers) is maintained by the data warehouse team and the user cannot create new definitions. If not, the reports can not be supported and maintained.
All users (and members of the data warehouse team) are allowed to report an incident.
In most cases the tool administrator is concerned with it. In case of a non-refresh of a cube/report it’s possible that other processes than the front-end tools are evolved (network problems, database problems). In that case the tool administrator needs the help of the system administrator/database administrator. To maintain the knowledge base the problem and solution must be documented.
GENERAL QUESTIONS
General questions concern:
- (power) user training
- request for authorization
- end-user-tool functionality explanation
- report functionality explanation
- definition explanation (i.e. ‘How is revenue calculated’, ‘What’s a work breakdown structure’?)
User Support should answer all these questions. User Support should have documentation to deal with this kind of requests. If it concerns definition explanation, they can refer to the key user who should be equipped with the data warehouse definition handbook. If such a handbook does not exist or the definition is not entered in the book User Support can raise a ticket and assign it as low priority ticket to administration. Training is planned by the team lead of User Support (or in case of no competence, the team lead of administration).
Only Key users may raise general questions.
To maintain the Frequently Asked Questions (FAQ) the question and answer must be documented.
PROBLEM REQUESTS
A problem request arises when – according to the user - a report does not correspond with the original specifications. Other data is reported then expected. If the data warehouse team is organized in the way that sources in production are maintained by administration a ticket will be assigned to the administration manager. If development is also working on production sources the ticket will be assigned to the development manager or directly to a certain developer. If the data warehouse team is implemented with substantial functional en technical documents with configuration management and release management procedures it’s possible to hand over development knowledge to an administration team. If not, production development should stay at the original developers (although, in a practical situation developers leave, roles don’t). The administration team can also report a problem request (i.e. reconciliation outcomes). The owner or the report (key user) needs to be reported.
To equip the assignee with all information and for maintain purposes the problem information (i.e. attachments of error messages) must be saved in a content management system (and if possible in the problem/change management tool).
In case of a problem report or a change request the developer
- contacts customer to make an inventory of the specifications;
- writes a functional design;
- makes a planning and gives feedback to requester;
- delivers functional design to requester;
- chases requester to approve the functional design (redesign can be result);
- adapts the design in case of disapproval;
- executes feasibility study (if not attainable requester should get well founded reply).
- writes the technical design in case of approval;
- a colleague judges the technical design (redesign can be a result)
- checks out all necessary sources.
- adds the new sources to the source id document;
- develops on the client (with connection to the development database\dwh)
- creates test specifications
- checks in all sources;
- writes the deliver-to-administration document so administration can publish all sources;
- promotes the sources to test landscape so testers can perform technical tests.
- writes an migration request and acceptance test criterions.
- request administration to promote (after approval) the source to acceptance and asks test management for an acceptance test request
- creates/maintains technical documentation;
- hands over input to User Support create/maintain user documentation/training material;
- request administration to promote (after approval) the source(s) to production and communicate to requester that problem is solved.
AD HOC REPORTS
Sometimes users wish to have once-only information, which is not provided by the existing reports. We distinguish two kinds of ad hoc reports:
User needs real-time, detailed information.
User needs detailed/aggregated information (currency equals frequency of the data warehouse refresh or less)
Ad 1. In this case a ticket must be assigned to a (sql-) developer in the development team. If a report (which meets the need of the requester) already exists in the application of the source, User Support must refer to this. The (sql)- developer must be able to connect to the original source because a data warehouse cannot provide real-time information. If User Support / development finds out that the query is similar to an existing request (i.e. with an another filter) User Support can advice the user to let it published as a structural report. Then a change request has to be raised.
Generally, these kinds of ad hoc reports have a high priority character because it’s a real-time need.
The developer saves the query, the result of the query and the metadata so User Support has access to it and can reuse it.
Ad 2. If the report already exists User Support must refer to this. If not, User Support can analyse whether the report can be deployed on an existing OLAP-cube. If so, User Support creates a ticket and assigns it to themselves. User Support must be aware that it should be a task of the user (as much as possible). If the report cannot be deployed on an existing OLAP cube and User Support assigns a ticket to a data access developer. User must confirm that the request is a once-only request. Otherwise, a change request has to be raised.
In case of assignment to development a data access developer builds a query on the source system (in case of a real time need) or on the DWH (not real time).
The developer
- contacts customer to make an inventory of the specifications;
- adds the new sources to the source id document;
- develops the query on development;
- asks requester to evaluate the draft;
- runs the query on production;
- saves result and query in content management system;
- maintains definition repository.
A migration to production process through change management won’t be used because of the high priority character of an ad hoc report request. In fact, nothing will be published but the result will be send to the requester (i.e. in xls format). If a report on a cube is published, the cube model itself has not changed. Users themselves can create reports on cubes and publish them without using the acceptance environment.
CHANGE REQUESTS
Change requests cannot be raised by all users. This is only admitted to the owner of a (set of) reports (a key user) or the lead of the User Group if the request concerns more key users.
A change request is a request for new information that is not provided by the existing report/cubes. If a change requests has a certain cost (i.e. above € 10.000) it will be classed and managed as a project. If less it can be raised as an individual ticket, assigned to development.
In case of a problem report or a change request the developer
- contacts customer to make an inventory of the specifications;
- writes a functional design;
- makes a planning and gives feedback to requester;
- delivers functional design to requester;
- chases requester to approve the functional design (redesign can be result);
- adapts the design in case of disapproval;
- writes the technical design in case of approval;
- a colleague judges the technical design (redesign can be a result)
- checks out all necessary sources.
- adds the new sources to the source id document;
- develops on the client (with connection to the development database\dwh)
- creates test specifications
- checks in all sources;
- writes the deliver-to-administration document so administration can publish all sources;
- promotes the sources to test landscape so testers can perform technical tests.
- writes an migration request and acceptance test criterions.
- request administration to promote (after approval) the source to acceptance and asks test management for an acceptance test request
- creates/maintains technical documentation;
- hands over input to User Support create/maintain user documentation/training material;
- request administration to promote (after approval) the source(s) to production and communicate to requester that problem is solved.
DATA MANIPULATIONS
A Data Manipulation is a change of data in the Data Warehouse. Data Manipulations request cannot be raised by all users. This is only admitted to the owner of a (set of) reports (a key user) or the lead of the User Group if the request concerns more key users. In most cases definitions are dynamical. In some cases a definition is determined by hard-coded data. For instance, the rule to determine whether a cost object is billable or not, cannot be automated. Regularly the cost objects must be updated in a specific table. Another example is the definition of the bridging day: there is no formula to calculate this, every year the policy can change. If possibly, the requester must have the means to do it himself (i.e. ASP).
Changes must be maintained in the Data Warehouse Definition Handbook.
EMERGENCY REQUESTS
In case of emergency it can be needed to publish a change as soon as possible in the production environment. In this case the data warehouse team manager has to approve. This kind of requests has origin in the development team. It’s an accelerated migration to production procedure. If necessary, technical documentation can be written afterwards. To check whether everything went ok a (short) evaluation with the DWH manager is needed.
Furthermore an emergency request can arise if a user wants the result of an ad hoc report immediately. The team lead for data access has to approve.
