Business Intelligence Testing

Business Intelligence Testing

Enterprises uses various BI tools like Cognos, Oracle Business Intelligence Enterprise Edition (OBIEE), Tableau, SQL Server Reporting Services (SSRS) etc to analyse swathe of data and convert raw data into more presentable format like Dashboard, Key Performance Indicators(KPIs) to make well informed decision. The success of any BI project depends on: trust on the data shown in the BI dashboard/ repots, performance of BI system, data integrity, data security, easiness to use application and graphical representation of data.

Testing of large scale BI system faces challenges in the form of: Data Quality Assurance(DQA), metrics/aggregation rule test, data grain etc – Success of any BI system depends on the traceability of BI functionality to Business Requirement (BR), issue/anomalies fix log and how good test cases are developed.

What is BI testing:

BI testing plays a pivotal role in the success of large scale BI roll out in enterprises. BI testing ranges from: data validation, formatting, security, performance and Graphical User Interface (GUI). Emphasis on a thorough BI is pivotal for improving the quality of data, user interface and meeting business requirements.

Testing Strategy:

The purpose of BI/DWH/ETL testing to get credible data for end-users. The credibility of BI can be increased by making sure a comprehensive testing strategy was adopted.

A comprehensive test strategy is a stepping stone for the effective test cycle of BI application. Tet strategy should cover any stage of data flow to make that data input to data output is effectively tested.

To ensure testing readiness, following key area of testing strategy should be focused on:

• Testing Scope – Testing categories and types to be used
• Test Environment
• Test Data Availability
• DQ and Performance benchmark
• Issue/Enhancement log and traceability

Testing Categories:

DWH/ETL Testing:

ETL testing is quite different from conventional testing. There are many challenges we faced while performing data warehouse testing. Here is the list of few ETL testing challenges:

• Incompatible and duplicate data.
• Loss of data during ETL process.
• Unavailability of inclusive test bed.
• Testers have no privileges to execute ETL jobs by their own.
• Volume and complexity of data is very huge.
• Fault in business process and procedures.
• Trouble acquiring and building test data.
• Missing business’s data transformation rules.

Data is important for businesses to make the critical business decisions. ETL testing plays a crucial part validating and ensuring that the business information is exact, consistent and reliable.

Functional Testing:

When a new report or dashboard is developed for consumption by other users, it is important to perform a few checks to validate the data and design of the included reports.

Verify that the new report or dashboard conforms to the report requirement / design specifications. Some of the items to check are:

• Verify that the report or dashboard page title corresponds to the content of the reports.
• For reports with charts, the axis should be labelled appropriately.
• The aggregation level of the data in the reports should be as per the report requirements.
• Verify that the report or dashboard page design conforms to the design standards and best practices.
• Validate the presence and functionality of the report download and print options.
• Where applicable, verify that the report help text exists and is appropriate for the report content.
• Verify the existence of any required static display text in the report such as FOIA text.

Regression Testing:

BI Stress testing is similar to testing of any other web/desktop application with an objective to simulate concurrent users accessing reports with different prompts and understand the bottlenecks in the system.

A typical BI user will login to the system and navigate to reports (or dashboards), apply prompts and drills down to other reports. After the report is rendered, the BI User reviews the data for a certain time called think time – Conducting a stress test requires simulation of the above BI user behaviour concurrently for different user loads. So it is important to have a list of different user logins for the stress test. When executing the reports, each user can pick a different set of prompt values for the same report.

Security Testing:

BI applications also have authentication and authorization security requirements, often integrated with SSO (Single Sign On) or Lightweight Directory Access Protocol (LDAP) or other mechanism like Oracle Weblogic. The objective of security testing is to validate that the BI user’s access to the BI reports, subject areas and dashboards is limited according to their access levels. Access to the reports is generally controlled by application role or LDAP based security in the BI tool.

Single Sign On is often used as the authentication mechanism for BI applications in large enterprises – objective of this testing is to ensure that users are able to access BI applications using their single Sign On access (or windows authentication).

Report/BI Testing:

BI tools such as OBIEE and Business Objects empower the business users by providing the capability to create their own reports without the help of a developer. These tools generate the database queries automatically for the reports based on the measures and dimension – For OBIEE, this model is defined in the RPD while Business Objects stores the model in the form of a universe.  Business users can select any combination of dimension and measure attributes available in the subject area to come up with their own ad-hoc report.

About Author:

I have more than 10 years’ experience of Business Intelligence (BI) in two leading BI stacks (Oracle and Microsoft), cross platform data migration and integration. I always explore possibilities of integrating cross platform BI technologies for clients to stitch heterogeneous systems to save cost, cut waste and improve efficiencies.