Background

The success of a BI and Analytics system or platform is characterised by how well it has been architected and built, by considering a variety of factors. Some of these are listed below:

  • Data access: Does it account for volume, variety, velocity, veracity etc of all types of relevant data
  • Modelling and analysis: Does it support data and mathematical modelling followed by analysis? Ease of development by end users: Can end users themselves build reports and analytic use cases?
  • Humans control the process: Given that it is important to consider this as a user driven process, how much of a control do humans have in the overall platform build and use?
  • Stand-alone, integration, and Web-based: Does it support all these three modes of use? Semi structured or unstructured problems: Sometimes the business decision making is around solving structured problems where all boundary conditions are known. Sometimes not. Does the platform support even such cases?
  • Support: Does the system support managers at all levels? How about individuals and groups?
  • Variety of decisions supported: Does it support interdependent or sequential decisions?
  • Process support: Does the platform lend itself to be used in all stages of the BI and Analytics process including intelligence, design, choice, and implementation?
  • Variety of decision processes and styles: Does the platform support diverse decision making processes and styles in different enterprises?
  • Effectiveness and efficiency: How performing and cost optimal is the platform?
  • User experience: Is the platform Interactive, easy to use, adaptable and flexible?

It may be noted that not all the above features can be embedded in enterprise class BI and Analytics platforms. However, it may be attempted to incorporate some of the very important features as deemed by the organisation, during the design phase itself. This will ensure quicker adoption and use of the platform itself.

Technology in Action

A typical enterprise class BI and Analytics Platform is built on the architecture shown in the figure below. As indicated, all internal data from enterprise systems such as ERP, CRM including Finance, Marketing, Production, Personnel and other functions are first ingested. Data from these systems can be ingested with well known API’s. On the other hand external data such as macro economic indicators, weather, etc also may be very relevant for the BI process. These are also selected, sourced, and then appropriately ingested. Many times these could either be batch uploads or real time ingested if it is from social media. There is also typically an enterprise scale knowledge base that helps in documenting and storing the nuances of operations and decisioning. The data that is ingested is typically subject to standard ETL processes. A couple of use cases arise: One, a Corporate data warehouse, which is a structured data holding only as much as required for operations and related decisions. Second, a Decision support database, usually taking the form of a Data Lake, which is much larger in size and tends to have both structured and unstructured data and information. It is important to build in features like querying and data dictionary services for the data lake as it is not as robust as the Data warehouse, which is usually a read only type of system for end users. The data lake then supports adhoc procedures for data retrieval, inquiries, updates, report generations and the like. It may be noted that any analytics and related models work on the data stored herein.

The role of AI in BI

With the proliferation of data and their increasing speed of availability, traditional approaches for performing ETL in batch and creating canned reports is a time consuming and hence non-effective way for delivering reports to managers. The inability of the central CIO function to quickly adapt to changing requirements from business users is best dealt with by embedding a self service based approach for report generation. Ideally, a natural language processing and natural language generation engine will be required to serve this need. For instance, the business analyst may simply type an English statement like “Show me the last 3 quarters sales of Product X for all HNI customers who have highest risk of attrition” and the NLP engine will automatically figure out the schema and query to be built to fetch this report in a dynamic manner. The NLG engine could speak out the results where required. These trends are now adopted by several off the shelf BI tools that support AI.

Conclusions

BI and Analytics are bound to ubiquitously used by all stakeholders within an enterprise across all decision-making levels. It is thus extremely important to ensure the architecture to support the myriad requirements is a robust, scalable, and performing one. Likewise, all the required analytical models and reporting schemas should be thought through and provisioned as part of the BI and Analytics platform. With the right combination of data technologies and BI technologies, embellished with business use cases, the ROI will only be very quickly realised by the business stakeholders.