BI, Simplified!
Business intelligence (BI) has two different meanings related to the use of the term "intelligence", first is the human intelligence capacity applied in business affairs/activities and the second relates to the intelligence as data assets i.e. information valued for its currency and relevance.
The Business Intelligence, thus might be defined as interactive cohesion of multiple human and non-human tenets that deliver two key objectives, it uses the Datafication framework and Algorithms to deliver business insights in predictive and/or prescriptive ways ( Read, outcomes) to provide meaningful context. This Context enables better-informed decisions that improve performance, create new strategic opportunities for growth, gain competitive market advantage or simply knowing the state of business, markets and customer dynamics thereof. As famous Sun Tzu once cited .
"...what enables the wise commander to strike and conquer, and achieve things beyond the reach of ordinary men, is foreknowledge"
In a nutshell BI is Foreknowledge, all about accessing, harvesting and knowing thus transforming your organization's current data assets and more concerned with the what's and the how's than the why's. The Whys can further be explored with Business Analytics (BA) or Data Science driven by set of specialized algorithms, tools and services, that engages statistical analysis and predictive analytics, real-time modelling etc to establish future trends and possible patterns that helps businesses figure out why things are happening the way they are and making an educated guess about how things will pan out in the future ( Read, forecasting!).
The BI deals with historical data till present showcasing what happened or happening and BA deals with futuristic outlook or trends that suggest what to expect, both consume organizational data assets in different context and ways!
Now that we know a clear distinction between BI and BA, let us focus on BI who's sole purpose is to support better business decision making and remained informed of current context.
Keep it Simple, BI is not Data Science
We talked about the benefits of BI system adaptation for a while, since over time the tools and solutions have become less expensive and thus have a greater base as a useful way of ingesting and collating business data and correlating that in varied datasets in a way that will provide useful insights and observations.
We still need to gain clarity between BI and Data Science, when the traditional BI works hand in hand with predictive analytics, it helps derive future predictions that will deepen the understanding of say your customers, intrinsic comprehensive responses to deliver greater user experience and competitive advantages as one of the example. There are quite a few areas that will help prove our point.
- Since the BI ecosystems help business look backwards based on real-time data points from real transactions, interaction and events.
- Data Science ( Read, Business analytics master set) in a way helps build on this theorem looking forward, setting up forecasts, trends and thus helps interpret logical modelling of the insight to predict the future possibilities. Its all about Perspective, isn't it?
- The reports, KPIs and trend charts delivered by BI ecosystem might not provide guidance on the patterns and designs that may take into account the various inputs and apply algorithms unless driven by data science framework, this is indeed another distinguisher
- Also to note traditional BI systems were static or rigid on comparative sets of data, once the dataset is build ( read, data cubes) it was difficult to bring about the change in the parameters the dataset is built upon.
- The data science modelling can easily parse the data into virtual data sets built point in time via data transformation capabilities and deliver results that are dynamic as the underlying datasets changes.
Five "C" - Factors Influencing Business Intelligence
The BI influencers to the business can be classified into five "C"s, they are Customers, then Competitors further Collaborations, Current Context and finally Cost Structure. It is thus obvious that any business that will take up BI as a strategic imperative will harvest these as benefits derive from the initiative.
Customers are the most critical aspect to a business success, without knowing them no business can succeed . Will it not be prudent to know your customers, their preferences, and their changing demands. BI enables this insights that will foster retention, acquisition and stickiness of customers proactively via current marketplace, innovation and adapting new products and services and come up with innovative strategies in anticipation of customer's changing demands
Competitors can make and break your equations with markets, customers and profitability and can be your biggest barrier for growth and success. Their objectives are the same as yours and that is to maximize profits and customer satisfaction. BI can provide you insights on the market trends, expansion and who's leading the curve and can help you succeed and stay one step ahead of your competitors taking informed decisions. No business wants to play the catch up game possibly losing valuable market share.
Collaborations are key for Go to market strategies ( Read, GTM) and key business partners must possess the same strategic information among themselves so that there is no miscommunication that can lead to inefficiencies or loss of opportunities. The integration of supply chain can be a good example for sharing current inventories, reorder levels and production / delivery schedules along with BI enabled performance metrics, KPIs and other supply chain data in order to collaborate externally towards continuous improvements and agile / dynamic logistic operations.
Current Cost Structure, bottom-line baggage ( read, liabilities) economic & regulatory environment, key economic indicators are important considerations when making key business decisions and evaluating market potential. This insights are vital while planning a roll out of new line of products or services during an economic recession, changing regulatory framework. BI provides data points and on the state of the economy to make informed and prudent decisions as to when is the right time to maybe expand or scale back your business operations.
Current Context is all important baseline of the business, what happening on the floor, internal operations, efficiencies at all levels, in depth knowledge about the internal workings of business functions from top to bottom. If one makes an arbitrary decision without knowing how their entire organization works it could turn out to be a risk that can result in inefficiencies and delays on important business plans. BI can help establish score cards, data points about the real-time KPIs and information on how the entire organization and partner ecosystem works and the yields.
The BI Stack, explained!
Since the foundation of BI is resting on the Key performance indicators (KPI) that present and prescribe real-time current course of actions and helps recalibrate goals / strategies making business more agile and adaptable. The technologies that fuel this adaptation and contribute more and more success stories of BI use cases has a major bearing on this initiative. Let us look at the big picture, the diagram below explains the logical landscape from the point of "proof of value".
The complexity of the above landscape wrt, BI is further classified into four key components,
- Data Hub / Warehouse - a fundamental for any BI ecosystem, the enterprise data hub is a concept that acts as a data repository and designed specifically for data analysis instead of standard transactional processing. It acts as the conduit between operational data stores and the gaining of insight based on composite data. Slices of data from the warehouse-usually summary data for a single department like sales or finance-are stored in a "data mart" or columnar databases for quicker access. Read more on why have an enterprise data hub?
- Extract, Transform, Load (ETL): The first important task is to stream/ scoop / extract the data from the various data sources and load/ingest it into a data warehouse where it is normalized (organized into tables while cleaning the data and removing redundancy and inconstancies). Once it has been appropriately structured it is available for querying and analysis. The modern outlook to ETL/ELT is now converging with BigData systems such as Hadoop. ETL jobs that make simple aggregations, summations, and calculated values in large volumes and scale of millions of records are well-suited use cases for Hadoop/Spark environment to deliver, and these challenges can be deployed for a fraction of the cost of a high-end ETL tool currently in the market. Some ETL jobs need basic relational capabilities and can be supported as Hbase row store or a Hive table. The Cassandra, PostgreSQL are the running higher adaptation curve simply because of their ability to perform ETL/ELT legacies far more effectively and in real time supporting most BI tools via ODBC/JDBC connectors.
- OLAP or ad-hoc query tools: OLAP (Online Analytical Processing), and its close cousin ROLAP-(relational) online analytical processing, is a technology that allows users to query data across multiple dimensions, for building standard reports or for enabling users to ask a specific business question. It is the foundation for many kinds of business applications for Business Performance Management, Planning, Budgeting, Forecasting, Financial Reporting, Analysis, Simulation Models, Knowledge Discovery, and Data Warehouse Reporting. OLAP enables end-users to perform ad hoc analysis of data in multiple dimensions, thereby providing the insight and understanding they need for better decision making. Apache Kylin is an open source Distributed Analytics Engine designed to provide SQL interface and multi-dimensional analysis (OLAP) on Hadoop supporting extremely large datasets. Kylin is designed to reduce query latency on Hadoop for 10+ billions of rows of data and offers ANSI SQL on Hadoop and supports most ANSI SQL query functions. Users can define a data model, interact with Hadoop data via Kylin at sub-second latency, better than Hive queries for the same dataset and even offers seamless integration with leading BI tools such as Tableau.
- Presentation layer: Custom Dashboards, scorecards and reports presenting the data to business users in a visually appealing way that is easy to understand. In response to the great success of data discovery and visualization tools like Tableau and QlikView, many full-stack vendors have also built their own data discovery and visualization tools to render the insights as better user experience and usability thereof.
The BI landscape of offering's...
There are three key verticals we can segregate the BI enablers, the "End to End" stack vendors, The "Pure Play Vendors" and finally component vendors, it is left to the businesses to determine how they wish to engage and evolve their BI journey and stay relevant to technologies / markets and dynamic needs of the business. The following depiction can help map the market and the players but caution, this is an evolving field and most disruptive context within and outside the ecosystem thus lot of innovation and highly integrated solutions lurk in from nowhere and drive the changes, the open source stacks are as fast evolving to dilute to boundaries and go beyond the known limitations.
Technically all these products are cloud ready and offer pay as you go models in some or the other fashion yet these are limited by need to manageability and acumen of the user organization as the picture is now shifting from IT owned to business owned paradigm.
More about BI stack, technology and deployment model..
While we discussed the not technical aspects, it is prudent we also talk about typical deployment (Read, logical schematic) stack. The aspects that drive the BI are now seen interwoven with modern open source tools that are largely crowdsourced developed to solve much larger and much wider use cases...here is a nice illustration from Wiki, the verticals cater to different tenets of BI, Analytics toolsets and solutions.Each block represents a competency and varied options are now streaming to solve the use cases catering to each stage, as said earlier there are players who offer end to end respite and there are specific verticals that align to niche domains or verticals in the transformation process, for few tech savvy enterprises who are willing to invest and build the ecosystem themselves there are open source technologies or components to focus on refinements, all in all the complexities and evolution is fast paced and expeditiously expanding most of the times. The face of change is dominating the underlying technology and business users are averse in knowing what it runs on or where it runs on as long as it meets the security standards and efficient for access and ability thereof.
Though the ecosystem is modular yet tightly integrated offers great value proposition when it comes to the digital transformation, the verticals play key roles liberating the data from the clutches of legacy applications, transactional systems that hold the data into native silos. The approach is to evolve the legacy the OLTP, MIS and DSS system interconnects and make them cohesive retaining the value they bring to the table. Unlike the full stack systems the component based systems are thus agile and easy for technology refresh in bits and pieces providing better ROI. The impact of disruptions in this fast paced ecosystem can be dealt with ease and resource skilling, deployments ( Read patching and upgrades) remain granular. The most important aspect is open interfaces that allow standard API based interconnects between modules and data exchange and workflow handoffs thus delinks the stack from dependency on various applications, operating platforms, cloud or data center locations where they run and even the flavour of underlying hardware as it leverages commodity hardware rather than the monoliths. This is further even leverage micro services that runs "Code as a Service" thus higher optimization and better utilization of available resources adds up to the value chain.
In summary, the BI continues the change the way we use and even look at technology that drives business KPIs and help us take informed decisions, fundamentally focuses on data management and transformation which is collecting data, information which is transforming the data, derive knowledge ( Read, Insights) and action , take decisions yield outcomes, it is indeed a fundamental building block in your digital transformation strategy!!
