Phases of Organizational Business Intelligence


What is your business intelligence level?


As you travel down the road of business intelligence, it is good from time to time to take stock of where you are at as an organization. Though organizations obviously differ in many ways, they usually take predictable paths of growth when it comes to the use of data. This guide should give you an idea of where you are at and what the next step is on your journey.

1. Paper and basic spreadsheets


The very first level of business intelligence is the spreadsheet phase. This phase encompasses organizations that are generally in the very early startup phase, perhaps the first year, with less than five staff members. In this phase the priority is often getting the very first customers, and all energy is directed to that end.

In this phase data is mostly being tracked on paper and in very basic spreadsheets. There is little data analysis going on because there’s little data to analyze. Because of the small size of the organization this fits the basic organizational profile well. Organizations generally stay at this level for less than two years – they either go out of business for lack of growth or necessity forces them to the next level.


Benefits of this stage:

  • Low IT Spend

  • Data is fairly accessible provided you don’t lose your spreadsheets or pieces of paper

Challenges of this stage:

  • No business rules on how data should be entered or validated

  • Lost pieces of paper

2. A few discrete point systems with little to no integration


If an organization succeeds in growing out of the first year or two, one of the first decisions made is how to keep track of basic data in a sustainable way. This usually means replacing some of the spreadsheet and paper based methods of tracking data with some free or low-cost data systems that enforce basic business rules and provide some nice features over the spreadsheet. Typically the first systems to be implemented are basic accounting/finance packages and customer relationship management systems.


These basic systems usually offer very simple internal reporting functions and that’s generally enough for the organization to function. Organizations in this mode are in early growth mode and there’s generally not a lot of effort put into analyzing data because there is still not enough data to really change how an organization does business.


The core leadership team in these organizations consists of a few people who are wearing many hats so data sharing is not particularly cumbersome. If the organization grows, however, and more hires are made, it begins to segment itself into departments. That’s where the third level of data organization kicks in.


Benefits of this stage:

  • Systems that enforce some level of data integrity

  • Some packages have basic data analytics functions for the data stored in that system.

Challenges of this stage:

  • Data silos begin to develop – no integration between packages

3. Established packages for most major systems, basic integration


As an organization grows it begins to segment into discrete departments that function somewhat independently, and these departments all start to purchase packages to manage their job functions. These packages are likely to include the following:

  • Accounting/Finance package

  • Customer Relationship Management Software

  • Marketing Campaign Management Software

  • Website Content Management and Analytics Software

  • Inventory Tracking

  • Customer Support/Ticketing Software

Also, beyond the packages that are purchased, departments create spreadsheets and small departmental databases to support any function which is not easy in their core system. Each department in a sense becomes a mini-organization of its own and recapitulates phases 1 and 2 trying to find the right set of tools to run as a distinct department.


At this phase the organization’s founders and executives have now shifted their focus from trying to create momentum to managing the momentum that has been created. Looking at the bigger organizational picture requires gathering data from all the various departments, and this is also where the first real pains of data analytics begins to be felt.



The process of collecting data at this stage tends to follow the following template:

  • Senior executives request certain data from department heads

  • Department heads go into their data systems and export data into spreadsheets (if export functions are available) or manually enter data into spreadsheets

  • Spreadsheets are sent to senior management for review

  • Senior executives or some administrative assistants take the spreadsheets and merge them into bigger spreadsheets.

  • Basic formulas and graphs and charts are created based on these spreadsheets.


This procedure is time consuming and so it usually only happens a few times a year. The taste of real data analytics that this process provides spurs the desire for more real-time data analysis. However, the pain of producing the spreadsheets leads organizational leaders to understand that if they want more timeline information they have to hire people whose full time job revolves around producing that data.


Benefits of this stage:

  • Departments generally have a dedicated system in place to handle their departmental data

  • Lots of data is being tracked so there is enough material for analysis

Challenges of this stage:

  • No “seamless” integration yet built between systems

  • Cumbersome and time consuming to get all the data needed to make decisions.

  • No “data specialists” who are proficient at getting data into usable formats

4. Hiring of business analysts


Once the organization grows to a certain point there is a realization that analyzing business data is time consuming enough that specialized staff should be hired. The organization then brings on board the first business analysts who are tasked with the collection and analysis of all of these spreadsheets that are flying around the organization. This frees up executives and department heads from the more arduous tasks of data analysis, but it also highlights the need for more data oriented IT infrastructure.


The newly hired business analysts are armed with a series of tools – Excel, Tableau, PowerBI, even perhaps Jupyter Notebooks. However, the realization soon sets in that all the ANALYTIC tools in the world can’t save an organization from the pain of having to have real data governance and data strategy policies that include tools and workflow to consolidate key data into one location.



This pushes organizations to the next phase – the creation of organizational data warehouses and data governance procedures.


Benefits of this Stage:

  • More staff to help with business intelligence

  • Data visualization tools make building complex reports easier.

Challenges of this Stage:

  • More people in the data analysis pipeline means it can take even more time time to define and create reports

  • Organization spend on data analytics goes up with the acquisition of people and tools

4. Data Warehouses / Data Marts / Data Governance


As the organization grows it begins to grapple in a serious way with questions of data governance.

Some of the issues that push an organization in this direction are as follows:

  1. More real-time data is needed to make decisions but it is still a slow process to get data from departments.

  2. Data security becomes a serious issue – the organization needs to analyze sensitive information regarding employee data, healthcare data, proprietary process data – and not everyone should have access to the data. This means that policies and systems to enforce those policies have to be put into place.

  3. The organization may be exposed to certain regulatory frameworks (environmental, national security, etc) that require certain data to be reported on a regular basis.


The business analysts tasked with producing reports quickly begin to discover the benefit of having some sort of centralized data store that they can connect to using their reporting engines. Thus the first organizational data warehouse is born. This creates a pivot in data strategy where individual departmental data systems are looked upon more as terminals for data entry but the real analysis takes place in the data warehouse system and the surrounding toolsets.


Creating a data warehouse is no easy task and it can take years to create one. However, a good data warehouse enables decision making on a much higher level.


Benefits of this Stage:

  • Finally a centralized place to store and get data across the organization.

  • Data warehouses bring increased redundancy and security to organizational data.

Challenges of this Stage:

  • Data analytics spend goes up significantly in terms of people, tools, and cloud

  • Programming resources are often necessary to take advantage of this stage.



5. Creating a data lake


Eventually if the volume of an organization’s data grows at a fast enough pace it becomes clear that there are limitations even on the organization’s data warehouse that must be solved. These limitations stem from the fact that relational data warehouses need some massaging and management, but the rate at which data grows often dictates the need to “just throw the data somewhere” until it can be processed further by analytics tools. It also isn’t strictly necessary to load all the organization’s data into a data warehouse; cloud based data engines are making it easier and easier to create “virtual databases” atop high volume text data. This leads to the formation of a cloud based data lake that can serve as the receptacle for the data until it can be processed into more useful formats.



An organizational data lake opens up a whole new set of benefits for the organization. One of the biggest benefits of a data lake is the shift of mentality from loading THE data warehouse to loading many different data marts and data warehouses depending on the particular security and functionality needs.


Benefits of this stage:

  • Finally a place to “dump” data until you decide what to do with it.

Challenges of this stage:

  • Easy to go crazy on cloud spend if you aren’t careful

  • Data lake can turn into “data swamp” – data is unstructured and hard to reconcile.

  • Data retention policies have to be looked at and applied to data lake.

6. AI/Machine Learning algorithms


The end state of business intelligence often happens when there is so much data streaming into the organization that it becomes necessary to enlist the help of machine learning algorithms to help make sense of things. These algorithms expend tremendous processing power to extract insight from the largest of data sets.


What generally happens is that data lakes are used to store raw data in text format until a tool such as TensorFlow is deployed over the data. Cloud systems such as AWS, Azure, and Google Cloud Platform offer systems where an organization can pay for compute time to create models.



Benefits of this stage:

  • Unexpected insights often come out of machine learning algorithms

  • Models generated can significantly impact business and possibly allow full or partial automation of certain business functions.

Drawbacks of this stage:

  • Significant spend on computing resources to crunch data.

  • Staff with machine learning expertise are very expensive



Where are you at in your data journey? With our broad range of services, we at Synthelize can join you at any stage of your technological growth and help you get to the next level.

Contact Us

322 North Shore Drive
Building 1B, Suite 200
Pittsburgh, PA 15212

412-230-8552

I would like info on the following services:

412-230-8552

  • LinkedIn

Linkedin

  • Facebook

Facebook

  • Twitter

Twitter

Providing Data Services and Business Intelligence Solutions to organizations of all shapes and sizes

Data Strategy

Cloud Migration

Big Data, Data Lakes & Data Warehouses

Data Integration

Data Visualization

Business Intelligence

Data Governance

Machine Learning & IoT

322 North Shore Drive
Building 1B,  Suite 200

Pittsburgh,  PA 15212

+1  (412)  230-8552

  • LinkedIn
  • Facebook
  • Twitter

©2020 by Synthelize. All Rights Reserved

  • LinkedIn
  • Facebook
  • Twitter