Design BiAnl- Business Intelligence - Analytics

Explaining - predicting, processing operations.

The world of BI and Analytics is challenging.
The Application Life Cycle model2 is underpinning decisions for in the boardroom. To know what is going on and what is needed, distance on details is required.

🔰 Most logical back reference.

---

Contents

Reference	Topic	Squad
Intro	Explaining - predicting, processing operations.	01.01
Why Bi&A (I)	Why BI & Analytics (I).	02.01
Why Bi&A (II)	Why BI & Analytics (II).	03.01
BI_proces	Business Intelligence Proces.	04.01
ANA_proces	Analytical Proces.	05.01
What next	Inventing changing Operations.	06.00
	Combined pages as single topic.	06.02

---

---

Progress

THis chapter is partially fresh, partially converted. Topics are:

• 2019 week 41
  1. Changed location to a sub page.
  2. Realizing Analytics, optimizing the proces is like: lean TPS.
• 2019 week 18
  1. Adjusting page frame.
  2. Realizing ALC model2 can use analytics for reporting, but that is different to using analytics following ALC model3.
• 2019 week 15
  1. This page set up with old illustrations, added Machine learing versus BI.
  2. To do: Check what is missed and still relevant to convert.

---

Why BI & Analytics (I).

Supporting Business proces optimization.
Business Intelligence (BI), Management Information System (MIS), Executive Information System (EIS) are alle the same, just using different words.
 

Informing the decision makers (business).

The goal is only informing management with figures so they can make their mind up with what do in the future.
The spreadsheet use by mangement doing the analyses for decisions.

At other layers "BI & analytics" is used:

• optimizing the middleware layer. Servicing the business balancing performance and cost factors. Optimization of ICT.
• The datacenter optimizing using the machines, analyzing possible generic security threats. Optimization of ICT.

BI & Analytics is not a "one size fits all" solution.

---

Informing by descriptive reporting.

In the years before BI was normal, the standard reports were delivered as being part of the proces job log. The reports being printed on paper were archived for a long period, requiring a lot of space and cabinets.

Extracting those numbers, figures, was later done by archiving those prints in an electronic way (datasets). The conversion of those electronic datasets was the first dwh being build. The spreadsheet being a key enabler replacing most of manual work.

📚 short list:
* standard
* ad hoc
* drill down
* alerts

---

Bi as the next thing of report log

First BI&A usage:

1. Moving the evaluation log to BI tools
   ⚙ Every Job did have an associated rprot printed what has happend.
   ⚒ The change into collection of those reports into:
   📚 " Intelligence".
   

---

Why BI & Analytics (II).

Analytics, Operations Research (BI) is coming wiht a lot of uncertainties. Monte Carlo simulations being optional.
 

Informing by predictive, prescriptive reporting.

This is augmenting reporting, doing extrapolations, regresions in probablities what could happen.

⚖ short list:
* statistics
* forecasting
* predictions
* optimizations

---

Analytics and Bi differences

Different type of BI&A usage:

1. Operations using BI tools
   ⚙ No preparation on what is needed neither evaluation what has done.
   ⚒ Focus: running the operations.
   


2. BI, Business Intelligence (dashboard-report)
   📚 Information is being gathered wiht a discussed goal to achieve.
   ⚙ ⚒ Data is being processed. Prepare for the goal.
   🎭 Results are evaluated, planning what do next.
   


3. AI, BI enhanced with analytical tools (dashboard-report)
   ⚖ ⚙ Information is being gathered having no clear goal to achieve.
   📚 ⚒ Modelling - analyzing using data, to find new unknown opportunities.
   🎭 Results are evaluated, planning what do next.


4. AI, ML Machine Learning, operational scoring
   ⚖ ⚙ Information is being gathered with a discussed goal to achieve.
   📚 ⚒ Modeling - analyzing data for the best approach.
   🎭 Results associated wiht tested models are evaluated.
   ⚙ Data is processed using accepted chosen models.
   

---

Business Intelligence Proces.

The classic approach is delivering just reports dashboards.
The historical connotion is that a dwh DataWareHouse has an 1-1 association wiht this approach.

ALC model2 BI&A.

In this infographic flow:

• There is a need to build a DWH (DataWareHouse).
  Being the only customer for a single purpose it is not a shared service.
  No As A Service approach (DWH-AAS).
• The delivery is a dashboard (type of report) informing the customer that is responsible for continuity and / or change.
• There are many intermediate steps in extracting to an partial area of the information (datamarts), using possible many tools to achieve the presentations. Those many intermediate steps easily causing delay in execution - delivery moment in time.
• Results are not delivered back electronical to operational systems.
  The dashboards/reports serve as arguments to change the operations by decision makers.
• The dashboard/report only validated by manually review, segregation in duties.
  There is no versioning release management with a DTAP.
• The data modelling, dedicated for each, is rather strict being as least in 3nf.
  Not allowing for incomplete inconsistent data.
• Time / dates is the standard horizontal axis when plotting them in defined fixed time periods.

Using Bi-tools

💣 The tools originally reserved for BI only use have become far more generic. With the type of BI&A usage there is already a list of four different type of users a "datawarehouse" - "data lake" should able to provide.

---

Analytical Proces

Automatization of decisions within predefined settings and predefined limits of variation. Any automated decision has to be explainable and able to possible be correct by human intervention. (profiling GDPR)
 

ALC model3 BI&A.

In this infographic flow:

• There is a need to collect a central datastore, DataLake.
  The datamart of a DWH could be used but:
  1. That one is too late with data delivery, availablity
  2. Having modified data where sparse pecualiar information can be the one that is searched for
• The delivery can be
  - a dashboard (type of report) informing the customer that is responsible voor continuity and / or change. He can make the decisions on that.
  - an automated proces of profiling - scoring giving a list of observations being expected the best candidates (ML).
• The automated logic generation is asociated with numbers like lift accuracy expected profit. Those are the number and figures helping the responsible person in decisions on what to do.
• There are four lines (vertical green + external connection) for versioning and release management. The list:
  1. Preparing data that can be used by data scientists.
     Having modified data containing sparse pecualiar information can be the one that is searched for
  2. Preparing data by datascientist in a way that they are suited for the statistical processing.
  3. The resulting generated code on the transformed data by the datscientist.
  4. Delivery of the scoringresults associated to subjects.

• Scoringresults with the input are new important information to be archived along wiht the used version of the model.
• Make this proces reliable controllable several checkpoints monitoring the scoring proces are required.
• The data model looks to be a star-schema in 3nf. It is an ER relationship allowing missing cardanalities and invalid data.

---

Inventing changing Operations.

💣 Analysing data is requiring real operational production data. Building decisons on faked information would generate very wrong results. Whether it is basic analytics doing reporting or automatized ML it is =level 3= (orange).

---

EMC - Big Data infographic (2013)

A nice review on this, "The big data journey rivisited" Bill Schmarzo 2016.

Combined pages as single topic.

---

🔰 Most logical back reference.

---