BI Business Intelligence - Analytics
BI Business Intelligence is very wide area within Information Technology.
The word BI was Hyping and now being replaced with Big-Analytics.
Many suppliers are active on the market.
They are not always clear in their relationships.
- The financial health of a company should be assured. Predicting forecasting of it is good behavior.
- Operational processes should be optimized (quality / Sales / financial).
- Product should be reliable underpinned with research. Especially to human health. (cdisc)
- Analyzing predicting forecasting is a good behavior with all.
Reporting, Analyzing, Forecasting are key factors of BI (Business Intelligence).
Analytics and Big Data as the new hyping words. >
Data mining, text mining , operational research are more difficult to understand.
Ever wondered why so much has gone wrong within financial services?
There have been regulations developed, when reading this these must be missed by the banking-crisis started with the wrong qualifications of loans.
My glossary (gathered references)
health care regulations
My glossary (gathered references)
IT disasters related to BI
Spreadsheet horror stories
(link as part of mistakes & abuse of statistics)
Collection of Generic failed operations
(link to gathered collection)
Classic BI, Business Intelligence
The dwh (DataWareHouse) concept was designed in the time computers did not have that much capacity.
IT was (capacity) and still is (responsiblity) important to segregate the operational process from decision support systems.
According to the pciture it is always about dataprocessing in several steps, enriching the data, quality validation and tranforming it.
The way primary operational process is used is requiring a quite different approach of how to handle and store the data at a DWH.
Normalized forms (third) of an DB are used at operational processes, eliminating duplicates as much as possible.
Cubes Olap Star-schema are used at dwh level. Having many duplicates to get fast easy views.
A common pitfall is to use BI-tools in the primary operational process becaus of the nice look and feel or the way of easy to use.
Data Ware Housing
My glossary (gathered references)
Data Mining, Operations research, Forecasting, Business Intelligence, Statistics
Business & security
The security to business data can´t be the same as the normal segregation ion operational processes.
Of course it must be facilitate the normal conventions with the segregation of duties.
Some reasons are:
- Analyzing the business isn't possible on faked data.
Searching for the unknown in the business (productions) data is the goal of doing analyses.
This is quite different to normal operational software. These are well defined in a development process steps (dtap).
- Building analyses to business figures is having a point of acceptance of these results.
This is quite different to normal operational software.
The acceptance of business figures is bringing a lot of confusion. It is not a software configuration process, as in that context the same word acceptance is also used.
- Analysts know the business figures well and are evaluating the results with this knowledge in their heads.
Analytics, Big data
see why BI. It is also visualisation forecasting.. optimization
seven dirty secrets data visualisation
(netmagazine feb 2013)
It is analyses of Business Data that is the main source to analyze.
Just having a database in not enough to do analytics.
big datas management revolution
predictive - modeling
It is analyzing all kind of data own busienss external data, gis, to get to some predictions by models.
Data Mining Group
is a consortium with an exchange standard (pmml).
This is the area of all trends in positive and negative. Customer behavior (churn rate) to dedicate proposals. Stock market predictions with profits or losses by mistakes.
Degradation of BI
In the beginning of usage of BI (Business Intelligence) it was assumed as the next thing solving everything. As the promise is used by marketing effort even the most simple report-listing is classified as BI.
The originally goal of well defined information to based major decision on is lost. So a new name buzz-words must be introduced.
These are: Analytics and Big-Data.
Type of analytics can be different.
The GPU (CUDA DirectX) ia having a lot of capacity measured within flops. (Floating operations- calculations).
CUDA is a different stage of analytics compared to analyses of databases. Cuda can be used to do Matrix computations, it can´t be used to analyse databases.
In the analyses of data you mostly have to deal with strings not only crunching floating point data.
The degradation of BI, has been evolved, so the lifting of new Buzzing words.
Why this change?
- BI is more then the presentation of a nice image
The origin and analyses used should make sense
- BI is more then coding a SQL-query for a report.
Of course SQL can be just a small step of the complete BI-process, SQL is a part of it not the whole of BI.
- BI is more then working on small datasets.
To days we are used to still growing bigger amounts of data.
Renew by versioning
A common try out to renew is the addition of version number.
The OLTP and SQL are classic ways of Storing data. A DBMS (DataBase Management System) is a common used tool.
Is meant to deal with a relative very small number of updates and retreives, should perform well.
A common approach is:
- to spread the record randomly
- having a fixed of amount of space/storage. Space/storage is managed by numbered blocks of fixed size
- Data is stored in blocks, Blocks are expected to get filled up as mean to 50%
- indexes used to block pointers in segregated managed ranged blocks.
With the goal of better performance the Partition_(database)
(wiki) is done.
Still a DWH used to retrieve all data often in a predefined sequentially way does not fit the requirements an OLTP DBMS was designed for.
An other the get data better served is searched in the way of storing data. Emerging “vertical” database systems in support of scientific data (2008)
Keeping the variables as close as possible together in the storage.
With predefined grouping (dimensions levels hierarchies) of real numeric variables (mweasures) is having the option of pre calaculation of some basic simple stochastic statistics. Aggregations like: N (count), Mean/Sum, sum of squares.
With the grouping an explosion of the needed storage can occur. With the same implemetnation the performance of retreiving and drill down is able getting tremenously improved.
The DWH and SQL is getting out of the hype. Instead NoSQL and optional direct access to all kind of data.
- NoSql (wiki) In computing, NoSQL (mostly interpreted as "not only SQL") is a broad class of database management systems identified by its non-adherence to the widely used relational database management system model; that is, NoSQL databases are not primarily built on tables, and as a result, generally do not use SQL for data manipulation.
- cassandra-vs-mongodb-vs-couchdb-vs-redis (kkovacs)
(wiki) Today, many large databases, such as those used for credit card fraud detection and investment bank risk management, use this technology because it provides significant performance improvements over traditional methods
Technical details SAS in database
(documented at SAS opertional life chapter)
Thinking about modern BI is also changing.
Thinking about modern BI is also changing the way of storing data. Buzzing -- Big data
Business Struggling BI - Analytics
Business view - IT strategy
historical grown knowledge
Struggling arround with IT as not knowing to translate their MIS requirements to BI Solutions.
Corporate Performance Management
Customer Intelligence , Datamining , Text mining
Today, accounting is called "the language of business" because it is the vehicle for reporting financial information about a business entity to many different groups of people.
Balance Score Card Dashboarding
Customer Relations Management
Online Analytical Processing, Pivot-tables (cube) is just a part of it.
IT Staff - Business
The way of communication:
- Not understanding each other (smart) (wiki SMART_criteria)
- Needing a break through.
The ignoring approach: "Outsourcing to .... , will solve everything "
- Planning with STAR is failing STAR (wiki Situation,_Task,_Action,_Result)
The goal of BI, Analytics, was your own business continuity and optimization.
Business - real customers
The goal Business is having customers and get some profits. A well running business having al lot of both.
building great customer experiences
IT Technical view:
Common blocking by requiring to copy the data to own maintained systems (file transfers)
Duplicating the business data. Not knowing anymore te real information
Common blocking issue by the approach: when not understood it is not allowed.
DBMS – DBA’s
Blocking the free way (DDL / DML / MDL ) of data creation by analysts
Just knowing the SQL approach
Commonly not in place or even blocking by conflicting technologies
Scheduling, Publishing Channels & More
Olap - dashboards
3 reasons to hate BI dashboards
(zdnet aug 2012)
Business data - value strategy
Big data CEO view
Business information papers
There was a good presentation about Data_rEvolution:
The changes on the market is very nice presented.
The "shared nothing architecture" to be understood by the seti@home
Not all data - information processing will be suitable for this approach.
Even better where the questions you should have asked.
Big data a blog
- whatsthebigdata (Gil Press)
Big Data is….
…the rapid growth in the amount of digital data created around the world
…the new tools helping us find relevant data and analyze its implications
…the availability of on-demand computing and storage resources in the cloud
…the new theory and practice of management
…the convergence of enterprise and consumer IT
…the shift (for enterprises) from processing internal data to mining external data
…the shift (for individuals) from consuming data to creating data
…the convergence of Madame Olympe Maxime and Lieutenant Commander Data.
Big Data is some of, all of, or much more than the above and this blog explores its impact on information technology, the business world, government agencies, and our lives.
- Big Data News of the Week (Gil Press 2012 Sandy and erp R like basic)
classic ETL as history
Common sense is staying important. Emcien is focussing on manufacturing lines. Organizations around the world are unknowingly under-utilizing their most important asset: data. Most organizations store and manage this data in departmental silos, so garnering a complete view requires teams of analysts pouring over spreadsheets containing data that is already old.
To gain a holistic, real-time view, the silo walls must come down. Herein lies the problem – and opportunity – of Big Data.
Business Analysts , Data scientist
The management language, touching Bi is at:
Six sigma , abc
Role Jobs - IT strategy
The Nature of Big Data and the Skills of Data Scientists
(smartdatacollective.com dec 2012 Ling Zhang)
The job title Data Scientist was invented by DJ Patil and Jeff Hammerbacher when they tried to name people in their data team who work on big data and they did not want to limit people’s functions because of improper job title like business analyst or research scientist
Building Data Science Teams...(see the book)
Data Scientist at work
- big_data_making_complex_things_simpler (executieve.mit.edu feb 2013)
In the past, IT worked primarily with finance to run reports, which were often used to justify decisions leaders had already made. That is changing with big data. Managers are analyzing enormous data sets to discover new patterns and running controlled experiments to test hypothesis. Decision making that was once based on hunches and intuition is now driven by data and knowledge."—Erik Brynjolfsson
Older functions Data Scientist
The Information analys, The Business analyst are some names of the same kind of work in some area.
The statistical analyst, actuary are other parts
TCO, Total Cost Ownership
Managing the Total Cost of Ownership of Business Intelligence
(SAP WP, Dr W. Applebaum 2010
For many companies, total cost of ownership is out of control. And the problem is growing, fueled by
ever-increasing demands from the user community, massive new sources for data, new capabilities,
shadow IT landscapes, and the cost of keeping people abreast of all the changes.
dwh Mid sized Business
Institutes, Consultancy , Expertise
© 2012 J.A.Karman (21 apr 2012)