Data representation, presentation, performance & tuning
Technical topics realizations.
The most technical topics, not ready in their basic nature for operations, are slowly made ready for getting used at operations.
Visions as futurists from the business leaders are not always sensible approaches. Mostly they are stressing hypes.
🔰 Too fast ..
previous.
Contents
| Reference | Topic | Squad |
| Intro | Technical topics realizations. | 01.01 |
| Hypes | New & old hypes for attention. | 02.01 |
| Coding | Programming languages. | 03.01 |
| Encode | Data representatation. | 04.01 |
| Proces | Processing Information. | 05.01 |
| What next | Following steps. | 06.00 |
| | Ongoing to do it | 06.02 |
| basics | Proces building. | 07.01 |
| abstract | Hardware abstraction. | 08.01 |
| layout | Presentation logical set up. | 09.01 |
| CSS | Presentation, technical CSS. | 10.01 |
| (un)happy | Unicode emoticons. | 11.01 |
| to solve | Issues. | 12.00 |
Progress
- 2020 week:28
- Lays out redesign starting wiht css.
- Just basic adjustments no storage popups scripts or whatever.
- Split page in two part of six squads. Generic six first, the site related details next.
- 2020 week:04
- Conforming lays out to other pages.
- Doing the hype, programming and data strcutres paragraphs.
- 2019 week:12
- First page only had the personal technical building references.
New & old hypes for attention.
Doing mathemics science is not my objective. What is developped in that area will come down in applications at some moment.
Being practical in some cases is giving "doing your math".
EU industry ministers have signed off on a eur 1 billion project to build the world´s fastest computer by 2023 in an effort to compete with China, the US, and Japan.
According to the commision, the new initiative is "crucial for the EU´s competitiveness and independence in the data economy."
Blockchain is one of the fastest-growing technologies in today´s digital world that have come into the limelight in the last few years. Undoubtedly, we have heard that this technology has revolutionized finance, real estate, and many other sectors.
In the last few decades, small-scale unmanned aerial vehicles have been used for many applications. The need for aircraft with greater maneuverability and hovering ability has led to a rise in quadcopter research. The four-rotor design allows quadcopters to be relatively simple in design yet highly reliable and maneuverable.
We created Parquet to make the advantages of compressed, efficient columnar data representation available to any project in the Hadoop ecosystem.
Parquet is built from the ground up with complex nested data structures in mind, and uses the record shredding and assembly algorithm described in the Dremel paper. We believe this approach is superior to simple flattening of nested name spaces.
The Raspberry Pi is a series of small single-board computers developed in the United Kingdom by the Raspberry Pi Foundation to promote teaching of basic computer science in schools and in developing countries
All may be promising but possible not useful in the way they are presented in the very technical first releases.
Blockchain was the promise solving anything in data governance in a way like bitcoin. Maybe is is more useful when using the axioma´s in an other way solving the data governance issues.
Programming languages.
Programming machines, computers, isn´t that easy. They are doing exactly what has been instrcuted to do, even when the intentions are clearly somewhat different.
Algol Cobol Fortran Rexx JCL
In the anciënt days when computers were far more expensive than the cost of labour to program them, you had to think very well do desktop testing before being allowed to run something.
The design of programs, algorithms, could be very deliciated to get acceptable performance and behaviour.
The language mandatory for use:
- Cobol: Common Business Oriented Language ➡ business
- Fortran: Formula Translation ➡ technical science
- Algol: Algorithmic Language ➡ educational
- For some others in a special case use ➡ assembler Rexx JCL
The goal was solving easy to understand functions in a modular way. The focus on optimizing the business process.
With an sequential instruction for the process, the approach with this 3GL´s using program flows.
The integration with other modules and programs to achieve a complete system doing in later steps.
The most important part: the business logic.
C SQL Java HTML bash
Most modern programming languages - C, C++, VB, Delphi, Java, Python, and even Perl - use syntax and concepts that originated with the AlgolLanguage. So they´re called AlgolFamily languages.
In an Algol-like language, with semicolons as statement separators, the semicolon is often used as a synchronizing word.
A move has made to focus on the language itself as the solution for solving business problems. That is weird.
I would expect the focus to be on the required business logic. Obviously a lot of business are suffering by this.
Many are still running Cobol not knowing for sure what and why the progams have been coded that way.
R Python julia SPSS Watson matlab SAS
These are new and older languages.
With these language it self, it is about all those packages (modules) you can use and run. The programming effort decreased but introduced new dependies.
low code (wikipdia) approaches. Some new names are Mendix and Outsystems.
Additionally, some CIOs have expressed concern that adopting low-code development platforms internally could lead to an increase in unsupported applications built by shadow IT
For shadow IT:
Some people consider shadow IT an important source of innovation, and such systems may become prototypes for future approved IT solutions.
On the other hand, shadow IT solutions are not very often in line with organizational requirements for control, documentation, security, reliability, etc.
Data representatation.
Information is translated and transformed in many phases. What the intention and meaning is, is needing a common ground.
In the world of using computers everything is digitized, assuming just two values 0 and 1.
The size of any measurement is having limits. In the technical approach understanding those is necessary.
Characters, National Language support
Morse and the first realisations the 8-bit single byte as basic unit aloowing for 256 (0-255) characters.
Ebcdic and Ascii two major competitors for the translation of what nummber to what symbol is intended.
The number of possible symbols was far too low to satisfy everybody. One solution for that is using different code pages.
That is the same number can have a different meaning when used in an other location. Transcoding between codepages is loosing information by not having the all of the same symbols.
Base64 is a ay to convert all 256 binary values into 64 common used characters. The goal is avoiding unwanted effects while doing an exeact binary copy.
This single byte rperesentation still is often used.
👓 Unicode is a different approach, it has abondoned that single byte approach. There are five ways of how to code and there are diifferent versions of the standard.
Those different versions is known with the emoticons being added as new ones.
Transcoding, converting those numbers to have the correct meaning, is a hidden problem cause. Somtimes the
replacement char is shown.
Basic unicode tables:
- utf-8 uses 1 up to 4 bytes. variable left to right
- utf-16 uses 2 or 4 bytes. Order in big or little endian (left to right is different)
- utf-32 uses 4 bytes Order in big or little endian (left to right is different)
A new issue to deal with: not every binary combination is valid!
Data information has to be relasised with the appropiate characterset in use.
Text, National Language support
It doesn´t stop having the right characters, there are much more conventions that are having language differences. One of those the ordering left to right or right to left.
another important one is the decimal point, instead of a point a comma is often used. Ohter measrures and units also exist.
The byte-order mark (BOM) in HTML (w3.org)
You can see that the order of the two bytes that represent a single character is reversed for big endian vs. little endian storage. The byte-order mark indicates which order is used, so that applications can immediately decode the content.
Dates time, National Language support
iso8601
When dates are represented with numbers they can be interpreted in different ways. For example, 01/05/12 could mean January 5, 2012, or May 1, 2012.
On an individual level this uncertainty can be very frustrating, in a business context it can be very expensive.
Organizing meetings and deliveries, writing contracts and buying airplane tickets can be very difficult when the date is unclear.
ISO 8601 tackles this uncertainty by setting out an internationally agreed way to represent dates
Number Data elements.
Real world numbers are floating numbers, analog with uncertainties. Impossible to implement those in a digital representation only some aspects covered.
An old type, packed decimal, stores two numbers in a single byte. Using the first 10 out of 16 hex values.
⚠ License numbers (cars) and bank account nummbers are NOT number types. They contain other symbols and are not used to calculate a sum or mean.
The 8-bit single byte as basic unit. Unsignend and Signed in two´s complement.
- 1 byte - 8 bit : 0 ⬅➡ 255 , -128 ⬅➡ 127
- 2 byte - 16 bit : 0 ⬅➡ 65536 , -32768 ⬅➡ 32767
- 4 byte - 32 bit : 0 ⬅➡ 4Gb , -2Gb ⬅➡ 2Gb
- 8 byte - 64 bit : -9 EB
⬅➡ 9 223 372 036 854 775 807
Working with real life numbers is the technical floating representation. It is hardware dependent, not bound to a programming language.
IEEE_754 (wikipedia)
⚠ note the limit of 15 (12 are save) decimal digits.
⚠ loss of precison is a cause for surprising not valid results.
Processing Information.
When an business process is set up, the major question is on the goal and flow of information.
It starts at a high level (conceptual), with the intention to achieve something to use at a presentation for interactions.
There are several obvious questions: When, How, Who, Where, Why, What. The sizing, capacity (how many) is hidden.
A top down design starts with the questions about information and will search for the best way in presentation(s).
A bottom up design will start with how soemthing is presented and is trying to invent what and how the information should be processed.
Conceptual Information (Back end)
A conceptual schema is a high-level description of informational needs underlying the design of a database. It typically includes only the main concepts and the main relationships among them.
Logical Information (Back end)
A logical data model or logical schema is a data model of a specific problem domain expressed independently of a particular database management product or storage technology (physical data model) but in terms of data structures such as relational tables and columns, object-oriented classes, or XML tags.
This is as opposed to a conceptual data model, which describes the semantics of an organization without reference to technology.
Physical Information (Back end)
A logical data model or logical schema is a data model of a specific problem domain expressed independently of a particular database management product or storage technology (physical data model) but in terms of data structures such as relational tables and columns, object-oriented classes, or XML tags.
This is as opposed to a conceptual data model, which describes the semantics of an organization without reference to technology.
Beam data storming
Using the 7 w´s for data modeling putting the informations questions from business perspective central has this source of modelstorming.
The technical logical and physical ideas is classic dwh. Aside Data Types (Cn, DTn, DN, Nn.n Tn B) there are Key types, Dimensions Attribute types and Fact types.
Asking with 7W´s how the data should look like for an informational result and asking with 7W´s how the processes should work to achieve goals should work in a generic way.
Dynamic Information (front end)
Online transaction processing systems increasingly require support for transactions that span a network and may include more than one company. For this reason, modern online transaction processing software uses client or server processing and brokering software that allows transactions to run on different computer platforms in a network.
Static & Meta Information
The descriptive information for the dynamic data is not required to build up over and over again. It can have an static layout. It will go along with the life cycle of programs - code.
Presentation (front end)
How the the Static & Meta Information and Dynamic Information is implemented in a Physical way is the delivery act of an information system.
Following steps.
Managing people isn´t my priority. Getting aligned to practice at organizations to get things done are the daily way of life.
The daily stand-up
Knowing what the colleagues are doing is important with a shared business goal.
The implementation of a stand-up can be brought in mandatory. I would assume sitting in each neighbourhood and doing that during working on items, would be sufficiënt.
From the common ground in a technical approach there is something strange going on:
- Forced is using hyped modern technologies although there are bleeding edges not solved well.
- The bleeding edges are not marked to solve and avoided for the time being but are a trigger to get in more modern technology with bleeding edges.
- Asked is a the keep it as simple and cheap as possible, lean agile.
The diagonal connections at both ends in the sitemap are defined instead of the normal vertical siloed lines.
The surprise is: they are practical in understanding the misunderstandings and big distances at organisations with processes.
Epilogue - Operational
Building and / or doing the operational work.
Making your mind up how to present all information was the first step.
⏳Starting and finding yourself in trouble by the work to be done and still not knowing what is to do for better, time passes.
Epilogue - Tactical
The usual discussions are daily management involvement.
There is an elephant, the big boss likes to be bothering with technical details when he is having a problem.
⌛The simple solutions are " build a database" - " roll out an app "
Ongoing to do it
These are practical data experiences.
technicals - math generic - previous
data, generic toolings 👓 next.
Others are: concepts requirements: 👓
Data Meta
Math
Building site, an example with challenges & Issues
Website building.
I am trying to build this site by coding just plain html and css. There are tools helping in coding the html other are generating code using templates.
There are many frameworks like WordPress Magento and interfaces with advanced packages. They all claim to make the work easier.
Why doing the basics?
The choice not going for the most usual and easy approach looks weird, it hurts.
The disadvantage are:
- Hard work, many technically surprises.
- Understanding of a lot technical details is needed before seeing some results.
- Review with care for Syntax errors typo´s. There is no autocorrection, although modern tools are supporting it to some level.
- Time consuming investing in researching the technical details.
- Specific technology seems having little value for business questions being .
The advantages:
- Learning a lot what can go wrong in frameworks and using advances packages.
- Minimized technical dependencies.
Lean manufacturing is achieved by focussing on core requirements and the details that are adding value. Using only that what others are already doing is a difficult competition on doing it cheaper better faster.
The change of HTML / CSS
A big conceptual change is made.
html css intro
HTML was NEVER intended to contain tags for formatting a web page!
HTML was created to describe the content of a web page, like:
-
<h1>This is a heading</h1>
-
<p>This is a paragraph.</p>
To solve this problem, the World Wide Web Consortium (W3C) created CSS.
CSS removed the style formatting from the HTML page!
All what has done in recent years will have to be rebuild to get aligned with this.
Analysing a CSS to generate new HTML with inline style code is a bad practice although giving quick results. A new problem is arising.
When defining all those CSS classes in a specific in house style, how to align the dynamic generated html code using those classes?
Hardware abstraction.
Any application should have as less as possible dependicies on the hardware used.
Using a browswer (HTML CSS) the knowledge on viewport is the most important for how to present what there is.
Old hardware was limited in the sizing and the performance. Those limitations are changed.
RWD Responsive Web Design
What is Responsive Web Design?
- Responsive web design makes your web page look good on all devices.
- Responsive web design uses only HTML and CSS.
- Responsive web design is not a program or a JavaScript.
Different type of endpoints, PC, tablet, smartphone
Some links on the technical coding:
Storing information, calculations
What was missing once was:
The heavy usage of java-script could avoided and more getting more decreased.
Windows Sizing - Positioning
The content is presented in floating blocks on a page. The layout is done css only with well chosen options.
Character size & type, borders margins and colors in a fixed size where others can resize.
To my surprise the hardware viewport wiht small devices is 400px or even smaller. The 25% of that size is not much. leafing very little rrom for text.
An example for testing
Complexity in several ways.
Calculate 75% of the available viewport width with a minimum of 180. It should show a decreased size of an image.
For this the medium layout figure is used.
Presentation logical set up.
Every application has his own set of requirements. Just documenting ordering like written book is very different to an interactive approach using one ore more backends.
Information for reading should noet be to have a great width. Scrolling vertical is the most logical seeing some text having read and new text coming in.
2,3,4,5 Natural whole numbers
For ease of dividing in whole numbers the following are practical layouts evenly spread.
- 6 = can be spilt in 3*2 2*3 and 1*6
- 12 = can be spilt in 6*2 4*3 and 2*6
- 60 = can be spilt in 6*10 5*12 4*15 3*20
Decison on the number of divisons
For different devices the goal is to have 1 - 2 or 3 blocks of column wise text aside each other.
Having a multiple set of 6 is supporting 12 ordered divisions.
Adding an header Footer and navigation for the pages in a repeated standard layout should result in a consistent look & feel.
Fixed and dynamic sizings
Images can be fixed when used to help understand text in a small inline elaboration. When they are over the 240px size they should resize wiht the section and viewport size.
Presentation, technical CSS.
The abstraction in the presentation html - css has resulted in a requirement for understanding and building the css code.
Having a goal of dynamic behavior showing and processing information the css presentation is causing challenges.
Setting text divisions disable flow (css)
The content of min-width is deliverd by the browser. In a wide screen wiht 3 columns over 2 rows the definition for css attribute clear is:
@media only screen and (min-width: 1200px) { :root { --divtxt-clr1: both ; --divtxt-clr2: none ; --divtxt-clr3: none ; --divtxt-clr4: both ; --divtxt-clr5: none ; --divtxt-clr6: none ; } ; }
Also is needed are six block class definitions wiht a class in the order of the six divisions.
div.tx-p%{ clear: var(--divtxt-clr%); }
Viewport, calaculations (css)
A responsive desing calculating the size of elements:
:root { --divtxt-width: calc(100vw /2 - 24px) ;
Paragraph breaking up in segments (html)
Just wanting a clear breakup within a division (
clear:both ), then a css attribute must be used, not html. An easy approach:
<hr class="clear">
Adding this in non floated block elements will solve the height for the borders when used with floated sub elements.
Issues, mistakes & not weel understood
- Doing calculations in CSS is very limited. No rounding or choosing a max/min value.
- Nested calculations in CSS is not transparant, better is repeating in basic simple statements.
- Id did probably not choose the bet fit settings in css in calculating.
Borders ouline are inlcuded or not or testing of added calculated values is at the edge of just excess a limit.
- When an element is above the max_width of the container adjustment in presentation can do unexpected things.
When an element somewhere is causing hat the size of it is to be reviewed.
That is work to do, just need a plan when there is time doing that.
Chapters
After some trials, I was happy to have three chapters on each page with four separations blocks.
The number three is aligned with the nine plane. This is a fit for the intended content.
Paragraphs
For human reading paragraphs are shown in a limited width.
A paragaph is filled with content up to a maximum length.
Using six paragraphs at each chapters allows dynamic reordering in different layouts:
- 2*3 A good fit for big screens on pc´s.
- 3*2 For limited width screens.
- 6*1 For small width devices like smartphones.
Each paragraph is having divisions. Ordering the divisions over paragraphs is challenging.
Disadvantage: there is no standard canvas, no standard layout. Hard work to do it in basics.
Frictions nine plane laysout.
A web page (html) is technical a Latin-1 basic communication protocol. Using extended characters is needing encoding.
Encoding: Using basic representing for other characters
When using multiple programming languages mixed up in a single document, quoting escape sequences are similar.
Frictions positioning - choices
Frictions are growing at point where the flows in relationships voor communication meet each another in a contrary direction.
Having big circles wiht al lot of stations in the relationships problems caused by misundertanding confusion and diifferent targets.
With a growing number of involved people the size of circle needs adjustments.
In pictures where the small circle loops in the horizontals are not drawn:
| 2 levels |
| + Less frictions but a bigger circle. |
| - Every friction is more difficult. |
| + Hierarchical distance is small. |
| - more direct involvmentare required. |
| 3 levels |
| - More frictions in smaller circles. |
| + Every friction is less difficult. |
| - Hierarchical distance increases. |
| -/+ Direct involments can get avoided.. |
Frictions positioning - choices
⚠ The Technology that is to get in house for improvement and new products is often confused with the technology used at the core business line.
Dumping your core business out of the organisation doesn´t make any sense.
- Defining a strategy for improving your core business, possible by external parties.
- Improving processes was always confusing who is doing that and impact by a wrong advice.
- Advices that resulted in bad results are not impacting advisors (ICT advices).
Outsourcing product improvement activity vertical (IM) in images:
| 2 levels |
| + Internal contacts for issues. |
| - Frictions at external tech support. |
| + Better long term continuity. |
| - Possible missing tech developments. |
| 3 levels |
| - Every issue is an external order. |
| + No frictions external tech support. |
| - Missing Long term strategy. |
| -/+ tech development alignment. |
🤔 Thoughts: The AIM model is used by accountancy firms in advices to implement a dedicated IM (Information Management) vertical staff department.
It is the central position in the AIM figure information / tactics. In my mindmap there is no location for a single IM.
The idea of an IM is that ICT is too difficult to understand and/or to manage and this new IM staff will solve to burden in management.
💣 Creating the IM vertical staff and not having made clear all the activities are expected to fullfil by them will create a bunch of new challenges while not solving the old ones.
Removing bottlenecks
Optimising work, seeing and understanding what is going on is what I do. The first project doing this was JST although I did not recognize it that way.
Seeing a ToC "Theory of Constraints" book. Reading the chapter "11 workplace bottleneck - eve" seeing a happy recognition.
The reference: 👓
The Bottleneck Rules (Clarke Ching)
A reaction on a review:
I´m so glad you liked the book ---- . It took a lot of work to write such a short book!
Issues.
Having tested validated the technical solutions sometimes behavior in practice is changing or in different browsers doing unexpected things.
A screen locked by a an uninteded loop (refresh rate) or changing charactersets, adjusted borders impacting text flow are frustrating experiences.
Chekcing CSS code validity functionality
This classic old style hard work. Defining and coding followed by testing for all stages with the page working on.
Time consuming and easy introducing new errors. The argument for using frameworks instead of this is a logical one.
List bullets not following block flow wiht figures
- Still to sort out why this is happening. Expecting all of the Html will follow the automatic flow of text around pictures.
- Replacing the bullets by own text is circumventing the wrong placed bullets when there a picture at the right side.
Having the pciture at the left is also avoiding this. (2018)
Floating elements and borders
A dynamic layout and dynamic positioning rises questions in the order of processing the lay out. When borders are added ore removed after the positioning of floating elements the positioning itself will change.
This is an unsolvable situation. Better is avoiding that ordering question.
A common problem with float-based layouts is that the floats´ container doesn´t want to stretch up to accomodate the floats.
If you want to add, say, a border around all floats (ie. a border around the container) you´ll have to command the browsers somehow to stretch up the container all the way.
Syntax & spacing
The required and resulting spacing is not always clear. CSS: Calculating a number and doing a substract - 32px and -32px would both look correct. The second one is not doing the calculation, no error/warning.
Imagemaps
I have used that For choosing in a map representing the site structure. The figure self is big (480 reference).
The coords attribute on the imagemap html tag does not support percentages. Percentages would be the way allowing dynamic adjusted content. Approaches are solving it scripted.
Someone is mentioning CSS.
fora
Video MP4 autoplay
Html 5 is supporting the videa tag, seems easy to implement. Having wondered why it did not autostart on the mobile and it did on the desktop.
Than noticed it didn´t start when it is the entry page, only when coming from another page of my onwn pages it start. Uhhh, what is going up? Web pages should not know what has happened before (stateless).
Emoticons not as expected
- Different devices are not showing exactly the same figures for the same emoticon.
- Unicode is evolving adding and changing them by time. The support on different devices could be not the same.
Sometimes a fall back to ancient figures is seen, not using the mos recent character set.
Segoe Windows charset is colored ancient, black white (july 2020 Windows).
Expectation is that at some moment it will be corrected again. The Mobile is still showing all colored (different).
Html evolution
- The language has been extended with many new tags going from version 3 to 5, others tags are depecreated.
Some recurring work in redesign en maintenance is required. The biggest change seggregation presentation from static content.