home-OS    Why an OS     Basics OS     Networking     P&T OS     P&T Middleware     OS security     MY OS Notes
Tune OS    Plan HW    Virt Server    Virt Desktop    Unix    Windows    Mainframe    links    top  bottom

Performance & Tuning basics


netwerk
Analyzing, this can be done as goal of:
Computer resources is hot stuff as SAAS PAAS are used

The Classic latency still can happen.
The hardware usage can be well optimized.

The growth of information still is excelaring. The ammounts of data stored still is growing
The speed of processor doesn´t ips (wiki) the growth in transistors moore s law is coming to an end.
Tuning and smarter solving the problems as grid (parallelism) will become more important. The popular GPU is a form of parallelism.
The Free Lunch Is Over: A Fundamental Turn Toward Concurrency in Software (gotw herb sutter)- http://download.intel.com/corporate/education/emea/event/af12/files/cownie.pdf (intel james Cownie)



Tuning Performance

Generic

Tuning Generic OS level


The basic are: It always has to do with some limitations on the building blocks of you computing environment.

Is it common job doing and doing it more frequent? Are you small or big?
Are you doing it just once as research or is it that big that a good plannig is needed?

Having the answers on those questions a requiremment planning is able to be made.


mislead

trashing

Trashing congestion are phenomenon´s to be understand. It is meaning the computer/network is doing a lot of work. But all the work is for internal housekeeping not useful for your intentions are being served anymore.

The tuning related direct to SAS is the next chapter.
doc cabins

Moore´s Law + Hadoop

What Is Apache Hadoop? hadoop Users guide (apache)     The Apache Hadoop software library is a framework that allows for the distributed processing of large data sets across clusters of computers using a simple programming model. It is designed to scale up from single servers to thousands of machines, each offering local computation and storage. Rather than rely on hardware to deliver high-avaiability, the library itself is designed to detect and handle failures at the application layer, so delivering a highly-availabile service on top of a cluster of computers, each of which may be prone to failures.
hadoop introduction (developer yahoo - apache)
No speed improvement to be expected anymore of CPU-s. Distribution of data and processing is hyping.

Hadoop is designed as a file system. The file system is not located on one machine but distributed over many machines. Is is not meant to be used with update processing, write once and read many times. With PIG it is becoming a NOSQL approach.

doc cabins

Percolator, Dremel and Pregel: Alternatives to Hadoop



doc cabins

Compressing data or not

Compressing data is saving space but at the cost of cup time. Sometimes total execution time and wall clock time can be improved, better results shorter wall clock time, by using compression. This is because the IO system takes the biggest part of time to handle the data.

Many tools are build with compression. Most of them are using the word zip.



Tune OS    Plan HW    Virt Server    Virt Desktop    Unix    Windows    Mainframe    links    top  bottom

Plan the hardware for expected usage

datacenter datacenter


archive

Datacenters

Datacenters are changing fast. In the old days they were big racks. Changed to shrinking cabins. And back again to big racks. Some datacenter information can give an idea. (Google Microsoft and a less known).
The are all

archive

Industrial Computers

Grid (wiki) computing is something that will replace the classic big-computers. They are composed with a lot of small pizza-size rack mounted computers. The smallest measurements in 1U server) sizing, bigger sizes are 3u or 5u.
Within the datacenters you can order for co-location, dedicated, hosted.

The Small are limited in their capacity. This is to be taken care of with capacity, perfomance, tuning. Not all processing is needed to be done on 1 server it is easy to use more servers. See: pizza limit

With this big industrial approach of datacenters is becoming hard to do it yourself. You must be really big or doing it as your core-business.

archive

Computer hardware components

CPU GPU
The CPU once was simple to understand. Today a processor socklet and virtualized cpu has made it difficult. A good explanation: vcpu-pcpu-lcpu

The GPU was invented to do graphics. 3d calculations are very common so it became an alternative in faster floating points interfaces.

archive

Performance comparisons

IBM Power - Intel Nehalem Xeon
Intel is getting into the area of the IBM power. tcp (ideas blog 2009 Gary Burgess) Ideas is been bought by Gartner
tcp (IBM note 2008)
P system Intel Xeon, driving out sparc itamium (clabbyanalytics 2010)
TPC
Hardware can perform different on "eTransaction Processing - OLTP"e (tpc-e tpc-c) or "eDecision Support"e (tpc-h) tcp



Tune OS    Plan HW    Virt Server    Virt Desktop    Unix    Windows    Mainframe    links    top  bottom

Cloud buzzing

Platform As A Service PAAS - Desktop As A Service DAAS

Server virtualization

The server can be virtualized.
This will share the original hardware capacity between multiple logical machines. It will gain a lot if the original hardware usage was low. An other aspect is that virtual machines can be migrated more easily.

In the case of original heavy resource utilization the same or more hardware resources are needed. With the usage of the newer datacenters approach a grid solution is the most obvious way.

Desktop virtualization

The desktop can be virtualized. VDI Virtual Desktop Infrastrcuture. There are Pros and contras. Defining VDI users: Who should really be using virtual desktops? (techtarget)

The desktop can be left away. BYOD Bring Your own Device. IT pros mix VDI with BYOD The VDI desktop is not as powerfull as the whole machine. Normally until 20 users are sharing one phyical machine. A portion of the hardware must be divided.


So wat you can expect as a VDI is like (2011):

Compared to a full highend desktop environment (2011)

Compared to a full server environment (2011)

This also is indicating that with VDI the processing capacity should be located to a server environment.
The personal computing era as IT solution is coming to an end in this way.



Cloud buzzing

Software Defined Data Center

SDDC




Tune OS    Plan HW    Virt Server    Virt Desktop    Unix    Windows    Mainframe    links    top  bottom

linux logo

Unix

latencies img

generic

The technical issue are meant for specialists.
finding information: wiki tuning
quicksheet (tablespace.net)


SAS

When starting with analyses of SAS performance this is a good start: employing host based tools




open files
IBM Usage and examples of lsof, pfiles, procfiles commands (IBM- United States)


To kill open files (locking)) fuser is a UNIX command used to show which processes are using a specified file, file system, or unix socket.

standard commands
vmstat
What to Look for in vmstat (Burleson) Normal Oracle oriented. Oracle has to run on a well behaved machine.


Load

Load_(computing) (wikipedia).


Nmon
nmon performance: A free tool to analyze AIX and Linux performance (IBM) nmon (wiki)
nmon for Linux released to open source under GPL - it was an internal project at IBM for many years. (IBM 2009)

With MXG nmon is included. Alle analyses can be done in SAS. Mentioned Postprocessing are: RRDtool

Zenoss
zenoss (open source) monitoring cloud



Linux Redhat
This Linux version is getting most attention (2012).



Redhat IO tuning
(sas)


Redhat SAS papers
The same as with Ethernet 10Gb values.



AIX, Unix version IBM
With this version is got the most experience while using it.
The relationship IBM and SAS is not very clear. I find partnership relations, the most evolved implementations within in SAS to AIX. At the same time they are competitors. IBM positions hsit Unix version with Linux in the way as described: AIX6+Quick+Ref+Sheet.pdf
AIX, Unix IO tuning
The most critical part the Performance of IO is described in this quick reference.
At SAS some detailed information can be found: AIX+Tuning+1 (2009)   Performance Monitoring Tips and Techniques nigels   AIX_Disk_IO_Tuning_093011.pdf (2011)  
WikiPtype/LVM   wlmrestypes.htm  
AIXBestPractice SAS eis   AIX_Practice_for_SAS_eBI_Users_S1b.pdf  

Mountpoint / Storage tuning:

AIX, procsesses
The operationasl part of running processes.

AIX, Unix virtualization
Links about pcpu vcpu entitlement:
like hell
Commands Unix
Unix has many commands, they are sometimes different to Unix versiosn. Them manual (help is mostly online). Some commands listed related to tuning here are:
A quick list of volumes in a volume group.
lsvg -p tmpdatavg |awk '{print $1}'|xargs -n1 lspv -l


like hell
Aix networking
I am suspicous about: tcp_nodelayack.... Streamingperformance...

Tcp_recvspace and sendspace are having values of 32K this could be set higher. The io-buffers of SAS has been set to 128K of datasets Ther 2Gb fiber adapters (rfc1323 ...) should have much higer values according to manuals.
The same as with Ethernet 10Gb values. tcp_nodelayack (Aix tcpip stack tuning IBM)


Tune OS    Plan HW    Virt Server    Virt Desktop    Unix    Windows    Mainframe    links    top  bottom

Windows logo

Windows


Some things can be done withoud changing everything.
Faster harddisk (SSD)
More Memory can be build in.
Using hacking tools to get a little more speed (up-clocking) with needed extra cooling
The use of better software, using a GPU more processors parallel is the most effective.

Common tools are perfmon and broncontrole giving the views on all technical resources.



Tune OS    Plan HW    Virt Server    Virt Desktop    Unix    Windows    Mainframe    links    top  bottom

Old days computing
Mainframe img

Mainframe SMF SDSF

SMF MXG
SMF System Management Facilty, is the classic source to analyses alle events and performace of the old IBM mainframe.
Merills MXG support System Performance Analyses - SAS-based. Is more or less build up by a community like the open source approach. Open source does not mean it is for free, just the source-code is free to verify. See: mxg.com

IFAs are engines that execute only Java code, are not included in MSU capacity ... can force all Java workload to execute only on IFA engines ...
OLD SAS notes


Tune OS    Plan HW    Virt Server    Virt Desktop    Unix    Windows    Mainframe    links    top  bottom

Links

When found nice links.... (here)



Tune OS    Plan HW    Virt Server    Virt Desktop    Unix    Windows    Mainframe    links    top  bottom
home-OS    Why an OS     Basics OS     Networking     P&T OS     P&T Middleware     OS security     MY OS Notes

© 2012 J.A.Karman (17 apr 2012)