home-SAS    SAS-SAAS    First steps    Installation    Hardening    Operational    Using    My Notes
Understand Bu    Support Bu    Run IT    SAS Advanced     Other Experiences     top  bottom

Operational life with SAS


Operational support The administrator(2):
This is what he is doing all the time. Keeping the service running. Getting the new things working.


For backgound of DTAP, software librarie for the business, see DTAP lcm scm chapter. For a glossary see that chapter. My Gathered Samples for detailed engineered and developed situations.

More information about the SAS Administrator is found at: My notes.
Versions (aug 2012)


business

Understand Business

Every group of users has his own approach and own requiements. Business Information can be required to be shared.

Security policies and other requirements can lead to have a shared implementation. The RBAC approach (security) is commonly shared.
learning paths the different roles can become more clear.

enterprise1

Classical defined processes

This is most people think IT is limited to. Predefined processes to be run as bulk. The operational execution is not intresting.
The way to design the process the business logic the segregation of duties is the job to do. That is about the DTAP design with the hardening part.

Standard solutions for common processes exist. SAP is well known for this.
SAS is positoning the solutions line for analytics. However own home made code also applies. The training paths are constantly changing. Of the beginnning of 2012 I made some snap-shots.

A Business User Business User I access existing information as well as create my own reports and analyses using point-and-click applications. I use SAS Add-In for Microsoft Office, SAS Web Report Studio, and the SAS Information Delivery Portal. .

Content must in some way be delivered for this business user. Candidates are the other types of users.
This type of user has been removed from traning paths at the end of 2012.
enterprise2

Analysts: regular processes

Annual or qurterly report accounting is an example of this. The process is wel defined but the environment can change faster than IT implementations can follow.

The approach of a classical defined process is distubrbing the time deadlines. Escapes are by different project management (Agile / RUP) or bypassing the CIA requirements executing the process by hand made own owned IT. EUC not validated as reliable.

When possible the same approach of DTAP with Agile project management can solve the business requests. The classic or modern analyst will be a fit.
Other logical traningpaths are:

A BI Content DeveloperBI Content Developert I understand my organization's data and the applications I need to create reports and analyses for myself and others in my organization. I use SAS Information Map Studio, SAS BI Dashboard, SAS Enterprise Guide for the creation of stored processes. I may also use SAS OLAP Cube Studio and SAS Visual BI (powered by JMP software).

A BI Applications DeveloperBI Applications Developer I´m responsible for building, implementing, and customizing applications. I use SAS development applications such as SAS/AF, SAS AppDev Studio, and SAS/IntrNet.


enterprise4

Analysts: research - conversions - auditing

Doing analyses with conversions one time solving. A single executed analyses will do. Mostly very simple analyses for checking integrtity of the data, searching standard deviations.

There is no special background analyses in this area. The classice Programmer/analyst is the most likely Fit.
A Programmer/AnalystProgrammer/Analyst I´m responsible for accessing and managing data and for performing complex queries and analyses. I write SAS programs or use SAS Enterprise Guide for these tasks.
enterprise5

Data mining - marketing, forecasting, fraude preventing

These people are highly educated. IT ashould help them working more efficient
Some subjects are at:
Many advanced analysticts learning paths exist. Just two of them are:
A Forecaster Forecaster I'm responsible for developing forecasting models to empower the organization to make informed decisions. I write SAS programs, or I use the Time Series Forecasting System or SAS Forecast Studio.

A Data MinerData Miner I'm responsible for extracting information from large databases to construct data segments and predictive models. I use SAS Enterprise Miner, SAS Credit Scoring, or SAS Text Miner, or I write SAS code.



Understand Bu    Support Bu    Run IT    SAS Advanced     Other Experiences     top  bottom

how to support BU

SAS setup in grid old dino Do you still think that getting the business data connected should be done by the analyst or user himself?

Watch well at the concepts with centralized metadata midtier all servers defined.
The analyst can indicate the data and where to find. The SAS administrator has to register it in SASmetadata or in other ways.

Still doing it youself? This approach is now falling into the hacking category (black-hat)

enterprise3 The administrator(1):
Keeping his eyes on normal usage by his customers, keeping his eyes on the hardening part of the installation. Compliancy not to be violated by someone.

Business Data & Logic Guidelines


Business technical


Business releases


Technical support - Configuration



Business Data & Logic


Logic Coding & Usage (1)


handy

SAS Base language

There are many sub languages with SAS in use.
  1. commands
  2. statements
  3. macros
  4. procedures
  5. AF SCL
  6. metadata BI DI
  7. others
commands, statements, macros, procedures
This is the classic way of thinking using SAS.
The Commands are getting away with BI solutions Line.
Statments and procedures are getting hidding with the Eguide project approach

procedures
Every procedure can be implement his own language.
Proc SQL, as example, is a very complex procedure and documented as an entity.
Metdata - SAS integration technologies is complete world

metadata BI DI
The new environment for the user interfaces. Aside web interfaces the major component are AMO (Addin Microsoft Office) and Eguide.

geniaal
Classic: comment header
Coding clear code is always said to be done by including clear comments

With the drag and drop approach like Eguide, the comment cannot be placed in the code. With Eguide the notes documenation can be included in the project. In that way much more is achieved. (see using notes)

With SAS 9.3 Soure can be stored at the metadata database. No need for palin text code.
I have not seen it yet (jul 2012). I am expecting the usage of catalogs type source somewhere stored as librarie members.
This long existing technical approach of catalogs is making SAS machine-type independant. By that it is used by SAS solutions.

geniaal
SAS Base language, subtype macro
SAS macros are more difficult to understand. They are however the part to make all things efficient. Coming back everywhere in SAS.



concatening vaules into macro-var (P.Matter)

proc sql noprint;
select strip(name) into :Concat_Name_String separated by ","
from sashelp.class ;
quit;
%put The result is: %bquote(&Concat_Name_String);

Data _null_;
Set sashelp.class;
Call symput('Concat_Name_String',Symget("Concat_Name_String")||strip(Name)||',');
Run;
%put The result is: %bquote(&Concat_Name_String);



A list of possible usable written macro sources:
geniaal
AF SCL
The component language was thougt to be gone with java and app dev studio rebuilding approach. This did not happen.

It is a classic approach at first sight.
Coding clear code is always said to be done by including clear comments

The variant in its desigg is the use of a preview-buffer. All code is generated before it is processed by something else.
preview (SAS(R) Component Language 9.3: Reference)
This handling with a preview buffer can also cause new issues. Some have warned in advance. like naming of sas-macro-vars and SCL-vars with the resolving of namings in the preview buffer.


complex
libnames/filenames
Hard coded libname filename (physical names) settings (macro-variables) options etc should be isolated to a startup source.

The old autoexec is a good place to place this information as is the metadatabase. Eliminating the physical names to settings make sources/code easy to get into versioning.

complex
password/keys
To hide/restrict passwords to the data is a requirement working with possible sensitve data. Many hints can found like 013-2009 Secret Sequel presentation.

The solution for te doormat paradox is having in inside in a secure environment. That is my proposal having those keys / password in secure environment and the sql access stored in password encrypted file (sas dataset). The os (host) security is providing the secure environment
Worked out as sample source. hiding key-pswd




Logic Coding & Usage (2)


Debugging

every sub-languge has debugginh features:
complex
base
The use of altlog sytemoptions is only possible at startup.
Proc printto is able to switch while running. Also possible to use with web-services.
Tips: at Unix is posiible tos user ~ (home location) and %-modifiers.
filename replog '~/Dev1_%y%m%d_%I%0M%0S.log' new;
proc printto log = replog new; run;

And stopping the printto.
proc printto log=log; run;

complex
base log4sas
aside arm-logging with a logserver. local logging is possible.

The SAS language enables you to use the SAS logging facility in a DATA step and in macro programs

geniaal
macro debugging
Is within SAS/base (and connect) easy with the options mprint mlogic source2 symbolgen.

To be used with complex macro-s as standard fucntions an macro call options to debug is to be parameterized.
options mprint mlogic source2 symbolgen ;



Within Eguide usage (4.1) the normal SAS macro behavior is disturbed.

geniaal
Intrnet - STP debugging
Is within SAS/base (and connect) easy with the options mprint mlogic source2 symbolgen.

SAS Intrnet

Stored Processes




complex

Date Time

With formats
kb42057 Writing a datetime value as m/d/y h:m:s AM/PM.
Use of: %m/%d/%Y %I:%0M:%0S %p as modifiers

Date Time indication (mynotes) OS date time variables.
Knowing SAS is written in C (java) and using Unix the %-modifiers also work with using external dataset names. Just as a "!" is indicating an operating system environtvariabele to be used. Or the "~" usage in a Unix datasetname.

complex

Time outs

WEB sessions
fire
Error processing in SAS
error processing (lrcon). The checkpoint restart is a new feature in 9.3. Having the erorr procssing design changed again. It was not stable in 9.1.3 never solved.




Logic Coding & Usage (3)


handy
Using ODS


handy
Using DI , BI


handy
Using Miner


handy
Using BI dashboard Flex


handy
Using IML


handy
Proc metalib


handy
SAS Stored Process





Logic Coding & Usage (4)


complex
Good habits
Programming and coding is a piece of art ot do it well.
Some of making it well documented being reusable by others are obvious.

complex
Parallel test
One of the most challenging parts to impmented as it cot possible to be doen by testers.

complex
Obfuscate / compiled
Obfuscation target is to hide the source code. There are Legal reasons like not wanting to let it show up easy or debug by the wrong people. A lot of options like: All catalog types of SAS can be placed in a encrypted password protected library. In that way the contents itself are protected.

SCL is licensed by SAS/AF is a supported way: kb 12488 . In my hardening part is the reference of advice using with SAS/intrnet because of not able breaking down the compiled code by users.

complex
WEB based / sessions
Using Sessions (SP Stored Processes: Developer's Guide)
Due to the stateless desing of the web. Sessions must be solved in other way. The SP documentation is the same approach, same calls, same naming conventions, as is used by SAS/intrnet. conversion SAS/Intrnet to SP shows the logical evolvment.
This type of functionality is not possible by WS servers/processes.

The direct generating of HTML is still a valid approach. kb37050 Sample 37050: Creating a custom HTML input form for a SAS(R) Stored Process.


WEBDAV is the component for storing and (versioning) the content.

Converting SAS/IntrNet into SAS Stored Processes. (SAS 9.1.3 Integration Technologies » Developer's Guide - STP


Converting SAS/IntrNet into SAS Stored Processes. (Angela Hall 2005)
Earlier in the year, I attempted to convert to a SAS Stored Process a SAS IntrNet program that used statements.
There is an undocumented workaround available (that I didn't tell you - wink wink). Just modify your SAS IntrNet file's ods html body statement to:
ods html body=_webout(no_bottom_matter) path=&_tmpcat (url=&_replay) style=sasweb;
ods html close;

start your data _null with:
data _null_;
file _webout;

and add prior to the ods html close statement:
ods html body=_webout(no_top_matter) path=&_tmpcat (url=&_replay) style=sasweb;


complex
Soap (internet) Wb service
Proc soap With PROC SOAP, you can include an optional SOAPEnvelope element in your XML file. Do this if you want to include custom information in the SOAPHeader element

sgf2008/soap Using SAS BI Web Services and PROC SOAP in a Service-Oriented Architecture



OS utilities use

geniaal
Using jar for zip
You should probably mention that jar is part of the Java installation, and creates compatible zipfiles which can be extracted with unzip and so on

jar cfM /tmp/sub_dir/pertinent_dir.zip -C /tmp/sub_dir pertinent_dir


To extract only certain files from a jar file, supply their filenames:

C:\Java> jar xf myFile.jar foo bar



geniaal
SAS piping
A most simple and handy way to get OS taks done started with SAS.

filename hulptar pipe " cd %sysfunc(pathname(&prjctpf.d)) ; tar -cvf StoredProcess.tar StoredProcess " ;
DATA WORK._NULL_ ;
  infile hulptar ; input; put _infile_;
run;quit;



geniaal
dir list
New functions in 93 are: Dopen Dinfo Doptname. Also you have Fdelete
With these functions there should be noe need to uses the x-cmd or piping methods anymore.
doptname (lefunctionsref).



geniaal
dataset info
data fileatt;
  length name $ 20 value $ 40;
  drop rc fid j infonum;
  rc=filename("myfile",My-file-spec""); put rc= ;
  fid=fopen("myfile"); msg = sysmsg();
  infonum=foptnum(fid);
  do j=1 to infonum;
   name=foptname(fid,j);
   value=finfo(fid,name);
   put 'File attribute ' name 'has a value of ' value;
   output;
  end;
  rc=filename("myfile");
run;

File attribute Filename has a value of /sas/student/assess
File attribute Owner Name has a value of melliott
File attribute Group Name has a value of stassmt
File attribute Access Permission has a value of rwxrwxrwx
File attribute Last Modified has a value of Tue Jul 24 10:15:22 2012
File attribute File Size (bytes) has a value of 831266816





Tuning your SAS application

geniaal
programmer responsible
As SAS programmer you have the biggest influence on the resulting performance. The subject is part of the advanced programmer training and certification.
The following is a selection of presentations (proceedings) on this subject.
357-2012 performance top ten
257-2012 performance 50 Ways more Efficiently
333-2009 performance
041-2009 performance energy hog


complex
optimize code
The first thing to achieve: Don´t let the machine do something that has no meaning. Do you have performance problems with new code?

First review this code to find options to save resources. These options have possible the biggest impact saven factors of 3,10,20 times giving faster results. See: coding & usage.

The code factors in savings are hardly possible to achieve within the OS level. The OS-levels are mostly are already tuned to run acceptable. Processor speeds are almost impossible to change,and when just by a small percent of it.

complex
Sort processing
With SAS 9 has become multithreaded. Alos Utilloc is introduced
Externalsort is possible by syncsort interface.
140-2010 Dear Miss SAS®Answers: A Guide to Sorting Your Data


complex
In database processing
SAS has Access modules. These modules are connecting to data-ptoviders with usage of the technical unique specifications of the DBMS/storge methos used.
ODBC and OLE/DB are able to access data in a standard uniform way. These interfaces are failing with more advanced requirements. SQL Pass-Through Facility Specifics for Netezza (acreldb)

some links (concepts):
Attention should be given to the correct fucntioning of advanced interfaces. some links (errors/fix):
geniaal
ARM
The Analyzes of the SAS application can start by 219-30 analyzing SAS-logs an old method but in certification news 2011 still valid.
Building own arm-logging (arm-2 with 913 , arm-4 with 93) is more advanced. ARM has its own challenges.
See the tuning SAS chapter for more details.

geniaal
SQL statement
Sql guidelines:

One thing you need, is to avoid the Cartesian joining of tables. As a pitfall this is the default of joining tables with SQL.

geniaal
SQL options
Sql libname & db options
Tuning and understanding response times
One thing you need, is understand the third party behavior.





geniaal

Guidelines: business data Usage

As IT-staff you don´t want to have uncontrolled access to business data.
Some actions has to be done:

secure bypass
indentifcation/autorisation misunderstanding
Misunderstandings: User administration or registration - resulting to audit-checks (SAS institute - consulting 913)
See hardening and SAAS chapters. Not able to solve.


secure trace
logging auditing apm tool
366-2012 Logging 101: Leveraging the SAS Enhanced Logging Facility
ARM - Application Response Measurement
ebiapm92.faq SAS® 9.2 Audit and Performance Measurement


Wil need a lot of configuration.
like rats
passwords in configuration files
Metadata maintenance - no way found (documentation) getting passwords changed/updated in the configuration. Someon change policies and we needed these updates. See hardening chapter. With SAS 93 feature updating passwords is added. That procedure will not fit to common policies having passwords of service accounts central managed.



geniaal

Education - Knowledge

This is an on going activity. Sharing the knowledge to users and managers.

complex
Tagsets
This is a part of ODS this is very living.
It is part of an installation but updated many more times. These updates are falling out of normal support focus (disadvantage). The advantages are: Usage Note 32394: Installing and Storing Updated Tagsets for ODS MARKUP
SAS handles tagsets similarly to templates. The tagsets that are shipped with SAS are located in the SASHELP.TMPLMST item store. When you create or update tagsets, they are saved by default to the SASUSER.TEMPLAT item store.
ODS MARKUP Resources Whenever SAS R&D issues new or updated tagsets between releases, they are available for download below, under Get the Tagsets.
More information in links, pandp ,a collection of presentations like: 003-2010 data to pivots in excel.

geniaal
Strings , Perl - cat
Normal string processing is enhanced with Perl string. the PRX fucntions.

Classic string handling is enhanced (cat). 205-2010 enhanced string concatenations



complex
bigdata , number of observations
kb36112 Managing large SAS® data sets that exceed the maximum number of observations.
A SAS data set stores the number of observations it has as an integer in the C programming language signed long-data type. This means that the maximum number of observations that can be counted for a SAS data set is limited by the long integer size for the operating environment.

There is a dependicy with 32-bit and 64-bit operating systems. The SPD engine and oher external data do not have this counter, processing big data (saswork) must be done with care.

geniaal
Remote catalogs - run multi machine
Remote catalogs are qworking fine (SAS/connect RLS). Even when running different SAS-releases
The only issue: updating catalog remote is not possible.

Updates are to be done with doing uploads (SAS/connect)



Business Releases / Version management


confused

Artifacts SAS versioning & life cycle management

There are many types of artifacts in a SAS environment. Not all of them are applicable in a versioning process. Every artifact has to reviewed for versioning options and techniques.




confused

Versioning or LCM context

The word versioning is used in different contexts. Very confusing. Versioning and LCM (Life Cycle Management) words.



IT specialists tend to thinking of the coding problems of developers when many are working in big projects.
Focus: the detailed changes programmers are doing.

Auditors tend to think on retention policies. Wich version of the truth was valid at a dedicated moment.
Focus: the detailed changes on the production environment and possible impact.

The focus should be on underpinning the state of a production environment.
The working approach of developpers is a minor question.


SAS tools supporting versioning


papers
Metadata evolving
376-2012 Evolution of Best Practices for Metadata Change Control in SAS®9 = Diane Hatcher


SAS is moslty an IDE, see IT words - IDE chapter - subject Integrated Devlopment environment
Some parts of SAS explicit are meant for developing. They have the special options mentioned for IDE-s.

Other parts are meant for "end user computing". End users are out of control with versioning.

papers
DI Studio
DI developers A team of SAS Data Integration Studio users wants to work simultaneously with a set of related metadata. They want to avoid overwriting each other's changes.

User transformations can be defined within metadata or as sas-macro Code Storage Options for SAS DI Studio Custom Transformations (amadeus)

papers
Eguide
137-2010 This promotes many new features of Eguide (4.3). C.Hemedinger is involved. About versioning:
Customers often request for an easy way to use source management/version control systems with their SAS Enterprise Guide projects. The challenge is that a project file (with the .EGP file extension) is a single binaryformatted file that is not compatible with change management and comparison tools. However, you can maintain your important SAS content, such as data files and SAS programs, as files that are external to the project file. In this scenario, the EGP file is simply a recipe for how your work fits together, but the main ingredients are accessible on your file system.

papers
AF SCL
009-31 (app dev) Everyone trying to update at same time? Lets try source control manager SAS/AF.
Guide to SAS/AF(R) Applications Development (appdevgd 9.3) Migrating Your Application from Testing to Production.
Note: Although you can use operating environment commands to copy SAS files, you should use the COPY procedure instead. After you edit catalogs in the build environment, some entries can become fragmented. When the COPY procedure executes, it reconstructs and reorganizes all SAS files, including data sets and catalogs SAS/AF software enables you to create an application in one development environment and to port that same application to the operating environments of your users.
Note: Although SAS/AF software does not support a build-time environment for FRAME entries on a mainframe, applications can be developed in desktop environments such as Windows and ported to run on a mainframe.


papers
Metadata generic
MetadataServerchngmgmt (papers sas) All metatadata related artifacts
UsageVersion-id metadatabase OMA reference and usage

A complete inventarization with dev test prd: GKA100307 (tata consult) Life Cycle of SAS Intelligence Platform Project

Metadata records
source in metadatbase versioned records. There are version indictions in the records, how to work with them is not found.

Metadata objects
replication-wizard-deprecated (paul)



papers
WEBDAV
bimtag 93, About the SAS Content Server Webdav Web-based Distributed Authoring and Versioning
Examples of such content include reports and documents created by users of SAS Web Report Studio and the SAS Information Delivery Portal.



papers
Models
modelmanager/manager/factsheet Modelmanger is managing models. Not only from SAS miner, als other modelling tools. PMML is the standard of a model defnition. In that way a model has become an artifact.


papers
Versioning add-ons
Scope
SAS is used in many ways, there is no dictate of wich way should be used. The following cite:
Most organisations will use the benefits of a local network drive, a mounted share or a dedicated SAS server file system to store and archive study data in multiple formats, analytical programs and their respective logs, outputs and deliverables.

A manual process is most often implemented to retain versions and snapshots of data, programs and deliverables with varying degrees of success. Although not perfect, the process is sufficient to a degree. The step from a local file system to enterprise environments can be a fair investment and a high degree of change management if you already have an evolving analytics environment.
(365-2012)

Having basic plain SAS code on a file system is a different approach compared to the complete BI-server approach.
SVN (Apache Subversion) is an open source sample that can be used within this scope. The way of working with many copies introduces a new problem of the correct copy. Everyone should have a local copy to the coorect version. 365-2012 Simple Version Control of SAS® Programs and SAS Data Sets
This highlights one of the minor nuances of Subversion and the working copy as users have to ensure that the working copy remains up-to-date with the latest or pre-specified revision.



Versioning basic approach


geniaal
CM - file locking
271-2009 (sys architecture) IEEE Std-729-1983 defines Configuration Management (CM) as “the process of identifying and defining the items in the system, controlling the change of these items throughout their lifecycle, recording and reporting the status of items and change requests, and verifying the completeness and correctness of items.”

File locking is a single update in time to developers. Avoiding the versioning problems with copies by many developpers.
The simplest approach is to avoid private working copies and use file locking to prevent concurrent development
The SAS system has been successfully integrated with many SCCS including CVS, Subversion and ClearCase. However, we point out that the integration of SAS9 with a SCCS is not straightforward, largely because of the metadata component.

geniaal
Classic SAS code - Dev Test Prod
Some developing links: Some test links: Some design links:

geniaal
Windows
Using SAS Foundation for projects: 372-2011 Setting Up a Different Windows Environment for Each SAS® Project

geniaal
DATA sets
cc14 nesug07 Audit trail, versioning of records in a SAS-dataset and SAS datasets as whole SF09209P triangle figure of audit trail




Bu Life Cycle Management

Software library, parallel development, promote, ship deploy
confused
Software library, parallel development, promote, ship deploy
The concept is explained in DTAP lcm scm
The detailed steps are listed here.


handy
SCM: Promote D,T
When the developer is ready, the artifacts can be promoted to Test.
In a simple approach with little developpers it can be left to the developer. When the developer and testers are physical placed together on one location this is smart to do.
A ship/deploy action is not needed if Develop and Test are sharing the same machine

handy
SCM: Promote T,A
When: The most simplified approach is promoting every artifact of the project to Acceptance stage. The developper/tester is responsible for the test environment
SCM: deploy - ship A
A ship/deploy action normally is needed. If the Acceptance environment is managed separately (as expected) this is the most smart way.

handy
SCM: Promote - ship- P,Z Archiving
Before overwriting the production version a save/archive is needed in normal releases.
When applying a fix or minor update this step can be passed having complete versions saved as total.

handy
SCM: Promote - ship- A,P
When Acceptance testing is succesfull finished,the customer is wanting it to use it. Production is the final stage. A change process with approvals is commonly involved.
A fall-back sceneriao must be present. Having a backup of all artifacts at 1/ the promote and 2/ the deploy/shipping proces.
SCM: deploy - ship P
A ship/deploy action is needed. An exact image as tested will be roll out.

handy
SCM: Special actions -1
Configuration central managed having effect to the business

  1. Defining libraries filenames or macrovars
  2. Deletion of artifacts followed by a deploy -ship


handy
SCM: Special actions -2
Configuration central managed having effect to the business

  1. Setting up/adjusting secuerity (host layer)
  2. Setting up/adjusting space resources




Whats new, transitions

Every version release of SAS is bringing new features and functions. Even fixes are sometimes changing a lot as they are used to add fucntionality.





handy

Multi threading

With parallel processinga major architectural change has been implemented from 82 to 91. It is implementing the multi threading.

handy

Strange OS effects

CSV

wtabort






handy

PROC SQL

Should be easy as no new features are expected when following ANSI SQL. The truth is some undocumentedal changes are implemented. With V9 it has implementing multi threading. The SQL optimizer is changing, debug options are changing.interfaces are changing.
Overwriting input Table
proc sql
select work.data001n
from work.data001n
where upcase(name) = 'TEAM' and ..
;

Was working until SAS V8. It was never allowed with Ansi-SQL. With the introduction of threaded kernal (V9). The syntac generates an error.
Schema option
Was allowed to code in SQL explicit pass through code but did not have any function. Is not allowed anymore since SAS V9
: or ; in SQL syntax
This was allowed to code in V8. The : was corected in a ; if necessary. That does not work anymore with V9 generating an error.
Order in result Table
When coded without an order option SQl is free to give the results in anys order.
Until SAS V8 the SQL process was a single process resulting in keeping the order of the input table.
With V9 SQL became multi-threaded. The order is not any more followed like the input table.
More / enhancements
Additonal extensions
An early extension to PROC SQL development was the CALCULATED keyword.



complex

ODS

With ODS a major architectural change has been implemented from 82 to 91. It is implementing the CSS of html standard. Former dedicated SAS approaches are set aside.
_webout
tagsets
HTML version - themes

odstemplates is a command like regedit to view and change setings. It will show active templates/styles.
Itemstore sas7bitm
Templates are store in a .sas7bitm file. There are restrictions with versions.
When the wrong type is copied nor warnings or errors occur, just when updating with a source a message is given.
ERROR: File DATAFRNT.TMPLMSTW.ITEMSTOR was created for a different operating system.
ERROR: Template 'Styles.Global' was unable to write to template store!


proc template;
path sashelp.tmplmst;
list base.univariate / sort=path descending;
run;




handy

Encoding

Encoding and NLS is changing constantly. With SAS 9 encoding has become a table attribuut.




fire

Language changes

The language requirements may change. The new features may result in a clonflict with old code. Fucntions mya behave different
syslput
A new option was added with V9. With an option the remote machine is named. When having a / used before as text, this does not work any more. The text has to be quoted now.
strings
A string was allowed to start immediate after an other special character without a blank. Starting in 913 this space saving approach is generating an error. Expatation is it go into an error with newer versions.
Substr
When having an pointer error offset of staring string. The result with this error is undefined. V8 did leave the string untouched and continues V9 does a cleanup of the string.

debugging - file-io - webbased
For some reason some programs run in a webbased environment and using Unix(Linux) and doing X commands are seeming to go into indefinite waits. The IO destination of the running prorgran is unclear possible causing the issues. The other solution with executing with pipes is working good.
When a SAS-intrnet debug-session is hanging alos with debugcode 131 , the log with debugcode 128 can open up.

call symput
Macro variables can be filled using data, the ,macorvarible itself can be varied.
DATA _NULL_;
   SET last_used_relbehcd;
   CALL SYMPUT(regio, TRIM(LEFT(PUT(relbehcd_max, BEST.))));
RUN;


Starting with 9 the generated macro-variable name must be valid.
ERROR: Symbolic variable name #N/B must begin with a letter or underscore.
NOTE: Invalid argument to function SYMPUT at line - column 20.
regio=#N/B relbehcd_max=. _ERROR_=1 _N_=1
NOTE: The SAS System stopped processing this step because of errors.

Starting with 9 the generated macro-variable name must be valid.
NOTE: Invalid argument to function SYMPUT at line - column 20.
regio=#N/B relbehcd_max=. _ERROR_=1 _N_=1
NOTE: There were 8 observations read from the data set WORK.LAST_USED_RELBEHCD.


(more)





fire

new procedures (xcmd needing)

SOAP, GROOVY (93)
Soap   Groovy that are also added. Groovy note: requiring noxcmd=off. Background
Many procedures and solutions are requiring "xcmd". This "xcmd" option is getting set off by design (Eguide IOM)

FCMP SCAPROC OPTSAVE-OPTLOAD (93)
FCMP SCAPROC OPTSAVE-OPTLOAD are new procedures that should be known. Less to be searched for are:




Technical support


like pioneer

Configuration: Reporting CSS

Every business company has his own indentity his own logos.
Generating reports this identy should be shown within these reports. That is where the CSS is coming in. The CSS is inhirited from the Html approach.

changing default css


Lays out in prompts





like pioneer

Configuration: Reporting distribution lists

Every business company has his own contacts.
Generating reports contacts can get informed by mail. The distribution list has to be defined.


(to be come)
like pioneer

Autoexec - Metadatalibnames

Remember the hardening chapter? On users request SAS is opening up again "autoexec.sas" usage, blog autoexec . More to know about eguide 4.3 libraries metadata - SAS admins
autoexec like using Eguide - (startup code whenver connected) (amadeus)

It solved an autoexec from Eguide. Not from batch or other starts. Smart approach to know: blog autoexec (may 2014).

Why is a simple question like this project-autoexec that difficult?
solution proposal:
The autoexec just need to do:

%let prjct= %sysget(prjct) ;
%&prjct ;


Identification Users/Groups

confused1
password limitations
/bi/2007/03/22/password-limitations Special Character problems in passwords continue to cause issues, and many organizations require the use of at least 1 to pass security requirements.

Specific ones to stay away from ...
@ ... Just today this caused problems in remote connections to Meta Engine Libname. & ... This can affect users connecting to the SAS Web based Products (Portal, Web Report Studio, etc)


Per SAS install doc (http://support.sas.com/documentation/configuration/biig.pdf), only one $ in a password. So Pa$$w0rd is no longer a viable option.

Special characters

confused1
Naming Metadatarecords
An error occurs if the stored process name contains an ampersand (&&)


The error message associated is misleading. There is an illegal character in the entity name.

Metadatasecurity

(to come)



Understand Bu    Support Bu    Run IT    SAS Advanced     Other Experiences     top  bottom

how to run IT

SAS setup in grid

netwerk

Security support


geniaal

metadata BI Enterprise

Table 1 in the following document is mentioning the type of users. Only information consumers are set to external-users as option.
It mentions also prblems with the more difficult to manage developpers environment. The security holes mentioned in the hardening chapter are not recognized. The unexpected deletion of metadata artifacts is a noted experience. 200-2007 "How Shall we Secure the BI Enterprise"

geniaal

metadata Eguide

Best Practices for Administering SAS ® Enterprise Guide:
297-2012 SAS® Enterprise Guide® Implementation and Usage
s long as users have the necessary host operating system permissions, they will be able to directly access and read a SAS table. So, what this boils down to is that any real data security must be put in place at the physical host operating system layer.

That is what I have documented in the hardening part. It also an operational issue.

like dark
Keys and passwords
Unlock an Internal Account (bisecag) We recommend that you establish individual metadata administrators rather than sharing the predefined SAS Administrator account
Knowing the password and waiting for an hour is an other option.

Update a Managed Password (bisecag)
like hell
startup sequence
Share-server needs to be administrated. A dedicated startup and sepearte procedure to allocate/free libraries is to be setup.
This is a omission from normal proceures described by SAS





Life Cycle Management IT

archive

LCM: Setinits

Mostly every year the license needed to be updated.
The renewal fees have the payed for first. This process getting handled in time can be frustrating as many people want to influence something.

archive

LCM: Hot fixes - releases -

On a very regulat times hot-fixes are becoming available. Every 3 month´s it can be necessary to do an update. Sometimes the client and server sides needed to be updated.
SAS information: 354-2012 the Art of SAS® Maintenance
hotfixes
Hot fixes can be analysed with a tool starting with 92. ftp SASHFADD   usage reading the goal/environment. With 93 this tool is outdated as de Sas deployment Manager will take over.

geniaal

LCM: Status - releases

Release status can be analysed with tools starting with 92. View registry and product_status.

geniaal

LCM: whatsnew

Eguide
egwhatsnew51
302 2012 Up Close and Personal with SAS® Enterprise Guide® 5.1


geniaal

LCM: Migration

focus area
Migration
Know what you are doing.




Technical Support

like hell
startup sequence
Servers
Metadata / object spawner - start up sequence failore caused by a bad script (SAS institute -913)
Solved by making a copy of the script. Added automated check metdata-server is running before (re)starting other servers.

like pioneer
Autoexec - Design change
Beginning with version 9.2 SAS installs with a set of config files and autoexecs which are not meant to be modified. It says clearly in the header of such files:
* Do NOT modify this file. Any additions or changes should be made in autoexec_usermods.sas.


Support users:
Users are encouraged to use the autoexec project in Eguide. It will act as a autoexec of the classic SAS.




geniaal
Eguide
Many links to using Eguide.


Addins are possible from other parties.




Install: Adding Nettezza
geniaal Adding a part of the complex of all SAS components is a difficult task. First thing to check is the versions and compatiblity. Matric view SAS version - Netezza driver - OS The software can not be easily added. Fixes and confoguration must be checked. After all the installation and testing must be done in a not by users commonly used machine. The installation/configuration stage is getting back here.




Tuning Monitoring

Logging

archive

Logfiles

Logfiles scattered all over different locations is very difficult to monitor and analyse. advantages-abound-by-centralizing-all-of-those-logs
Very advanced tools can be used for monitoring the machines and performance. Simple approaches can do a lot. 010-2007 Building Your Own Real-Time SAS Server Monitor under Unix
Grid processing is making it a little more complex 370-2012 RTM and SASGSUB, the Power to Know… what your Grid is doing

geniaal

IO Storage (OS level)

The IO, the speed to the storage, is the most limiting resource in the building blocks as it is the slowest of all. So this has the most attention in speeding up.

Usage Note 42197: A list of papers useful for troubleshooting system performance problems
P&T Middleware is made a dedicated chapter.


The sizing requirements of storage will increase. No issue as the sizing of hard disks had been grown the last years. You can easily get a 1Tb disk drive or 300Gb SSD. But how to get it into your central IT is a question.

geniaal

IO Storage (SAS level)

aligning on 8 bytes and padding can increase datasetsize with conversions. (since v8, note 41214)

Observation Length, Alignment, and Padding for a SAS Data Set A list of papers useful for troubleshooting system performance problems


The compress option of SAS is decreasing the datasets size at the cost of cpu.

archive
SAS Work
The sas work has to be cleaned up. Atlernatives and failures in normal clean-up with a day selection. unix script 3 days
manage sas work usage - mail users


Mail SMTP
like pioneer This means sending mail by using SAS. Having a mail client on the desktop no additional configuration is needed.

Used in coded way (program script) it is done by Filename SMTP (9.3 lestmtsref).
There is a need with system options to get the stuff working. A mailserver. Portnumber (normally 25) and a dns-adres of the mailserver. The mailserver must be configured to allow sending mails of your server. See: mail options (9.3 lrcon).
Getting to rights to relay messages on a mail-server is the biggest challenge to get it working.
Public mail-poviders are often deliver this option requiring an authorization (user/pswd) phase.

like pioneer
Ftp Sftp
filename ftp options (9.3 movefile).
Watch the umask setting with the command.

SFTP is for better security also possible FILENAME Statement, SFTP Access Method (9.3 lestmtref).
geniaal

Olap tuning

Olap is nothing more than having calculated in advance a lot of summary statistics.
With many combinations ver large datasets can be created. A 32-bit counter limit is kb45491 . kb32278 Calculating the maximum tuple size for rendering relational data in crosstabulation tables in SAS® Web Report Studio and SAS® Web Report Viewer

modern

Eip Web-report

Web-report and all other middle-tier clients (servers) are heavily related to the EIP platform.
Some special usefull links with information for the role platform admin.




ARM - performance - Security access

armarch
geniaal
ARM
ARM is en open source standard set up by Tivoli. Tivoli is now owned by IBM. With SAS 9 the Arm 4 version is used for all kind of monitoring. It is the log-server doing this.

Bij changing the configuration options (xml-files) it can be tailored to specific requirements.

See:

footstep

Monitoring apm

Still evolving.

See:

footstep

Monitoring rtrace

Still available usefull.

See:

footstep

Monitoring ostools

Available and usefull as doing OS level.

See:

geniaal

Grid Connect

The planning of a grid I have put into P&T middleware chapter
Operational life to be filled in later.

Grid is using LSF and SAS/connect. The most complex parts to do.

like pioneer
Parallel processing
With the free e-learing module (amo,eguide 5.1 - Stored processes 9.3) got some attention points. MP-connect was a pittfall with getting to run em 5.3
With SAS/intrnet (application broker 2004) the same approach of executives and parallel processing was implemented.

like hell
Threaded Kernel
Performance improvement in a way circumventing the processor speed limits.
The promise of better performance is nice. When on the other hand reliability and behavior is affected in the wrong way, as it looks to be, we can only become very unhappy.
The issue with share-server (see ..) could be originating at this design change.

like hell
Spawner - stopping unexpexted
Sometimes the tcpip stack is troublesome.

Users complaining about lost sessions after about 2 hours. As the direct terminal access without SAS also behaves this way, the root-cause should be within networking firewalls.

The service process is not always reliable. 31969 Spawner can stop by tcpip errors. No recovery other than restarting. No option to recognize/monitor withoud bothering users.
17753 Poort number mainframe and service name not nice coded (warning).
11240 Account code mainframe only can be made hard coded, not user dependent as should be.
11240 Port defined to task (tcp.profile) is different as naming resolution (etc.services) mainframe.
11240 telnet is poort 23. This is the origig of SAS/connect still present as default value.



geniaal

Report & analyses & configuration validate

It is the unwanted (ore not) that should be avoided. The work of this avoiding is ....

like pioneer
SRX - performance (Eguide)
The generation of a SRX report is performing too slow. No clue why. (2012)

like pioneer
CSS - business template
Should be set up, made available. According to used house-style of business.

The location is setting up hidden in the default installation. Normally closed not able to update.




Disaster & recovery - Business continuity

DR help help
Disaster & recovery


work outside
Business continuity


Understand Bu    Support Bu    Run IT    SAS Advanced     Other Experiences     top  bottom

SAS advanced

Way problems
tocv dvd 93 12.1 -= bookstore


sas logo

94 documents - not found in previus collections

SAS Base & Procedures
SAS(R) 9.4 DS2 Language Reference


Changed color scheme
The new analytical views (LSAR) is has given a complete new range of problems that are possible.
SAS Analytical 12.1
New version announced as 12.1 (end aug 2012).
Analytical Products 12.1
This analytics parts still refer to 9.3 as base SAS platform (eip) used (deployment outline).
Color scheme is changing to a grey world like the HPA forecasting.

servers (services) 9.3
The approach of many dedicated servers started with SAS 9. This has given a complete new range of problems that are possible.
Without manuals to review work can not be done.




sas logo

SAS base - standard products (9.3)

SAS Base & Procedures
SAS(R) 9.3 Language Reference: Concepts
SAS(R) 9.3 Statements: Reference
SAS(R) 9.3 Data Set Options: Reference
SAS(R) 9.3 Formats and Informats: Reference
SAS(R) 9.3 SAS(R) 9.3 Functions and CALL Routines: Reference
SAS(R) 9.3 System Options: Reference
SAS(R) 9.3 Macro Language: Reference
SAS(R) 9.3 Component Objects: Reference

SAS(R) Guide to Applications Development
Step-by-Step Programming with Base SAS(R) Software

SAS/AF(R) 9.3 Procedure Guide
SAS(R) Component Language 9.3: Reference
Base SAS(R) 9.3 Procedures Guide
Base SAS(R) 9.3 Procedures Guide: Statistical Procedures

SAS(R) 9.3 Output Delivery System: Users Guide
SAS(R) 9.3 ODS Graphics Editor: Users Guide
SAS(R) 9.3 ODS Graphics Designer: Users Guide

SAS(R) 9.3 National Language Support (NLS): Reference Guide
SAS/CONNECT(R) 9.3 Users Guide
SAS/SHARE(R) 9.3 Users Guide
Communications Access Methods for SAS/CONNECT(R) 9.3 and SAS/SHARE(R) 9.3
Moving and Accessing SAS(R) 9.3 Files
Application Messaging with SAS(R) 9.3, Second Edition

SAS(R) Universal Viewer 1.3: Users Guide
Download universal viewer

SQL MDX OLAP
SAS(R) 9.3 SQL Procedure User's Guide
SAS/ACCESS(R) 9.3 for Relational Databases: Reference, Second Edition
SAS/ACCESS(R) 9.3 Interface to PC Files: Reference
SAS(R) 9.3 Drivers for JDBC: Cookbook
SAS(R) 9.3 OLAP Server: Users Guide
SAS(R) 9.3 OLAP Server: MDX Guide pdf
SAS(R) Scalable Performance Data Server(R) 4.53: User's Guide
SPDS is still in focus or not?. Within scalability - in memory analytics
SAS(R) Scalable Performance Data Server(R) 4.53: Administrator's Guide

SAS(R) 9.3 In-Database Products: Users Guide

Statistics-1
SAS/STAT 12.1 index documentation
SAS/STAT 12.1 User's Guide
SAS/IML(R) 12.1 index documentation
SAS/IML(R) 12.1: User's Guide
SAS/OR(R) 12.1 index documentation
SAS/OR(R) 12.1 Users Guide: Project Management
SAS/OR(R) 12.1 Users Guide: Mathematical Programming
SAS/OR(R) 12.1 Users Guide: Constraint Programming
SAS/OR(R) 12.1 Users Guide: Bill of Material Processing
SAS/OR(R) 12.1 Users Guide: Local Search Optimization
SAS/ETS(R) 12.1 index documentation (forecasting arima spectra loan model timeseries)
SAS/ETS(R) 12.1 User's Guide
SAS/QC(R) 12.1 index documentation

Statistics-2
SAS enterprise Miner KB Product documentation.
With Example data for the getting started examples.
Several others like text mining referenced
SAS(R) enterprise Miner 12.1 index documentation Documents are secured now, needing user/pswd
7.1 documentation is still acceptable (aug 2012)
Getting Started with SAS(R) Enterprise Miner(TM) 7.1
SAS(R) Enterprise Miner(TM) 12.1 Extension Nodes: Developer’s Guide
SAS(R) Enterprise Miner(TM) 7.1: Administration and Configuration, Second Edition
SAS(R) Model Manager index documentation
SAS(R) Model Manager 12.1 Users Guide PMML 4.0 can be imported/exported

SAS(R) Enterprise Miner(TM) 12.1: JMP Extension Nodes

SAS Intelligence Platform -1
SAS(R) 9.3 Overview (biov)
SAS(R) 9.3 System Administration Guide
SAS(R) 9.3 Security Administration Guide (bisecag)
SAS(R) 9.3 SAS Web Infrastructure Platform
SAS(R) 9.3 Application Server Administration Guide
SAS(R) 9.3 Data Administration Guide, Second Edition
SAS(R) 9.3 Desktop Application Administration Guide


SAS(R) 9.3 Guide to BI Row-Level Permissions


SAS(R) 9.3 Logging: Configuration and Programming Reference


SAS(R) 9.3 Web Application Administration Guide
SAS(R) 9.3 Guide to Metadata-Bound Libraries
SAS Intelligence Platform -2
SAS(R) 9.3 Integration Technologies: Overview
SAS(R) 9.3 Stored Processes: Developer's Guide
SAS(R) 9.3 BI Web Services: Developer's Guide
SAS(R) 9.3 Publishing Framework: Developer's Guide
Application Messaging with SAS(R) 9.3
SAS(R) 9.3 Integration Technologies: Java Client Developer's Guide
SAS(R) 9.3 Integration Technologies: Windows Client Developer's Guide
SAS(R) 9.3 Integration Technologies: Directory Services Reference
SAS(R) 9.3 Foundation Services: Administrator's Guide


SAS Intelligence Platform -3
SAS(R) Data Integration Studio 4.5: Users Guide


SAS(R) BI Dashboard 4.31: User´s Guide
SAS(R) Information Delivery Portal 4.3: Help
SAS(R) 9.3 SQL Query Window User's Guide


SAS® Add-In 5.1 for Microsoft Office: Getting Started in Microsoft Excel, Microsoft Word, and Microsoft PowerPoint (pdf)


SAS Intelligence Platform -4
SAS(R) 9.3 Audit, Performance and Measurement
Audit, Performance and Measurement Package for SAS 9.3 Enterprise Business Intelligence
Administering Logging for SAS Servers (bisag)




SAS metadata - structure interfaces

The Internal structure of the SAS-metadata is a special SAS topic. It is their intellectual property and is allowed to change as they wish. There is a still a lot documetned as Within the namespace of the omamodref the structure/realations of eg Transformations (Di studio) is given.

SAS miscellaneous

Data quality (data flux)
SAS(R) 9.3 Data Quality Server Reference

Work flow.
SAS(R) Workflow Studio 1.2: User's Guide

SAS intrnet the first web enablement.
SAS/IntrNet(R) 9.3: Overview
SAS/IntrNet(R) 9.3: Application Dispatcher

The Logging server is difficult to find, strange as it is the key factor with monitoring and logging.
See:
Servers in the SAS Intelligence Platform - Logging and Monitoring for SAS Servers (biov)
SAS(R) 9.3 Logging: Configuration and Programming Reference
SAS® 9.3 Interface to Application Response Measurement (ARM): Reference


SAS encryption

The encryption tasks are also not easy to find (documentation):
See:
Encryption in SAS(R) 9.3
About SAS/SECURE (bisecag)



Reading all this you could think "it is not good". The problem of SAS is that is that many, overwhelming.
You can find all documentation. Compared to others: most suppliers do not even provide this level of information.


SAS new evolvments

SAS Studio

SAS studio (web-based SAS base)

SAS JMP

JMP Product Documentation (Coming in with Miner license)

JMP / Visual analytics

SAS(R) Visual Analytics 6.1: User's Guide


SAS Visual Analytics (index page)
SAS Visual Analytics (demo-trial and video)

SAS HPC Analytics

Changing to a new approach. "In memory analytics". The old SAS/base and the Intelleigence Platform are not visiable yet (2012). Is sold as appliance.
SAS(R) 9.3 Interface to HadoopReference pdf

hadoop-tips (news sascom) Most enterprise data warehouses are constrained by the cost and scalability of relational databases. Storage might be cheap, but without a relational database to make sense of it, it is nothing more than a storage unit stuffed to the rafters with random artifacts.
Relational databases, designed to organize those artifacts, can only deal with hundreds of gigabytes to a few terabytes, and can only support at most 1,000-2,000 variables. The problem is that when you're doing true predictive analytics you often want to test a few thousand variables, or analyze tens of thousands of terms in a text mining example.
>



sas logo

SAS Operating System connections

Companion
SAS(R) 9.3 Companion for Unix
SAS(R) 9.3 Companion for Windows
SAS(R) 9.3 Companion for z/OS

Index Syntax
SAS(R) 9.3 index Syntax
Scheduling, LSF
platform (LSF) (Coming in with many products)

Platform Suite for SAS 6.1 Reference Documentation:
System_requirements.html - System Requirements for Platform Suite for SAS Version 6.1
Arm_instrumentation.pdf - Enabling ARM Instrumentation for Platform Suite for SAS 6.1
RTM2.07_admin_guide.pdf - Platform RTM for SAS Administrator Guide Version 2.0.7
RTM2.06_release_notes.pdf - Platform RTM for SAS Release Notes


LSF Version 7.06 Reference Documentation:
Lsf7.06_release_notes.pdf Release Notes for Platform LSF Version 7.06
Lsf7.06_foundations.pdf - An Introduction to Platform LSF Version 7.06
Lsf7.06_admin.pdf - Administering Platform LSF Version 7.06
Lsf7.06_qrefcard.pdf - Platform LSF Quick Reference Version 7.06
Lsf7.06_command_ref.pdf - Platform LSF Command Reference Version 7.06
Lsf7.06_config_ref.pdf - Platform LSF Configuration Reference Version 7.06
Lsf7.06_users_guide.pdf - Running jobs with Platform LSF Version 7.06
Lsf7.06_windows_using.pdf - Using Platform LSF Version 7.06 on Windows
Lsf7.06_intract_logon.pdf - Configuring Interactive Logon with Platform LSF Version 7.06
Lsf7.06_security.pdf - Security Considerations with Platform LSF Version 7.06
Lsf7.06_hpc_using.pdf - Using Platform HPC with Platform LSF Version 7.06
Lsf7.06_upgrade_unix.pdf - Upgrade to LSF Version 7.06 on UNIX
Lsf7.06_upgrade_win.pdf - Upgrade to LSF Version 7.06 on Windows


Process Manager Version 8.1 Reference Documentation:
PM8.1_release_notes.pdf - Administering Platform Process Manager Version 8.1
PM8.1_admin.pdf - Administering Platform Process Manager Version 8.1
PM8.1_using.pdf - Using Platform Process Manager Version 8.1
Using_templates.pdf - Using Templates with Platform Process Manager Version 8.1



Understand Bu    Support Bu    Run IT    SAS Advanced     Other Experiences     top  bottom

Other Experiences, OS related

Way problems Other issues related to behavior of SAS on machine platforms. The "Companion" is used by SAS to document the machine platform differences.

A lot of thing are behaving different as normally expected. Causing more problems that should not occur.
Mainframe Subject
Regionsize memory
Bound libraries storage properties
Eguide or 3270 user interface
Unix Kernal memory
Help key - PF1 storage properties
SASwork mainframe /tmp sashelp error
System- User abend -..-> return code error handling
SAS/Share .. Central service System abends, errors


linux logo

Unix

Linux and other on the same principles based included.
like hell
Quoting - pointers tot strings
When setting up scripts there is the string handling. Strings are parsed in substrings with pointers to it. Nothing to be seen while debugging. Still the scripts don´t work. The list of pointers is causing the problems. Hard to get it working correct.
Within the SAS delivered scripts the same problem exist causing several problems
It is the same challenge as quoting within the SAS-macro environment.

handbook The Single UNIX ® Specification, Version 2

like hell
Case sensitive - file names
This is causing many unexpected troubles. Advices:

kb15989 In UNIX environments, the names of SAS® data sets and external files must contain only lowercase characters
kb33772 Names of autocall macro .sas files must be lowercase in the UNIX environment
kb8592 Mixed-case Unix filenames listed incorrectly with PROC DATASETS and PROC CONTENTS




like hell
Locking of files
This is causing many unexpected troubles. Even access from SAS get involved at unexpected moments.
Advices:
When code sources as SAS macros are stored in catalogs. This can cause locking problems.
The catalog shoul not be accesses by its owner (Unix acces rights), but by a group member only having read-access. (2012 SdeW)



The backgrounds are referring C I/O components of C I/O



complex
Unix scripting







complex
32-bit 64-bit - versions OS - Java
A migration of 32-bit to 64-bit based code is happening. The planning is badly done (AIX 4 5 6)
In 2007 I had to go back form 64-bit SAS 82 to a 32-bit SAS 82 because of new 64-bit options in the OS AIX (53)
After having done a consolidation with SAS 82 SAS 913 was coming. SAS 913 AIX is 64-bit no 32-bit version anymore.

The SAS catalog´s are required to migrate from 32-bit to 64-bit as they are not readable between the two versions. Just with SAS/connect or SAS/share a conversion is possible. (hidden notes in Proc Migrate)
A lot of scl-code (SAS catalogs) has been written, facing: needed the migrating plan/project.

complex
32-bit 64-bit - Unix tuning
On OS level a lot of settings are possible tuning/optimiziong.
A parameters exist indicating 32-bit / 64-bit usage or memory optimization. IBM AIX tuning , Multiple page size application support gives information to 32-bit code and 64-bit-code. LDR_CNTRL=... gives the 32-bit code 64K internal memory pages. With 64-bit code it looks like the LARGE_PAGE_TEXT=Y influencing this. Even to Unix specialist a most difficult item to get to understand how it is working.






Windows logo

Windows


complex
Windows installation issues
Within this environment there are the most dependicies. .NET versions Java versions com and other Windows basic run-time versions. The interference with other tools on the desktop is amazingly disturbing a lot
complex
Windows profiles
SASwork
Having the SASwork located to you user or temp or to a single location for everyone...

complex
Office jet ace
With Windows-NT MDAC and Jet were bundled by microsoft.
The Jet engine is needed to access native MS-Excel MS-Access formats. In the case of exporting data to office also tagsets can be considered.
The problem with Jet/ACE is that is must be present before installing SAS (access PC-files) on Windows. cc959828 (microsoft refering to IANA)


complex
Spawner / SAS connect
The spawen is designed at telnet. Poort 23. This poort should not be used anymore.
kb/18/941 SAS/CONNECT® spawner on Windows stops responding when there are a high number of concurrent connections)


The started remote proces can not elevate rights. As the spawmer is forking a process it cannot fork to a process getting local admin rights.

The spawner runs out of heap. When the spawner runs as a service, it creates an invisible (noninteractive) desktop for the SAS/CONNECT remote sessions to use so that the icons aren't displayed on the interactive desktop kb/18/941 SAS/CONNECT® spawner on Windows stops responding when there are a high number of concurrent connections)


complex
32-bit 64-bit - versions Eguide Office
What's New in SAS Enterprise Guide 5.1 Eguide is now 32 and 64 bit available. For integration with Microsoft office (32-bit or 64-bit version) corresponding bitness should be reviewed. (oct 2012)



Mainframe img

Mainframe


Regionsize
like hell In the old day´s IBM used a 24-bit addressing scheme using 16Mb as upper limit. When moving to a 32-bit addressing scheme with a 2Gb upper limit they made some not understandable conversion actions.
Giving a region size below 16Mb You´ll 32Mb extra. Given a regionszize above 16Mb You´ll 8Mb extra. Values between 8Mb and 16Mb are forbidden.

This gives see: A brief history of virtual storage and 64-bit addressability Without this background not ables to explain. With java used on the mainframe some hundreds Mb of memory is needed. The values of 4Mb or 6MB or even 64Mb are totally out of date as even your telephone comes in with more. old time logo
As you are still working with this situation this must be managed.
Bound libraries
geniaal 2 This it the classic approach of this machines. Many limits are of the 16-bit age (64k/32k maximum numbers), Blocksizes are limited up to 32Kb. A half track szie 3380 (27640) is an optimal sizing. Volumes are limited to ca 9Gb (3390 model-3) sizing. With bigger datasets multivolume allocations is needed.
Dataset are by default limited in size with pre-allocations. This is different to Windows or Unix where size is not determined in advance.
The allocation is based on numbers in tracks. No more tan 64Kb tracks can be allocated on a volume for a dataset. 64K tracks - excp old time logo

HFS support a Unlix like approach is possible.
SMS and HSM as storage management tools can make working with mainframe a lot easier.


oldtime
Eguide or 3270
Eguide has been designed to segregate the SAS system from a Server. It replaces the terminal software. You need the license Integration Technologies which is part of BI-server to be able to do that.
The 3270 approach with Classic mainframe dinosaurs don´t have that license.

confused
Unix Kernal
The Mainframe Kernel is now running in a Unix part of it. This is not well known by classic mainframe support. (2006-2012)

confused
Help key - PF1
The Help-key interaction has been changed to interaction with an internet browser on your desktop requiring an available web server to the help. This is not well known by classic mainframe support. (SAS93 2012)

like hell
SASwork mainframe /tmp - sashelp error
45492 /tmp causing installation problems on a mainframe

On a mainframe SAS is installed since 9 in the Unix kernal (BPXAS Open/MVS Ishell). Needed was a X-server client. A line mode is also allowed in the latest version.



System abend, User abend, return code
quichot IBM has made many segregate roles within the classic mainframe management support. These functions should also exist in other environments like Windows or Unix, but no background can be found easy. IBM has the longest history in running big datacenters.

Some of these roles are:

Search google: " system user abend site:ibm.com " is giving:
zos/basics zproblem abovr The way an abend (ABnorma END) should be handled. Search google: " return code site:ibm.com " is giving:
User completion codes - RMF Handling of User Abends within RMF. RMF as example of middleware/subsystem.
Clearly in te list, the segregation of operator response and System programmer response.
return exit - SDK Director guideline of well defined meanings. Interacting Unix - Windows
Cobol - return code with stop statement no abend statement exists. 


Why I made this note within SAS usage?
SAS is not conforming anymore to this approach (system abend / user abend / return codes) on mainframes.


Problem
SAS/Share (mainframe) fails technical and no support to recover.

Analyses
Background, see above.
Server administrators Managing a SAS/Share Server.

Possible is a relationsship with the threaded kernal and Unix services approach (since 913).

Almost every error will result in a USER998 or a returncode 4/8 with no further indication. Needing to go to the JOB-log or SAS-log searchin the real cause.
Normal system abends like SB37 / SE37 are redirected to the same USER999 abend or even returncode 8. S0C4 or S0C7 errors are also or ignored or giving the return-code not the system abend.

The normal procedures to a Mater Control Room, automated task, an application progammer with job-instructions are possible failing.
A subsytem like SAS/Share as a DBMS fucntionality (middleware) will become a challenge
Special attention to this is needed.

Solution
Not to be expected...(SAS tracks have been done)

This all is a central task to be solved, server administrators.
The service-desk support error handling (SAS intitute) is getting just to a single user or programmer approach. Not seeing a failing server as a major problem.
Perhaps with a SAS log-server connected to arm-monitor it is solvable. Im this case leaving the SAS/Share log as is, and moving to an EIP appoach.
Leaving the mainframe migrating to .. is a to be option.


2012 J.A.Karman (medio 2010)



fire
SAS/Share failures
Problem
SAS/Share (mainframe) fails im some task to server clients.

Messages in SAS/Share server log are found.
20JUN2012:10:15:57.900 RESOURCE ENQUEUE: Erroneous internal condition encountered: CANNOT RESOLVE INDEX. In "DMS Process"(1) of user ....(596).
10.15.57.90: ERROR: FAILURE IN RESOURCE ENQUEUE: INDEX ENTRY NOT USED, VALUE: 30, FCODE: 280
20JUN2012:10:16:00.320 --.--.DATA(129) opened for update/2 via engine V9 by "AF"(6) of user --(511).
20JUN2012:10:16:00.320 --.--.DATA(129) reopened for update/R by "AF"(6) of user --(511).
20JUN2012:10:16:00.340 --.--.DATA(129) closed by "AF"(6) of user --(511).


And an other, this one can be harmless stating communication is broken and to be recovered.
23DEC2010:16:30:21.600 User --(36) has terminated "DATASTEP"(15) (under "DMS Process"(1)).
NOTE: Data set --.--.---.-- has been deallocated.
23DEC2010:16:31:02.330 Server library -- (accessed as --) released by "DMS Process"(1) of user --(36).
23DEC2010:16:31:02.330 User M04C111(36) has terminated "AF"(4) (under "DMS Process"(1)).
23DEC2010:16:31:02.330 User M04C111(36) has terminated "DMS Process"(1) (under "CONFIG"(0)).
23DEC2010:16:31:02.340 TERMINATE CONV: Unexpected ERROR C0005B24 from WQRQST - CTERM in "CONFIG"(0) for user --(36):
ERROR: A communication subsystem partner conversation takedown request failure has
occurred. ERROR: Communication Subsystem Request Parameter Error: Invalid conversation identifier.


This one open a librarie V9/V9 getting the SAS/Sahre server stopped.
22DEC2010:10:02:06.060 043100 --.--.DATA(11) opened for input/S via engine V9 by "DATASTEP"(5) of user --(11).
ERROR: System abend 0C4 occurred in module SASYHRSV function YHRCQA at offset 000066.
ERROR: Traceback will be attempted...

NOTE: Traceback called
NOTE: Module Function Offset Line
NOTE: -------- -------- ------ ----
NOTE: SASYHRSV YHRSERV 004CAC 0
NOTE: SASSERVE SASSERVE 0012E2 0
NOTE: SASXA1 UNKNOWN 0F3C38 0

ERROR: Delete current task.
NOTE: The SAS System stopped processing this step because of errors.
22DEC2010:10:02:06.110 043144 User --(11) has terminated "DATASTEP"(5) (under "Program"(1)).
22DEC2010:10:02:06.110 043102 --.--.DATA(11) closed by "DATASTEP"(5) of user --(11).
22DEC2010:10:02:06.110 043102 --.--.DATA(10) closed by "DATASTEP"(5) of user --(11).
22DEC2010:10:02:06.110 043102 --.--.DATA(9) closed by "DATASTEP"(5) of user --(11).


This one using an older V6 version dataset. Was later converted. Same module got corrupted.
23DEC2010:11:56:14.530 043069 Server library -- ('--.--.--.LIB' V6) accessed as -- by "Program"(4) of user --(88).
23DEC2010:11:56:18.300 043021 User --(89) has connected to server SASSHR1.
23DEC2010:11:56:18.310 043143 User --(89) has created "Program"(1) under "CONFIG"(0).
23DEC2010:11:56:18.330 043069 Server library -- ('--.--.--.LIB' V9) accessed as -- by "Program"(1) of user --(89).
23DEC2010:11:56:18.330 043032 ASSIGN LIBRARY: Unable to allocate XHRDTL. In "Program"(1) of user --(89).
ERROR: System abend 0C4 occurred in module SASYHRSV function YHRCQA at offset 000066.
ERROR: Traceback will be attempted...

NOTE: Traceback called
NOTE: Module Function Offset Line
NOTE: -------- -------- ------ ----
NOTE: SASYHRSV YHRSERV 004CAC 0
NOTE: SASSERVE SASSERVE 0012E2 0
NOTE: SASXA1 UNKNOWN 0F3C38 0

ERROR: Delete current task.
NOTE: The SAS System stopped processing this step because of errors.
23DEC2010:11:56:18.340 043144 User --(89) has terminated "Program"(1) (under "CONFIG"(0)).
NOTE: Data set --.--.--.-- has not been deallocated because it is still associated with another libref.
23DEC2010:11:56:18.340 04306A Server library -- (accessed as --) released by "Program"(1) of user --(89).
23DEC2010:11:56:18.350 043022 User --(89) has disconnected from server SASSHR1.
23DEC2010:11:56:18.350 043151 Usage statistics for user --(89):
Messages processed: 4
Bytes transferred: 407
Active time: 446866:56:18.3879
Elapsed time: 0:00:00.0518



Analyses
The module name like YHR- are likely to be SAS/Share modules. Mainframe is limited to 8 character naming. The type of error(s) is indicating sommething is terrible wrong at the low-level routines. No clue what is wrong, it can be a victim of something different.

Also some errors have been seen with S0c7 codes. Looking like: Found note kb7937 old SAS 9 TS m0 note.
YHRCQA: Unexpected WARNING 00000002 from FCB.FCODE :
ERROR: INTERNAL ERROR - YP UNABLE TO FORMAT MESSAGE


Found note kb33892 not free librarie, client times out
kb35398 no altlog allowed

New searches beginning 2012:
Searching "SAS/Share Problem note" is giving many hits. Most of them are new, did not exist when starting with the problems with 9 migration.
New searches medio 2012:
New problems in this area. The mainframe (yes whole machine) went nearly down by a loop en locking problem. Checking again all possible information (medio 2012)
Some errors: kb6387 corrupted data The approach to the error is prefessional.
This simple error manifests a much larger problem whereby the lock synchronization on the data set fails. During the misalignment of locks, data can become unprotected and data integrity can be compromised. The message could be ingored but the real problem is bigger.
This approach of looking to the real problem or bigger impact looks to have vanished.

Searching fo dataset locking and=or SAS/Share Problem is giving hits.


Solution
Not to be expected... Reason: see abend-codes
The number of Problem notes are still relative small. The signal of what is going wrong and the solution is troublesome.


2012 J.A.Karman (medio 2010 started)



Understand Bu    Support Bu    Run IT    SAS Advanced     Other Experiences     top  bottom
home-SAS    SAS-SAAS    First steps    Installation    Hardening    Operational    Using    My Notes

© 2012 J.A.Karman (25 feb 2012)