home-SAS    Main-SAS usage    Main-SAS operational     Main-DTAP design
Base    Desktop    SQL    Stat-Miner    releases Business     Instal&check     OS level    top  bottom

SAS Blogging Notes


Jaap working

Blogging subjects

Bloggings can be found on the web by many more links.
The sastraining blog contains many tips of several writers.

Some experiences with tips and solutions of me are joined to a collection here.
The concepts, design and operational subjects are the main line of my knowledge documentation.


SAS Base , common usage

Way problems

Windows specifics


Start of base (Windows), Wich interface

complex It is possible to sart SAS Base, Foundation, in many ways. The interactive / Batch mode is a menu option by the mouse selection (92 and up) on a sas-source file. I remember reading somewhere it is because both SAS base and SAS in batch mode both appear as SAS 9.2 except one version the symbol is orange and the other is yellow.

The server mode is done at installation time of sas with a registry-setting. This registry setting defines the SAS startup with some options.

Autoexec and config files with options are possible influencing each other.

Advice: leave the installation of SAS to the default/standard as much as possible. Base/Foundation config options autoexec start and all settings to be set up on your own personal location. Eguide has become an autoexec project flow.

2011 J.A.Karman

lost window , Window is not accessable

fire Problem
In the program editor, action: Find. A window should pop-up to define the search-string. It is not coming.

Analyses
The color of the editor window is changing. Indicating a pop-up has started.
The profile settings ot the user is propagated (raoming profiles) flex desktops inused
Asking for change of monitor usage, user was first using two monitors now having one. We can find an other machine the user ia able to start.
The pop-up window is appearing on the other screen. Now we can move it back to the first screen.

Solutions
As described keep the windows visiable.
To be find yet

Open
There are no by-passes to move that window oin ohter way. Expectation is a windows registry setting to the position. User sas-profile and sas registry where cleaned but did not have any influence.



2012 J.A.Karman



Microsoft office, how to do Interactions

geniaal handys Problem
Most people are common to use Excel Word Outlook and Power Point (Microsoft Office) doing their work. SAS is not fitting into this.

Analyses
SAS offers many ways to interact with Microsoft Office.
To solve this is correction of lack of information. See also: Links to excel blogs .

Solutions
Addin Microsoft Office
requirements
License server side with Office included. A BI-server or a Solution normally has it.
Desktop package roll-out fully integrating on the normal standard workstation.
advantages
Ability to run predefined processes own defined processes generating reports (Word)
Interaction to data while running Excel to do analyses. Newer SAS versions (92) support updating server-side data of sources restricted by security rules server-side
Reporting of the results/reports direct into power-point
Newer SAS versions (92) support working within Outlook interactions. In that case outlook 2007 or higher is required
Eguide
requirements
License server side with Eguide included. A BI-server DI-server or a Solution normally has it.
Desktop package roll-out able to access necessary files.
advantages
All is included while working with Eguide.
PC-files server
Interfaces to PC-file formats: mdb access Excel etc. (collection paragraph)
156-28 PC files   031-2010 tagsets (xls).
requirements
License server side with Access/PC Files included.
Getting a windows machine installed and configured with PC-files Server
Authentication and Identification must be taken into design to access files. advantages
Can run in unattended process (classic batch). SAS Libname usage to tables (excel Access) in native office formats.
Tagsets ODS
...(more)... requirements
Base SAS usage (license). advantages
Usage of xml-based interfaces. No additional software needed.
Import Export Base procedures
...(more)... requirements
Base SAS usage running on Windows and with Access/PC Files (license) .
32/455 Needed up to date SAS version with up to date Office.
advantages
Usage of native Windows and Office formats.
Well known as this is the classic stand alone desktop approach

2011 J.A.Karman



PC files Unix

modern Problem
Mixing Incompatible OS levels like Windows-32bit desktop, Windows-64bit server, Unix server, are blocking the connection SAS tot PC-files.

Analyses
Design issues caused by incompatible OS.

Solution
Use PC-files server in 32-bit mode. Other or other options to connect to 32-bit windows used as desktop system.
Interfaces to PC-file formats: mdb access Excel etc. (collection paragraph)
kb45340 PC files, 32-bit 64-bit office interface  



2012 J.A.Karman (1 march 2012)

Addin Microsoft Office (AMO) doesn´t work anymore

like hell Problem
Having installed and used AMO at some moment is not working anymore.

Analyses
Design issues caused by Office design.
Addins can be set disabled. Technical parts can be get off version with shared workstations. Technical parts are sometimes single process.

Solutions
Disabled
Enable again
Second office Excel session
Multiple Excel sheets processes started.
Use the first/only Excel process version.



2012 J.A.Karman (1 march 2012)

Office DDE

like hell Problem
MS office DDE calls are required to interact.

Analyses
Design issues caused by Office design.

Solutions
DDE
X '"C:\Program Files\Microsoft Office\Office10\WINWORD.EXE"';
filename cmds dde 'Winword|System';
DATA _null_;
rc = system('start winword');
call sleep(2,1); /* wait word has started */



file cmds;
put '[FileOpen.Name = "' "&path_programfolder.&pre &rown %nrquote(&lettername)" '"]';
put '[FilePrint()]';
put '[FileClose]';
run;

(PT 2012)
Second office Excel session




2012 J.A.Karman (1 march 2012)



Advanced base


Perl regular strings

modern Problem - Question
How we can searching a numeric value(a number) in a string using SAS. That number may appear anywhere in the string(with delimiters as well).


Solution
data _null_;
t='hi there, I am using SAS 9.2';
if prxmatch('/\d+\.\d+/',t) then put 'gotya';
run;


(SAS network)

2012 J.A.Karman (ca 2011 - ... collected)

Unix tab delimited

modern Problem - Question
How we can code the tab delimtied of Unix (du df).


Solution
filename hulptar pipe "suexec /usr/local/etc/rc.sas du -g /home ; " ;
DATA WORK.du_home (Keep=size vertical mntpnt stage type); length applid vertical $8 stage type $1 mntpnt $128 ;
infile hulptar delimiter="09"x missover ;
input Size mntpnt ;



2012 J.A.Karman



SAS/Connect spawner error

ligth house Problem
Failed to get a connection with SAS/Connect to Server
10061 - WSAECONNREFUSED


Analyses
Normal users first line support:
The technical working is not understood sufficient to get around this
Needed Background
The classic SAS/Connect with a spawner is very nice to eliminate the terminal and download upload files challenge. It is working by A lot can go wrong. The message normally got is:
ERROR: A communication subsystem partner link setup request failure has occurred.
ERROR: connect call failed, system error is '10061 - WSAECONNREFUSED'.
ERROR: Remote sign on to ***** canceled.
No real clue of the root-cause is given. The user has to debug this.

Debugging
Question: Is the spawner correctly working?
As soon this is confirmed, go to next checks.
Question: Is the key/user correctly working with SAS?
Debugging: (tcpdebug.dat) = options set=tcpdebug 30 ; =
Debug files are expected to send to SAS tech support.

Solution
When debugged to the root-cause, this is to be solved. It can be very easy.

1996,2004,2008,2012 J.A.Karman (21 Apr 2012 - debug aug 2012)




Out of storage , class/by join processing

complex Problem
The process is long running.

Analyses
Looking to files if possible generating a lot of data. Another option is that a lot of internal memory is used.
Used is by or class processing (Base) or a complex SQL is running.

The effect of by/class or join processing can be a factorial or other power combination of data. Suppose you 60 by 100 by 20 by 50 by 15 by 8 classes to interact. this will lead to 60*100*20*50= 240.000.000 possible interactions. The more classes and the higher the ranges are the amount can become tremendous.

Using floating numbers as classes can by very tricky as floating is by nature not fixed. Rounding numbers to classes can help. It is better to use just character based variables as classes.
Tabulate means report
These procedures have the option of running with a class options (memory based) or by option (sorted order) choose appropriate.
Transpose SQL
Proc transpose an SQL (joining) can easily coded having the this effect.

Solution
Review the analysis.
Knowing your data and predict the effects of permutations combinations with all intermediate steps.



2009,2012 J.A.Karman (9 apr 2012)




fire

sashelp.vstable (v-members) - trampoline - OS dump

Problem
The sashelp.v- members are behaving slow and sometimes running in dump.
A trampoline error is a symptom with this errors
719 proc sql noprint;
720 select min(memname) as memname
721 into :sok_bestand 722 from sashelp.vstable
723 where upcase(libname) = 'TEAMWORK'
724 and upcase(memname) like 'SOK_NNVA_%'
725 ;
Segmentation Violation In Task [ SQL ]
Fault Occurred at [0x800000000fc9578(?)]

Task Traceback
o_conn+0x38 -- bridge stub
xdname2+0x38 -- bridge stub
sqlloop+0x38 -- bridge stub
_uc_trampoline+0xfffffffffffffffc


Analyses
When using remote library services or SAS-share remote connections the sashelp.v- routines have these problems causing this kind of dumps.

Using the sashelp.v- routines implies that at first all information on all tables is gathered. A selection is done afterwards.
The first step of gathering all information is possible causing much overhead and delays.
Solution
Recode the analyses using old fashioned SAS-procedures. Using just the information on tables you are needing.
proc contents data=teamwork._all_ out=inhoud noprint; run;

proc sql noprint;
select min(memname) as memname
into :sok_bestand
from inhoud
where upcase(libname) = 'TEAMWORK'
and upcase(memname) like 'SOK_NNVA_%'
;
quit;

%put sok_bestand = -&sok_bestand.-;




2007,2012 J.A.Karman (23 apr 2012 BD)




macro token-comment pitfall error

fire
Problem - Question
The macro in a autocall libraries compiles correct.
The first time it is not executing but the second and thereafter it works correctly.

A special case is the macro-code:

%macro test ;
%put macro test is running;
%mend ;
...
   %* /* ; /* */
...



It will show no error or any indication of this weird behavior.

Solution
Remove all statements outside %macro / %mend blocks
The token char handling of source execution shoul be got very clear.

Getting to some unoffcial explanation:
With the autocall set up the first time a macro is called when recognized not having a compiled version it will compile it.
It will be compiled in the same way as the source member (same name) was included. Then it resume processing.
In this way having normal code in an autocall source member wil get an unclear menaing.
The way commenting can also be unclear as there are 3 ways to do it and several way of parsers are being used.



2013 J.A.Karman




ODBC connections


CMS AVAYA

like hell Using SAS, connected to a CMS Avaya system.
The key approach is using ODBC as described by: 780701_1 (avaya)

2012 J.A.Karman (oct 2012 - P.Matter )





Changes / OS / Kernal


got page -1 instead

like hell Problem
Unexpected, Sometimes not reproducible, blocking (as not running)

Analyses
some several issues, same message
Corrupted profile dataset: kb 43100 Caused by: /a breaking down SAS at first pop-up, b/ network lines going down at having profile dataset at network drives.

93 profile describes profile issues in LRcon Language concepts).
kb 36664 SGIO caching can have influence on this.

Also something with caching on OS (synchio sgio dio) kb 7816 . Caching/no-caching should work as an option. Failing in this way is indicating a fundamental design error.
In SAS v6 and SAS v8 on mainframe there were issues with multi volume datasets storclas Guaranteed Space

Solutions Dependant on situation


2004-2009 J.A.Karman (1 march 2012)

input contains commas and double quotes

confused Problem - Question
How do I read a text column from a csv file which contains commas and double quotes?

Solution
The following options:
a/ Use an other delimiter. See: LE put example.
If the comma has the meaning of a/ digit indication but not always present and b/ field separation it will be hard to make the difference automated.
b/ ~ (tilde) format modifier is also used at the input definitions . See: LE input example 5



2012 J.A.Karman (ca 2010)


Base    Desktop    SQL    Stat-Miner    releases Business     Instal&check     OS level    top  bottom

desktop , system errors

Way problems All kind of notes and errors related to the desktop installation and the way of support on it.
  Subject
Install Clients shared storage
EGuide 4.1 libname Not understandable error trying to open
ora78 Access ORA-error
DI 3.4 Blocked start DI 3.4
VDI using not running programs as usual
  Subject
X Systask "Server Execution Failed"
Hangup while closing Base Foundation
Eguide export to errors, .. Excel
Dll´s registering - re-packaging


Install Clients - shared storage Program Files

unsafe Problem
not really
Background
Having installed the Desktop-clients not on "C:\Program Files" is behaving fine.
Some disadvantages and advantages to be known by the users.

Solution


2012 J.A.Karman (medio 2009)

Eguidel 4.1 libname

unsafe Problem
Eguide 4.1 starting up well. Giving error at unexpected point. Like open library on server.

Analyses
SAS Base is installed and many shared components are registered correct.
Eguide is using this. It will start up with no errors and fails somewhere on the missing parts.

This is the same behavior as an old wrong SAS base is present on the machine and Eguide is installed correctly.
Some pinpointing of the root-cause me be necessary.

Solution
(re)-install Eguide. In a deployment with shared storage the installation updating the machine part.
Be sure using the version that is delivered with SAS Base already present on the machine. (EG 4.1 = base 913 , EG 43 and 5 = base 92 93)

2012 J.A.Karman (april)

Access Ora-error

unsafe Problem
Not able to use Oracle from SAS Foundation running on desktop (Windows)

Analyses
When having installed an old oracle client interface (7-8) and using SAS 9.
The SAS/Access interface gives a generic ORA-error, no warning of the outdated Oracle client.

Solution
Remove the old client and replace it by a newer, supported, version solves the error.

2012 J.A.Karman (medio 2009)

Blocked start DI 3.4

unsafe Problem
DI studio does not start up. Other users can still work.

Analyses
Having a corrupted initialization file will block a start-up of DI (3.4).
The initialization file is named wadef.txt and can be found one level up in the %appdata% dir. Typing %appdata% in the explorer window will jump to your location.


Solution
removing wadef.txt will solve the startup problem.

The WA is inherited from the Warehouse Administrator product. That is Renamed to ETL and renamed to DI.

2012 J.A.Karman (medio 2011)

VDI using - not running programs as usual

unsafe Problem
On the Virtual Desktop normal used programs are not running.

Analyses
A VDI is often delivered as clean version at startup. Not having the normal desktop environment or previous settings installed.
Many things off SAS still are working (share file) but some things are missing, like: access Oracle , enhanced editor.

Solution
Install all needed additional programs on the VDI first.
Do not reboot a VDI as it will deliver a clean environment again. Nothing of the additional´s installed



2012 J.A.Karman (medio 2011)



XCMD Eguide - Local SAS

genius Problem
X functions (call the OS) do not work any more, using SAS Eguide as interface.

Analyses
Installation mistake.

Solution
See: installation chapter, experience, eguide xcmd  

2012 J.A.Karman (medio 2009)

Hang up while closing , Base Foundation

light House Problem
When closing the Base Foundation is stops at a grey screen, never closing.

Analyses
With SAS 8.2 using the macro call to end SAS (AF embedded) SAS Base got looping.
With SAS 9.1.3 having connections, connections are not closing, base looks waiting.

Solutions
Don not use the macro / dm statement endsas. (SAS 82)
Use first signoff to all remotes sessions then bye. Can be done the best way as dm-statement
Not a big problem to be escalated it is not nice behavior.



2006 2009 J.A.Karman

Eguide export to Excel

genius Problem
Excel and SAS don not react as expected or wanted.

Analyses
not applicable

Solutions
Bypasses listed for several situations
Configuration setting listed for several situations
Importing by Eguide to a server, wrong server destination
Root cause: In the export process no server selection is available (EG 4.1)
By pass: Set the default server in the connection options to the wanted one.
An error occurred saving the file, Object reference not set to an instance of an object
Resulting in doubling the records in the excel spreadsheet.
By pass: Send to Excel an save file in Excel.
Root cause probably some missing references of files libnames in the metadata.
(medio nov 2011 , thx GS)



2010,2011 J.A.Karman



Dll´s --- registering,re-packaging

unsafe Problem
The desktop client tool fails ore fails in some functions.

Analyses
error messages indicating failing Dll-s.

Solutions
Listed for the several desktop clients
Enhanced Editor
kb 32252 regsrv32 to register. (9.2) 2012 J.A.Karman (medio 2009 - 2012)

Office Addin
kb 30503   kb 43127   regsrv32 to register. Admin rights (?) required (4.3)
cd "I:\SAS\Add-In for Microsoft Office"

C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\regasm.exe /codebase /tlb SAS.OfficeAddin.dll

C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\regasm.exe /codebase SAS.OIBootstrap.dll

C:\WINDOWS\system32\regsvr32.exe SAS.OfficeAddin.Loader.dll

2012 J.A.Karman (medio 2009 - 2012)

Eguide
kb 40233 regsrv32 to register. atl.dll (NO admin rights required 4.1 ) 2012 J.A.Karman (medio 2009 - 2012)

OLE DB
SAS ABM dataproviders, Eguide OLE DB (9.1.3)
Small mistake in the repackage. To be solved by (NO admin rights):
regsvr32 "I:\SAS\sasv9shr\SAS OLE DB Data Providers\sasafshr0901.dll"
2012 J.A.Karman (medio 2009 - 2012)



--- J.A.Karman ---


Base    Desktop    SQL    Stat-Miner    releases Business     Instal&check     OS level    top  bottom

SQL ACCESS

Way problems

Tnsnames Oracle

believe Problem - Question
Alway´s have doing work to get the tnsname.ora file updated?
Not necessary all information can be given in the path not as short name but as the whole definition.

Solution
Must be a lot easier to maintain with the use of centralized registrations. Not Tnsnames involved anymore.

proc sql;
   connect to oracle
   (user=wr_<yourkey> password=*****

   path=´(DESCRIPTION=
     (ADDRESS = (PROTOCOL = TCP) (HOST = <dns oracle-server>) (PORT = <port oracle-server>) )
     (CONNECT_DATA = (SID=<ora sid>) )
     )´

   );

   select * from connection to oracle
   ( select *
   from wr_<yourkey>.testsas
   );

   disconnect from oracle;
   quit;


2012 J.A.Karman (1 march 2012, thx BD)

Views and Proc SQL

handy Problem - Question
Analyzing flat files.
Joins and selections are needed.

Solution
Having SAS views , Ffil01 Ffil02, defined to flat-files the PROC SQL will work.
An easy way to analyse flat-files with the SQL-query language.

PROC SQL ;
   SELECT *
   FROM LIB.Ffile01 a LIB Ffile02 b
   Where a.veld1 = b.var001 ;
   quit;

Looks simple, remember: you are accessing flat-files (-.txt) in this way, not a dbms!


2012 J.A.Karman (5 march 2012)

Performance-1 basics

handy Problem - Question
Getting data fron an to a classical OLTP database (Oracle DB/2 SQL-server) is always giving some performance problems.
The root cause is that a OLTP is designed with ACID operational requirements. All overhead by locking Committing fulsing buffers ans spreading of data are results of that.

Solution
The first things are to get all overhead minimized, see:

2013 J.A.Karman (2008,2010,2011)

Performance-2 Commit bulk load

handy Problem - Question
Getting data fron an to a classical OLTP database with higher volumes is getting more challeing to get all overhead of it eliminated.

Solution
Gettting the commit influencing IO/buffering and even bul load options are btter lukt

2013 J.A.Karman (2008,2010,2011)



pst pitfall

unsafe Problem
SQl query does not run. Table not existing. Visitable in libname list.

Analyses
Explicit pass through can be surprising.
The inner part does not know anything of the SAS libnames that are open. They cannot be used. The error will be an unknown table.
Just tables accessible by the DBMS connection are available.

Solution
Use the name Oracle knows with pst. Can be an other name than the SAS libname.
More blogs: SAS Shelly Goodin The hard coding of user/password should be avoided.


2012 J.A.Karman (5 march 2012)

Sort pitfall

unsafe Problem
Oracle can not be accessed anymore (was accessible). Strange Ora-errors. An error message may look like:
Oracle Database Error Code ORA-24316 Description : illegal handle type


Analyses
Tables in a DBMS cannot be sorted by PROC SORT. Trying to do this can result in a corrupted connection to the DBMS.

Solution
Restarting everything to get it cleaned up.


2012 J.A.Karman (5 march 2012 )

reserved name pitfall

unsafe Problem
Table can not be copied from SAS (imported from Excel) to Oracle.

Analyses
By exception the user was allowed to create tables into the DMBS. Creating and manipulating tables is normally a DBA task. A column field was used with the name "date"


Root cause: Tables and field in a DBMS are restricted by reserved names.
Using a reserved name like &quaotdate" of eg Oracle in Excel will block the storing the table into an Oracle table. Just saying ... error.

Solution
Do not use restricted names to a DMBS creating new fields. (see usage notes)
Challenge: How to educate Excel users not to use reserved Oracle-names in their column headings?


2012 J.A.Karman (27 march 2012 )



Base    Desktop    SQL    Stat-Miner    releases Business     Instal&check     OS level    top  bottom

Stat an Miner

Way problems



Miner installation 5.3 (53)

like hell Problem
Customer / Users can not work with this tool. Strange errors are popping up.
Analyses not applicable, see background history.

Solution
Repair and fix some of the issue were approproate. Goal is getting something working. The better solution is life cycle (upgrade version).
Progress
Having solved the Java run-time (32-bit 64-bit) problems Miner starts to work. The next issues popping up.

Some unclear errors are reported. First thing checking no errors/warnings are seen at startup (Workspace method). The start of SAS should be clean as clean can be. It looks to be critical as a start of classic Foundation.
After checking, this possibility is skipped.

How it should work is well defined: em53 getting started Getting the sample downloaded and the way following this document.
Working with the client gives a SAS-explorer program-editor SAS-log menu-s. It is normal SAS to be accessed in this way.
The project has a start-up source (like autoexec). The debugging can be started. (options mprint mlogic source source2 symbolgen)
All Miner macro-s are loaded with a proc display including sources from catalogs.
EM desktop Tree
The interactive tree desktop is not working. I will to have reinstall it. Checking the fix-levels again.
fix em53 kb33324 fixes base for Em.

CD CT35S7, but a new version is available: kb31759 limited access in bin. Download new version. download

To archive: Issues on terminal server exist. Must be run by the installers key (?). kb33993 install directly in terminal server.

miner docs to review again. 53 miner interactive The way it should work. Invoking by JWS not tried, not working. Expected, The java container is not functioning JWS start

The test scenarios are followed (sashelp.shoes). Miner 53 doesn´t behave well.
New projects are getting scattered in available App-servers.

Fixes to implement all first, and I will do the same tests again. A pity is the client software has to install from scratch (isolated). The repackaging process has missed something. The fixes needed to be restarted with a fresh install.
Installation of the desktop tree is found at " --\SAS\sasv9shr\Enterprise Miner Clients\V9\nls\eng" It is the shared location kb39598 , installation messages can be incorrect.
Getting some weird errors using this program standalone proves this is a VB program.

After all fixes the desktop tree install doing from scratch is giving the wanted reaction in miner-desktop .(Solved) Some more files found in the emtree directory.
The last thing to do is getting this to the customer. This is the last thing to solve. I have to start over again as some dll´s have got mixed up with my sas93 environment.

MP Connect
attribute EM_SASCMD in the logical workspace server should be set. kb14760

Em is updating personal records in the metadata and server context definitions, using the unrestricted admin.

Settings Seem to be ok. Mp connect is working. Question: what is missing?

Debugging: The mpconnect start give a correct start-up. Then a sysrput is visible. After status connection is false.
The issue is clear now. Running with the "noconnectwait" option active the logon check fails (no warning/error).
Setting "options connectwait;" in the startup-source is the proof, it is running now. Issue Solved.

Why I have no other way found to change options at em_variables is not clear.
Why I have set noconnectwait? With connections to other machines (SAS/connect) it was the most stable approach. Not polling continuously back over a not always reacting network.
Not expecting a mp-connect usage (asynchrony) would require synchrony start up.
Maxmemquery Pmbr
""Out of memory" messages with some procedures. Memsize was up to 4G should be no problem. Then my eye fall on maxmemquery=6M (default 913). This is not the memsize option. It has to do with in memory procedure. Searching what this setting does gives no clear statements. kb38650 text miner (not the case this situation). kb39249 Warranty Analysis is also using PMBR. This is a hit. emtmsas913 PMBR.pdf just says storing observatiosn into memory.
Have set to 500Mb, bigger as the 92 default of 256Mb. (Solved)
Floating errors, mixed hreg fit
Proc mixed kb42826 Proc Hreg kb7140 Proc fit kb44871 Proc svm: 913 preprod The proc SMV wil be reviewd for a solution (TLS SAS)
Is described to be solved by applying the fixes.
Lock - cleanup - stop
To be solved by a scripted program. To be started by an Eg-at the workspace-server? Lock EM information: The dir: ....(prjct)/Workspaces/EMWS/System is containing the inforamtion
Logfile (mpconnect) run.log and wsopen.lck if project is locked


Change / User instructions


background history
I have blogged this situation because it was caused by to many split ups in technical support. Lasting roll out in a too long time.


Root cause: Isolated server approach, isolated clients approach, no possiblity of verifying the functionality, not involving customer usage.

Following the actions in solving this issue, the server and client side have to be approached at the same time in sake of fixes and functionality
The hand over to customers in to be using the tool should be done after having verified the functionality. Following a sample is the necessary step.


New: working with models
Scoring from Eguide: kb18862 For some models this is exploding in Eguide. Instructions to run as SAS-code. I imagine that there are two phases of the problem:
  1. Scoring task wizard consumes a lot of resources during preparation (it builds list of variables, displays screen to map variables etc). (now a lot of RAM is already consumed)
  2. After this the actual scoring will need some RAM. If there is no memory available now, then it fails with error.
My suggestion is to try to score the data with scoring code, but without using scoring task wizard. It will be easy if you don't need to do the variables mapping, which is available in the wizard. They can copy-paste scoring code from EM results and run it in EG as SAS program. All they need to do is add three lines of code (define input and output tables and add run statement):

data library.out_table;
set library.in_table;
{ paste scoring code here }
run;


kb19873 slow building pictures multiplot
As altnative name analyses is also used by customers

The first reactions are hopefull. The change wat not totally completed.
New by-passes to get client updates to customers as temporay working information.

2012 J.A.Karman (1 maart 2012 - several weeks)




Base    Desktop    SQL    Stat-Miner    releases Business     Instal&check     OS level    top  bottom

Support versioning & releases Business

Way problems
  Advanced migrates conversions
CEDA SAS/connect 32/64 bit
Conversion SAS datasets Os limits
Conversions, change options SAS
  Subject
metdata artifacts - classical code promote ship/deploy
Replication - business to other machine


CEDA SAS/Connect , 32/64 bit

geniaal handys Problem Proc Migrate
"Proc Migrate" is said to be the solution.
issue: Failing 32/64 bit catalogs migation
issue: diificaulties maps/directories migations >

Problem CEDA
Failing CEDA usage by
  1. updating records
  2. using formats
  3. using indexes


Problem Migrate (using CEDA)
Failing migrates caused by
  1. When having sas-sadatset types and normal flat files been intermixed in same maps/directories
  2. is by passing bitmap-style SAS datasets. For example: Styles of proc template. AF generated slists
  3. changing security setting (Linux/Unix) of eg catalogs (locking)


Analyses 32/64 bit catalogs
Not common understanding the Migration issue. For Migration - files inventory. You make a planning

Look at Migrate concepts and Migrate proc (slibref) connect session. The 32 64 bit conversion is done by SAS-connect. Proc migrate is requiring this SAS-connect.
Other notes: kb38339 and kb12679 needing SAS/Connect or SAS/SHARE is documented clearly (93).
Other notes: migrate RLS is gving all steps. A lot of technical backgrounds needed, not to be bypassed.

Analyses CEDA
CEDA is decalaring access to data in other representation. In this way no immediate conversion of data is needed. Some exceptations: When this happen a converison is needed. As CPort/Cimport is alwasy avalailable this is the first mentioned option, hidden in proc migrate. SAS-connect has less disadvantages as it will aleays work and even is given the option to go back.

Solution
If the release management process of the business is using already SAS-connect. That will be a equal approach to do migration on that part.
It must be part of the overall migration traject.



2009,2012 J.A.Karman (9 apr 2012, aug 2012)




Conversions SAS datasets

geniaal handys Problem
Running conversions issues:
  1. Failing OS tools
    copying, moving datasets is limited in number of files


Analyses Linux/Unix
Not common understanding the: "cp" and "mv"p limitations.
Avoid common errors in UNIX and Linux (IBM) getconf arg_max.
/usr/bin/ksh: /usr/bin/cp: 0403-027 The parameter list is too long.

needing a work arround when having many files.

Analyses dir listing
SAS 9 is having directory functions. kb25074 and 009-2010 Group File Processing with SAS®Macros and the Data Step.
Using System calls (Unix) to process.

Solution
Use dir lsiting info to process a copy at file level



2012 J.A.Karman (aug 2012)

Conversions, change options

geniaal handys Problem
Running conversions issue:
  1. Different settings of options: compress.
    For big data changed usage changed default to Yes. Advice of: news55 (mxg)


Analyses compress
Pointobs with compress is possible: pointobs=
When hitting a small dataset a message is generated. The message is in SAS 9 a Note with SAS 8 it was a warning.
Proc SQl is not switching back:
175 ORDER BY bat_proc_volgnr ;
NOTE: Compressing data set WORK.X_BAT_PROC increased size by 100.00 percent.
Compressed is 2 pages; un-compressed would require 1 pages.
NOTE: Table WORK.X_BAT_PROC created, with 1 rows and 20 columns.

WARNING: Compression was disabled for data set WORK.PERIODE because compression overhead would increase the size of the data set.



Solutions
Inform the developers & users.

2009, 2012 J.A.Karman




Segregation of Duties

like hell Problem
The business logic needs to be versioned. Release management must be done on business logic.
Needed knowledge and hours is not planned, causing budget problems.

Analyses
The dependencies on used middleware is making it impossible to do this stand alone business task.
As there are many tasks requiring segregation at business side, at the same time not having that many people in a lean organization some tasks have to be delegated. The executing, not the request or planning, of versioning and releases of the business logic is executed by infrastrcuture staff.


By auditing and regulations (basel Sox Solvency) the requirements of business are to be met.

The high technical knowledge needed at infrastructure level of all components is another challenge.
Auditors are well informed on possible risks. It is their job to look for them.



Solutions





1996,2004,2009,2012 J.A.Karman



SAS metadata - classic sas code

unsafe Problem
Releases of business logic by: Export - import of metadata is resulting in an incomplete environment.

Analyses
With SAS 9 all focus has gone to the metadata contaning all the information.
Classic sas code is still developed with the old approach. Eguide is positoned as a tool able to build code in that way. Classic code is stored on a server location using a link in Eguide. The Eguide projects can be seen as binary artifacts and should be involved in teh release process.


Eguide projects are commonly positioned personal files on Windows. They can as easily be placed on a server in the project location.

Data-time stamps of physical files.
The metadata export process is creating the fils at the new location causing owner and date time stamps are changing.
Use the classic tools with zip- or tar- usage to handle all physical files. They will save and restore also date time stamps correctly.


Solutions
Use the correct information: both type of components a/ metadata and b/ classic code are involved.
Bring all artifacts also the Eguide projects into the releasement process.

Use classic tools to keep data/time stamps on physical files as correct indication.

Use a strict process an order of the metadata artifacts and classic code artifacts. With promote ship/deploy processes. By example:




2009,2012 J.A.Karman (note april 2012)



Replication - business to other machine

geniaal handys Problem
The Business should get running on an other (logical) machine.
What parts are involved? Steps to do?

Analyses
Using SAS there is no issues on the type of OS or where you run the processes. As long as a/ you have them up and running b/ they have a compliant setup.
The business code/logic with the business data can be copied to any type of machine and got running.

The zip approach is hidden used in many places. java sas -package , Microsoft oofic x-files.

Solutions
Get the SAS middelware running on a OS. In a SAS BI environment the Metadataserver, Workspace serverer and Stored proces server with the related clients are common required components. Other components as needed.

Remember the package file for metadata can be very limited. You will be missing a lot of code. I have seen 80% classic code (not metadata connected missing in package files) and 20% connected to metadata.




2009,2012 J.A.Karman (note april 2012)



Base    Desktop    SQL    Stat-Miner    releases Business     Instal&check     OS level    top  bottom

Installation & basic checks

Way problems
  Subject
Installation checks
Crash dummy-s
a/ check functionality
b/loggings (eip913)
Where stores SMC keys passwords Management Console
Corrupted metadatabase mdassoc errors
Corrupted metadatabase no stored proces running
  Subject
Eguide BI Mainframe Errors - installation
Eguide BI notes generic Errors - installation
SAS/Connect tty reply messages missing


unsafe

SAS user no space Personal Storage

Problem
SASUSER out of space

Analyses
No more space availabel at OS level. (out of memory?)
Several reasons and backgrounds.

Solutions
At Mainframe: Get help OS support.
At Windows: You are really using this with all that big data?


1996,2004,2009,2012 J.A.Karman

unsafe

SASUSER missing

Problem
run Failed due to missing SASUSER.

Analyses
If SAS does not find an allocation of the DDNAME SASUSER (as is typically the case in batch), SAS tries to assign a library, by data set name, using the value of the SASUSER option as the data set name. If SASUSER is not defined as a catalog alias on the system, this results in a dynamic allocation request with the following parms. Processing this dynamic allocation request may result in a SYSDSN enqueue request with a DSN of "SASUSER".


When using SAS Interactive with a 3270 terminal or by SAS/connect the TSO-profile datasets must be able to allocate. An alias must exist. Missing the alias (catalog) blocks alocating personal datasets

Solution
BatchCircumvention: Put a //SASUSER DD to a small temporary data set in the SAS cataloged procedure with a suggested minimum SPACE of 3 tracks. KB17370
InteractiveDefinne the TSO User-Alias User to be able to let allocate the personal datasets.




1996,2004,2009,2012 J.A.Karman



SAS Work - Utility & Sizing & Mainframe

like hell Problem
Failing to run. Message: Out of space, System crash.

Analyses
Had to be done at LCM of SAS at production environment of business.

Utility messages
Utility dataset are used with proc sort - proc sql and others as intermediate data.
When errors ot this type happen, contact your sasadmin.
Mainframe HFS work
SAS 9.1 allows the WORK library to reside in a directory within a UNIX file system or a zSeries File System on z/OS. This feature simplifies the process of specifying disk space for the work libraries that are created for client processes in various SAS server environments. This feature may also be valuable for single-user SAS sessions. kb11159
Utility Unix
Utility datasets spacing on Unix Not always given te correct message. Old (82) was failing the option of utilloc. kb4914
Utilty z/OS multi volume

ERROR: Write to WORK.'@T000001'n.UTILITY failed. File is full and may be damaged.

See: kb16416
Hidden the descripton of an utility file is no to span volumes. If UTILLOC=WORK and the work library is allocated to span multiple volumes, the maximum for each utility file will only be one volume. kb11174
Mainframe multivolume
Not using next volume when running out of space on current B37-04. kb4916
Normal data -Mulitvolume z/OS. (old 82)
File is full and not switching to next volume. kb31737
multi-volume redesigned in SAS 82 kb9850
And the "DISP=MOD" with EOV ginving problems kb5054
Mainframe multivolume
On the OS/390 operating system, this resource environment file is allocated as a temporary data set with a system generated data set name. The amount of space allocated for the resource environment file is based on the following SAS options: FILESPPRI= FILESPSEC= FILEDEV= FILEUNIT= kb9531


Solution
Working out in SAS-options and startup scripts by Platform Admin.
Would be nice when available in the infrastructure acceptance testing (QA).


2009 J.A.Karman



like hell

Memory - Out of.. & tuning - Mainframe

Problem
"Out of memory messages" Not running the processes anymore.

Analyses
Had to be done at LCM of SAS at production environment of business.
limits z/OS
Memsize: On z/Os mainframe dependent of region-size hosto390 memsize
Memleave: Depends on the application and available system resources at the site. hosto390 memleave

spawned
The spawned SAS Workspace Server is a UNIX System Services (USS) process, and it follows all USS rules that govern memory/storage allocation for a USS process. The order of precedence (lowest to highest) for allocating memory for a USS address space is:
o the MAXASSIZE parameter in the 'SYS1.PARMLIB(BPXPRMxx)' file.
o the ASSIZEMAX parameter specified in the RACF USS/OMVS segment that is associated with the authenticated user
o exits (for example, IEFUSI) that affect USS processes
180-2008
amode rmode
26493 - ABEND 0Cx in loadm for an application linked AMODE=24 TSOE TSOAE ISPF

An 0Cx ABEND may occur in loadm for a program linked with AMODE 24. The ABEND can occur during program initialization, in which case the normal LSCX041 ABEND message may not appear. The abend may also occur via TSOE/ kb26493

28069 - All-resident programs may be linked RMODE(SPLIT)

All-resident programs used to be required to be linked with RMODE(24). If all-resident applications were linked with RMODE(ANY) then IEW messages such as IEW2646W or IEW2635I would result. That restriction has been ... c nt28069

other
28015 - ABEND 0C4 occurs using system in MVS/XA with TSO: prefix

This ABEND can occur and is usually intermittent, that is, the ABEND does not always occur. The problem is ... c nt28015   clug zallrest
The graphics support is possible using something causing "out of memory" kb15775
Indexing a dataset hits something causing "out of memory" kb5579.html


Solutions
Mainframe


2007,2009,2012 J.A.Karman



unsafe

SASUSER & Eguide -eip

Problem
SASUSER not accessable or available

Analyses
With enterprise using process with mainframe, the sasuser is set to work. See: 180-2008
With enterprise and other clients using a Workspace Server process (IOM-server) is setting SASUSER to "read-only".

Works as designed. (multiple processes could exist)

Solutions



2009,2012 J.A.Karman

unsafe

SASUSER no space

Problem
SASUSER out of space messages
SASUSER not accessible for update

Analyses
No more space availabel at OS level. (out of memory?)
Several reasons and backgrounds.

Solutions
At Unix: At Windows: You are really using this with all that big data?

2009,2012 J.A.Karman



like hell

SAS Work - Utility & Sizing & dio sio sgio

Problem
Failing to run. Message: Out of space, System crash.
Bad Performance throuhput low. Complaint is: same job running now 30 hours was 13 hours (on older development system).

Analyses
Had to be done as operational support shoud be solved by design.
Two users at the same are running. The sas-work (lspv Unix) is slow and exhausted.

The lay of storage has been changed. It is one big device now. In te past it were many small ones. This was not communicated as a change to SASWORK.
Segregation of usage
Need for segregation could be recognize as the used Non Personal keys at the used SAS-Work location.
NMON performance
Slow performance could be regconize as devices in the volume group of the saswork are most of the time ver-busy.
Cpu useage is indicating very long wait-times.

After having a new SASWORK connected and running by the customer still slow response is seen.

Getting help of storage specialists. Analyzing wat is happening.
Storage of SASWORK has been converterd to new but slow storage. Storage specialist is able to link inforamtion of device to a location in the SAN.
SAN is very smart, relocating data not used that much to slow devices. SASWORK files is classified in that way.

Getting more detailed Information:
One Fiber-channel attached is limited to 2 Gbit/s. A device running at 250Mbyte/s wil fully overload a chanhel, 1 Byte is 8 Bit multiply 250 by 8 to get bits.
Machine is causing having loads as seen in history graphs.

Solutions
Segregation of usage
Segregation in work implemented as third file-system for SASWORK. Activated to the NPA needing performance.
Round Robin option not available, SAS version tol old.
Performance -1-
DIO-CIO was activated on SAS-workk to get the memory-caching by-passed (10GB cache 50 Gb files).
Change is SAS-config was missed on this machine. Buffers were higher and memory was st higher. Is made equal to the other machine.
Small volumes (SAN) of 50Gb were asked in stead of the big volumes of 200Gb.
Performance -2-
Relocating of SASWORK to fast (faster) devices in the SAN.


2012 J.A.Karman (Apr 2012)



like hell

Memory - Out of.. & tuning - Unix

Problem
"Out of memory messages" Not running the processes anymore.

Analyses
Had to be done at LCM of SAS at production environment of business.
Ulimits Unix
File-size limit or memory limit not set by the visible user but are inherited by te spawning user-id. kb10797
Uncontrolled Changing of ulimits (by securitt) caused the enexpected issues of out of memory. Looking at the wrong key first. It is not the personal key settings.

Solutions
Unix


2007,2009,2012 J.A.Karman



Installation checks 913 eip

ligth house Problem
Not sure if all is working as should be.
Analyses
That is what is asking for.

Troubleshooting SP servers
kb13078 913 servers (I) Make some "crash dummy" sources to validate WS and SP startup.
/* crash dummy WS and SP Servers */
/* Version: NA.jak.1.0 */
libname _all_ list;
filename _all_ list ;
%put _all_ ; /* not working at 913 note 31887 */

kb31877 Missing normal correct working of macro-vars (IOM 913 servers)
SP standard macros you know already...
Troubleshooting WS servers
Run the "crash dummy WS and SP Servers" directly from Eguide
Troubleshooting Web-server
Define static HTML-pages containing:
Let these html-page be called by the wanted web-adres.
<html> <head> <title> crash dummy HTML </title> </head> <body>
<h1> crash dummy HTML </h1>
<h3> Version: NA.jak.1.0 </h3>
Dear tester <br>
When seen this text, The test has succeeded<br>
If the text is not visiable, call your tech support for analysing the problem <br>
</body> </html>

Troubleshooting SAS/intrnet services
Define as "dummy.sas" as text-file and "dummy.dummy.source" as source-file in a SAS-catalog. Let these programs run in the business environment by calling the http-adress and evaluate the results.
/* crash dummy .sas .scl */
/* Version: NA.jak.1.0 */
Data dummy ;
vartest="NA.jak.1.0 was here (print var)";
Proc print;

%put "NA.jak.1.0 was here (log macro)";

Indeed: Program is doing almost nothing. It should run, not more than that.
Use and activate Loggings
For a Unix based system:
-log "./WorkspaceServer/logs/WRS#d#b#y#v.log" -logparm "rollover=session open=replaceold write=immediate" -sastraceloc SASLOG -sastrace ',,,dq' -nostsuffix -fullstimer


For a Microsoft Windows based system:
-log "c:\....\logs\ws.%b%d@%H%M_%v.log" -logparm "rollover=session open=replaceold write=immediate" -sastraceloc SASLOG -sastrace=(,,,dq) -nostsuffix -fullstimer

For a z/OS system: o(''log="./WorkspaceServer/logs/WS_%Y.%m.%d_%H:%M:%s_%v.log" logparm="rollover=session write=immediate" fullstimer '')


(Note use of double and single quotes!)
Note: Ensure that every user that might execute a workspace server session has Write permission to the directory to which the logs will be written.

For Object Server Parameters, append the following:
iomlevel=1 applevel=2 JNLLINEMAX=10000 JNLSTRMAX=10000



can be activated at SP and WS Servers. Do not forget to restart object spawner.

Solutions
After having defined the "crash dummy" approach and explained to business support, these checks must be able to execute by them.



2009 J.A.Karman



Where stores SAS Management Console keys passwords

legal Problem
Auditing and reviewing the implmentation to policies. The User/Customer just can say SAS Management Console (SMC) stores it.

Analyses
This question can not be answered. The SAS ManagementConsole (SMC) is just tool to maintain a database. It is the metadatabase and the configuration files where all keys and password are stored
Other interface to metadata, Open Metadata Interface (OMI) exist. Java and SAS fucntions are documentend. SMC it is not the only interface.

Customer/User has probably been fooled. Selling the environment and presuming maintenance and support is easy. Solution
See the hardening part. If necessary restart everything with a fresh new installation to get it well secured.




2009,2012 J.A.Karman



reposmgr - mdassoc errors

like hell Problem
Users could not working well anymore. Miner was a big issue. One / more other users got failing password synchronisation to the server. At the moment errors occured while adding users (as admin) a more clear message did popup. Mdassoc failed.

Analyses
Having migrated the repository to an other location - machine, no errors are occuring. The error situation can not be reproduced.

The mdassoc errors hardly can be solved and recongized. Looking like corrupted metadata. the object cannot be added to the container TMP000.mdassoc.DATA is a clear message.
How to solve is hard to find. Starting with -Manually restore- : bisag manually restore and note 18027: reposmgr stop seperately, are giving some instructions.

The reposmgr directory contains the references to the other repositories and the mdassoc. It is descibed with omaport. Some text:
In order for %OMAPORT to work, the repository manager pathname must exist, and the rposmgr directory must contain valid repository manager data sets for the target host. The SAS Metadata Server populates the rposmgr directory with repository manager data sets upon invocation.
Starting the Sas Management Console (smc) with the unrestricted admin recreates the reposmgr. Some text:
When you are prompted for the repository path, specify the repository´s new location. The SAS Metadata Server will read the existing files in the specified location to create the new repository´s registration.

The clue of reposmgr. Some text: The datasets that live in the repository manager location are created for you and without your knowledge as the Open Metadata Server is initialized. Therefore, it is easy enough to remove all the datasets from that location and let the Open Metadata Server create them for you during initialization (when the server is started). What's the catch? None, other than the fact that you will need to re-register or add your existing repositories to the new repository manager.
deprecated omaprt with 92. note 40834 , new is smu Open Meta Architecture PORTen or Sas Migration Utility, just naming?
93 recovery the note mentioned cannot be found at this moment (feb 2012). At the chapter-Moving a Metadata Repository to a New Location on the Same Metadata Server- the approach of deleting the repository manager can be found.

Solution
The approach of deletion of reposmgr. Rebuilding and cleaning up the metadata, has resulted in a stabel working environment again.


2012 J.A.Karman (januari)

reposmgr - no stored proces running

like hell Problem
No stored processes were able to do any work.
People were able to login with Eguide all things looking to be ok to the metadata.
Another client was complaining about unavailable web-services.

Analyses
It happened the monday morning. Asking for some special events in the last days.
In the weekend the server was interupted by a unplanned power down, hardware power supply failure.
As the saswork wasn't cleaned up also, this problem was to solve first. As the metadatabase is memory-based cached it won't have got the oppurtunity to synchronise the data to disk. It will have gor corrupted when updates are doen after friday. A check call confirmed their actions.

Solution
The approach of deletion of reposmgr. Rebuilding and cleaning up the metadata, has solved the problem again.


2012 J.A.Karman (june)




papers

Eguide BI on Mainframe

Problem
Eguide and BI having some issues
Analyses
Not all requirements are known and full filled. Solution
Take care of notes like:


2012 J.A.Karman

papers

Eguide BI notes generic

Problem
Eguide and BI having some issues
Analyses
Not all requirements are known and full filled. Solution
Take care of notes like:


2012 J.A.Karman


SAS/Connect tty - messages missing

like hell Problem
SAS/connect session cannot operate together with mandatory "switch user" routines.

Analyses
Test command to start directly (ssh terminal) and from tty SAS with trace/debug
/usr/local/sas/batch/sasv9.ksh SX=myrgs_sd =CNCT -config /usr/local/sas/config/cing.cfg -config /usr/local/sas/config/eip91it1.cfg

$ fig/cing.cfg -config /usr/local/sas/config/eip91it1.cfg <
sasv9 20120406_140452 == suexec setting: myrgs_sd from user:fn46ge
oracle profile.....
profile for SAS users ...
rn05d2
oracle profile.....
profile for SAS users ...
drives-maps = /usr/local/sas -- /home/myrgs_sd/MY_SASfiles -- /home/myrgs_sd/MY_SASfiles --
config lic = -CONFIG /usr/local/sas/config/sasv9.cfg --
sasoptns =

SAS(R) TCPIP REMOTE LINK PORT=61334 SESSION ESTABLISHED.




Normal proceeding after "hello>"
Hello>
TRACE: udoneap:
TRACE: udone:
Type '/usr/local/sas/batch/sasv9.ksh =CNCT -config /usr/local/sas/config/cing.cfg -config /usr/local/sas/config/eip91it1.cfg ' lf;
TRACE: Goto u_stat2;
TRACE: u_stat2:
TRACE: Waitfor 'SESSION ESTABLISHED': rnsas,
TRACE: 'set home directory.': ufailhd,
TRACE: 'Cannot start remote process': ufailhr,
TRACE: 'Goodbye': ufailr,
TRACE: 145.0 seconds: ufail;
SAS(R) TCPIP REMOTE LINK PORT=61358 SESSION ESTABLISHED
TRACE: rnsas:
TRACE: Log 'NOTE: IT supprt-it TO_SHUIT SAS/CONNECT conversation established.';
NOTE: IT supprt-it TO_SHUIT SAS/CONNECT conversation established.



And now with the SX option, as worked well at Unix prompt.

Hello>
TRACE: udoneap:
TRACE: udone:
TRACE: Type '/usr/local/sas/batch/sasv9.ksh SX=myrgs_sd =CNCT -config /usr/local/sas/config/cing.cfg -config /usr/local/sas/config/eip91it1.cfg ' lf;
TRACE: Goto u_stat2;
TRACE: u_stat2:
TRACE: Waitfor 'SESSION ESTABLISHED': rnsas,
TRACE: 'set home directory.': ufailhd,
TRACE: 'Cannot start remote process': ufailhr,
TRACE: 'Goodbye': ufailr,
TRACE: 145.0 seconds: ufail;
........ (failing the text SESSION ...)
......


Where is the message response?.... it does not come.
Background
I have tried to use "su - " in the past. Other Unix commands to start. There is a limitation what can be used to start.
Missing the reason why. Looks like the SAS-spawner has something build in. Can be alsos soemthing with a line communication.

I am suspecting something with tty no-tty or display (file-1 file-2) but did not get the clue.. Some links: . oreilly ch08   mkssoftware secshd   mkssoftware stty  
The SAS/connect script is a documented way to use.

Using a scritp to start SAS is not unuasual. Somentimes no other way to get it working correctly...


Solution
Searching for a solution continues.

2004,2006,2009,2001,2012 J.A.Karman (apr 2012)



Base    Desktop    SQL    Stat-Miner    releases Business     Instal&check     OS level    top  bottom

OS level releases & security & tuning

Way problems
SAS Mainframe
Return codes ok=0 B37 E37 c008 U998
Mainframe Connect missing open segment definition Racf
Screen for input not coming blocked 3270 applications
Os networking exchange
Timed out, connection lost Firewalls
NL , CR-LF Unix Windows difference
Blocked ports network


SAS Java SCL


fire

entityExpansionLimit - DI SMC (Java)

Problem
When user logs into the DI 3.4 Studio software you is getting Clase errors and when you try to right click on a transform in a job, it will not generate the code.
Code Generation Error
Error Type: Code Generation Error
Cannot find class: 1dc9f77d-0a46-525b-0119-e58372b595cb.
...Code generation aborted...

Analyses
271-2009 5.2 Memory limitations advice to increase Xss Smx and Denttity...
DI Studio – Missing Nodes – DentityExpansionLimit=1000000 (blog.saasinct.com - optimalbi.com)




javase/1.5.0 Compatibility (oracle) Security Issue Posed by Nested Entity Definitions.
XML Denial of Service Attacks and Defenses. (msdn)
Solutions (possible)
Java runtime parameters
Add the following options
CommandLineArgs=-Xmx1028m -Xss2m -DentityExpansionLimit=1000000

in the etlstudio.ini file
kb20286 A code generation error that contains the string "cannot find class" occurs when the Process Library Tree is not displayed and your job contains one or more SAS® transformations
kb20991 Parser has reached the entity expansion limit "64,000" set by the application


2012 J.A.Karman (dec 2012 JB)

Java Popup screens

confused Problem
Windows of your desktop application is not reacting anymore. There should be a response, where?

Analyses
In Windows a pop-up will normally appear in the foreground (visible)
With Java the ordering can be less sure. It can be somewhere but not visible. The Windows screen-list does not help, it is Java.

Solution
When not able to get a reaction, try to find pop-ups hidden in the background.



2012 J.A.Karman (ca 2010)



fire

AF.AFGO SCL usage run-time

Problem
Conversion and uasage of SLC code, known as AF.


Analyses
Application Facilty has developed and changed much. Using SCL-code (Sytem Component Language) has advantages as it is compiled code and can run on all SAS environments.
SAS is promoting the usage of the BI appoach wich does not have SCL. Strange enough are the Solutions of SAS often build with the SCL-usage. The AF usage is not abandoned, see Operational life.

Problem
Solving issues with SCL AF at conversions.

Analyses
Knowledge of AF environment is not common. Some notes:

Solutions
SCL 93:
afproc Passing Options to an Application. The cmdlist is stored at af.afgo . A different catalog can be specified.

Object Declare
Within SCL "declare" (dcl) is used to define variables. Is it valid in certain positions, not in the header of a method. When defined in the header of a method the most strange errors occur.
The declare statement of the hashing,als object oriented, could be a hidden reason. Hashing is using the same naming conventions, objects, as SCL.
Double quotes (imbedded macro)
An imbedded macro is giving errors on call syput calls
Changing the fixed variable names to be used as single quotes runs without error. The macro in SAS-base is not having this issues.
The difference: something with encoding and/or macro expanding.
X-task (intrnet)
Refuses to continue afters issuing a X-cmd.
Bypass the usage with piping is not giving any problem.
The difference is: gathering the datasream of the called system process.


2012 J.A.Karman (june-august 2012 SW)



SAS Mainframe


Return codes - ok=0 B37 E37 c008 U998

oldtime Problem
The return codes have changed in the time. RC=0 still means everything should be ok. The warning level, originally on mainframe based as value 4, now is more over value 1. The differences: Win error levels   Unix completion status   z/OS Solving Problems

Analyses
z/OS was unique in having system abends like S0C4 S0C7 and handling with SB37 SE37 and User abends. This difference has been gone for some reason, probably TK, with SAS. The testing can be done with IEBGENER copying a file to a short location causing a SB37. The same logic can be done with SAS.
When C008 is the result something is wrong to schedulers. With an option (NOSTAE) a U998 is achieved.
//....
// EXEC SAS,OPTIONS='NOSTAE'
//....


A list of MVS Ssytem abend-codes MVS SYSTEM CODES a 0c7 is indicating memory corruption. Solution
(waiting)

2004-2009 J.A.Karman (1 march 2012)

Mainframe, Open segment Racf

unsafe Problem
SAS/Connect failing for some users and other users still succeeding. This in the migration roll-out to SAS 9


Analyses
Root-cause: There must be an "open segment" in RACF defining the Unix kernel be defined to all users using SAS/Connect (mainframe)
This is documented in the installation/configuration. At roll-out not checked was present to all users.

Solution
Solved: setting correct global system wide by security administrators.

2010 J.A.Karman (21 march 2012), 2006


Screen for input not coming

like hell Problem
At some unpredictable moments no input can be entered anymore. The screen is not coming another screen behaving abnormal.
With Windows (and X11-Unix) there is a change to find the screen on an other position. With 3270 mainframe this is impossible. Blocking the normal process.

Analyses
This looks like some very nasty timing problem in the basics. The order of screens seems to be get incorrect. An informational screen overlaying the input screen.
The problem is getting this to a repeatable scenario. I found one scenario by accident.
Doing two remote connects each quickly followed by a remote screen. It will dump the second input request in a wrong order. The remote displays is coming afterwards. Blocking the input request password/key of the second request
This scenario is of course not a practical request. It only proofs that getting the response screens out of order is happening.
Suspecting more of this kind, very difficult to recognize.

Solution
Not available. Waiting for admitting and solve it.

2009 J.A.Karman (2012 Still Waiting)


-Nix security


Change pswd -- using X command

like hell Problem
Wanting to update a password in Unix in a scripted way Analyses
Got A hit: 790-changing-password-via-a-script (en.kioskea.net)
echo -e "new_password\nnew_password" | (passwd --stdin $USER)

Note that the method can be use to change the password on Samba based system:
echo -e "new_password\nnew_password" | (smbpasswd -a -s $USER)


The method is failing in SAS:

Solution
Not possible

2012 J.A.Karman (25 oct 2012), 2006


Sudo - boks: getting secure (su)

like hell Problem
The usage of "high privileged keys" should be monitored. These "high privileged keys" are indicated as all "Non Personal Accounts".
This problem is not well defined as coming in. The risk analyses of the different types of NPA´s is missing. The usage of NPA´s to lower the privileges and risks is misunderstood.

Analyses
2007 su - usage
We were forced to use it and connect NPA´s to a nosshd group. The SAS/Connect doesn´t succeed something bothering to get a password replied within scripting. All NPA´s implemented by a nosshd group, trying to force "su - " usage.


2009 sudo - usage
"Switch User DO" is a well known tool to implementation for auditing on "Non Personal Keys". As SAS is using many NPA´s Sudo usage become mandatory.
There are more things to do. The login to a shell must be disabled an at the same time login with SAS must be continued.
sudo is behaving well and also SAS/connect setup could be faked to use it. With "sudo -l" all personal options/access is visible.
sudo Nearly implemented for all NPA´s.

2011 Conversion to Boks done () October
Boks should be better in auditing, adding keystroke logging. All commands and instructions are changed (3 months work).
Then he problem of implementing it in SAS/connect (remote command) failed. Looks missing a terminal device connection. Tip(s): do a reset of the HOME variable it is containing the wrong old originiting user. Script in a ssh shell is working with sx.

2012 continuing to solve
It is the same problem as the "su" has. Trying to use this behind SAS/connect is failing somewhere in the communication of port numbers using the first line-communication tty?
A lot of this can be traced by using SAS connect scripts
The environment used with suexec is not set properly.
Tip(s): unset TMPDIR.


Solution
(to be continued)
Back to attempting solving it in a SAS/connect script with TTY and Unix script.

2012 J.A.Karman (April 2012)



Aix 5 - Aix 6, getting password problems.

like hell Problem
A Simple migration of a OS-level, should give no problems. Is done and ..... Some keys cannot be used anymore to login. Why?

Analyses
Getting to debug it. Personal keys have no problem, the NPA´s have issues. Password-s of 12 long and some special characters like _ and @ are used.
First suspect: different password hashing

Hitting: Unix is just using 8 chars ignoring the longer phrase and the special chars are influenced by the "crypt()" routine (default hashing).
Action to be done now is resetting all the keys-passwords containing these special chars. Lucky I have some using as "crash dummies". After having done these password changes, nothing solved. Checking the hash on the systems (privileged access) showed no difference.

Seeking further. four of my six key-s are working fine two are failing. Some differences found between the machines. min-age and max-age are zero on the old Aix version 1 and 5 on the new version. The failing two keys have a valid last-login-date the four well working have no date. Trying to login in SMC meta data-server had a message of expired password for the key in the metadatalog. SAS-connect was just saying "invalid password".


Boks is replacing the SSh shell. Behavior of the OS utilities can become failing. There is a difference between the three OS/Aix Boks and SAS.


Tip(s): use options tcpdebug (?). use permtest to check.
Solution
Solved: setting max-age min-age to zero by security/Unix administrators.
Solved: settin expiration date correctly in BOKS.

J.A.Karman 6 April 2012 august 2012.

Who was / is

unsafe Problem
Who is was doing something? Last login?

Analyses
this should help with analyses

Solutions
Command last
$ last

m00008g pts/3 wpnld8958186.nl.europe.intranet May 18 10:04 still logged in.
fn46ge pts/2 wpnld9309244.nl.europe.intranet May 18 09:15 still logged in.
ro33gg pts/0 wpnld9315936.dhcp.ing-ned May 15 08:23 still logged in.


2007,2012 J.A.Karman


OS networking exchange


unsafe

Timed out, connection lost

Problem
Getting unexpected lost connections.

Analyses
It is happening just in one building. Suspecting migration with segregation in networking.
After some time a statement came back of networking and firewalls (gateway related):
When no data is entered for a period of 2 hours the statefull database will be flushed. This could be the issue.

It works as has been implemented using a gateway and segregated networking.
The network repsonse is also lower compared to other buildings, same gateway as difference. Solution
Pay attention to this pitfall

2012 J.A.Karman
unsafe

NL , CR-LF

Problem
Getting corrupted files after transfers.

Analyses
Although Unix and Windows share the Ascii character set, they did something different with record-indications.
This results in the requirement to recognize binary and character up/downloads. When doing wrong, getting corrupted files.
If you can open the file with "vi" you will see ^M symbols. All other access does not give a clue like this. NL new-line       CR-LF Carriage Return , Line feed       both simulating the typewriter.


Solution
Pay attention to this pitfall

2007,2012 J.A.Karman
unsafe

Blocked ports network

Problem
Getting no connectiosn started.

Analyses
A port was reporting back as in use (see below)
netstat -an|grep 49421
tcp 0 0 10.20.18.110:38320 10.20.18.113:49421 ESTABLISHED

The application associated with the port 49421 was no longer active and we needed to clear this contention to re-start the application without having to reboot the server (to clear inactive processes)

Solution
Use lsof to find and clear contended ports (*NIX Flavors) without rebooting your server
By using lsof command to find the process ID (pid) associated with the port and removing the pid the port contention is cleared and no server re-boot was needed.
lsof -i TCP:49421 COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME java 3229 jbadm02 1315u IPv4 4740019 0t0 TCP aiilnx64:38320->v3aiilnx64:49421 (ESTABLISHED)

And now removing the pid 3229 (2nd column in output above)
kill -9 3229

will clean up the contention and you no longer need to reboot server.
* note: use of lsof. may require root priveleges, ask your admin.


2013 J.A.Karman (SAS Architects/Administrators/Implementation Specialist - Rafi jan)




Base    Desktop    SQL    Stat-Miner    releases Business     Instal&check     OS level    top  bottom
home-SAS    Main-SAS usage    Main-SAS operational     Main-DTAP design