About FriedEgg

FriedEgg · ‎08-23-2011

data input; input key $ data $; cards; Abc123. 077123 Abc123. 098546- Abc123. 875433 Def123. 345423 Def123. 344565x ; run; proc sort data=input; by key; run; /* Option 1 */ proc print data=input noobs; by key; id key; run; /* Option 2 */ data output; length all_data $256; set input; by key; retain all_data; if first.key then all_data=data; all_data=catx(' ', all_data, data); if last.key then output; run; proc print data=output; run;

FriedEgg · ‎08-18-2011

If you are looking to collect a variable list for a given table you can also use the sashelp.vcolumns table instead of proc contents and upcase the name as part of the extract.

FriedEgg · ‎08-17-2011

The SPDE library engine is a included feature as part of you base license, which is quite nice. It is somewhat of an introductory product to the SPDS product.

FriedEgg · ‎08-17-2011

SAS has a procedure for doing exactly that. http://support.sas.com/documentation/cdl/en/proc/61895/HTML/default/viewer.htm#a000316288.htm You can also send the resulting file via ftp using the filename statement: http://support.sas.com/documentation/cdl/en/lrdict/64316/HTML/default/viewer.htm#a000178980.htm As far as scheduling is confirmed you can install a crontab task to execute the job as a SAS batch process if you wanted.

FriedEgg · ‎08-17-2011

The SPDE library engine works very well for this. As long as the I/O subsystem on the machine is sufficient enough to support it. The SPDE engine also provides a number of other benefits, like being able to split data sets over multiple I/O channels and disks, I utilize the engine for extremely large data. Tables in the trillions of rows for example. Here are some metrics from a job I ran just the other day: I extract a table from Oracle Data Warehouse. The data is column dimensioned. 12.3T rows and 4 columns. Extract in 8 threads to single dataset using dbslice... 2 hours Proc sort entire set by primary and secondary key... 5 hours Datastep to transpose data using by group processing and arrays... 9 hours The resulting dataset in a row dimensioned SAS table with 200M rows and 5500 columns. Without utilizing the SPDE library engine these processes took so long that they were basically deemed useless... Most significantly the final transpose step took more that 3 full days to run. The machine used to create the above numbers is a 64bit Linux, 8 cores, 42 GB ram, storage is a 24 disk DAS with two 12 disk raid 5 groups that are then in a raid 0 configuration. SPDE engine is using 8 data store points and 24 max threads. In short, I LOVE the SPDE library engine! it is an incredibly useful tool for big data. Other methods for data step multithreading is to split the data into logical groups as others said and utilizing the MP-CONNECT tools (available if you have SAS/CONNECT license) then run multiple asyn steps. I personally prefer the simplicity the use of the SPDE engine allows for comparatively.

FriedEgg · ‎08-17-2011

data temp; input id q1 q2 q3; cards; 1 1 1 1 2 2 2 2 3 3 3 3 4 4 4 4 5 1 2 3 6 2 3 4 ; run; data _null_; do i=1 to 3; call execute( 'proc freq data=temp;' || ' tables q' || strip(i) || '/out=percent' || strip(i) || '(keep=percent) noprint;' || 'run;' ); call execute( 'proc transpose data=percent' || strip(i) || '(rename=(percent=q' || strip(i) || '))' || ' out=ptran' || strip(i) || '(drop=_label_ rename=(_name_=question)) prefix=pct;' || 'run;' ); end; run; data percent; set ptran:; run; could also be done in a macro, but meh... EDIT: Just had an additional thought to the above. You could significantly reduce the I/O requirements of the frequency step by replace the multiple calls of the frequency procedure by instead creating multiple tables statements instead... Gains would become noticeable with increasing quantities/variables.

FriedEgg · ‎08-17-2011

This paper provides some insight into the splitting utilizing hash object method as well another example from the pre-v9 era. A nice read. http://www2.sas.com/proceedings/sugi30/236-30.pdf And to Art's point it is authored in part by Paul Dorfman.

FriedEgg · ‎08-17-2011

data temp; input id upc_code year ; cards; 2000351 1230070413 2004 2000351 . 2004 2000351 1230000018 2004 2000351 1230070499 2004 2000351 . 2004 2000351 . 2005 2000351 1230011813 2005 2000351 . 2005 2000351 . 2005 2000351 . 2005 2000351 1230070413 2005 2000351 . 2005 2000351 . 2005 2000351 . 2005 2000351 . 2005 2000351 1230011813 2005 ; run; proc format; value upcfmt 0-high='1' other='0'; run; proc freq data=temp; tables upc_code /out=want(drop=upc_code percent) noprint; by id year; where upc_code > 0; format upc_code upcfmt; run; or proc means data=temp n noprint; var upc_code; by id year; output out=want(drop=_type_ _freq_) n=count; run;

FriedEgg · ‎08-17-2011

Hi, I understood your intended result slightly differently as tring to 'event' by your dt var. Your question is not really specific to your intended problem/solution. data a; input sasid cul dt ddmmyy10.; format dt ddmmyy10.; cards; 1 23 12/10/10 1 23 15/10/10 1 23 16/10/10 2 44 16/10/10 2 34 12/10/10 2 44 16/10/10 3 14 12/10/10 3 24 15/10/10 3 56 16/10/10 3 18 10/10/10 ; run; proc sql; create table b as select dt, min(sasid) as f_s, max(sasid) as l_s, min(cul) as f_c, max(cul) as l_c, count(1) as cnt from a group by dt order by dt; quit; proc print data=b; run; obs dt f_s l_s f_c l_c cnt 1 10/10/2010 3 3 18 18 1 2 12/19/2010 1 3 14 34 3 3 15/10/2010 1 3 23 24 2 4 16/10/2010 1 3 23 56 4

FriedEgg · ‎08-17-2011

Since you appear to try and execute this code as a batch process your should add the -batch system option. All system options should be defined by -[option] <[value]> C:\Program Files\SAS\SASFoundation\9.2\SAS.EXE" -batch -lrecl 3000 -icon -noterminal -nosplash -noxwait -noxsync -CONFIG "C:\Program Files\SAS\SASFoundation\9.2\SASV9.CFG" -SYSIN "X:\Paul\Programs\Major\Onboarding\Code\onboarding_trial_030_em.sas" -LOG "X:\Paul\Programs\Major\Onboarding\Code\onboarding_trial_030_em.log Usually I prefer to keep lrecl option parameters inside my programs and use an options statement or on my filename statement or infile statement. Options Statement syntax: http://support.sas.com/documentation/cdl/en/lrdict/64316/HTML/default/viewer.htm#a000289431.htm Filename Statement syntax: http://support.sas.com/documentation/cdl/en/hostwin/63285/HTML/default/viewer.htm#chfnoptfmain.htm Infile Statement syntax: http://support.sas.com/documentation/cdl/en/lrdict/64316/HTML/default/viewer.htm#a000146932.htm

FriedEgg · ‎08-15-2011

Ah, that's right, thank you for the clarification.

FriedEgg · ‎08-15-2011

When was it changed that these types of group by summarizations no longer required a retain statement containing the summarizing variables? I tested both ways and results are the same (using SAS under Unix 9.2). However I remember in previous versions or possible different circumstances where if the retain statement was not used results were incorrect. data cig.Merged2_bna_law; SET cig.Merged2; by household_id; retain npb4w npb4r npb4b; if first.household_id then do; npb4w=0; npb4r=0; npb4b=0; end; if purchase_date<work_date then npb4w+1; if purchase_date<restaurant_date then npb4r+1; if purchase_date<bar_date then npb4b+1; if last.household_id then output; run;

FriedEgg · ‎08-15-2011

Hi, I never received any reply's to this posting and so I decided to go about figuring it out myself. First, I created a keystore using the 'keytools' application. This is what I will use later to config Djavax.net.ssl keytool -genkey -keyalg RSA -alias mycert Second, I altered my config file to contain the following: -jreoptions (-Dtkj.app.launch.config=!SASROOT/picklist -Dsas.app.class.path=/usr/local/SAS/SASVersionedJarRepository/9.2/eclipse/plugins/tkjava.jar -DPFS_TEMPLATE=!SASROOT/misc/base/qrpfstpt.xml -Djava.security.policy=!SASROOT/misc/base/sas.policy -Djava.security.auth.login.config=!SASROOT/misc/base/sas.login.config -Djava.class.path=/usr/local/SAS/SASVersionedJarRepository/9.2/eclipse/plugins/sas.launcher.jar -Djava.system.class.loader=com.sas.app.AppClassLoader -Dsas.ext.config=!SASROOT/misc/base/sas.java.ext.config -Djavax.net.ssl.trustStore=/home/&sysuserid/.keystore -Djavax.net.ssl.trustStorePassword=q1w2e3) This link was helpful but somewhat incomplete in explaining what needed to be done. http://support.sas.com/documentation/cdl/en/proc/61895/HTML/default/viewer.htm#a003286920.htm Thanks, Matthew Kastin

FriedEgg · ‎03-30-2011

Not sure if this will aide you since i do not have a ftp with which to verify but you could try to use fileref function: %macro check_ftp_connection(inf=); %if %sysfunc(fileref(&inf)) ne 0 %then %put %sysfunc(sysmsg()); %mend; filename indir ftp user='user' pass='pass' host='host' dir; %check_ftp_connection(inf=indir); Since you said it is not generating a error I am not sure that this will benefit. Message was edited by: Fried Egg

FriedEgg · ‎03-30-2011

I agree with Linus, running via cron should result in a different system environment. You should check relative system and sas environment from cron vs. running manually on your account. Also you can try to have cron load your user profile prior to executing the script: 0 8-20 * * 1-5 . /home/testuser/.shell_profile your_script_here.sh Additionally you could task cron to su to a different user and run the script as a given user.

Online Status	Offline
Date Last Visited	‎03-31-2025 06:28 PM