About FriedEgg

FriedEgg · ‎08-26-2011

using the ''dt will convert your text date to a sas numeric date value which may not correctly represent the same date in your oracle machine. %let startdate=01jan2010; %let enddate=31dec2010; data _null_; call symput('start', "'" || "&startdate.:00:00:00" || "'"); call symput('end', "'" || "&enddate.:00:00:00" || "'"); run; proc sql; /* stuff */ &start <= t3.file_date_of_report <= &end /* otherstuff */ quit;

FriedEgg · ‎08-26-2011

DATA input(SORTEDBY=customer); INFILE cards; INPUT customer $ product_type $; CARDS; C1 A C1 B C1 C C2 D C3 B C3 E C4 B ; RUN; /* using normal datastep */ DATA output(KEEP=customer); SET input; BY customer; IF FIRST.customer THEN DO; b_buyer=0; other_buyer=0; END; IF product_type='B' THEN b_buyer+1; IF product_type NE 'B' THEN other_buyer+1; IF LAST.customer and b_buyer>0 and other_buyer=0 THEN OUTPUT; RUN; /* with proc sql */ PROC SQL; CREATE TABLE output2 as SELECT customer /* not sure that having the distinct keyword here is necessary, in this limited case it is not */ FROM input GROUP BY customer HAVING MIN(product_type='B') and MAX(product_type='B'); QUIT;

FriedEgg · ‎08-26-2011

Using the code above I end up truncating the data in the last comment field which contains a '|' the same as the delimiter being used.

FriedEgg · ‎08-26-2011

Since my real data contains over 100 million rows and about every 20 many rows there is just a erronious line of data with just the + and - marks I would rather save the time having to reread the file after a proc import to remove those lines. So i guess the answer is no, I did not try using proc import, it may work, I'm not well very well versed with the procedure. Can it automatically determine layouts on fixed length files? I would think not? How would it interpret the text fields that contain the faux-delimiter pipes?

FriedEgg · ‎08-26-2011

Problem: I have a very long running process, due to inconsistent server load determining when the process will complete is difficult based on wall time alone. The process runs in batch mode on a unix server. Possible Solution: Here is an example of the functionality I would like to replicate (code below is perl). Put a note in the log each time a given number of records has been processed. while (<INFILE>) { $counter++; if ($counter%500000 == 0) { print "Working... $counter \n"; } #Do Stuff; } print "All Done... $counter \n"; The above will read a file and each time it reaches an increment of 500,000 records will print a status msg while the program is working to let the user know the progress of the script. This is what I want to replicate in SAS. Should I add -logparm "write=immediate" to my config file, does this actually print immediately while still processing a datastep? I am not currently able to test but I know in my current enviornment the following will not work: Curent Enviornment: LOGPARM=WRITE=BUFFERED ROLLOVER=NONE OPEN=REPLACE data _null_; do i=1 to 10000000; call ranuni(1234); if mod(i,50000)=0 then putlog i=; end; run; If the write=immediate option doesn't resolve the issue my next thought is to call sysexec and cat a note onto the end of the logfile (if I even can incase there is a lock on the file held by sas while using write=buffered? Has anyone tried something like this? Have any other ideas to implement a similar function?

FriedEgg · ‎08-24-2011

Problem: I receive a file from outside client that is a extract from mysql dumped to text. As the database grows the format of the file is bound to alter based of the data input into the clients survey application. Because of the boundless nature of the input a dynamic fixed length reading step seems to be the most productionable way to convert this data. In other words - The input column will not change in any way from day-to-day however the length of the fields will change as data is collected (to be the length of the longest entity of the column). As you can see there are delimiters between the columns however there is no restriction to the data entered so the delimiter cannot be necesarily relied upon from a row-to-row basis. Desired Outcome: Properly converted sas data to later be utilized for reporting purposes. Obs list_name original_filename email_addr member jobno job_name ae question choice comment 1 08232011 survey_list_20110823.csv this_is@anemailaddress.com MEM1 FT93 This is a jobs name hjsimpson How satisfied were you with the accuracy of your order? 4 2 08232011 survey_list_20110823.csv this_is@anemailaddress.com MEM1 FT93 This is a jobs name hjsimpson How satisfied were you with the timeliness of your order? 3 This is some sort of random comment that the user has decided to leave in response to this question. 3 08232011 survey_list_20110823.csv this_is@anemailaddress.com MEM1 FT93 This is a jobs name hjsimpson How satisfied were you with you account manager? 4 4 08232011 survey_list_20110823.csv this_is@anemailaddress.com MEM1 FT93 This is a jobs name hjsimpson How satisfied were you with the products and services Company XX offers? 3 This is another example of a random comment. This is is quite long so yeah, I dunno, what do you think about it? I was just starting to think to myself that this is just about long enough. Okay, well mayke just a little bit longer then.... Hmm, okay. 5 08232011 survey_list_20110823.csv this_is@anemailaddress.com MEM1 FT93 This is a jobs name hjsimpson How satisfied were you with the overall experience of working with Company XX? 4 The comments could potentially contain any characters they wanted so I am not going to be able to use the | character to delimit them. My Solution and Purpose for Posting: This is my conclusion to the problem at hand, I thought of a few others but this is the one I ended up writing. It works as expected, just looking to see what others might come up with. Enjoy data input; infile cards truncover; input data $512.; array _end[10] _end1-_end10; if _n_=1 then do _i=1 to dim(_end); retain _end:; if _i=1 then _end[_i]=find(data,'+',2); else _end[_i]=find(data,'+',_end[_i-1]+1); end; if substr(data,1,1)='+' then delete; if _n_=2 then delete; length list_name original_filename email_addr member jobno job_name ae question choice comment $256; array _var[10] list_name original_filename email_addr member jobno job_name ae question choice comment; do _j=1 to dim(_var); if _j=1 then _var[_j]=strip(substr(data,2,_end[_j]-2)); else _var[_j]=strip(substr(data,_end[_j-1]+2,_end[_j]-_end[_j-1]-2)); end; drop data _:; cards; +-----------+--------------------------+----------------------------+--------+-------+---------------------+-----------+--------------------------------------------------------------------------------+--------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | list_name | original_filename | email_addr | member | jobno | name | ae | content | choice | comment | +-----------+--------------------------+----------------------------+--------+-------+---------------------+-----------+--------------------------------------------------------------------------------+--------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | 08232011 | survey_list_20110823.csv | this_is@anemailaddress.com | MEM1 | FT93 | This is a jobs name | hjsimpson | How satisfied were you with the accuracy of your order? | 4 | | | 08232011 | survey_list_20110823.csv | this_is@anemailaddress.com | MEM1 | FT93 | This is a jobs name | hjsimpson | How satisfied were you with the timeliness of your order? | 3 | This is some sort of random comment that the user has decided to leave in response to this question. | | 08232011 | survey_list_20110823.csv | this_is@anemailaddress.com | MEM1 | FT93 | This is a jobs name | hjsimpson | How satisfied were you with you account manager? | 4 | | | 08232011 | survey_list_20110823.csv | this_is@anemailaddress.com | MEM1 | FT93 | This is a jobs name | hjsimpson | How satisfied were you with the products and services Company XX offers? | 3 | This is another example of a random comment. This is is quite long so yeah, I dunno, what do you think about it? I was just starting to think to myself that this is just about long enough. Okay, well mayke just a little bit longer then.... Hmm, okay. | | 08232011 | survey_list_20110823.csv | this_is@anemailaddress.com | MEM1 | FT93 | This is a jobs name | hjsimpson | How satisfied were you with the overall experience of working with Company XX? | 4 | The comments could potentially contain any characters they wanted so I am not going to be able to use the | character to delimit them. | +-----------+--------------------------+----------------------------+--------+-------+---------------------+-----------+--------------------------------------------------------------------------------+--------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ ; run;

FriedEgg · ‎08-24-2011

proc format; invalue blahfmt (fuzz=.09999999999) .1 = .A .2 = .B .3 = .C .4 = .D .5 = .E other = .; run; data _null_; nmiss=0; do i=1 to 1000; blah=input(ranuni(1234),blahfmt.); if blah=. then nmiss+1; end; putlog nmiss=; run;

FriedEgg · ‎08-24-2011

I would double check your language setting through the sas pass-through session: SELECT * FROM NLS_DATABASE_PARAMETERS WHERE parameter IN ( 'NLS_LANGUAGE', 'NLS_TERRITORY', 'NLS_CHARACTERSET'); You can change these on a per session basis by: execute by oracle(ALTER SESSION SET NLS_LANGUAGE = FRENCH_FRANCE.WE8ISO8859P1); You might also look into altering the NLS parameters on your executing server for example in unix: %sysexec(export NLS_LANG=FRENCH_FRANCE.WE8ISO8859P1); You can lookup your valid languages using: SELECT parameter, value FROM V$NLS_VALID_VALUES; More simply: You can pass the optional NLS_DATE_LANGUAGE parameter to your TO_DATE function SELECT TO_DATE('1-Août-01','DD-MON-YY','NLS_DATE_LANGUAGE=FRENCH') FROM DUAL;

FriedEgg · ‎08-24-2011

The remaining string I believe is the "&NBSP" which is a non-collapsing space tag. You could modify the regular expression to remove this additional tag as well. s/<[^>]+>|\x26[^<>]*;//

FriedEgg · ‎08-23-2011

Not really certain what it is you are attempting to accomplish here but you might find this helpful: ods output LinearDiscFunc=discrim_results; proc discrim data=discrim; class=group; var var1-var5; run;

FriedEgg · ‎08-23-2011

You can access the EG scheduler from the File>Schedule Project (the document above says it is under the Tools drop-down). I have never used it before but basically anything would be preferable to a constantly running loop to execute a subsequent step once every 10 minutes...

FriedEgg · ‎08-23-2011

This is just to provide sample data for your example as well as a duplicate of your SQL. data recover(rename=(i=customer_id) sortedby=customer_id); do i=1 to 10000; nb_created_date=round(ranuni(1234)*100,11)+'01AUG2011'd; output; end; run; data nbcase(rename=(i=customer_id) sortedby=customer_id drop=j k); do i=1 to 10000; j=round(ranuni(1234)*10,1); do k=1 to j; nb_first_date=round(ranuni(1234)*100,11)+'01AUG2011'd; nb_last_date=round(ranuni(2345)*100,11)+'01AUG2011'd; output; end; end; run; proc sql; delete from _recover r where exists ( select * from nbcase nb where r.customer_id=nb.customer_id and ( r.nb_created_date < nb.nb_first_date or r.nb_created_date > nb.nb_last_date ) ); quit; Message was edited by: Matthew Kastin Reason: Previous posting was erroneous after testing.

FriedEgg · ‎08-23-2011

.A - .Z are valid numeric invalues: data _null_; input x @@; cards; .A .B .C .D .E .F .G .H .I .J .K .L .M .N .O .P .Q .R .S .T .U .V .W .X .Y .Z ; proc print; where x <= .E; run; Obs x 1 A 2 B 3 C 4 D 5 E

FriedEgg · ‎08-23-2011

proc format; invalue chtono (default=1) 'Y'= 1 'N'= 2 'X'= .N other= .U; run; data want; input @1 x chtono. /* since defined default in proc format do not need to define chtono<w>. */ @2 y chtono. /* could just use chtono1. instead for same result */; cards; 12 YN XY NX 21 ; proc print data=want noobs; run; x y U U 1 2 N 1 2 N U U

FriedEgg · ‎08-23-2011

Unix Scheduler: Unix$> crontab -e 10 * * * * /path/to/sas -batch -noterminal -config path/to/sasconfig.cfg -sysin /path/to/code.sas -log /path/for/log.log -print /path/for/print.lst SAS Macro (would not recommend): %do until(&exception=1); %execution_macro data _null_; slept=sleep(60*10); /*sleep for 10 minutes */ run; /* sas statements to catch an exeception to stop loop */ %end;

Online Status	Offline
Date Last Visited	‎03-31-2025 06:28 PM