About FriedEgg

FriedEgg · ‎10-05-2011

Here is a functional implementation of the simplest cipher I know, the Caesar cipher. Note that this code is very similar to the macro implementation available in the pdf I linked in the original post. I am still continuing work on additional implementations of more complex algorithms. procfcmp outlib=work.func.cipher; function caesar(var $, shift, mode $) $; lengthcipher $26; alphabet='ABCDEFGHIJKLMNOPQRSTUVWXYZ'; cipher=substr(alphabet,shift+1); array c[100]$ c1-c100; do i=1to shift; c=substr(original,i,1); end; cipher=cats(of cipher c1-c100); ifupcase(mode) in ('E','ENCRYPT')then do; encrypt=translate(strip(var),strip(alphabet),strip(cipher)); return(encrypt); end; elseif upcase(mode) in ('D','DECRYPT')then do; decrypt=translate(strip(var),cipher,alphabet); return(decrypt); end; elsereturn('***ERROR***'); endsub; quit; %letcmplib=%sysfunc(getoption(cmplib)); optionscmplib=(work.func &cmplib); datatest; inputt $; e=caesar(t,5,'e'); d=caesar(e,5,'d'); cards; TEST REST BEST ; run; options cmplib=(&cmplib);

FriedEgg · ‎10-05-2011

Okay, I think I better understand the issue you have at hand. The issue is that you have a number of ETL processes that possible run in parrallel of eachother and want the to have the added intelligence to wait for eachother to release their locks on datasets. There are several options you can use to see what works for your specific situation. The libname statement I beleive has option FILELOCKWAIT and I think related system option FILELOCKWAITMAX. Whether the lock statement works with SPDE, I do not know. But apperances, based on your comments, lead me to believe that it is not compatible. Try resolving the automatic variable &syslckrc after attempting to perform a lock on your SPDE dataset.

FriedEgg · ‎10-04-2011

Make sure the delimiter is actually a tab. Also that part of my code was only to recreate your example data files. This is the part that is actually revelant to your situation: filename in ('/temp/*.csv'); data have; length _file $256; infile in dsd dlm=',' filename=_file; input date :??mmddyy8. (px_last px_low px_high) (:8.4) volume :7.; ticker=scan(scan(strip(_file),-1,'/'),1,'.'); run;

FriedEgg · ‎10-04-2011

Any open on a file with the V9 or SPD Engines causes a lock to be applied to the file. Unlike the SPDS which allows for multiple opens. The reason to use a lock statement or command usually to prevent other programs from altering a dataset while it is in the middle of some multistep processing.

FriedEgg · ‎10-03-2011

without really knowing your exact usecase scenario I would not forsee there being any issues with the macro code resolution.

FriedEgg · ‎10-03-2011

filename tmp temp; data _null_; infile cards missover; input data $512.; file tmp; put _infile_; cards; 7647 Dummy Date 11-Jun 11-Mar 10-Dec 10-Sep 10-Jun 10-Mar 9-Dec 9-Sep 9-Jun 9-Mar 8-Dec 8-Sep 8-Jun 8-Mar 7-Dec 7-Sep 7-Jun 7-Mar 6-Dec 6-Sep 7647 Dummy Cash 374,999.00 355,305.00 344,476.00 421,880.00 419,419.00 362,088.00 335,474.00 363,165.00 350,761.00 350,637.00 124,905.00 138,088.00 153,846.00 169,608.00 183,856.00 178,935.00 180,308.00 182,232.00 185,859.00 178,840.00 7647 Dummy Credit 903,945.00 892,582.00 898,555.00 890,329.00 910,922.00 929,207.00 862,928.00 878,434.00 908,463.00 947,960.00 908,375.00 922,330.00 853,334.00 858,979.00 864,756.00 784,002.00 749,575.00 714,901.00 697,474.00 660,277.00 7647 Dummy Supplies 22,567.00 27,059.00 37,032.00 35,536.00 40,547.00 37,694.00 46,079.00 42,035.00 52,795.00 41,942.00 33,281.00 29,246.00 593 494 351 192 108 89 69 76 7647 Dummy Other @NA @NA @NA @NA @NA @NA @NA @NA @NA @NA @NA @NA @NA @NA @NA @NA @NA @NA @NA @NA 7647 Dummy Total @NA @NA @NA @NA @NA @NA @NA @NA @NA @NA @NA @NA @NA @NA @NA @NA @NA @NA @NA @NA 7647 Dummy Equipment @NA @NA @NA @NA @NA @NA @NA @NA @NA @NA @NA @NA @NA @NA @NA @NA @NA @NA @NA @NA ; run; proc import datafile=tmp out=have dbms=dlm replace; getnames=no; delimiter='09'x; run; proc transpose data=have out=want(drop=_:) prefix=xyz; by var1 var2; id var3; var var4-var23; run; data want; set want(rename=( var1=id var2=name xyzother=other xyztotal=total xyzequipment=equipment)); cash=input(xyzcash,comma10.); credit=input(xyzcredit,comma10.); supplies=input(xyzsupplies,comma10.); xyzdate=strip(xyzdate) || '-2010'; date=input(xyzdate,date11.); format cash credit supplies dollar12.2; drop xyz:; run;

FriedEgg · ‎10-03-2011

The proc import procedure or whatever other method you would utilize would likely determine the input column formats as character, just as I have here. The fact that they are mixed on the input file is why. To adjust to the proper formats you would need to perform additional actions in a datastep after the transpose to fix it.

FriedEgg · ‎10-03-2011

I am not entirely sure I understand the exact format by which you expect your output to conform, but here is what my guess is that you are trying to achieve. data have; infile cards dlm='09'x; input id (name var data1-data20) (:$20.); cards; 7647 Dummy Date 11-Jun 11-Mar 10-Dec 10-Sep 10-Jun 10-Mar 9-Dec 9-Sep 9-Jun 9-Mar 8-Dec 8-Sep 8-Jun 8-Mar 7-Dec 7-Sep 7-Jun 7-Mar 6-Dec 6-Sep 7647 Dummy Cash 374,999.00 355,305.00 344,476.00 421,880.00 419,419.00 362,088.00 335,474.00 363,165.00 350,761.00 350,637.00 124,905.00 138,088.00 153,846.00 169,608.00 183,856.00 178,935.00 180,308.00 182,232.00 185,859.00 178,840.00 7647 Dummy Credit 903,945.00 892,582.00 898,555.00 890,329.00 910,922.00 929,207.00 862,928.00 878,434.00 908,463.00 947,960.00 908,375.00 922,330.00 853,334.00 858,979.00 864,756.00 784,002.00 749,575.00 714,901.00 697,474.00 660,277.00 7647 Dummy Supplies 22,567.00 27,059.00 37,032.00 35,536.00 40,547.00 37,694.00 46,079.00 42,035.00 52,795.00 41,942.00 33,281.00 29,246.00 593 494 351 192 108 89 69 76 7647 Dummy Other @NA @NA @NA @NA @NA @NA @NA @NA @NA @NA @NA @NA @NA @NA @NA @NA @NA @NA @NA @NA 7647 Dummy Total @NA @NA @NA @NA @NA @NA @NA @NA @NA @NA @NA @NA @NA @NA @NA @NA @NA @NA @NA @NA 7647 Dummy Equipment @NA @NA @NA @NA @NA @NA @NA @NA @NA @NA @NA @NA @NA @NA @NA @NA @NA @NA @NA @NA ; run; proc transpose data=have out=want; by id name var; id var; var data1-data20; run; Obs id name _NAME_ Date Cash Credit Supplies Other Total Equipment 1 7647 Dummy data1 11-Jun 374,999.00 903,945.00 22,567.00 @NA @NA @NA 2 7647 Dummy data2 11-Mar 355,305.00 892,582.00 27,059.00 @NA @NA @NA 3 7647 Dummy data3 10-Dec 344,476.00 898,555.00 37,032.00 @NA @NA @NA 4 7647 Dummy data4 10-Sep 421,880.00 890,329.00 35,536.00 @NA @NA @NA 5 7647 Dummy data5 10-Jun 419,419.00 910,922.00 40,547.00 @NA @NA @NA 6 7647 Dummy data6 10-Mar 362,088.00 929,207.00 37,694.00 @NA @NA @NA 7 7647 Dummy data7 9-Dec 335,474.00 862,928.00 46,079.00 @NA @NA @NA 8 7647 Dummy data8 9-Sep 363,165.00 878,434.00 42,035.00 @NA @NA @NA 9 7647 Dummy data9 9-Jun 350,761.00 908,463.00 52,795.00 @NA @NA @NA 10 7647 Dummy data10 9-Mar 350,637.00 947,960.00 41,942.00 @NA @NA @NA 11 7647 Dummy data11 8-Dec 124,905.00 908,375.00 33,281.00 @NA @NA @NA 12 7647 Dummy data12 8-Sep 138,088.00 922,330.00 29,246.00 @NA @NA @NA 13 7647 Dummy data13 8-Jun 153,846.00 853,334.00 593 @NA @NA @NA 14 7647 Dummy data14 8-Mar 169,608.00 858,979.00 494 @NA @NA @NA 15 7647 Dummy data15 7-Dec 183,856.00 864,756.00 351 @NA @NA @NA 16 7647 Dummy data16 7-Sep 178,935.00 784,002.00 192 @NA @NA @NA 17 7647 Dummy data17 7-Jun 180,308.00 749,575.00 108 @NA @NA @NA 18 7647 Dummy data18 7-Mar 182,232.00 714,901.00 89 @NA @NA @NA 19 7647 Dummy data19 6-Dec 185,859.00 697,474.00 69 @NA @NA @NA 20 7647 Dummy data20 6-Sep 178,840.00 660,277.00 76 @NA @NA @NA

FriedEgg · ‎10-03-2011

There is no procedure specific option to directly limit the quantity the IMPORT procedure will read in, to my knowledge. You can use the OBS= system option to limit the number of records that SAS will process. Setting OBS=10 will limit the processing to the first 10, then set it back to OBS=MAX so subsequent operations consider the full data. options obs=10; proc import datafile="G:\Departments\Research\MAP\1112\Oconee - &admin. - Clean.txt" out=map_export dbms=dlm replace; delimiter='09'x; run; options obs=max; Since this is a text file, you should also consider using the DATA step with an INFILE statement to read the data. It requires more knowledge of the data fields as you read them, but that approach provides much more control over how much data is read in. See this topic for more guidance and troubleshooting. (Editor's note: modified this reply for clarification and more detail on this popular topic.)

FriedEgg · ‎10-03-2011

put your sql query into a macro and account for the all condition in a dynamic where clause construction. for example %if %upcase(&cust)=%str(ALL) %then %do; where abc; %end; %else %do; where cust like %oraquote(parm=&cust); %end;

FriedEgg · ‎10-03-2011

Hi LinusH, Here is a very quick writeup on the differences between the methods you have mentioned. The MD5 function as part of base SAS: -- MD5 is a one-way function. A given string of any length is passed to the hashing function which produces a unique string of 32 characters in return. The encrypted string cannot be reversed to produce the original string. SAS/Secure offers some additional hashing algorithms such as sha-1. The PWENCODE Procedure: --The pwencode procedure available in SAS has three available encryption methods (2 with base SAS and 1 additional with SAS/Secure). The first is a basic base64 encoding, the second is a 32bit encryption that SAS reports to be a proprietary methodology and the third uses a 256bit variant of AES --The pwencode procedure basically produces an encrypted variable that the macro interpreted will decrypt to plain text. It is useful in situations where for example you wish to store an encrypted password for a connect statement. Basically the task at hand is a method of producing your own version of this functionality with a different methodology. The encrypt dataset option: --This is an option through which you can choose to obsfucate the data enables methods by which to secure the information stored within a dataset by using a variety of methods of encryption that utilize a provided key (password) to decrypt. In base SAS the method utilized is a SAS proprietary algorith and SAS/Secure adds additional industry standards such as RC4, DES, AES and more. This in my experience is a useful technique when sharing datasets between companies over the internet where the information stored in sensitive. It provides an additional layer of security to the data layer itself. There are of course a lot more to each one of these items mentioned. There are also a few other encryption related topics in SAS that have not been mentioned here but also do not necessarily pertain to the topic at hand. --This is a good document which outlines a lot about the methods mentioned here and pwencode procedure. http://support.sas.com/documentation/cdl/en/secref/62092/PDF/default/secref.pdf As a final thought to tie this back to the original topic. By implementing your own cipher what do you gain? Well all of these methods provided by SAS are great and work as expected. The one issue I have with the pwencode option is as follows: Anyone with SAS can decrypt the string, simply, and the outputs are easily identifiable to methodolgy used. For example I will share a encrpyted string use the procedure here: {sas002}F1CB951853BCC0632380806E1603DC310868278F Now you can take that string and even without me telling you how I produced it, pretty easily identify how to decrpyt utilize it to exploit what it represents. If however I were to use a non-standard cipher, the method itself by which to utilize the string becomes part of the security it offers. So with a self implemented cipher unless you share the method of encrpytion and/or the code with which to decrypt the message it is to some degree safer from being missused outside the originating codebase or at least this is my thinking at the moment. Really I post this just for fun. Implementing a cipher is a nice exercise in both computing and mathmatics.

FriedEgg · ‎10-02-2011

The following probably will not be helpful unless you know perl, but here is an implementation of the playfair cipher: $,=42;for(34,0,-3,9,-11,11,-17,7,-5){$*.=pack'c'=>$,+=$_}for(reverse split//=>$* ){$%++?$ %%2?push@C,$_,$":push@c,$_,$":(push@C,$_,$")&&push@c,$"}$C[$#C]=$/;($#C >$#c)?($ c=\@C)&&($ C=\@c):($ c=\@c)&&($C=\@C);$%=$|;for(@$c){print$_^$$C[$%++]}

FriedEgg · ‎10-02-2011

KSharp, this is most definitly a challenge. Espeically if you are not already versed in crytography and ciphers. This may help put you onto a starting path: The following is a implementation of the solitaire cipher in python: 01 from itertools import chain, ifilter, imap, izip 02 03 A = 53 04 B = 54 05 N_CARDS = 54 06 ORD_A = ord ( 'A' ) 07 08 class Deck( list ): 09 def __init__( self , key = ''): 10 list .__init__( self , range ( 1 , 55 )) 11 12 self .step() 13 for ch in key: 14 self .counted_cut( ord (ch) - ORD_A + 1 ) 15 self .step() 16 17 def move( self , card, distance): 18 fm = self .index(card) 19 to = fm + distance 20 if to > = N_CARDS: 21 to - = (N_CARDS - 1 ) 22 23 self .insert(to, self .pop(fm)) 24 25 def triple_cut( self ): 26 a = self .index(A) 27 b = self .index(B) 28 s, e = (a, b + 1 ) if a < b else (b, a + 1 ) 29 self [:] = self [e:] + self [s:e] + self [:s] 30 31 def counted_cut( self , count): 32 self [:] = self [count: - 1 ] + self [:count] + self [ - 1 :] 33 34 def step( self ): 35 self .move(A, 1 ) 36 self .move(B, 2 ) 37 self .triple_cut() 38 self .counted_cut( min ( self [ - 1 ], A)) 39 40 def __iter__( self ): 41 while True : 42 ndx = min ( self [ 0 ], A) 43 44 if self [ndx] < A: 45 yield self [ndx] 46 47 self .step() 48 49 def encrypt(text, key = ''): 50 encode = lambda c,k: chr (( ord (c) - ORD_A + k) % 26 + ORD_A) 51 52 padded_text = chain(text, [ 'X' ] * ( - len (text) % 5 )) 53 54 cyphertext = imap(encode, padded_text, Deck(key)) 55 56 groups = imap(''.join, izip( * [cyphertext] * 5 )) 57 58 return ' ' .join(groups) 59 60 def decrypt(text, key = ''): 61 decode = lambda c,k: chr (( ord (c) - ORD_A - k) % 26 + ORD_A) 62 63 letters = (c for c in text if c.isupper()) 64 65 return ''.join(imap(decode, letters, Deck(key)))

FriedEgg · ‎09-30-2011

I'm not sure I understand what you are asking. I do some minor cleanup and remove an erronious %end; Are you wanting to determine the value of bins_needed or explicitly define it, like I have here? The proper syntax for an array statement is available here: http://support.sas.com/documentation/cdl/en/lrdict/64316/HTML/default/viewer.htm#a000201956.htm %let bins_needed=15; data default; set default; if _N_ eq 1 then do; set Ts; set Quants; end; array T Ts1-Ts&bins_needed; array Q Qs1-Qs&bins_needed; if quick_ratio lt qs[1] then T_QUICK = ts[1]; else if quick_ratio ge Qs&bins_needed then T_QUICK = ts[&bins_needed]; else do i = 1 to (&bins_needed - 1); if quick_ratio ge Q and quick_ratio lt Q[i+1] then T_QUICK = T; end; drop Ts1-Ts&bins_needed Qs1-Qs&bins_needed i; run;

FriedEgg · ‎09-30-2011

data _null_; input edateset yymmdd8.; call symput('edateset',put(edateset,date9.)); do i=1 to 6; call symput('e' || strip(i) || 'monthago',put(intnx('month',edateset,-i),date9.)); end; cards; 110930 ; run; %put &edateset &e1monthago &e2monthago &e3monthago &e4monthago &e5monthago &e6monthago; 30SEP2011 01AUG2011 01JUL2011 01JUN2011 01MAY2011 01APR2011 01MAR2011

Online Status	Offline
Date Last Visited	‎03-31-2025 06:28 PM