About FriedEgg

FriedEgg · ‎10-24-2011

Yes, the elusive sashelp.base.master.dictnary, I cannot figure out how to utilize this catalog either... It seems to maybe be stored in a similar method to stored format, maybe.

FriedEgg · ‎10-21-2011

I will explain so maybe you can work out the issue you have. First I create some data to work with. data foo; input dte $ mgf3654 emtcol1 ntd8475 col3; cards; 2001.01 5 0 23 45 2001.02 89 0 4 2 2006.12 12 0 8 6 ; run; I have a character field (dte) and four numeric variables without a standard naming convention (mgf3654 emtcol1 ntd8475 col3). The first thing I want to do is collect the metadata I want to work with. data _null_; length names nsums $32000; retain names nsums; set sashelp.vcolumn(where=(libname='WORK' and memname='FOO' and type='num')) end=eof; names=catx(' ',of names name); nsum=strip(name) || '_sum'; nsums=catx(' ',of nsums nsum); if eof then do; call symputx('names',names); call symputx('nsums',nsums); call symputx('vars',put(_n_,best.)); end; run; I will collect, dynamically the names of my numeric variables. I will create a second copy of these names with '_sum' added to them for a calculation step next. I will collect the number of variables I'm working with into macro variable 'vars' Now I will run a sum for all columns. If the columns sum is 0 then I will want to drop that variable. data _null_; set foo end=eof; array v &names; array s &nsums; do i=1 to &vars; s+v; end; length drops $32000; retain drops; if eof then do; do i=1 to &vars; if s=0 then do; drop=scan("&names",i,' '); drops=catx(' ',of drops drop); end; end; call symputx('drops',drops); end; run; Create arrays using the variable lists I collected in the previous step The first loop will calculate the sums for each column When I reach the last record in the file I will loop through each variable again. If I have calculated this columns sum to be 0 I will collect the name of the variable into a macro variable 'drops' to use later. Now that I have found all the variables I want to drop I will perform the action. proc sql noprint; alter table foo drop column &drops; quit;

FriedEgg · ‎10-21-2011

I read a very interesting article today. 'How to Write a Spelling Corrector" by Peter Norvig. In this article he outlines the process Google uses to to do it's 'Did you mean:' new spelling suggestions. Here is a link to the article: http://norvig.com/spell-correct.html So proc spell can identify misspelling and even provide suggestions of new words if you use the suggest option but it cannot, as far as I know, replace identified text with a new word. So what I am suggesting is that taking the vast information and algorithms outlined in the paper by Peter Norvig and you create, ideally a version or the correct function using proc fcmp, or another process by which to suggest a proper spelling to a given string or return the original string if it is a proper word.

FriedEgg · ‎10-21-2011

You get the information from the html of the form you are working with: <form name="form1" method="post" action="results.php" onsubmit="return validate(this)"> <p>License Types with Disciplinary Actions: <select name="type" value="">License Type<option value="ALL">ALL</option><option value="Ambulance Service">Ambulance Service</option><option value="Assisted Living Residence Administrator">Assisted Living Residence Administrator</option><option value="Athletic Trainer">Athletic Trainer</option><option value="Audiologist">Audiologist</option><option value="Barber">Barber</option><option value="Barber Apprentice">Barber Apprentice</option><option value="Barber Shop">Barber Shop</option><option value="Chemical Dependency Professional">Chemical Dependency Professional</option><option value="Chiropractor">Chiropractor</option><option value="Clin. Lab Sci.-Generalist">Clin. Lab Sci.-Generalist</option><option value="Controlled Substance Registration">Controlled Substance Registration</option><option value="Dentist">Dentist</option><option value="Dietician/Nutritionist">Dietician/Nutritionist</option><option value="Doctor of Acupuncture">Doctor of Acupuncture</option><option value="Drug Wholesaler In State">Drug Wholesaler In State</option><option value="Drug Wholesaler Out of State">Drug Wholesaler Out of State</option><option value="Emergency Medical Technician">Emergency Medical Technician</option><option value="Esthetician">Esthetician</option><option value="Funeral Director/Embalmer">Funeral Director/Embalmer</option><option value="Funeral Establishment">Funeral Establishment</option><option value="Hair Design Shop">Hair Design Shop</option><option value="Hairdresser/Cosmetologist">Hairdresser/Cosmetologist</option><option value="Hearing Aid Dealer/Fitter">Hearing Aid Dealer/Fitter</option><option value="Hypodermics">Hypodermics</option><option value="Interpreter for the Deaf Grandparent">Interpreter for the Deaf Grandparent</option><option value="Manicuring Shop">Manicuring Shop</option><option value="Manicurist">Manicurist</option><option value="Marriage & Family Therapy/Mental Health Counselors">Marriage & Family Therapy/Mental Health Counselors</option><option value="Massage Therapist">Massage Therapist</option><option value="Medication Aide">Medication Aide</option><option value="Midwife">Midwife</option><option value="Nurse">Nurse</option><option value="Nursing Assistant">Nursing Assistant</option><option value="Nursing Home Administrator">Nursing Home Administrator</option><option value="Occupational Therapist">Occupational Therapist</option><option value="Occupational Therapist Asst.">Occupational Therapist Asst.</option><option value="Optician">Optician</option><option value="OptometristTX RX Amplified Glaucoma">OptometristTX RX Amplified Glaucoma</option><option value="Pharmacist">Pharmacist</option><option value="Pharmacist Limited (Intern)">Pharmacist Limited (Intern)</option><option value="Pharmacist Tech Trainee-In Training">Pharmacist Tech Trainee-In Training</option><option value="Pharmacist Technician">Pharmacist Technician</option><option value="Pharmacy Tech 1">Pharmacy Tech 1</option><option value="Pharmacy Tech 2">Pharmacy Tech 2</option><option value="Pharmacy-Non-Resident">Pharmacy-Non-Resident</option><option value="Pharmacy-Retail">Pharmacy-Retail</option><option value="Physical Therapist">Physical Therapist</option><option value="Physician">Physician</option><option value="Physician Assistant">Physician Assistant</option><option value="Podiatrist">Podiatrist</option><option value="Psychologist">Psychologist</option><option value="Radiographer">Radiographer</option><option value="Respiratory Care Practitioner">Respiratory Care Practitioner</option><option value="Social Worker">Social Worker</option><option value="Speech Lang. Pathologist">Speech Lang. Pathologist</option><option value="Transliterator for the Deaf Grandparent">Transliterator for the Deaf Grandparent</option><option value="Veterinarian">Veterinarian</option></select> </p> <h2>Find Only</h2> <p>Last Name <br> <input type="text" name="name" id="name"><br> If you are not sure of spelling, enter only as many letters as you know. Please include any punctuation marks in the licensee's name (e.g. hyphens, apostrophes).</p> <p>Sanctions Since (YYYYMMDD)<br> <input type="text" name="sanction_date_from" id="sanction_date_from"> </p> <div class="cssbuttons"> <p> <input class="positive" type="submit" name="submit" id="submit" value="Get Actions >>"> </p> </div> </form> From the form tag you get that we are working with the url: http://www.health.ri.gov/lists/disciplinaryactions/results.php And the we are using the post method The select tag shows us that this field is called type and the value of the field we want is called ALL The first input tag shows us that this text field is called name, we do not want to use this field so it's value is null The second input tag shows us that this text field is called sanction_date_from, we do not want to use this field so it's value is also null The last input tag shows us that this field is called submit and has a value of Get Actions >> (or >> or %3E%3E since to properly submit this it will need to be properly encoded for url) So this is how I know out post should send: type=ALL name= sanction_date_from= submit=Get Actions >> and to properly write for use type=ALL&name=&sanction_date_from=&submit=Get+Actions+%3E%3E The ct option stands for content-type, the value 'application/x-www-form-urlencoded' is the content type we are dealing with for this post. The in parameter is what tells proc http what to send in the post. Apparently this needs to be a fileref and cannot be a text string...?

FriedEgg · ‎10-21-2011

data foo; input dte $ mgf3654 emtcol1 ntd8475 col3; cards; 2001.01 5 0 23 45 2001.02 89 0 4 2 2006.12 12 0 8 6 ; run; data _null_; length names nsums $32000; retain names nsums; set sashelp.vcolumn(where=(libname='WORK' and memname='FOO' and type='num')) end=eof; names=catx(' ',of names name); nsum=strip(name) || '_sum'; nsums=catx(' ',of nsums nsum); if eof then do; call symputx('names',names); call symputx('nsums',nsums); call symputx('vars',put(_n_,best.)); end; run; %put &vars &names &nsums; data _null_; set foo end=eof; array v &names; array s &nsums; do i=1 to &vars; s+v; end; length drops $32000; retain drops; if eof then do; do i=1 to &vars; if s=0 then do; drop=scan("&names",i,' '); drops=catx(' ',of drops drop); end; end; call symputx('drops',drops); end; run; proc sql noprint; alter table foo drop column &drops; quit;

FriedEgg · ‎10-21-2011

Thanks Art, I do not have ability to test at the moment...

FriedEgg · ‎10-21-2011

Hi, The first thing to notice is the proper method to use. You will notice the line in the html code for the form: <form name="form1" method="post" action="results.php" onsubmit="return validate(this)"> You can see that it is using the post method to results.php, the onsubmit action in this case is irrelevant, it is a javascript call to validate that you entered a name or a date and not both... This next thing to understand is what it is that you are sending in your post. In the form there are three fields: type, name, and sanction_date_from There is also a value given to the submit action which will act as a fourth variable in our post. type=ALL name=[blank] sanction_date_from=[blank] submit=Get Actions >> The next step is to properly form these variables into a post command so that the form will understand them. type=ALL&name=&sanction_date_from=&submit=Get+Actions+%3E%3E Now, I will usually test using wget... wget --post-data="type=ALL&name=&sanction_date_from=&submit=Get+Actions+%3E%3E" http://www.health.ri.gov/lists/disciplinaryactions/results.php I check the results, and I am getting what I expect! So just convert this now to work with proc http: proc http in="type=ALL&name=&sanction_date_from=&submit=Get+Actions+%3E%3E" url="http://www.health.ri.gov/lists/disciplinaryactions/results.php" method="post" ct="application/x-www-form-urlencoded"; run;

FriedEgg · ‎10-20-2011

I agree with the modifications here, they are all beneficial to performance. There is one issue with the exact implementation here that I see however. The cards statement is unbuffered and cannot use the end option on the infile statement. You could instead use the parmcards statement. If this is not the case in a different OS or version of SAS I would be interested in knowing.

FriedEgg · ‎10-19-2011

To attract the proper audience try posting you question in the forum for JMP software.

FriedEgg · ‎10-18-2011

I agree with Art, to get the result you say you now retrieve you would have to be using code other than that which we specified. data foo; infile cards dlm='|'; input (name data) (:$20.); do i=1 to count(data,',')+1; column2=scan(data,i); *change made here to remove specified delimiter, it will still produce the same result but now properly scans instead of truncating to length; output; end; keep name column2; cards; Wells|D2E,D3R,D5H,H6J Dons|H5N,J9T,F4S Gens|L2G 7H9 Leads|K9P 5F9 ; run; Wells D2E Wells D3R Wells D5H Wells H6J Dons H5N Dons J9T Dons F4S Gens L2G Leads K9P

FriedEgg · ‎10-18-2011

data foo; infile cards dlm='|'; input (name data) (:$20.); do i=1 to count(data,',')+1; column2=scan(data,i,','); output; end; keep name column2; cards; Wells|D2E,D3R,D5H,H6J Dons|H5N,J9T,F4S ; run;

FriedEgg · ‎10-17-2011

Not quite sure from your question what type of maximum value you are looking for? Here are two possible answers to two possible questions... find the by row maximum value: the max function. ie. ymax=max(of y1-y8); find the by column maximum value: one option is proc summary. ie. proc means data=out_2 max; var y1-y8; run;

FriedEgg · ‎10-17-2011

Indexing does no always provide an advantage. In some of the larger table I deal with creating indexs actually hurt performance. There isn't necessarily a rule of thumb to knowing, more of trial and error. Also the vast majority of databases that I can think of do have hash joins, merge joins, and cartesian products joins available to them and as I mentioned earlier the default join mechanic in proc sql, for an inner join, is the hash method.

FriedEgg · ‎10-14-2011

Here is another thought specifically pertaining to hash joins and proc sql, in case I have previously missed what you are trying to ask. proc sql with use a hash join method for inner joins between tables. you can see this by adding the _method option to your proc sql statement. However it will not use this method for other types of joins. You can manually produce a hash join method using the hash object available to data steps. I do not know how data being stored in an outside rdbms would effect this. try running your query with the _method option and seeing what the explain plain says it is doing. proc sql _method; * your query here; quit; if you see 'sqxjhsh' then it is using a hash join. if you see 'sqxjm' then it is using a join-merge.

FriedEgg · ‎10-14-2011

Could you provide a more specific example to what you are asking. Depending on the complexity of the join and other conditions in a query, just because you are dealing with large tables doesn't mean a join will take a substantial amount of time, especially with the proper hardware to support such data. Earlier today for example, I ran a query that joins together 12 sas tables of about 600million records each and it completed in about 5 hours. For my scheduling, this is more than fast enough, and has a lot of room for improvement if needed. It is all relative to what you are trying to accomplish...

Online Status	Offline
Date Last Visited	‎03-31-2025 06:28 PM