I am trying to use PROC HTTP to pull the following website: http://www.health.ri.gov/lists/disciplinaryactions/ If you click 'Get actions' then a whole data will populate. It's that information I want to download. However, when I run the following PROC HTTP code, I get nothing.
filename file 'c:\file.txt';
PROC HTTP
method='post'
url='http://www.health.ri.gov/lists/disciplinaryactions/results.php'
out=file;
RUN;
Can someone please help. I find the support documentation to be a litte 'advanced.' So please bare with me as I'm new to proc http.
I couldn't get FriedEgg's code to run, but figured out how to use the code he proposed to created the desired file. I ran:
filename file 'c:\file.txt';
filename infile 'c:\infile.txt';
data _null_;
input;
file infile;
put _infile_;
datalines4;
type=ALL&name=&sanction_date_from=&submit=Get+Actions+%3E%3E
;;;;
proc http
in=infile
url="http://www.health.ri.gov/lists/disciplinaryactions/results.php"
method="post"
ct="application/x-www-form-urlencoded"
out=file;
run;
I'm not sure how to send the get action command, but I did notice that the site has a number of preformed result pages. You could get one of them with:
PROC HTTP
method="get"
url="http://www.health.ri.gov/lists/disciplinaryactions/resultsbydate.php"
out=file;
run;
To send the request you actually wanted, someone has to let us (I would like to know as well) how to send the command: input type="submit" name="submit" id="submit" value="Get Actions >>"
Somehow I don't think I have the command quite right as i tried to include it in a file with the in= option, but without success.
Thanks for the advice and the hlep.
I looked at that link earlier, but it did not have the whole history of data.
Hi,
The first thing to notice is the proper method to use. You will notice the line in the html code for the form:
<form name="form1" method="post" action="results.php" onsubmit="return validate(this)">
You can see that it is using the post method to results.php, the onsubmit action in this case is irrelevant, it is a javascript call to validate that you entered a name or a date and not both...
This next thing to understand is what it is that you are sending in your post.
In the form there are three fields: type, name, and sanction_date_from
There is also a value given to the submit action which will act as a fourth variable in our post.
type=ALL
name=[blank]
sanction_date_from=[blank]
submit=Get Actions >>
The next step is to properly form these variables into a post command so that the form will understand them.
type=ALL&name=&sanction_date_from=&submit=Get+Actions+%3E%3E
Now, I will usually test using wget...
wget --post-data="type=ALL&name=&sanction_date_from=&submit=Get+Actions+%3E%3E" http://www.health.ri.gov/lists/disciplinaryactions/results.php
I check the results, and I am getting what I expect!
So just convert this now to work with proc http:
proc http
in="type=ALL&name=&sanction_date_from=&submit=Get+Actions+%3E%3E"
url="http://www.health.ri.gov/lists/disciplinaryactions/results.php"
method="post"
ct="application/x-www-form-urlencoded";
run;
I couldn't get FriedEgg's code to run, but figured out how to use the code he proposed to created the desired file. I ran:
filename file 'c:\file.txt';
filename infile 'c:\infile.txt';
data _null_;
input;
file infile;
put _infile_;
datalines4;
type=ALL&name=&sanction_date_from=&submit=Get+Actions+%3E%3E
;;;;
proc http
in=infile
url="http://www.health.ri.gov/lists/disciplinaryactions/results.php"
method="post"
ct="application/x-www-form-urlencoded"
out=file;
run;
Thanks Art,
I do not have ability to test at the moment...
Worked like a champ. I think one of the challenges is having domain knowledge in both SAS and web development.
Can you explain what the in parameter does as well as the ct parameter? Also, how where you able to find what needed to be specified for type? Can I put ****= anything? Or does PROC HTTP know that type is a keyword in the file?
Thanks.
You get the information from the html of the form you are working with:
<form name="form1" method="post" action="results.php" onsubmit="return validate(this)">
<p>License Types with Disciplinary Actions:
<select name="type" value="">License Type<option value="ALL">ALL</option><option value="Ambulance Service">Ambulance Service</option><option value="Assisted Living Residence Administrator">Assisted Living Residence Administrator</option><option value="Athletic Trainer">Athletic Trainer</option><option value="Audiologist">Audiologist</option><option value="Barber">Barber</option><option value="Barber Apprentice">Barber Apprentice</option><option value="Barber Shop">Barber Shop</option><option value="Chemical Dependency Professional">Chemical Dependency Professional</option><option value="Chiropractor">Chiropractor</option><option value="Clin. Lab Sci.-Generalist">Clin. Lab Sci.-Generalist</option><option value="Controlled Substance Registration">Controlled Substance Registration</option><option value="Dentist">Dentist</option><option value="Dietician/Nutritionist">Dietician/Nutritionist</option><option value="Doctor of Acupuncture">Doctor of Acupuncture</option><option value="Drug Wholesaler In State">Drug Wholesaler In State</option><option value="Drug Wholesaler Out of State">Drug Wholesaler Out of State</option><option value="Emergency Medical Technician">Emergency Medical Technician</option><option value="Esthetician">Esthetician</option><option value="Funeral Director/Embalmer">Funeral Director/Embalmer</option><option value="Funeral Establishment">Funeral Establishment</option><option value="Hair Design Shop">Hair Design Shop</option><option value="Hairdresser/Cosmetologist">Hairdresser/Cosmetologist</option><option value="Hearing Aid Dealer/Fitter">Hearing Aid Dealer/Fitter</option><option value="Hypodermics">Hypodermics</option><option value="Interpreter for the Deaf Grandparent">Interpreter for the Deaf Grandparent</option><option value="Manicuring Shop">Manicuring Shop</option><option value="Manicurist">Manicurist</option><option value="Marriage & Family Therapy/Mental Health Counselors">Marriage & Family Therapy/Mental Health Counselors</option><option value="Massage Therapist">Massage Therapist</option><option value="Medication Aide">Medication Aide</option><option value="Midwife">Midwife</option><option value="Nurse">Nurse</option><option value="Nursing Assistant">Nursing Assistant</option><option value="Nursing Home Administrator">Nursing Home Administrator</option><option value="Occupational Therapist">Occupational Therapist</option><option value="Occupational Therapist Asst.">Occupational Therapist Asst.</option><option value="Optician">Optician</option><option value="OptometristTX RX Amplified Glaucoma">OptometristTX RX Amplified Glaucoma</option><option value="Pharmacist">Pharmacist</option><option value="Pharmacist Limited (Intern)">Pharmacist Limited (Intern)</option><option value="Pharmacist Tech Trainee-In Training">Pharmacist Tech Trainee-In Training</option><option value="Pharmacist Technician">Pharmacist Technician</option><option value="Pharmacy Tech 1">Pharmacy Tech 1</option><option value="Pharmacy Tech 2">Pharmacy Tech 2</option><option value="Pharmacy-Non-Resident">Pharmacy-Non-Resident</option><option value="Pharmacy-Retail">Pharmacy-Retail</option><option value="Physical Therapist">Physical Therapist</option><option value="Physician">Physician</option><option value="Physician Assistant">Physician Assistant</option><option value="Podiatrist">Podiatrist</option><option value="Psychologist">Psychologist</option><option value="Radiographer">Radiographer</option><option value="Respiratory Care Practitioner">Respiratory Care Practitioner</option><option value="Social Worker">Social Worker</option><option value="Speech Lang. Pathologist">Speech Lang. Pathologist</option><option value="Transliterator for the Deaf Grandparent">Transliterator for the Deaf Grandparent</option><option value="Veterinarian">Veterinarian</option></select> </p>
<h2>Find Only</h2>
<p>Last Name <br>
<input type="text" name="name" id="name"><br>
If you are not sure of spelling, enter only as many letters as you know. Please include any punctuation marks in the licensee's name (e.g. hyphens, apostrophes).</p>
<p>Sanctions Since (YYYYMMDD)<br>
<input type="text" name="sanction_date_from" id="sanction_date_from">
</p>
<div class="cssbuttons">
<p>
<input class="positive" type="submit" name="submit" id="submit" value="Get Actions >>">
</p>
</div>
</form>
From the form tag you get that we are working with the url: http://www.health.ri.gov/lists/disciplinaryactions/results.php
And the we are using the post method
The select tag shows us that this field is called type and the value of the field we want is called ALL
The first input tag shows us that this text field is called name, we do not want to use this field so it's value is null
The second input tag shows us that this text field is called sanction_date_from, we do not want to use this field so it's value is also null
The last input tag shows us that this field is called submit and has a value of Get Actions >> (or >> or %3E%3E since to properly submit this it will need to be properly encoded for url)
So this is how I know out post should send:
type=ALL
name=
sanction_date_from=
submit=Get Actions >>
and to properly write for use
type=ALL&name=&sanction_date_from=&submit=Get+Actions+%3E%3E
The ct option stands for content-type, the value 'application/x-www-form-urlencoded' is the content type we are dealing with for this post.
The in parameter is what tells proc http what to send in the post. Apparently this needs to be a fileref and cannot be a text string...?
I haven't had time yet to investigate it, but I have to presume that for it to be a text string one would have to mask at least the ampersands and probably the percents.
Back to the OP's original comment, the documentation doesn't provide any clues regarding the various options. If it did, one might be able to deduce some of these needs on their own. FriedEgg, your post was extremely helpful and I'd like to carry on the conversation offline. However, I don't know your email address.
Please send me a real email. You can reach me at art297 at rogers dot com.
Hi @mlogan - This is an old thread. If you have a new specific question, please post a new topic.
For PROC HTTP and REST, I have several examples on my blog:
http://blogs.sas.com/content/tag/rest-api/
http://blogs.sas.com/content/tag/proc-http/
I'd like to get one more out there. I feel I'm close - can anyone help. The link is: http://www.dgsweb.state.pa.us/Debarment_list/default.aspx
My SAS Code is here:
filename file1 'C:\out.txt';
filename file2 'C:\in.txt';
DATA _NULL_;
input;
file file2;
put _infile_;
datalines4;
__EVENTTARGET=&__EVENTARGUMENT=&__LASTFOCUS=&__VIEWSTATE=%2FwEPDwUKMTQ4NjEyMDk4NQ9kFgICAw9kFgYCAQ9kFgoCAQ8QZBAVAwpDaG9vc2UgT25lBkFjdGl2ZQZMb2cgSW4VAwpDaG9vc2UgT25lBkFjdGl2ZQZMb2cgSW4UKwMDZ2dnFgECAWQCAw8QZGQWAWZkAgsPEA8WAh4HVmlzaWJsZWhkZBYAZAINDw8WBB4EVGV4dAUiQ2hvb3NlIG9uZSB0byBzcGVjaWZ5IHNlYXJjaCBhcmVhLh8AaGRkAhUPEGRkFgECAWQCAw9kFgYCCQ8QZA8WRGYCAQICAgMCBAIFAgYCBwIIAgkCCgILAgwCDQIOAg8CEAIRAhICEwIUAhUCFgIXAhgCGQIaAhsCHAIdAh4CHwIgAiECIgIjAiQCJQImAicCKAIpAioCKwIsAi0CLgIvAjACMQIyAjMCNAI1AjYCNwI4AjkCOgI7AjwCPQI%2BAj8CQAJBAkICQxZEEGVlZxAFBUFnaW5nBQIxMGcQBQtBZ3JpY3VsdHVyZQUBNGcQBQdCYW5raW5nBQE1ZxAFG0NlbnRlciBmb3IgSnV2ZW5pbGUgSnVzdGljZQUEODEwN2cQBRhDaXZpbCBTZXJ2aWNlIENvbW1pc3Npb24FAjMyZxAFC0NvcnJlY3Rpb25zBQIxMWcQBQREQ0VEBQIyNGcQBQREQ05SBQIzOGcQBSZEZXBhcnRtZW50IG9mIEVudmlyb25tZW50YWwgUHJvdGVjdGlvbgUCMzVnEAUeRGVwYXJ0bWVudCBvZiBMYWJvciAmIEluZHVzdHJ5BQIxMmcQBQlFZHVjYXRpb24FAjE2ZxAFG0Vudmlyb25tZW50YWwgSGVhcmluZyBCb2FyZAUCMzdnEAUYRmlzaCBhbmQgQm9hdCBDb21taXNzaW9uBQIyMmcQBQ9HYW1lIENvbW1pc3Npb24FAjIzZxAFEEdlbmVyYWwgU2VydmljZXMFAjE1ZxAFBkhlYWx0aAUBN2cQBQlJbnN1cmFuY2UFATlnEAUUTGlxdW9yIENvbnRyb2wgQm9hcmQFAjI2ZxAFHU1pbGl0YXJ5IGFuZCBWZXRlcmFucyBBZmZhaXJzBQIxM2cQBRRNaWxrIE1hcmtldGluZyBCb2FyZAUCMjdnEAUYT2ZmaWNlIG9mIEFkbWluaXN0cmF0aW9uBQQ4MTAxZxAFGU9mZmljZSBvZiBHZW5lcmFsIENvdW5zZWwFBDgxMDJnEAUbT2ZmaWNlIG9mIEluc3BlY3RvciBHZW5lcmFsBQQ4MTAzZxAFFE9mZmljZSBvZiB0aGUgQnVkZ2V0BQQ4MTA0ZxAFFk9mZmljZSBvZiB0aGUgR292ZXJub3IFATFnEAUdUEEgSHVtYW4gUmVsYXRpb25zIENvbW1pc3Npb24FBDgxMDVnEAUUUEEgUHVibGljIFRWIE5ldHdvcmsFAjM0ZxAFBFBDQ0QFBDgxMDZnEAUEUEVNQQUCMzFnEAUtUGVubnN5bHZhbmlhIEhpc3RvcmljYWwgYW5kIE11c2V1bSBDb21taXNzaW9uBQIzMGcQBShQZW5uc3lsdmFuaWEgTXVuaWNpcGFsIFJldGlyZW1lbnQgU3lzdGVtBQI3MWcQBRlQZW5uc3lsdmFuaWEgU3RhdGUgUG9saWNlBQIyMGcQBQhQRU5OVkVTVAUCMzNnEAUUUHJvYmF0aW9uIGFuZCBQYXJvbGUFAjI1ZxAFKVB1YmxpYyBTY2hvb2wgRW1wbG95ZWVzIFJldGlyZW1lbnQgU3lzdGVtBQI3MmcQBQ5QdWJsaWMgV2VsZmFyZQUCMjFnEAUDUFVDBQIxN2cQBQdSZXZlbnVlBQIxOGcQBRVTZWN1cml0aWVzIENvbW1pc3Npb24FATZnEAUFU3RhdGUFAjE5ZxAFIVN0YXRlIEVtcGxveWVlcyBSZXRpcmVtZW50IFN5c3RlbQUCNzBnEAUXU3RhdGUgRXRoaWNzIENvbW1pc3Npb24FAjQwZxAFHFN0YXRlIFRheCBFcXVhbGl6YXRpb24gQm9hcmQFAjM2ZxAFDlRyYW5zcG9ydGF0aW9uBQE4ZxAFBlNlbmF0ZQUCNDFnEAUYSG91c2Ugb2YgUmVwcmVzZW50YXRpdmVzBQI0MmcQBQ9BdWRpdG9yIEdlbmVyYWwFATJnEAUIVHJlYXN1cnkFATNnEAUQQXR0b3JuZXkgR2VuZXJhbAUCMTRnEAUVTHQuIEdvdmVybm9yJ3MgT2ZmaWNlBQIyOGcQBQVQSEVBQQUCMzlnEAUmTGVnaXNsYXRpdmUgQWdlbmN5IG9mIEdlbmVyYWwgQXNzZW1ibHkFAjQ1ZxAFGkxlZ2lzbGF0aXZlIFNlcnZpY2UgQWdlbmN5BQI0NmcQBSZMZWdpc2xhdGl2ZSBCdWRnZXQgJiBGaW5hbmNlIENvbW1pdHRlZQUCNDdnEAUWSnVkaWNpYWwgQ29uZHVjdCBCb2FyZAUCNTFnEAUSQ29tbW9ud2VhbHRoIENvdXJ0BQI1OGcQBQ1TdXByZW1lIENvdXJ0BQI2MGcQBSdJbmRlcGVuZGVudCBSZWd1bGF0b3J5IFJldmlldyBDb21taXR0ZWUFAjYzZxAFF1BBIEdhbWluZyBDb250cm9sIEJvYXJkBQI2NWcQBSNQaGlsZGVscGhpYSBSZWdpb25hbCBQb3J0IEF1dGhvcml0eQUCODhnEAUgU3RhdGUgU3lzdGVtIG9mIEhpZ2hlciBFZHVjYXRpb24FAjkwZxAFE1R1cm5waWtlIENvbW1pc3Npb24FAjkxZxAFIFB1YmxpYyBTY2hvb2wgQnVpbGRpbmcgQXV0aG9yaXR5BQI5OGcQBS9Hb3Zlcm5vcidzIE9mZmljZSBmb3IgTWFuYWdlbWVudCAmIFByb2R1Y3Rpdml0eQUEODEwOGcQBRNDb3VuY2lsIG9uIHRoZSBBcnRzBQQ4MTA5ZxAFHkNhcGl0b2wgUHJlc2VydmF0aW9uIENvbW1pdHRlZQUCOTlnEAUJUEEgQ291cnRzBQI1M2cWAWZkAhUPEGRkFgFmZAIdDxBkZBYBZmQCBQ9kFgQCAQ8QZGQWAWZkAgUPPCsADQBkGAIFEWdyZERlYmFybWVudExpc3QyD2dkBRBncmREZWJhcm1lbnRMaXN0DzwrAAoBCAIBZDtIKQzuOUvkzIcQ22nPMsJDDYX4&__EVENTVALIDATION=%2FwEWEQLH2ujdCgLRoIiFCALf68jbBQL4oL9tApnvpbIHAvmXzo0MAun45GMC9vjkYwKT9%2FrLBwL%2B6YyoCgLCi9reAwKtkuWiCgKXjq%2BFCAKIjq%2BFCAKY4YXrBALC34jvAwL6o4PRA5xSrzJgIdRAZfigZJrvkhhVoNXL&ddlSelect=Active&ddlSearchBy=0&txtContractor=&btnSubmit=Search&rblPaging=0&hidEdit=False&hidLogin=
;;;;
RUN;
PROC HTTP out=file1
in=file2
url='http://www.dgsweb.state.pa.us/Debarment_list/default.aspx'
method='POST'
ct='application/x-www-form-urlencoded';
RUN;ok
Registration is now open for SAS Innovate 2025 , our biggest and most exciting global event of the year! Join us in Orlando, FL, May 6-9.
Sign up by Dec. 31 to get the 2024 rate of just $495.
Register now!
Learn the difference between classical and Bayesian statistical approaches and see a few PROC examples to perform Bayesian analysis in this video.
Find more tutorials on the SAS Users YouTube channel.
Ready to level-up your skills? Choose your own adventure.