BookmarkSubscribeRSS Feed
🔒 This topic is solved and locked. Need further help from the community? Please sign in and ask a new question.
Jay1
Calcite | Level 5

I am trying to use PROC HTTP to pull the following website: http://www.health.ri.gov/lists/disciplinaryactions/ If you click 'Get actions' then a whole data will populate.  It's that information I want to download.  However, when I run the following PROC HTTP code, I get nothing.

filename file 'c:\file.txt';

PROC HTTP

method='post'

url='http://www.health.ri.gov/lists/disciplinaryactions/results.php'

out=file;

RUN;

Can someone please help.  I find the support documentation to be a litte 'advanced.'  So please bare with me as I'm new to proc http.

1 ACCEPTED SOLUTION

Accepted Solutions
art297
Opal | Level 21

I couldn't get FriedEgg's code to run, but figured out how to use the code he proposed to created the desired file.  I ran:

filename file 'c:\file.txt';

filename infile 'c:\infile.txt';

 

data _null_;

  input;

  file infile;

  put _infile_;

  datalines4;

type=ALL&name=&sanction_date_from=&submit=Get+Actions+%3E%3E

;;;;

proc http

  in=infile

  url="http://www.health.ri.gov/lists/disciplinaryactions/results.php"

   method="post"

   ct="application/x-www-form-urlencoded"

  out=file;

run;

View solution in original post

11 REPLIES 11
art297
Opal | Level 21

I'm not sure how to send the get action command, but I did notice that the site has a number of preformed result pages.  You could get one of them with:

PROC HTTP

  method="get"

  url="http://www.health.ri.gov/lists/disciplinaryactions/resultsbydate.php"

  out=file;

run;

To send the request you actually wanted, someone has to let us (I would like to know as well) how to send the command: input type="submit" name="submit" id="submit" value="Get Actions >>"

Somehow I don't think I have the command quite right as i tried to include it in a file with the in= option, but without success.

Jay1
Calcite | Level 5

Thanks for the advice and the hlep.

I looked at that link earlier, but it did not have the whole history of data.

FriedEgg
SAS Employee

Hi,

The first thing to notice is the proper method to use.  You will notice the line in the html code for the form:

<form name="form1" method="post" action="results.php" onsubmit="return validate(this)">

You can see that it is using the post method to results.php, the onsubmit action in this case is irrelevant, it is a javascript call to validate that you entered a name or a date and not both...

This next thing to understand is what it is that you are sending in your post.

In the form there are three fields: type, name, and sanction_date_from

There is also a value given to the submit action which will act as a fourth variable in our post.

type=ALL

name=[blank]

sanction_date_from=[blank]

submit=Get Actions >>

The next step is to properly form these variables into a post command so that the form will understand them.

type=ALL&name=&sanction_date_from=&submit=Get+Actions+%3E%3E

Now, I will usually test using wget...

wget --post-data="type=ALL&name=&sanction_date_from=&submit=Get+Actions+%3E%3E" http://www.health.ri.gov/lists/disciplinaryactions/results.php

I check the results, and I am getting what I expect!

So just convert this now to work with proc http:

proc http

  in="type=ALL&name=&sanction_date_from=&submit=Get+Actions+%3E%3E"

  url="http://www.health.ri.gov/lists/disciplinaryactions/results.php"

   method="post"

   ct="application/x-www-form-urlencoded";

run;

art297
Opal | Level 21

I couldn't get FriedEgg's code to run, but figured out how to use the code he proposed to created the desired file.  I ran:

filename file 'c:\file.txt';

filename infile 'c:\infile.txt';

 

data _null_;

  input;

  file infile;

  put _infile_;

  datalines4;

type=ALL&name=&sanction_date_from=&submit=Get+Actions+%3E%3E

;;;;

proc http

  in=infile

  url="http://www.health.ri.gov/lists/disciplinaryactions/results.php"

   method="post"

   ct="application/x-www-form-urlencoded"

  out=file;

run;

FriedEgg
SAS Employee

Thanks Art,

I do not have ability to test at the moment...

Jay1
Calcite | Level 5

Worked like a champ.  I think one of the challenges is having domain knowledge in both SAS and web development.

Can you explain what the in parameter does as well as the ct parameter?  Also, how where you able to find what needed to be specified for type?  Can I put ****= anything? Or does PROC HTTP know that type is a keyword in the file?

Thanks.

FriedEgg
SAS Employee

You get the information from the html of the form you are working with:

<form name="form1" method="post" action="results.php" onsubmit="return validate(this)">

          <p>License Types with Disciplinary Actions:

            <select name="type" value="">License Type<option value="ALL">ALL</option><option value="Ambulance Service">Ambulance Service</option><option value="Assisted Living Residence Administrator">Assisted Living Residence Administrator</option><option value="Athletic Trainer">Athletic Trainer</option><option value="Audiologist">Audiologist</option><option value="Barber">Barber</option><option value="Barber Apprentice">Barber Apprentice</option><option value="Barber Shop">Barber Shop</option><option value="Chemical Dependency Professional">Chemical Dependency Professional</option><option value="Chiropractor">Chiropractor</option><option value="Clin. Lab Sci.-Generalist">Clin. Lab Sci.-Generalist</option><option value="Controlled Substance Registration">Controlled Substance Registration</option><option value="Dentist">Dentist</option><option value="Dietician/Nutritionist">Dietician/Nutritionist</option><option value="Doctor of Acupuncture">Doctor of Acupuncture</option><option value="Drug Wholesaler In State">Drug Wholesaler In State</option><option value="Drug Wholesaler Out of State">Drug Wholesaler Out of State</option><option value="Emergency Medical Technician">Emergency Medical Technician</option><option value="Esthetician">Esthetician</option><option value="Funeral Director/Embalmer">Funeral Director/Embalmer</option><option value="Funeral Establishment">Funeral Establishment</option><option value="Hair Design Shop">Hair Design Shop</option><option value="Hairdresser/Cosmetologist">Hairdresser/Cosmetologist</option><option value="Hearing Aid Dealer/Fitter">Hearing Aid Dealer/Fitter</option><option value="Hypodermics">Hypodermics</option><option value="Interpreter for the Deaf Grandparent">Interpreter for the Deaf Grandparent</option><option value="Manicuring Shop">Manicuring Shop</option><option value="Manicurist">Manicurist</option><option value="Marriage &amp; Family Therapy/Mental Health Counselors">Marriage &amp; Family Therapy/Mental Health Counselors</option><option value="Massage Therapist">Massage Therapist</option><option value="Medication Aide">Medication Aide</option><option value="Midwife">Midwife</option><option value="Nurse">Nurse</option><option value="Nursing Assistant">Nursing Assistant</option><option value="Nursing Home Administrator">Nursing Home Administrator</option><option value="Occupational Therapist">Occupational Therapist</option><option value="Occupational Therapist Asst.">Occupational Therapist Asst.</option><option value="Optician">Optician</option><option value="OptometristTX RX Amplified Glaucoma">OptometristTX RX Amplified Glaucoma</option><option value="Pharmacist">Pharmacist</option><option value="Pharmacist Limited (Intern)">Pharmacist Limited (Intern)</option><option value="Pharmacist Tech Trainee-In Training">Pharmacist Tech Trainee-In Training</option><option value="Pharmacist Technician">Pharmacist Technician</option><option value="Pharmacy Tech 1">Pharmacy Tech 1</option><option value="Pharmacy Tech 2">Pharmacy Tech 2</option><option value="Pharmacy-Non-Resident">Pharmacy-Non-Resident</option><option value="Pharmacy-Retail">Pharmacy-Retail</option><option value="Physical Therapist">Physical Therapist</option><option value="Physician">Physician</option><option value="Physician Assistant">Physician Assistant</option><option value="Podiatrist">Podiatrist</option><option value="Psychologist">Psychologist</option><option value="Radiographer">Radiographer</option><option value="Respiratory Care Practitioner">Respiratory Care Practitioner</option><option value="Social Worker">Social Worker</option><option value="Speech Lang. Pathologist">Speech Lang. Pathologist</option><option value="Transliterator for the Deaf Grandparent">Transliterator for the Deaf Grandparent</option><option value="Veterinarian">Veterinarian</option></select>          </p>

          <h2>Find Only</h2>

            <p>Last Name  <br>

            <input type="text" name="name" id="name"><br>

            If you are not sure of spelling, enter only as many letters as you know.   Please include any punctuation marks in the licensee's name (e.g. hyphens,   apostrophes).</p>

            <p>Sanctions Since (YYYYMMDD)<br>

              <input type="text" name="sanction_date_from" id="sanction_date_from">

            </p>

          <div class="cssbuttons">

            <p>

              <input class="positive" type="submit" name="submit" id="submit" value="Get Actions &gt;&gt;">

            </p>

          </div>

        </form>

From the form tag you get that we are working with the url: http://www.health.ri.gov/lists/disciplinaryactions/results.php

And the we are using the post method

The select tag shows us that this field is called type and the value of the field we want is called ALL

The first input tag shows us that this text field is called name, we do not want to use this field so it's value is null

The second input tag shows us that this text field is called sanction_date_from, we do not want to use this field so it's value is also null

The last input tag shows us that this field is called submit and has a value of Get Actions >> (or &gt;&gt; or %3E%3E since to properly submit this it will need to be properly encoded for url)

So this is how I know out post should send:

type=ALL

name=

sanction_date_from=

submit=Get Actions >>

and to properly write for use

type=ALL&name=&sanction_date_from=&submit=Get+Actions+%3E%3E

The ct option stands for content-type, the value 'application/x-www-form-urlencoded' is the content type we are dealing with for this post.

The in parameter is what tells proc http what to send in the post.  Apparently this needs to be a fileref and cannot be a text string...?

art297
Opal | Level 21

I haven't had time yet to investigate it, but I have to presume that for it to be a text string one would have to mask at least the ampersands and probably the percents.

Back to the OP's original comment, the documentation doesn't provide any clues regarding the various options. If it did, one might be able to deduce some of these needs on their own.  FriedEgg, your post was extremely helpful and I'd like to carry on the conversation offline.  However, I don't know your email address.

Please send me a real email.  You can reach me at art297 at rogers dot com.

mlogan
Lapis Lazuli | Level 10
Hi art297, Look like you know PROC HTTP in this forum. I am new at PROC HTTP. I am trying to learn it for extracting data with REST API. Can you please tell me where can I start at to learn?

Thanks,
ChrisHemedinger
Community Manager

Hi @mlogan -  This is an old thread.  If you have a new specific question, please post a new topic.

 

For PROC HTTP and REST, I have several examples on my blog:

 

http://blogs.sas.com/content/tag/rest-api/

http://blogs.sas.com/content/tag/proc-http/

It's time to register for SAS Innovate! Join your SAS user peers in Las Vegas on April 16-19 2024.
Jay1
Calcite | Level 5

I'd like to get one more out there.  I feel I'm close - can anyone help.  The link is: http://www.dgsweb.state.pa.us/Debarment_list/default.aspx

My SAS Code is here:

filename file1 'C:\out.txt';

filename file2 'C:\in.txt';

DATA _NULL_;

input;

file file2;

put _infile_;

datalines4;

__EVENTTARGET=&__EVENTARGUMENT=&__LASTFOCUS=&__VIEWSTATE=%2FwEPDwUKMTQ4NjEyMDk4NQ9kFgICAw9kFgYCAQ9kFgoCAQ8QZBAVAwpDaG9vc2UgT25lBkFjdGl2ZQZMb2cgSW4VAwpDaG9vc2UgT25lBkFjdGl2ZQZMb2cgSW4UKwMDZ2dnFgECAWQCAw8QZGQWAWZkAgsPEA8WAh4HVmlzaWJsZWhkZBYAZAINDw8WBB4EVGV4dAUiQ2hvb3NlIG9uZSB0byBzcGVjaWZ5IHNlYXJjaCBhcmVhLh8AaGRkAhUPEGRkFgECAWQCAw9kFgYCCQ8QZA8WRGYCAQICAgMCBAIFAgYCBwIIAgkCCgILAgwCDQIOAg8CEAIRAhICEwIUAhUCFgIXAhgCGQIaAhsCHAIdAh4CHwIgAiECIgIjAiQCJQImAicCKAIpAioCKwIsAi0CLgIvAjACMQIyAjMCNAI1AjYCNwI4AjkCOgI7AjwCPQI%2BAj8CQAJBAkICQxZEEGVlZxAFBUFnaW5nBQIxMGcQBQtBZ3JpY3VsdHVyZQUBNGcQBQdCYW5raW5nBQE1ZxAFG0NlbnRlciBmb3IgSnV2ZW5pbGUgSnVzdGljZQUEODEwN2cQBRhDaXZpbCBTZXJ2aWNlIENvbW1pc3Npb24FAjMyZxAFC0NvcnJlY3Rpb25zBQIxMWcQBQREQ0VEBQIyNGcQBQREQ05SBQIzOGcQBSZEZXBhcnRtZW50IG9mIEVudmlyb25tZW50YWwgUHJvdGVjdGlvbgUCMzVnEAUeRGVwYXJ0bWVudCBvZiBMYWJvciAmIEluZHVzdHJ5BQIxMmcQBQlFZHVjYXRpb24FAjE2ZxAFG0Vudmlyb25tZW50YWwgSGVhcmluZyBCb2FyZAUCMzdnEAUYRmlzaCBhbmQgQm9hdCBDb21taXNzaW9uBQIyMmcQBQ9HYW1lIENvbW1pc3Npb24FAjIzZxAFEEdlbmVyYWwgU2VydmljZXMFAjE1ZxAFBkhlYWx0aAUBN2cQBQlJbnN1cmFuY2UFATlnEAUUTGlxdW9yIENvbnRyb2wgQm9hcmQFAjI2ZxAFHU1pbGl0YXJ5IGFuZCBWZXRlcmFucyBBZmZhaXJzBQIxM2cQBRRNaWxrIE1hcmtldGluZyBCb2FyZAUCMjdnEAUYT2ZmaWNlIG9mIEFkbWluaXN0cmF0aW9uBQQ4MTAxZxAFGU9mZmljZSBvZiBHZW5lcmFsIENvdW5zZWwFBDgxMDJnEAUbT2ZmaWNlIG9mIEluc3BlY3RvciBHZW5lcmFsBQQ4MTAzZxAFFE9mZmljZSBvZiB0aGUgQnVkZ2V0BQQ4MTA0ZxAFFk9mZmljZSBvZiB0aGUgR292ZXJub3IFATFnEAUdUEEgSHVtYW4gUmVsYXRpb25zIENvbW1pc3Npb24FBDgxMDVnEAUUUEEgUHVibGljIFRWIE5ldHdvcmsFAjM0ZxAFBFBDQ0QFBDgxMDZnEAUEUEVNQQUCMzFnEAUtUGVubnN5bHZhbmlhIEhpc3RvcmljYWwgYW5kIE11c2V1bSBDb21taXNzaW9uBQIzMGcQBShQZW5uc3lsdmFuaWEgTXVuaWNpcGFsIFJldGlyZW1lbnQgU3lzdGVtBQI3MWcQBRlQZW5uc3lsdmFuaWEgU3RhdGUgUG9saWNlBQIyMGcQBQhQRU5OVkVTVAUCMzNnEAUUUHJvYmF0aW9uIGFuZCBQYXJvbGUFAjI1ZxAFKVB1YmxpYyBTY2hvb2wgRW1wbG95ZWVzIFJldGlyZW1lbnQgU3lzdGVtBQI3MmcQBQ5QdWJsaWMgV2VsZmFyZQUCMjFnEAUDUFVDBQIxN2cQBQdSZXZlbnVlBQIxOGcQBRVTZWN1cml0aWVzIENvbW1pc3Npb24FATZnEAUFU3RhdGUFAjE5ZxAFIVN0YXRlIEVtcGxveWVlcyBSZXRpcmVtZW50IFN5c3RlbQUCNzBnEAUXU3RhdGUgRXRoaWNzIENvbW1pc3Npb24FAjQwZxAFHFN0YXRlIFRheCBFcXVhbGl6YXRpb24gQm9hcmQFAjM2ZxAFDlRyYW5zcG9ydGF0aW9uBQE4ZxAFBlNlbmF0ZQUCNDFnEAUYSG91c2Ugb2YgUmVwcmVzZW50YXRpdmVzBQI0MmcQBQ9BdWRpdG9yIEdlbmVyYWwFATJnEAUIVHJlYXN1cnkFATNnEAUQQXR0b3JuZXkgR2VuZXJhbAUCMTRnEAUVTHQuIEdvdmVybm9yJ3MgT2ZmaWNlBQIyOGcQBQVQSEVBQQUCMzlnEAUmTGVnaXNsYXRpdmUgQWdlbmN5IG9mIEdlbmVyYWwgQXNzZW1ibHkFAjQ1ZxAFGkxlZ2lzbGF0aXZlIFNlcnZpY2UgQWdlbmN5BQI0NmcQBSZMZWdpc2xhdGl2ZSBCdWRnZXQgJiBGaW5hbmNlIENvbW1pdHRlZQUCNDdnEAUWSnVkaWNpYWwgQ29uZHVjdCBCb2FyZAUCNTFnEAUSQ29tbW9ud2VhbHRoIENvdXJ0BQI1OGcQBQ1TdXByZW1lIENvdXJ0BQI2MGcQBSdJbmRlcGVuZGVudCBSZWd1bGF0b3J5IFJldmlldyBDb21taXR0ZWUFAjYzZxAFF1BBIEdhbWluZyBDb250cm9sIEJvYXJkBQI2NWcQBSNQaGlsZGVscGhpYSBSZWdpb25hbCBQb3J0IEF1dGhvcml0eQUCODhnEAUgU3RhdGUgU3lzdGVtIG9mIEhpZ2hlciBFZHVjYXRpb24FAjkwZxAFE1R1cm5waWtlIENvbW1pc3Npb24FAjkxZxAFIFB1YmxpYyBTY2hvb2wgQnVpbGRpbmcgQXV0aG9yaXR5BQI5OGcQBS9Hb3Zlcm5vcidzIE9mZmljZSBmb3IgTWFuYWdlbWVudCAmIFByb2R1Y3Rpdml0eQUEODEwOGcQBRNDb3VuY2lsIG9uIHRoZSBBcnRzBQQ4MTA5ZxAFHkNhcGl0b2wgUHJlc2VydmF0aW9uIENvbW1pdHRlZQUCOTlnEAUJUEEgQ291cnRzBQI1M2cWAWZkAhUPEGRkFgFmZAIdDxBkZBYBZmQCBQ9kFgQCAQ8QZGQWAWZkAgUPPCsADQBkGAIFEWdyZERlYmFybWVudExpc3QyD2dkBRBncmREZWJhcm1lbnRMaXN0DzwrAAoBCAIBZDtIKQzuOUvkzIcQ22nPMsJDDYX4&__EVENTVALIDATION=%2FwEWEQLH2ujdCgLRoIiFCALf68jbBQL4oL9tApnvpbIHAvmXzo0MAun45GMC9vjkYwKT9%2FrLBwL%2B6YyoCgLCi9reAwKtkuWiCgKXjq%2BFCAKIjq%2BFCAKY4YXrBALC34jvAwL6o4PRA5xSrzJgIdRAZfigZJrvkhhVoNXL&ddlSelect=Active&ddlSearchBy=0&txtContractor=&btnSubmit=Search&rblPaging=0&hidEdit=False&hidLogin=

;;;;

RUN;

PROC HTTP out=file1

in=file2

url='http://www.dgsweb.state.pa.us/Debarment_list/default.aspx'

method='POST'                                                   

ct='application/x-www-form-urlencoded';                                         

RUN;ok

sas-innovate-2024.png

Join us for SAS Innovate April 16-19 at the Aria in Las Vegas. Bring the team and save big with our group pricing for a limited time only.

Pre-conference courses and tutorials are filling up fast and are always a sellout. Register today to reserve your seat.

 

Register now!

What is Bayesian Analysis?

Learn the difference between classical and Bayesian statistical approaches and see a few PROC examples to perform Bayesian analysis in this video.

Find more tutorials on the SAS Users YouTube channel.

Click image to register for webinarClick image to register for webinar

Classroom Training Available!

Select SAS Training centers are offering in-person courses. View upcoming courses for:

View all other training opportunities.

Discussion stats
  • 11 replies
  • 4101 views
  • 3 likes
  • 5 in conversation