Help using Base SAS procedures

PROC HTTP error

Accepted Solution Solved
Reply
SAS Employee
Posts: 8
Accepted Solution

PROC HTTP error

Hello All,

I am trying to query textual comments data from the youtube api using proc http with the below code. However, i am getting a http error in the log and my request is not getting processed. I have referenced for related documentation why it is failing but other than the description of the error at http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html, i couldn't figure out the issue.  I have tried using my youtube login credentials in the webusername and webpassword parameters but it didnt work. Appreciate your help !

ERROR: java.io.IOException: Server returned HTTP response code: 415 for URL:

http://gdata.youtube.com/feeds/api/videos/pzI4D6dyp_o/comments?v=2&max-results=50&start-index=1

Below is the code i used, this is similar to the code that is available at http://sww.sas.com/toolpool/pool/Twitter_Web_Application/index.html which was used for retrieving twitter data.

GOPTIONS ACCESSIBLE;

/* this is where we will store the HTML output */

filename odsout "C:\Inetpub\wwwroot\youtube";

ods listing close;

/*The code to connect to twitter using SAS looks like this: */

filename REQUEST "C:\temp\blank.txt"; /*make sure to create this blank txt file before running the code*/
filename SchOut "c:\temp\Video_Meta.xml"; 

/* url is the location we pass to proc http */

%let url = "http://gdata.youtube.com/feeds/api/videos/pzI4D6dyp_o/comments?v=2&max-results=50&start-index=1";
%put &url;

/*connect to youtube */

proc http
    in=REQUEST
    out=SchOut
    url=&url
    method="get"
    proxyhost="inetgw.unx.sas.com"
    proxyport=80
    webusername='' /* not required here */
    webpassword='' /* not required here */
    ;
run; 

Thanks,

Murali


Accepted Solutions
Solution
‎09-23-2011 11:21 AM
Trusted Advisor
Posts: 1,301

Re: PROC HTTP error

I still am shakey as to why you are connecting through the proxy host?

Why not just use the filename statement for url access?

filename yt url 'http://gdata.youtube.com/feeds/api/videos/pzI4D6dyp_o/comments?v=2&max-results=50&start-index=1';

data _null_;

infile yt length=len;

input record $varying32000. len;

put record $varying32000. len;

run;

libname ytube xml xmlfileref=yt;

data ytube;

set ytube.entry;

run;

View solution in original post


All Replies
Trusted Advisor
Posts: 1,301

PROC HTTP error

Do you actually have access to the proxyhost you are attempting to connect through?

If all you are trying to go is download an xml form you could (if you are using unix) just use wget.  I assume windows has a similar tool but it does not come to mind at the moment.

x wget &url > /temp/schout.xml;

EDIT: you can get wget for windows here: http://gnuwin32.sourceforge.net/packages/wget.htm

SAS Employee
Posts: 8

PROC HTTP error

Thanks for your help ! I believe I have the access required, because when i try the same proxy server and port for twitter api, I can get it work. I have some third party free softwares currently which I am able to use to directly download the data from twitter in .csv or xml formats. However, I would like to be able to use SAS instead to showcase the capabilities even for data extraction through the standard api interfaces.

Super User
Super User
Posts: 7,039

PROC HTTP error

On my Unix box I have to explicitely add the port number to the URL to read it from SAS code.  Try adding :80 after the hostname.

http://gdata.youtube.com:80/feeds/api/videos/pzI4D6dyp_o/comments?v=2&max-results=50&start-index=1

SAS Employee
Posts: 8

PROC HTTP error

I tried it, didn't work Smiley Sad

Solution
‎09-23-2011 11:21 AM
Trusted Advisor
Posts: 1,301

Re: PROC HTTP error

I still am shakey as to why you are connecting through the proxy host?

Why not just use the filename statement for url access?

filename yt url 'http://gdata.youtube.com/feeds/api/videos/pzI4D6dyp_o/comments?v=2&max-results=50&start-index=1';

data _null_;

infile yt length=len;

input record $varying32000. len;

put record $varying32000. len;

run;

libname ytube xml xmlfileref=yt;

data ytube;

set ytube.entry;

run;

SAS Employee
Posts: 8

PROC HTTP error

Thank you for the help ! I ran this and it works perfectly fine excepting some missing details like author id of the comment posted. A colleague of mine also helped me solve the issue with proc http by just removing the proxy host and port details from the code. Thanks a lot again Smiley Happy

Trusted Advisor
Posts: 1,301

PROC HTTP error

Yes, it seemed like the likely cause of the issues with proc http was the proxy options you were using.  The variables will be better defined not using the defualt xmlmapper file and instead definind your own the fits the data apporiately.

http://support.sas.com/documentation/cdl/en/engxml/62845/HTML/default/viewer.htm#a002484837.htm

SAS Employee
Posts: 8

PROC HTTP error

Yes, I am using xmlmapper for this. Thanks!

PROC Star
Posts: 7,468

PROC HTTP error

Muali,

If you got it to run with your original code would you mind posting the code that you finally used?  I'm wondering as I haven't been able to get proc http to do it yet (other than using FriedEgg's suggested code).

Art

SAS Employee
Posts: 8

PROC HTTP error

Hello Art,

Here is the code.

%let url = "http://gdata.youtube.com/feeds/api/videos/pzI4D6dyp_o/comments?v=2%nrstr(&max)-results=50%nrstr(&sta...";

%put &url;

filename test "< --path-- >.xml";

proc http

/*  in=REQUEST */

    out=test

    url=&url

    method="get"

/*    proxyhost="inetgw.unx.sas.com" */

/*    proxyport=80 */

/*    webusername=''*/

/*    webpassword=''*/

    ;

run

I am yet to do the mapper, will post the rest of the code when i'm done with that.

Thanks,

Murali

PROC Star
Posts: 7,468

PROC HTTP error

Much appreciated!  That worked for me as well!

Trusted Advisor
Posts: 1,301

Re: PROC HTTP error

filename in temp;

filename out temp;

data null;

   file in;

   input;

   put infile;

   datalines;

pzI4D6dyp_o/comments?v=2%nrstr(&max)-results=50%nrstr(&start)-index=1

;

proc http in=in out=out url='http://gdata.youtube.com/feeds/api/videos'

  method='get'

run;

libname ytube xml xmlfileref=out;

data ytube;

set ytube.entry;

run;

Contributor
Posts: 28

Re: PROC HTTP error

Hello All,

I need help with the similar situation. Thanks to you all for your postings. It was very informative.

I was wondering on how to handle the proc http, if my URL is a variable in a data set called HTTP. I have used the url access method, but I was getting an error saying invalid server response. I need to run all the url links one by one and get the values stored on the server (some name), and send those values to the name variable back to my work data set. I was thinking to export the file to the temp folder on unix, and then use the

filename in temp;

filename out temp;

Please advise.

I have used your method here:

My environment is UNIX:

proc sql noprint;

select count(*) into :httpcnt

from work.URL_Test where substr(http,1,5)='http:';

quit;

%put &httpcnt;

data _null_;

  set Test;

call symput("cpe",trim(variable));

call symput("date_time",trim(variable));

run;

%let url = "http://host:80abcdetcetc%nrstr(&cpe)%nrstr(%&)incidentDate=%nrstr(&date_time)";

filename http_get "/test/test/http.csv";

%macro get_http;

/*%do i=1 %to &httpcnt;*/

proc http

    out=http_get

    url=&url

    method="get" ;

run;

/*%end;*/

%mend;

%get_http;

Btw, This method worked for me when I took just a single link from the variable to test its out. It worked great, and created forllowing valuesin the out=http_get.csv file.

Please let me know if its possible to get only "name I am looking for"? are there any options to remove the tags?? I need to run all the links, and include the results to my data set (in name variable). Thanks a lot in advance.

*****http_get.csv file.

<html>

<head>

<title>adcdef</title>

</head>

<body>

<h1>name I am looking for</h1>

</body>

</html>

🔒 This topic is solved and locked.

Need further help from the community? Please ask a new question.

Discussion stats
  • 13 replies
  • 3789 views
  • 4 likes
  • 5 in conversation