topic reading in web data in SAS Programming

reading in web data

rboire — Mon, 12 Jun 2017 22:06:03 GMT

see below SAS code and errors. any suggestions

filename foo url
169      'http://support.sas.com/techsup/service_intro.html';
170
171 data file2;
172     infile foo length=len;
173     input record $varying200. len;
174     put record $varying200. len;
175     if _n_=15 then stop;
176 run;

ERROR: Invalid reply received from the HTTP server. Use the debug option for more info.
NOTE: The SAS System stopped processing this step because of errors.
WARNING: The data set WORK.FILE2 may be incomplete. When this step was stopped there were 0
         observations and 1 variables.
NOTE: DATA statement used (Total process time):
      real time           0.48 seconds
      cpu time            0.09 seconds

Re: reading in web data

WarrenKuhfeld — Mon, 12 Jun 2017 23:26:42 GMT

Does the file you are reading exist? Here is an example of reading one of the SAS sample programs.

data _null_;
   %let url = //support.sas.com/documentation/onlinedoc/stat/ex_code/142;
   infile "http:&url/templft.html" device=url;

   file 'macros.tmp';
   retain pre 0;
   input;
   _infile_ = tranwrd(_infile_, '&amp;', '&');
   _infile_ = tranwrd(_infile_, '&lt;' , '<');
   if index(_infile_, '</pre>') then pre = 0;
   if pre then put _infile_;
   if index(_infile_, '<pre>')  then pre = 1;
run;

Re: reading in web data

BrunoMueller — Tue, 13 Jun 2017 07:39:54 GMT

As suggested in the error message, use the DEBUG option like so:

filename foo url
  'http://support.sas.com/techsup/service_intro.html'
  debug
;

This will return the communication between SAS and the web server, this will help you identify the problem. For the URL it looks like this:

NOTE: >>> GET /techsup/service_intro.html HTTP/1.0
NOTE: >>> Host: support.sas.com
NOTE: >>> Accept: */*
NOTE: >>> Accept-Language: en
NOTE: >>> Accept-Charset: iso-8859-1,*,utf-8
NOTE: >>> User-Agent: SAS/URL
NOTE: >>> 
NOTE: <<< HTTP/1.1 301 Moved Permanently
NOTE: <<< Date: Tue, 13 Jun 2017 07:36:15 GMT
NOTE: <<< Server: Apache
NOTE: <<< Location: https://support.sas.com/techsup/service_intro.html
NOTE: <<< Content-Length: 258
NOTE: <<< Connection: close
NOTE: <<< Content-Type: text/html; charset=iso-8859-1
NOTE: <<< Set-Cookie: BIGipServer~WEB~www.sas.com-pool=1074113941.20480.0000; path=/; Httponly
NOTE: <<< 
NOTE: >>> GET /techsup/service_intro.html HTTP/1.0
NOTE: >>> Host: support.sas.com
NOTE: >>> Accept: */*
NOTE: >>> Accept-Language: en
2                                                          The SAS System                               09:36 Tuesday, June 13, 2017

NOTE: >>> Accept-Charset: iso-8859-1,*,utf-8
NOTE: >>> Cookie: BIGipServer~WEB~www.sas.com-pool=1074113941.20480.0000
NOTE: >>> User-Agent: SAS/URL
NOTE: >>> 
NOTE: <<< HTTP/1.1 404 Not Found
NOTE: <<< Date: Tue, 13 Jun 2017 07:36:16 GMT
NOTE: <<< Server: Apache
NOTE: <<< Expires: Thu, 01 Jan 1970 00:00:00 GMT
NOTE: <<< X-Powered-By: Jetty(9.2.9.v20150224)
NOTE: <<< Content-Type: text/html; charset=UTF-8
NOTE: <<< Set-Cookie: renderid=rend03; path=/;
NOTE: <<< Set-Cookie: JSESSIONID=lqmlrqwb58pdwzzq8yvyei0w;Path=/
NOTE: <<< Connection: close
NOTE: <<< 
ERROR: Invalid reply received from the HTTP server. Use the debug option for more info.

Re: reading in web data

rboire — Sun, 18 Jun 2017 11:38:04 GMT

got the following error. any thoughts

2671 filename foo 'http://support.sas.com/documentation/onlinedoc/stat/ex_code/142/templft.html';
2672
2673
2674 data test;
2675   infile foo length=len;
2676      input record $varying200. len;
      ----
      180
2677      put record $varying200. len;
      ----
      180
2678      if _n_=15 then stop;
      ----
      180
2679 run;
      -
      180
ERROR 180-322: Statement is not valid or it is used out of proper order.

2680

NOTE: The SAS System stopped processing this step because of errors.
WARNING: The data set WORK.TEST may be incomplete. When this step was stopped there were 0
         observations and 0 variables.
WARNING: Data set WORK.TEST was not replaced because this step was stopped.
NOTE: DATA statement used (Total process time):
      real time           0.02 seconds
      cpu time            0.03 seconds

2681 proc print data=test;run;

Re: reading in web data

art297 — Sun, 18 Jun 2017 19:10:14 GMT

Will work a lot better if you include the url engine in your filename statement:

filename foo url 'http://support.sas.com/documentation/onlinedoc/stat/ex_code/142/templft.html';
data test;
  infile foo length=len;
  input record $varying200. len;
  put record $varying200. len;
  if _n_=15 then stop;
run;

Art, CEO, AnalystFinder.com

Re: reading in web data

Tom — Sun, 18 Jun 2017 21:23:46 GMT

You either have a missing semi-colon that has converted your DATA statement into a comment or part of the previous statement. In that case the INFILE and INPUT functions would be invalid if not part of a data step.

Or perhaps garbage characters in your source file on the lines that it is complaining about.