Data download from a web page

[Mr_Belvedere]

I am attempting to download a simple spreadsheet from a webpage into my SAS program and I have been unsuccessful.  I am not sure if my issue is due to our firewall or my code.  I am using this code within SAS EG7.1. 

Here is my code and subsequent error message:

 filename bob url "http://www.nasdaq.com/screening/companies-by-industry.aspx?exchange=NASDAQ"

 debug;

 data test6;

 infile bob lrecl=30000;

 input col1 $10000.;

 put col1 $10000.;

 run;

ERROR: Invalid reply received from the HTTP server.

The funny thing is, the following code works successfully:

filename eSUG url "http://www.sas.com/offices/NA/canada/en/edmonton.html";

data eSUG_achive(keep=pdffile);

length pdffile $200;

infile eSUG length=len lrecl=32767;

input line $varying32767. len;

if find(line,".pdf") then do;

pdffile=scan(line,2,'"');

output;

end;

run;

filename eSUG clear;

Any thoughts on alternative ways to download the spreadsheet within my SAS code would be appreciated.

Thanks. 

[Shmuel]

1) could it be a temporary problem ? did you try running it again some time later ?

    and yes, it could be a firewall isuue. Ask your system team.

2) what do you mean by PUT statemnet in your code?

data test6;
 infile bob lrecl=30000;
 input col1 $10000.;
 put col1 $10000.;
 run;

3) I would add the option TRUNCOVER to the infile statement.

[Cynthia_sas]

Hi:

  Your second link is working because what is being retrieved is an HTML file. Your first link is NOT an HTML file and it is NOT a spreadsheet. When I visit your FIRST link, without using a program, just in my browser, I see that the ASP script is building a list (not a spreadsheet):

There does seem to be a button on the page for downloading the list in CSV format, but you need to get to the page first and then click the link. The FILENAME engine won't "click" that download button for you. My suspicion is that you'll have to find THAT link and use it in your program and, if it truly is a CSV file, you might have to change your program accordingly, since what is on the page is not HTML and is not a spreadsheet.

cynthia

[Mr_Belvedere]

Thanks for the quick response.

So is it possible to load an 'Active Server Page' within a SAS program? I believe that I could use that data directly from that page, but as I attempt to pull that information, I also get the 'Invalid Reply' error message.

[Cynthia_sas]

Based on your experience, I would say that it's not working to use the ASP page. If you scroll to the bottom of that first page, you'll see a link to load the next group of rows. So I believe that while the web server is using ASP, you can view, but not directly access what is displayed on the screen.

Since the ASP page generates an error, you'd have to find out whether they have another type of interface for downloading data or another URL.

cynthia