topic Re: Downloading files from WEB in SAS Users Group in Israel

Downloading files from WEB

Assaf_Attas — Thu, 08 Jul 2021 08:06:32 GMT

hey guys,

i'm trying to automate a download process using SAS.

i found some posts on this but non of them worked for me..

the code runs but i dont get the expected resault

any help?

method 1

filename out "%sysfunc(getoption(WORK))\DL_File.xlsx";
 
proc http
 url='https://data.gov.il/dataset/branches/resource/1c5bc716-8210-4ec7-85be-92e6271955c2/download/snifim_heboi.org.il.xlsx'
 method="get" out=out;
run;

method2

filename myinfile url "https://data.gov.il/dataset/branches/resource/1c5bc716-8210-4ec7-85be-92e6271955c2/download/snifim_heboi.org.il.xlsx";
filename out "%sysfunc(getoption(WORK))\DL_File.xlsx";

data _null_;
	n=-1;
	infile myinfile recfm=s nbyte=n length=len;
	file out recfm=n;

	input;
	put _infile_ $varying32767. len;

run;

Re: Downloading files from WEB

Kurt_Bremser — Thu, 08 Jul 2021 09:35:29 GMT

Run your first code with a different outfile name:

filename out "/folders/myfolders/DL_File.html";

and you will find that you do not get a XLSX file, but a HTML file with lots of scripts in the header and no body.

Re: Downloading files from WEB

Assaf_Attas — Thu, 08 Jul 2021 09:39:58 GMT

that's what i referred as the undesired result.
what i need is the actual file the link refers to.

Re: Downloading files from WEB

Kurt_Bremser — Thu, 08 Jul 2021 11:11:58 GMT

The problem here seems to be this:

Instead of supplying the file, the website sends a script which then does the download. Since PROC HTTP is not a full-fledged web browser with a Javascript sandbox, this won't work.

IMO, the non-standard way that the website handles downloads means that you can't automate this in SAS, but I may be proven otherwise.

I suggest that you also open a track with SAS technical support while you wait for possible answers here.

Re: Downloading files from WEB

EyalGonen — Thu, 08 Jul 2021 14:14:04 GMT

Hi @Assaf_Attas ,

I tested your URL with a 3rd party tool called "curl" and I got back from the server a file that contains HTML with embedded JavaScript as @Kurt_Bremser suggested.

So it seems like this is not a SAS specific problem with handling this URL as it seems like "curl" is not downloading the XLSX file as well.

Cheers,

Eyal