topic Re: SAS to Download File from Dynamic URL in SAS Programming

SAS to Download File from Dynamic URL

dkewon — Thu, 29 Jun 2023 01:41:38 GMT

Hi all,

I have a link that looks like this - https:// ... (year)...(month).xslm.

The month part of the link changes every month. Is there any way SAS can click, download a file from this URL, and save it to our file system?

I can schedule a job, but I just can't figure out how to make SAS save this file to our file system. I tried PROC HTTP GET request but had no luck.

Thank you for your help!

Re: SAS to Download File from Dynamic URL

JosvanderVelden — Thu, 29 Jun 2023 09:02:49 GMT

Have a look at this post: https://blogs.sas.com/content/sasdummy/2017/12/04/scrape-web-page-data/

Re: SAS to Download File from Dynamic URL

Reeza — Thu, 29 Jun 2023 15:07:34 GMT

filename _myfile "/home/fkhurshed/downloaded_data%sysfunc(today(), year4.)_%sysfunc(today(), monname3.).xlsm";
 
proc http method="get" 
 url="https://www.noaa.gov........%sysfunc(today(), year4.)_%sysfunc(today(), monname3.).xlsm"
 out=_myfile
;
run;
 
filename _myfile clear;

First get this working for a single file with hardcoded dates, something like above.
Then use %SYSFUNC() with today() and the appropriate format for your date structure to have the date values calculated automatically.

Re: SAS to Download File from Dynamic URL

yabwon — Thu, 29 Jun 2023 15:33:01 GMT

If you want to get a binary copy of the "online" file, do it like this:

%let year=2023;
%let month=June;


filename in URL "https:// ... (&year.)...(&month.).xslm" 
recfm=N lrecl=1; /* this important for binary copy */

filename out "/path/to/store/the/file/ filename (&year.)...(&month.).xslm" recfm=N lrecl=1;

data _null_;
rc = fcopy("in", "out");
rctxt = sysmsg();
if rc then 
  put "ERROR:" rctxt;
run;

filename in clear;
filename out clear;

I'm using this technique in my programs.

All the best

Bart