topic Re: Download SAS Files from EG in SAS Programming

Download SAS Files from EG

Wang_Yajun — Mon, 15 Feb 2016 10:55:12 GMT

Dear friends,

I am trying to download a SAS file with .sas7bdat extension from EG Serve using the sample code below. While I successfully downloaded the file, but the file cannot be used. I suppose that the procecure to copy the files from EG Serve may need to be modified, so I am asking for your help. Thank you very much.


options mprint ;
%macro get_files ;
 
/* use filename FTP to obtain a list of the files in the remote directory */
 filename flist ftp ""
host='fs.tarp.com'
user="user"
pass="password"
ls
cd="remote/directory/path" ;
 
data remlist ;
length fname $100 ;
infile flist truncover ;
input fname $100. ;
/* Specifiy a pattern match here to subset the remote files that are to be moved */ if index(fname,"Test")>0 then output  ; run ;
 
/* create a list of macro variables holding the remote file names */ proc sql noprint ; select fname into :fname1-:fname32000 from remlist ; %let fcount=%eval(&sqlobs) ; quit;
/* determine count of files */
%put Number of files to be copied:  &fcount ; options nonotes nomprint ;
 
/* loop through the file list */
%do j=1 %to &fcount ;
 
/* assign a FILENAME FTP for the remote file */ filename remfile ftp "&&fname&j"
host="fs.tarp.com"
user="user"
pass="password"
cd="remote/directory/path" ;
 
/*FILENAME for the local copy of the file */ filename flocal "local/directory/path/&&fname&j" ;
 
%put copying file &j ;
/* data step to copy the remote file */
data _null_ ;
infile remfile ;
file flocal;
input ;
put _infile_ ;
run ;
%end ;
/* of loop */
options notes ;
%mend ;
%get_files ;

Re: Download SAS Files from EG

RW9 — Mon, 15 Feb 2016 11:29:28 GMT

Hi,

I am not sure I follow you? If they are datasets, then point a libname statement to the area where the datasets are located, a libname to where you want them to be and then datastep:

data want.data;

set eg.data;

run;

For each of them. I don't understand why you are FTP copying them?

Re: Download SAS Files from EG

LinusH — Mon, 15 Feb 2016 11:33:22 GMT

"...it may need to be modified". Why don't you try first, it will just a guessing game for us.

Perhaps off topic, but since the data is already in SAS format, isn't there another way to access it than using ftp? The name indicates it is a SAS server (perhaps EG stands for something else than enterprise guide). Either by a direct libref or using a workspace server.

Re: Download SAS Files from EG

Kurt_Bremser — Mon, 15 Feb 2016 12:42:58 GMT

What do you mean by "but the file cannot be used"?

If you have a .sas7bdat file, which is a native SAS dataset, but no local SAS software (aside from EG), you have no tool to open the SAS file.

If it is anything other, please provide the error message(s).

Re: Download SAS Files from EG

Wang_Yajun — Mon, 15 Feb 2016 14:41:22 GMT

Since the data set I want to copy reside in the EG serve, not in the local place, so I have no way to directly copy the data set.

Re: Download SAS Files from EG

RW9 — Mon, 15 Feb 2016 14:44:12 GMT

Do you have access to the EG system (what is this EG Serve)? If so, why can you not connect to the server via EG, point a libname to where you want the data locally, and then datastep as I mentioned. Am not following what the problem is, you have:

Source datasets - you should have access to these some how

A destination, on your machine or network, by path.

A method of moving the data, two libnames and a datastep.

Re: Download SAS Files from EG

Wang_Yajun — Mon, 15 Feb 2016 14:47:34 GMT

Dear Linush,

Thanks for your reply. To make it clear,EG stands for Enterprise Guide, and I want to download SAS data sets from EG serve.

Actually I tried to modify the coding, but I have no idea about how the final data step succeeds in copying the files, so I am here to ask for help.

Re: Download SAS Files from EG

Wang_Yajun — Mon, 15 Feb 2016 14:54:31 GMT

Dear KurtBremser,

Thanks for your reply. When I am downloading the SAS data set from the SAS EG Server into local PC, I use libname statement to read the SAS data set, and it appears a warning "This is not a SAS dataset, do you want to open another SAS dataset?". Also, while I download the SAS dataset from EG server, I found that the dataset is surprising smaller than it should be. Therefore, I suppose that the final data step needs to be modified, and I have no idea how to tackcle this problem. Thanks all the way for helping me.

Re: Download SAS Files from EG

Wang_Yajun — Mon, 15 Feb 2016 15:11:02 GMT

Dear RW9,

Thanks for reply, I will try your method. Actually I thought when I login into the serve, I cannot refer to the local path and remote path simulataneouly. Once it worked, I will let you know. Thanks again for your help.

Re: Download SAS Files from EG

LinusH — Mon, 15 Feb 2016 15:46:34 GMT

So if the resides on a SAS Server ("EG Serve"), why don't you use EG yourself? And why do you need to download it?

If yo still need to download for some reason, there is functionality within EG to do so.

Re: Download SAS Files from EG

ChrisHemedinger — Mon, 15 Feb 2016 19:47:47 GMT

You can download a batch of files from EG using the Data->Download SAS Data Sets task.

If you want even more flexibility, you can use the Copy Files custom task.

Both if these steps can be captured as steps within the SAS Enterprise Guide process flow.

If you're looking to copy the files outside of an EG process, then you should probably use a shell script or FTP script.

Chris

Re: Download SAS Files from EG

Doc_Duke — Mon, 15 Feb 2016 21:21:07 GMT

I assume that "EG Serve" is actually a SAS server running on a remote host.

To add to Chris' comment, there are two possibilities of problems with the code you first displayed.

1) you may need to add a 'binary' option to the FTP filename for "remfile". Text is fine for getting the filenames, but not for the data.

2) do the local copies have th "sas7bdat" extension? If not EG and SAS won't recognize them.

Re: Download SAS Files from EG

Wang_Yajun — Tue, 16 Feb 2016 00:50:32 GMT

Hello,

I know that in EG there is a point-click process to download the desired data, but now I want to use base SAS to automate this process. The sample code I provided use FTP method to download the data/files, but I found that even if I downloaded the data/files, and all the data/files has the right extension, but all the data/file size is 1kb, and cannot be opened locally. So I am asking for help to modify the coding.

Re: Download SAS Files from EG

Reeza — Tue, 16 Feb 2016 01:35:36 GMT

Are you planning to use Base SAS to run and develop the code? Base interacts differently and you can see both your remote and local servers and move files between them with a bit more ease.

Re: Download SAS Files from EG

Kurt_Bremser — Tue, 16 Feb 2016 09:25:55 GMT

Your code uses the ftp method, but without a recfm= option, files are treated as text files, and the default lrecl is 254. Therefore mainly binary files will read a "record" until a linefeed character is encountered, and the data will be truncated to 254 bytes which are written out. Rinse and repeat.

You may find that your files arrive as one 254 byte chunk.

If you want to copy binary files, look into the recfm=n option.

But why don't you use a standard ftp tool, which is much more suited to the job?

Re: Download SAS Files from EG

Wang_Yajun — Tue, 16 Feb 2016 12:07:24 GMT

Dear Duke,

As you said, I added the binary option to the FTP filename for remfile, and it successfully downloaded the SAS data set with .sas7bdat extension. However, when I specified the libname statment to try to view the data, it returned an error "FIle XXXXXX is destroyed. I/O process is not complete".

Re: Download SAS Files from EG

Wang_Yajun — Tue, 16 Feb 2016 12:08:11 GMT

Hi, KurtBremser

Thanks a lot for your helpful comment. Could you please show me the standard tool to download data from EG(Enterprise Guide) serve? Thanks a lot.

Best regards,

Yajun

Re: Download SAS Files from EG

Kurt_Bremser — Tue, 16 Feb 2016 12:41:41 GMT

It's the original "ftp" utility, present on all platforms. Just call it with the X statement or call system subroutine. Usage depends on the operating system.

For usage, on UNIX type man ftp, on Windows ftp /?

Re: Download SAS Files from EG

ChrisHemedinger — Tue, 16 Feb 2016 13:26:31 GMT

If you want to use a SAS program to drive the copy operation, here's a technique you can try. Two approaches: the DATA step to copy binary files, even from an FTP location, and then also the FCOPY function that might help (SAS 9.4).

Re: Download SAS Files from EG

Wang_Yajun — Tue, 15 Mar 2016 03:05:00 GMT

Hello,

I'm sorry for late reply, as I am asking the SAS Technical support for help. The final solution to download files from EG serve is in the following SAS code.

options mprint mlogic ;
%macro get_files ;
 
/* use filename FTP to obtain a list of the files in the remote directory */
 filename flist ftp ""
host='172.26.38.38'
user="sasdemo"
pass="Sas123"
ls
cd="\t1" ;
 
data remlist ;
length fname $100 ;
infile flist truncover ;
input fname $100. ;

/* Specifiy a pattern match here to subset the remote files that are to be moved */ 
/*if index(fname,"Test")>0 then output  ; */
run ;
 
/* create a list of macro variables holding the remote file names */ 
proc sql noprint ; select fname into :fname1-:fname32000 from remlist ; %let fcount=%eval(&sqlobs) ; quit;
/* determine count of files */
%put Number of files to be copied:  &fcount ; /*options nonotes nomprint ;*/
 
/* loop through the file list */
%do j=1 %to &fcount ;
 
/* assign a FILENAME FTP for the remote file */ 
filename remfile ftp "&&fname&j"
host="172.26.38.38"
user="sasdemo"
pass="Sas123"
cd="/t1" recfm=f debug;
 
/*FILENAME for the local copy of the file */
filename flocal "C:\temp\download\&&fname&j" recfm=n;
 
%put copying file &j ;
/* data step to copy the remote file */
data _null_ ;
N=1;
infile remfile NBYTE=n;
file flocal;
input ;
put _infile_ @@;
run ;
%end ;
/* of loop */
options notes ;
%mend ;
%get_files ;