Download SAS Files from EG

[Wang_Yajun]

Dear friends,

I am trying to download a SAS file with .sas7bdat extension from EG Serve using the sample code below. While I successfully downloaded the file, but the file cannot be used. I suppose that the procecure to copy the files from EG Serve may need to be modified, so I am asking for your help. Thank you very much.

options mprint ;
%macro get_files ;
 
/* use filename FTP to obtain a list of the files in the remote directory */
 filename flist ftp ""
host='fs.tarp.com'
user="user"
pass="password"
ls
cd="remote/directory/path" ;
 
data remlist ;
length fname $100 ;
infile flist truncover ;
input fname $100. ;
/* Specifiy a pattern match here to subset the remote files that are to be moved */ if index(fname,"Test")>0 then output  ; run ;
 
/* create a list of macro variables holding the remote file names */ proc sql noprint ; select fname into :fname1-:fname32000 from remlist ; %let fcount=%eval(&sqlobs) ; quit;
/* determine count of files */
%put Number of files to be copied:  &fcount ; options nonotes nomprint ;
 
/* loop through the file list */
%do j=1 %to &fcount ;
 
/* assign a FILENAME FTP for the remote file */ filename remfile ftp "&&fname&j"
host="fs.tarp.com"
user="user"
pass="password"
cd="remote/directory/path" ;
 
/*FILENAME for the local copy of the file */ filename flocal "local/directory/path/&&fname&j" ;
 
%put copying file &j ;
/* data step to copy the remote file */
data _null_ ;
infile remfile ;
file flocal;
input ;
put _infile_ ;
run ;
%end ;
/* of loop */
options notes ;
%mend ;
%get_files ;

[RW9]

Hi,

I am not sure I follow you?  If they are datasets, then point a libname statement to the area where the datasets are located, a libname to where you want them to be and then datastep:

data want.data;

  set eg.data;

run;

For each of them.  I don't understand why you are FTP copying them?

[Wang_Yajun]

Since the data set I want to copy reside in the EG serve, not in the local place, so I have no way to directly copy the data set.

[RW9]

Do you have access to the EG system (what is this EG Serve)?  If so, why can you not connect to the server via EG, point a libname to where you want the data locally, and then datastep as I mentioned.  Am not following what the problem is, you have:

Source datasets - you should have access to these some how

A destination, on your machine or network, by path.

A method of moving the data, two libnames and a datastep.

[Wang_Yajun]

Dear RW9,

Thanks for reply, I will try your method. Actually I thought when I login into the serve, I cannot refer to the local path and remote path simulataneouly. Once it worked, I will let you know. Thanks again for your help.

[LinusH]

"...it may need to be modified". Why don't you try first, it will just a guessing game for us.

Perhaps off topic, but since the data is already in SAS format, isn't there another way to access it than using ftp? The name indicates it is a SAS server (perhaps EG stands for something else than enterprise guide). Either by a direct libref or using a workspace server.

[Wang_Yajun]

Dear Linush,

Thanks for your reply. To make it clear,EG stands for Enterprise Guide, and I want to download  SAS data sets from EG serve.

 Actually I tried to modify the coding, but I have no idea about how the final data step succeeds in copying the files, so I am here to ask for help.

[Kurt_Bremser]

What do you mean by "but the file cannot be used"?

If you have a .sas7bdat file, which is a native SAS dataset, but no local SAS software (aside from EG), you have no tool to open the SAS file.

If it is anything other, please provide the error message(s).

[Wang_Yajun]

Dear KurtBremser,

Thanks for your reply. When I am downloading the SAS data set from the SAS EG Server into local PC, I use libname statement to read the SAS data set, and it appears a warning "This is not a SAS dataset, do you want to open another SAS dataset?". Also, while I download the SAS dataset from EG server, I found that the dataset is surprising smaller than it should be. Therefore, I suppose that the final data step needs to be modified, and I have no idea how to tackcle this problem. Thanks all the way for helping me.

[LinusH]

So if the resides on a SAS Server ("EG Serve"), why don't you use EG yourself? And why do you need to download it?

If yo still need to download for some reason, there is functionality within EG to do so.

[ChrisHemedinger]

You can download a batch of files from EG using the Data->Download SAS Data Sets task.  

If you want even more flexibility, you can use the Copy Files custom task.

Both if these steps can be captured as steps within the SAS Enterprise Guide process flow.

If you're looking to copy the files outside of an EG process, then you should probably use a shell script or FTP script.

Chris

[Doc_Duke]

I assume that "EG Serve" is actually a SAS server running on a remote host.

To add to Chris' comment, there are two possibilities of problems with the code you first displayed.

1) you may need to add a 'binary' option to the FTP filename for "remfile".  Text is fine for getting the filenames, but not for the data.

2) do the local copies have th "sas7bdat" extension?  If not EG and SAS won't recognize them.

[Wang_Yajun]

Dear Duke,

As you said, I added the binary option to the FTP filename for remfile, and it successfully downloaded the SAS data set with .sas7bdat extension. However, when I specified the libname statment to try to view the data, it returned an error "FIle XXXXXX is destroyed. I/O process is not complete".  

[Wang_Yajun]

Hello,

I know that in EG there is a point-click process to download the desired data, but now I want to use base SAS to automate this process.  The sample code I provided use FTP method to download the data/files, but I found that even if I downloaded the data/files, and all the data/files has the right extension, but all the data/file size is 1kb, and cannot be opened locally. So I am asking for help to modify the coding.

[Kurt_Bremser]

Your code uses the ftp method, but without a recfm= option, files are treated as text files, and the default lrecl is 254. Therefore mainly binary files will read a "record" until a linefeed character is encountered, and the data will be truncated to 254 bytes which are written out. Rinse and repeat.

You may find that your files arrive as one 254 byte chunk.

If you want to copy binary files, look into the recfm=n option.

But why don't you use a standard ftp tool, which is much more suited to the job?