Move Files (any type) between Unix servers using SAS and untar

[dart9596]

Is there a simple (and current) way to do the following - using SAS....

1) Copy file A (any type) from server X to Y - while holding a SAS Studio session on server Z

2) Untar (tgz format) file on server Y - while holding a SAS Studio session on server Z

Typically we use WinSCP for this process - but given the number of times (per week) we need to do so (and the number of files) - would prefer to have a simple SAS script to run - to do the whole process.

[Reeza]

Do you have x command enabled?
What version of SAS do you have?
I'm assuming you're programming as well, not using GUI/task?

[dart9596]

I can use X commands

SAS 9.4

Yes - programmer

[Reeza]

9.4? M?

[dart9596]

Sas Studio 3.8

SAS release: 9.04.01M6

[Kurt_Bremser]

Wgy don't you let the datacenter operators do it from the scheduler?

Anyway, you need to enable public/private key authentication from server Z to server Y, so you can run commands there without entering a password (because you cannot do that from SAS).

You need to do the same from Y to X for the copy.

Then you can use ssh to run commands on Y, first the scp to get the file, then the gzip.

[dart9596]

A co-worker suggested the following....

Processing appears to work. The file is slightly larger after transfer - per the UNTAR process - 'garbage' added to end of file.

PROC COMPARE indicates the datasets within the .TGZ align - are the same.

%macro mv(file=);
   filename indir   ftp 'LOCATIONX' DIR host="SERVERX" user="USERX"  pass="PASSX" prompt binary debug;
   filename outdir  ftp 'LOCATIONY' DIR host="SERVERY" user="USERY"  pass="PASSY" prompt binary debug;
   data _null_;
         infile indir(&file) ;
         input;
         file outdir(&file);
         put _infile_;
     run;
%mend mv;
%mv(file=FILEA.tgz);

I believe the approach is based on a SAS.com post.

Your thoughts/opinions?

[Kurt_Bremser]

It's a workaround, but it moves the data twice over the network (X to Z, Z to Y). Connecting to Y and getting the data from X moves it across the network only once.

But data steps usually try to read and write text data. So the larger size may be the result of writing a complete buffer (_infile_) that is only partially filled.

[dart9596]

Thank you.

Any thoughts on using SAS to UNTAR remotely?

From server X UNTAR tgz on server Y?

Below is a similar post - but i receive an '?invalid command' for the 'tar' put.

Otherwise commands such as ls and pwd work fine.

https://support.sas.com/kb/25/211.html

data _null_;
   file ftpcmds pad lrecl=200;
   put "user USERX PASSX";
   put "cd LOCATIONX";
   put 'tar -xz --directory="." -f "FILEA.tgz"';
run;

filename doftp pipe "ftp -n &host < &ftpcmds";
data _null_;
  infile doftp;
  input;
put _infile_;
run;

[Reeza]

For the zipped portion - if your files are text you can read them straight from the zipped files now. No need to unzip unless you want to.

[dart9596]

Need to pass the sas datasets to a application - will only take sas datasets

[SASKiwi]

Probably unlikely but if you have SAS/CONNECT licensed on both SAS servers a simple PROC UPLOAD will copy your SAS datasets or indeed whole SAS libraries a lot more easily. No fluffing around with tar, ftp, DATA step file processing or anything else.

Pity you are not using Windows servers with UNC drive mappings, then PROC COPY would work fine as you can define LIBNAMEs on the target server pointing to the source server folders. @Kurt_Bremser - hate to say this, but this is one area where Windows just works better

[Patrick]

@Kurt_Bremser 

Why gzip? Doesn't the scp -c flag allow to transfer files compressed if required?

[Kurt_Bremser]

---

@Patrick wrote:

@Kurt_Bremser 

Why gzip? Doesn't the scp -c flag allow to transfer files compressed if required?

---

because .tgz files are tar files compressed with gzip. In order to use the contained datasets with SAS, they need to be unzipped and un-tared.