Issue with spde files and data migration

[AM5]

We are moving from a Windows based SAS grid to a Linux based one in the next few months. At the moment we store all our SAS datasets as spde files because of their size. However the SAS program we have been given by the SAS Institute, cannot move spde files and they have advised us to uncompress all the datasets before migration. Is there a way to do this decompression in bulk? At the moment I can read in one dataset at a time then output it as a normal SAS dataset, migrate it across to the new grid, then recompress it there. We've estimated it will take a staff member 2 weeks full-time to do this. If we could do it in bulk it would greatly reduce our timeframes.

I've attached the migration code for reference.

For info we have gone from SAS EG version 7.15 64 bit, to 7.15 32 bit for Linux.

[ChrisNZ]

As of the latest maintenance version, SAS can use Cross-Environment Data Access (CEDA) to read SPDE files.

See here: https://documentation.sas.com/?docsetId=lrcon&docsetTarget=n0oj2nagtyy32yn17pj01t6vytaw.htm&docsetVe...

The distributed paths of a grid might be an issue, I don't know.

I am unsure if Remote Library Services (RLS) is still unsupported in that release for SPDE data. 

You could check this, as it would be a good alternative since you have the two sessions talking via SAS/Connect.

If it is, you could use proc copy directly. RLS is slow though.

Another option is proc cport where you can copy a whole library in one go. Again, distributed paths may be an issue. And file size...

[ChrisNZ]

> Is there a way to do this decompression in bulk? At the moment I can read in one dataset at a time then output it as a normal SAS dataset

Can you use PROC COPY on the full library? To decompress a data set generated by PROC COPY, use the COMPRESS=NO system option before the PROC COPY statement with the NOCLONE option. 

[Tom · Accepted solution]

Can you just use the fact that you have SAS/Connect to move the files directly from Windows to Unix?

libname IN spde .... ;

signon .... ;
rsubmit;
  libname OUT spde .... ;
  proc upload inlib=IN outlib=OUT; 
  run;
endrsubmit;

[AM5]

Thanks, this worked perfectly!

[ChrisNZ]

You are losing the timestamps with proc upload.