topic Re: Issue with spde files and data migration in SAS Programming

Issue with spde files and data migration

AM5 — Mon, 25 Mar 2019 21:44:38 GMT

We are moving from a Windows based SAS grid to a Linux based one in the next few months. At the moment we store all our SAS datasets as spde files because of their size. However the SAS program we have been given by the SAS Institute, cannot move spde files and they have advised us to uncompress all the datasets before migration. Is there a way to do this decompression in bulk? At the moment I can read in one dataset at a time then output it as a normal SAS dataset, migrate it across to the new grid, then recompress it there. We've estimated it will take a staff member 2 weeks full-time to do this. If we could do it in bulk it would greatly reduce our timeframes.

I've attached the migration code for reference.

For info we have gone from SAS EG version 7.15 64 bit, to 7.15 32 bit for Linux.

Re: Issue with spde files and data migration

ChrisNZ — Mon, 25 Mar 2019 22:35:03 GMT

As of the latest maintenance version, SAS can use Cross-Environment Data Access (CEDA) to read SPDE files.

See here: https://documentation.sas.com/?docsetId=lrcon&docsetTarget=n0oj2nagtyy32yn17pj01t6vytaw.htm&docsetVersion=9.4

The distributed paths of a grid might be an issue, I don't know.

I am unsure if Remote Library Services (RLS) is still unsupported in that release for SPDE data.

You could check this, as it would be a good alternative since you have the two sessions talking via SAS/Connect.

If it is, you could use proc copy directly. RLS is slow though.

Another option is proc cport where you can copy a whole library in one go. Again, distributed paths may be an issue. And file size...

Re: Issue with spde files and data migration

ChrisNZ — Mon, 25 Mar 2019 22:41:55 GMT

> Is there a way to do this decompression in bulk? At the moment I can read in one dataset at a time then output it as a normal SAS dataset

Can you use PROC COPY on the full library? To decompress a data set generated by PROC COPY, use the COMPRESS=NO system option before the PROC COPY statement with the NOCLONE option.

Re: Issue with spde files and data migration

Tom — Mon, 25 Mar 2019 22:48:06 GMT

Can you just use the fact that you have SAS/Connect to move the files directly from Windows to Unix?

libname IN spde .... ;

signon .... ;
rsubmit;
  libname OUT spde .... ;
  proc upload inlib=IN outlib=OUT; 
  run;
endrsubmit;

Re: Issue with spde files and data migration

AM5 — Tue, 26 Mar 2019 04:52:37 GMT

Thanks, this worked perfectly!

Re: Issue with spde files and data migration

ChrisNZ — Tue, 26 Mar 2019 05:00:05 GMT

You are losing the timestamps with proc upload.