topic Re: How to select data set in a folder based on the year they were created and move them in SAS Enterprise Guide

How to select data set in a folder based on the year they were created and move them

alepage — Sat, 02 Oct 2021 20:57:56 GMT

Hello,

It might happen that I have duplicate dataset in the same folder but not created on the same date. But due to the use of spde system, the name is not exactly the same (see the snapshot)

So if I use a proc copy as below

proc copy in=src1 out=dest1 memtype=data;
select e001_tracking_scenario e999_tracking_scenario
;
run;

I am not sure that I will pick the dataset produce on September 14, 2021.

What's the best way to do that?

Re: How to select data set in a folder based on the year they were created and move them

Tom — Sat, 02 Oct 2021 23:30:18 GMT

You cannot have duplicate DATASETS in the same library. You cannot have duplicate FILES in the same directory.

It kind of looks like you are trying to look a the FILES created by the SPDE engine.

Why not look at the DATASETS created by using the SPDE engine?

libname old spde './old/';
proc contents data=old._all_ noprint out=contents; run;

Re: How to select data set in a folder based on the year they were created and move them

Patrick — Sat, 02 Oct 2021 23:46:20 GMT

With SPDE the data gets partitioned and there is also additional metadata (in separate files) which has the information about these partitions.

If you update a table then only the partitions (the physical files) will change that have rows that changed.

If you need to extract data from a specific date then this needs to happen on data level and there is hopefully some update_date or creation_date column in your data which you can use for such a selection.

In short: You can't use these physical partitions directly. You need to access your data via the SPDE engine (libname ... spde ....).