how to delete spde files on a specific date

[alepage]

Hello,

I have found duplicate spde file into many folder. For example, in sas2023 folder, I see 

ha_auto_inforcefev2023.dpf.00010aa2.215.1.spds9   April 4, 2023

ha_auto_inforcefev2023.dpf.0000c9da.15.1.spds9    February 28, 2023

I have the same issue with premium dataset

ha_auto_prmfev2023.dpf.0000c9da.67.1.spds9  April 4, 2023

ha_auto_prmfev2023.dpf.00010aa2.23.1.spds9 February 28, 2023

I would like to delete those on April 4.

How do we do that ?

[Patrick]

What makes you think these are "duplicates"? The spd engine stores data in junks and it's very well possible that these junks get created at different times.

Consider below sample code:

libname test spde 'c:\temp\out' partsize=16M;
options compress=no;

data work.source;
  length var $1024;
  do k=1 to 16;
    do i=1 to 1024;
      output;
    end;
  end;
  drop i k;
run;

proc append base=test.target data=work.source;
run;quit;

data _null_;
  call sleep(60,1);
run;

proc append base=test.target data=work.source;
run;quit;

proc contents data=test.target;
run;

libname test clear;

I've defined the libname with the spd engine to create 16 megabyte junks.

The source table work.source is more or less 16 megabyte. I then append the work table twice to library test but wait one second between the two append operations. 

You can see in below screenshot that the two junks have been created/modified at different times. They belong to the same table with the spd engine and if you delete one of these junks (files) then you will corrupt the table.

Use Proc Contents to show you when the table has last been modified. This date should be the same than the max date of your junks.

If you really believe that there are junks that are not part of your table (unlikely) then one option for a clean-up:

1. Define another spde libname that points to different folders

2. Use Proc Datasets/Copy with Move. Any junk that remains after this operation in the source folders is then something can delete 

3. Use OS commands or SAS function fdelete to remove any file that you've identified as obsolete.

Btw: I believe the correct terminology for "junks" would be "buckets".