Get all the list file names in a S3 bucket

lpy0521 — Sat, 17 Jul 2021 18:36:36 GMT

Hey folks,

I have a S3 bucket that contains multiple csv files named with certain prefix and suffix, something like blabla_yymmdd.csv

I am wondering if we can get all the file names using proc s3 or any other alternatives?

Thanks in advance!

Re: Get all the list file names in a S3 bucket

Tom — Sun, 18 Jul 2021 01:05:22 GMT

You can use the LIST statement to get a list of the files with a particular prefix.

But unless they have updated PROC S3 in the last few years the only way I found to capture that into data was to redirect the SAS LOG to a file and read it back in.

filename list temp;
options nonotes nosource;
proc printto log=list new; run;
proc s3 config="&config";
list  "/&bucket/team_rwa/sas_download";
run;
proc printto log=log; run;
options notes source;

Then you can run a data step to read the new LIST file and parse out the filename size and datetime stamp.

topic Get all the list file names in a S3 bucket in SAS Programming

Get all the list file names in a S3 bucket

Re: Get all the list file names in a S3 bucket