topic Re: Read some zipped files from some folders in New SAS User

Read some zipped files from some folders

eduard1231 — Mon, 24 Feb 2020 02:41:56 GMT

Hi everyone,

I guess I have 3 issues. I need to read files, that have filepath in following format:

x1/x2/x3/x4/file

So, I want to iterate over SOME x1, ALL x2, SOME x3, ALL x4. Finally when I am in x4 folder, I have 2 types of file's name in following format:

file_name_date

file_name_date.new

So, from those I want to read only file_name_date files, and I want to skip all file_name_date.new.

file_name part of file is constant the only thing changes in file_name_date is the date part.

Then, this file is zipped. Can sas read zipped file?

Finally, when I read this file and did some manipulation with it, I want to save it and then start to work with new file. When I iterate over all files in folder, i will go back from x4 level to x3 level, change folder, go to new x4 level and repeat process.

Sorry, I guess it is confusing. You are more than welcome to ask extra questions, I will try to explain as hard as I can

Thanks,

Re: Read some zipped files from some folders

Reeza — Mon, 24 Feb 2020 02:45:52 GMT

Are the files zipped text files or sas7bdat files? If this is text files everything you want is possible. If it's sas formatted files then I don't think it's as easy.

There's a FILENAME ZIP option that supports reading from zipped files. If you google the term, Chris Hemidinger has written some good blog posts on reading from zipped files, have you explored those posts already?

Re: Read some zipped files from some folders

eduard1231 — Mon, 24 Feb 2020 15:19:56 GMT

Thanks for this part, yes, it is text files. I see how to read them now, but I am not sure how to iterate over folders I need iterate

Re: Read some zipped files from some folders

Tom — Mon, 24 Feb 2020 16:27:40 GMT

@eduard1231 wrote:

Thanks for this part, yes, it is text files. I see how to read them now, but I am not sure how to iterate over folders I need iterate

I usually find it easiest to generate the full list of files into a dataset. Usually by reading output of appropriate operating system command (dir, ls , find etc.). There are many examples on this site for how to do that. For example on Unix you might want to use the FIND command:

data filelist;
  infile "find /x1/" pipe truncover ;
  input filename $256. ;
run;

Then I can apply selection/exclusion logic to the file names either in that step or another step.

where scan(filename,3,'/') in ('log','saslog') ;

Once you have the list of file just use it to call the code to process one file. So if you had a macro that processed one file. Call it %ONEFILE() you can just use a data step to generate one call for each observation in the dataset with the names of the files to process.

data _null_;
  set filelist ;
  call execute(cats('%nrstr(%onefile)(',filename,')'));
run;