BookmarkSubscribeRSS Feed
🔒 This topic is solved and locked. Need further help from the community? Please sign in and ask a new question.
eduard1231
Fluorite | Level 6

Hi everyone,

 

I guess I have 3 issues. I need to read files, that have filepath in following format:

 

x1/x2/x3/x4/file 

 

So, I want to iterate over SOME x1, ALL x2, SOME x3, ALL x4. Finally when I am in x4 folder, I have 2 types of file's name in following format:

file_name_date

file_name_date.new 

 

So, from those I want to read only file_name_date files, and I want to skip all file_name_date.new.

file_name part of file is constant the only thing changes in file_name_date is the date part.

 

Then, this file is zipped. Can sas read zipped file? 

 

Finally, when I read this file and did some manipulation with it, I want to save it and then start to work with new file. When I iterate over all files in folder, i will go back from x4 level to x3 level, change folder, go to new x4 level and repeat process.

 

Sorry, I guess it is confusing. You are more than welcome to ask extra questions, I will try to explain as hard as I can

 

Thanks, 

Ed

1 ACCEPTED SOLUTION

Accepted Solutions
Tom
Super User Tom
Super User

@eduard1231 wrote:

Thanks for this part, yes, it is text files. I see how to read them now, but I am not sure how to iterate over folders I need iterate

 


I usually find it easiest to generate the full list of files into a dataset.  Usually by reading output of appropriate operating system command (dir, ls , find etc.).   There are many examples on this site for how to do that.  For example on Unix you might want to use the FIND command:

data filelist;
  infile "find /x1/" pipe truncover ;
  input filename $256. ;
run;

Then I can apply selection/exclusion logic to the file names either in that step or another step.

where scan(filename,3,'/') in ('log','saslog') ;

Once you have the list of file just use it to call the code to process one file.  So if you had a macro that processed one file. Call it %ONEFILE() you can just use a data step to generate one call for each observation in the dataset with the names of the files to process.  

data _null_;
  set filelist ;
  call execute(cats('%nrstr(%onefile)(',filename,')'));
run;

View solution in original post

3 REPLIES 3
Reeza
Super User
Are the files zipped text files or sas7bdat files? If this is text files everything you want is possible. If it's sas formatted files then I don't think it's as easy.

There's a FILENAME ZIP option that supports reading from zipped files. If you google the term, Chris Hemidinger has written some good blog posts on reading from zipped files, have you explored those posts already?
eduard1231
Fluorite | Level 6

Thanks for this part, yes, it is text files. I see how to read them now, but I am not sure how to iterate over folders I need iterate

 

Tom
Super User Tom
Super User

@eduard1231 wrote:

Thanks for this part, yes, it is text files. I see how to read them now, but I am not sure how to iterate over folders I need iterate

 


I usually find it easiest to generate the full list of files into a dataset.  Usually by reading output of appropriate operating system command (dir, ls , find etc.).   There are many examples on this site for how to do that.  For example on Unix you might want to use the FIND command:

data filelist;
  infile "find /x1/" pipe truncover ;
  input filename $256. ;
run;

Then I can apply selection/exclusion logic to the file names either in that step or another step.

where scan(filename,3,'/') in ('log','saslog') ;

Once you have the list of file just use it to call the code to process one file.  So if you had a macro that processed one file. Call it %ONEFILE() you can just use a data step to generate one call for each observation in the dataset with the names of the files to process.  

data _null_;
  set filelist ;
  call execute(cats('%nrstr(%onefile)(',filename,')'));
run;

SAS Innovate 2025: Register Now

Registration is now open for SAS Innovate 2025 , our biggest and most exciting global event of the year! Join us in Orlando, FL, May 6-9.
Sign up by Dec. 31 to get the 2024 rate of just $495.
Register now!

Mastering the WHERE Clause in PROC SQL

SAS' Charu Shankar shares her PROC SQL expertise by showing you how to master the WHERE clause using real winter weather data.

Find more tutorials on the SAS Users YouTube channel.

Discussion stats
  • 3 replies
  • 644 views
  • 0 likes
  • 3 in conversation