BookmarkSubscribeRSS Feed
jjames1
Fluorite | Level 6

Hello All,

 

I have a temporary folder from where I used read the different xml files from SAS.

 

Now I am trying to do something more complicated.

 

I want to store only 10K files at a single time in my temporary folder and once I am done with reading the first 10K files from SAS, I want to delete them from the temporary folder and store the next 10K files inside the temporary and so on.

 

Since I am a beginner to SAS, I would also like to know know can I dynamically read the files one by one as all my files are of different names.

 

Please help

 

Thanks in advance!

3 REPLIES 3
RW9
Diamond | Level 26 RW9
Diamond | Level 26

For point 1 - how are the files getting into the folder - that is where you should be limiting things if there are too many in the folder.

 

For point 2 - get a directory listing - many examples on here for this please search, for example:

https://communities.sas.com/t5/General-SAS-Programming/read-a-list-of-file-names-with-blanks-by-file...

 

You can then use that dir list to either generate code, or call macros etc.

Astounding
PROC Star

Here's the general idea, just to see if this fits what you are looking for.

 

1. Put 10,000 files into your temporary folder.  I have to assume you can do that already.

2. SAS can issue a system command to get a list of all the files in the folder.  (See the X statement, and the %SYSEXEC statement.)  Route that list to a file. The format of that system command changes, depending on the operating system.

3. Read the file to get a SAS data set holding the names of all files in your folder.

4. Process each file (possibly using macro language, depends on what "processing" means).

5. Issue another X or %SYSEXEC command to delete all files in the folder.

 

Repeat as needed.

ballardw
Super User

I have an addition to @Astounding's outline, before permanently deleting files perform some verification that you may have the data correctly read/ processed.

 

If you are contemplating using Proc Import 10K times expect to spend a lot of time getting variable types and lengths aligned afterwards.

sas-innovate-2024.png

Join us for SAS Innovate April 16-19 at the Aria in Las Vegas. Bring the team and save big with our group pricing for a limited time only.

Pre-conference courses and tutorials are filling up fast and are always a sellout. Register today to reserve your seat.

 

Register now!

How to Concatenate Values

Learn how use the CAT functions in SAS to join values from multiple variables into a single value.

Find more tutorials on the SAS Users YouTube channel.

Click image to register for webinarClick image to register for webinar

Classroom Training Available!

Select SAS Training centers are offering in-person courses. View upcoming courses for:

View all other training opportunities.

Discussion stats
  • 3 replies
  • 980 views
  • 0 likes
  • 4 in conversation