topic How to Input All Files in a Folder? in SAS Programming

How to Input All Files in a Folder?

jjames1 — Wed, 26 Jul 2017 19:18:17 GMT

I have a question that I suspect has an answer, but I have seached online for quite a long time and have yet to find one solution.

Suppose I have a folder called "MySasFolder" and inside MySasFolder I have N number of files: "a.xml", "b.xml", "c.xml","d.xml" etc. and so on.

I have to read this data into a SAS dataset.

Is there a way that SAS can loop through all files in a folder and input them into one SAS dataset without me having to tell SAS the names of each file ?

When I input each file into the SAS dataset I also need to keep the name of each file as a variable as well.

Any help would be much appreciated!!!

Thanks,

James

Re: How to Input All Files in a Folder?

Reeza — Wed, 26 Jul 2017 19:27:25 GMT

Are all the XMLs formatted the same way? They have the same structure?

Do you have a process built to import one file to start with, thats what you need first. Then you can create a list of files and call the macro using CALL EXECUTE.

I have an example here, but uses XLSX files:

https://github.com/statgeek/SAS-Tutorials/blob/master/Import_all_files_one_type

You can replace my PROC IMPORT with your import procedure.

Re: How to Input All Files in a Folder?

jjames1 — Wed, 26 Jul 2017 19:52:52 GMT

No the xmls are of different structure.

But my SAS code is able to handle all different types of XMLs that I have.

But all these cases I manually input my xml filename into a macro and then do my process.

For example if I need to read ABC.xml I do the following

%let variable= ABC

filename output temp;
libname output xmlv2 "/home/test/&variable..xml" automap=replace xmlmap=output;
proc copy in=output out=work noclone;

In my case I want to read from a folder that has thousands of xml files

So I know that this approach is not ideal.

Re: How to Input All Files in a Folder?

Reeza — Wed, 26 Jul 2017 20:01:07 GMT

The idea is still the same. Wrap your current code into a macro, in my example it's called IMPORT_FILE. The other macro LIST_FILES (first one) creates a list of files. The last data step then uses the file paths to import the files.

I updated some comments in the program that may help you work through it on your side.