DATA Step, Macro, Functions and more

batch process data files with contionout file names

Accepted Solution Solved
Reply
Super Contributor
Posts: 312
Accepted Solution

batch process data files with contionout file names

I have 12 files, file name like "xxxx201501.csv", "xxxx201502.csv", ... "xxxx201512.csv"

 

if I want to do the same opration on them, can I use a loop and macro varaible to do it?

 

Thanks


Accepted Solutions
Solution
‎01-20-2016 04:39 PM
Super User
Posts: 5,083

Re: batch process data files with contionout file names

OK, given that you can get the program to work for 1 file, here is a way to write the loop.

 

%macro run_me;

 

%local n n2;

%do n=1 %to 12;

   %let n2 = %sysfunc(putn(&n, z2));

   data output&n2;

   infile "xxx2015&n2";

   ...

   run;

%end;

 

%mend run_me;

 

Clearly, it would be possible to add other parameters such as names that replace "output" and "xxx".

 

It's your choice whether you want to name the outputs with or without leading zeros:  output&n2 vs. output&n

 

Good luck.

View solution in original post


All Replies
Super User
Posts: 17,837

Re: batch process data files with contionout file names

Yes, look into Call Execute.

Super User
Posts: 5,083

Re: batch process data files with contionout file names

What do you already know how to do with these 12 names?

 

  • Issue a system command to display the names?
  • Get the names of the files into a text file?
  • Get the names of the files into a SAS data set?
  • Nothing?  (In that case, we might need to know the operating system.)

Yes, you certainly can create a macro and loop through the names.  But depending on what you want to do with them, a macro might not even be necessary.

Super Contributor
Posts: 312

Re: batch process data files with contionout file names

for example, I want to read in these 12 files and output 12 new files:
data work.datast01;
infile "xxxx201501.csv" ...;




Super User
Posts: 5,083

Re: batch process data files with contionout file names

If macro language could get you the right INFILE statements, would you know how to write the INPUT statement?

 

Is your ultimate goal 12 separate SAS data sets, or 1 large SAS data set combining all the data from the 12 files?

Super Contributor
Posts: 312

Re: batch process data files with contionout file names

I want to have 12 output data sets

I was thinking about a do loop, but I do not know how to replace the 01, 02, ... 12 with the count variable
Super User
Posts: 5,083

Re: batch process data files with contionout file names

OK.  Here are two specific tasks you will need to complete in order to take this further.

 

1. What operating system command can take the names of all 12 files and put the names into a text file?  What does the text file look like?

 

2. What program would it take to read in a single one of the files, and create a SAS data set from that single file?  No macro language is involved.  Hard-code everything needed.

 

 

Super Contributor
Posts: 312

Re: batch process data files with contionout file names

[ Edited ]

The code below works. But only for files 01-09.

Is there a way to include 10, 11 and 12 in the same macro?

 

%macro run_me;
%let n=1;

%do %while (&n <= 9);
%let filein = xxxx20150&n..csv;

data output&n;
infile &filein;
<data step-->
run;

%let n=%eval(&n+1);
%end;
%mend;

 

Super User
Posts: 5,083

Re: batch process data files with contionout file names

A loop like that is possible.

 

Do you know what the rest of the DATA step would look like?

 

Do all the .csv files have the same structure?

Super Contributor
Posts: 312

Re: batch process data files with contionout file names

[ Edited ]

yes, they have the same structure. It is for the same data set in different months.

 

I inserted my code which was used to read a single file, and it worked. Now I need to think about how to include 10, 11 and 12 three files.

Solution
‎01-20-2016 04:39 PM
Super User
Posts: 5,083

Re: batch process data files with contionout file names

OK, given that you can get the program to work for 1 file, here is a way to write the loop.

 

%macro run_me;

 

%local n n2;

%do n=1 %to 12;

   %let n2 = %sysfunc(putn(&n, z2));

   data output&n2;

   infile "xxx2015&n2";

   ...

   run;

%end;

 

%mend run_me;

 

Clearly, it would be possible to add other parameters such as names that replace "output" and "xxx".

 

It's your choice whether you want to name the outputs with or without leading zeros:  output&n2 vs. output&n

 

Good luck.

Super Contributor
Posts: 312

Re: batch process data files with contionout file names

%sysfunc(putN(&n,z2) is the thing I was looking for.

Thank you very much!
☑ This topic is SOLVED.

Need further help from the community? Please ask a new question.

Discussion stats
  • 11 replies
  • 327 views
  • 1 like
  • 3 in conversation