import data set, process and output file then repeat in a loop

[tjc87]

Hi,

*update: what I need SAS to execute is an increment loop for 'output001' since the rest can be overwritten. This works if I can use the find and replace all option from ctrl + h then manually increment output001 to output002 but I would be doing this 100 times.

I have a loop I need to iterate 100 times, at the end of the loop I need to save the output to a .txt file.

Next I need to find all places where the two strings "response1" and "output001.txt" are located, increment the ending number by 1, complete the loop then save the output as an incremented file name. The following (4) lines of code are from various parts of the code that need to be changed.

(1) %let response1 = 'C:\Documents\F2\output001.txt';  /*the numbers in response1 and output001 needs to incremented up to 100 sets*/

(2) filename respt &response1;     /*the numbers in response1  needs to incremented each time, up to 100 sets*/

(3) data response1;     /*the numbers in response1 needs to incremented each time, up to 100 sets*/

(4) if _n_ = 1 then set parameters; set response1;    /*the numbers in response1  needs to incremented each time, up to 100 sets*/

After these have completed I need to save the output then repeat until I've done this with 100 data steps.

My code to save the data is the following:

data finish; set finalout;                                       /* these can be overwritten on each iteration, no changes need */

file "C:\Documents\F2\output001.txt";    /*  output001.txt needs to be incremented each time (output002.tx ...... output100.txt) */

put @1 codex 4. @10 error 6.;                      /*no changes need */

run;

Thanks for any help or direction to an efficient approach!

[Tom]

Do you really need to create the macro variables RESPONSE1, REPSONSE2, .... 100?  Or are you only using them to create the fileref RESPT?  Why are you pointing step (2) and the final step to the exact some filename?  Are you overwriting your input files?

Do you really need to create the datasets RESPONSE1, REPSONSE2, .... 100?

Or do you only really need to create the output files?

If not you could just use the same name for that dataset on every loop.  Just like you are using the same name for FINALOUT dataset.

Use the Z format to create the three digit numbers in the output filenames.

Make sure to use double quotes instead of single quotes around your filenames if you want to have macro variables expand to become part of the filename.

Make sure to use && when you want to use a suffix macro variable to generate part of the name of a macro variable. The macro processor will replace the && with & and then reprocess the resulting string to expand more macro variable references.

Make sure to use period to mark the end of your macro variable name when you are using it in the middle of a longer strings so that SAS knows where the macro variable name ends and the constant text begins.

%do i=1 %to 100;
%let j=%sysfunc(putn(&i,z3));
%let response&i. = "C:\Documents\F2\output&j..txt"; 
filename respt &&response&i.;
data response&i.;
  if _n_ = 1 then set parameters;
  set response&i.;

data finish;
  set finalout;
  file "C:\Documents\F2\output&j..txt";
  put @1 codex 4. @10 error 6.; 
run;
%end ;

[Reeza]

Sounds like a simple macro loop that @Tom has illustrated, though there's no mention of macros in your question so I'm not sure if you're familiar with the concept. Basically it's a code generation method. 

But there are also lots of ways to do this without macros, and in general, it's generally considered best to avoid macros until you're well versed in base functionality. You can use the filevar option for example to read/write multiple text files at once, in addition to BY group processing to run multiple models at once. 

In general, based on your description of the question, take your code and replace the number with an &i or &index if you need the leading zeroes.  Here's a quick demo of how that can work. 

%macro loop (n=);

%do i=1 %to 100;

    %let index=%sysfunc(putn(&i, Z3.));

    %put &i.;
    %put &index;
    %*used in line with text;
    %put output&index;

%end;

%mend;

%loop(n=100);

You would add your code in the middle of the %DO/%END loop. 

Although you don't need the intermediary tables I would highly recommend deleting them in between and cleaning up at the end of a loop so that the next iteration doesn't inadvertently use the 'leftovers'. And you really don't want to have 5 temp sets indexed from 1 to 100 lying around because that's 500 data sets. I use PROC DATASETS before the the %END to remove the temporary data sets. 

---

@tjc87 wrote:

Hi,

*update: what I need SAS to execute is an increment loop for 'output001' since the rest can be overwritten. This works if I can use the find and replace all option from ctrl + h then manually increment output001 to output002 but I would be doing this 100 times.

I have a loop I need to iterate 100 times, at the end of the loop I need to save the output to a .txt file.

Next I need to find all places where the two strings "response1" and "output001.txt" are located, increment the ending number by 1, complete the loop then save the output as an incremented file name. The following (4) lines of code are from various parts of the code that need to be changed.

(1) %let response1 = 'C:\Documents\F2\output001.txt';  /*the numbers in response1 and output001 needs to incremented up to 100 sets*/

(2) filename respt &response1;     /*the numbers in response1  needs to incremented each time, up to 100 sets*/

(3) data response1;     /*the numbers in response1 needs to incremented each time, up to 100 sets*/

(4) if _n_ = 1 then set parameters; set response1;    /*the numbers in response1  needs to incremented each time, up to 100 sets*/

After these have completed I need to save the output then repeat until I've done this with 100 data steps.

My code to save the data is the following:

data finish; set finalout;                                       /* these can be overwritten on each iteration, no changes need */

file "C:\Documents\F2\output001.txt";    /*  output001.txt needs to be incremented each time (output002.tx ...... output100.txt) */

@put @1 codex 4. @10 error 6.;                      /*no changes need */

run;

Thanks for any help or direction to an efficient approach!

---