Change dataset with loop count into append statement

[imdickson]

TOPIC: Change dataset with loop count into append statement

I have a macro that will loop and create a new dataset with a counter behind.

Code like this:

PROC IMPORT OUT=WORK.out&i  DATAFILE= "&dir/&name" 			
		/*excelout*/			
            DBMS=csv REPLACE;					
			delimiter='09'x;		
			getnames=no;		
		RUN;		

data test&i (drop=			
		%do k=1 %to &cnt;			
		&&col&k..			
		%end;			
		);	
			
		length station $10 voltage $10 year 8 month $20 transformer $10			
		Day $20 Date Time MW_Imp MW_Exp MVAR_Imp MVAR_Exp MVA Power_Factor 8;			
		format Time hhmm.;			
		set out&i. end=last;	

Currently the script will generate about 4 data sets if i have 4 external files by PROC IMPORT.

What i want is to eliminate the creation of multiple datasets but just append them into the master file. Is there a way to do so?

[PGStats]

Yes, if you read your data as text files. The infile statement can accept multiple file names. The files are simply read in sequence. Check the documentation for the infile statement.

[Astounding · Accepted solution]

You can, but you still need both the PROC IMPORT and the DATA step first.  For example, in PROC IMPORT, re-use the same data set:

proc import out=temp ......

Then in the DATA step, modify it:

data temp (drop= .... same as before ...,.);

set temp;

 ..... same as before .....;

run;

Then add it to a master data set:

proc append data=temp base=master;

run;

This change relies on the DATA step logic accurately guaranteeing the same structure for all the data sets.

[Reeza]

If your files are csv with the same structure you can read it in a single step and avoid a macro which is the better approach. PROC IMPORT may truncate records so I wouldn't recommend it either in a situation like this.