'm using the following commnad to concatenate 1000 files. But all 1000 files have header it them. How do i just merge data not header from all data files?
x %sysfunc(quote(copy "F:\Test\*monthly*.csv" "F:\Test\2017_monthlytsv")) ;
You're using OS commands there, so you need an OS command to process the files.
Are you planning to read these files into SAS or is this just a process of using SAS to execute OS commands? If you're reading them into SAS you don't need to combine it, SAS will use the wildcard to read all the files at once.
You'll probably get faster help on a forum dedicated to your OS scripting language, if that's Windows or Unix.
On UNIX, one would simply do
tail -n +2 file.txt > file.stdout
to remove the first line. Have no idea how to do it on a toybox.
The whole shell script would look like
cd /test OUTFILE=2017_monthlytsv rm $OUTFILE for FILE in *monthly*.csv do tail -n +2 $FILE >> $OUTFILE done
If you install a UNIX environment like cygwin on your Windows, you will get a lot of the useful UNIX utilities, including bash for shell scripting and commands like tail.
Thank you !
Hi,
You can append all the files very simply in one dataset - or actually read it all in in one go:
For both I had two csv files, just with:
abcd,def
1,2
This one just reads all the files and writes out one single file with all the data no headers:
data _null_; infile "s:\temp\rob\*.csv" dlm="¬"; file "s:\temp\rob\processed.csv"; input; if not(substr(_infile_,1,5)="abcd,") then put _infile_; run;
You could of course modify this to do all the processing:
data want; infile "s:\temp\rob\*.csv" dlm="¬"; file "s:\temp\rob\processed.csv"; input; if not(substr(_infile_,1,5)="abcd,") then do; a=scan(_infile_,1,","); b=scan(_infile_,2,","); output; end; run;
Are you ready for the spotlight? We're accepting content ideas for SAS Innovate 2025 to be held May 6-9 in Orlando, FL. The call is open until September 25. Read more here about why you should contribute and what is in it for you!
Learn how use the CAT functions in SAS to join values from multiple variables into a single value.
Find more tutorials on the SAS Users YouTube channel.