topic Re: Combine *.sas7bdat files based on a reference file in SAS Programming

Combine *.sas7bdat files based on a reference file

Abhi2008 — Thu, 30 Apr 2015 16:54:00 GMT

Hi All,

We have multiple sas7bdat files saved on a network drive and a reference file which lists the file names that need to be combined.

How can I do put them together using data steps?

Thank you for your time.

Re: Combine *.sas7bdat files based on a reference file

Reeza — Thu, 30 Apr 2015 17:17:11 GMT

Read the list of files from the reference file into a macro variable, if it's less than 64K characters and include that in your set statement.

proc sql noprint;

select name into :ref_list separated by " " from reference_list;

quit;

data want;

set & ref_list;

run;

Re: Combine *.sas7bdat files based on a reference file

Abhi2008 — Tue, 05 May 2015 19:44:14 GMT

And we need this job to run on weekly basis to check for new files.can you please list out the software that is required and the process. I 'm pretty new with SAS tool.

Thanks a lot for your help

Re: Combine *.sas7bdat files based on a reference file

Patrick — Tue, 05 May 2015 23:02:12 GMT

"Check on a weekly basis" sounds like you need to schedule your job. If it's just "run weekly" then any scheduler will do (Cron, Task Scheduler,....).

"check for new files": What identifies a new file? Safest would be you maintain a list of already processed files and then compare this list with your control file.

You seem to have a process where you add some weekly snapshots to a "analytical base table". Assuming this table is a SAS data set and if the structure of the weekly files is stable (same columns with same attributes) then you should eventually consider to use Proc Append instead of a data step. This would avoid processing ALL the data every single week.

An alternative approach: You could also create a view over the individual SAS files. This way you would only need to re-create the view once a week including all files available.