topic Re: Read in files from list in other file dynamically in SAS Programming

Read in files from list in other file dynamically

amar961 — Mon, 06 Jun 2022 20:29:46 GMT

I am reading in the following dataset:

  data list;
      set project.daily_list;
    run;

This gives:

I want to read in, for each Date, a file in the NBBO column.

For example, for 20140102:

data X;

set nbbo.NBBOM_20140102;

run;

With X, I will do some operations. Then I want to save X (see below).

However, I want to do this for each nbbo file. In Rstudio I'd just use a for loop. But I am stuck on how to proceed in SAS. Please help!

data project.final_nbbo_20140102;set X;run;

Re: Read in files from list in other file dynamically

Reeza — Mon, 06 Jun 2022 20:37:23 GMT

Same clarifications required as on SO.

Set them into one data set?

Individual one each?

I'm assuming you don't want them all called X? Show what the final code should look like please.

Re: Read in files from list in other file dynamically

amar961 — Mon, 06 Jun 2022 20:43:03 GMT

Hi Reeza,

The files are from an external server, from which I want to download the data. The list data contains the names of the specific files I would like to download.

Since I can't alter the data on the external server, I can't set them into one data set. Furthermore, the files are 2gb each. Hence, I want to read them one-by-one (I don't know how), do some operations to get what I need (don't need help with that), and how to save them by giving them dynamic names (I don't know).

Re: Read in files from list in other file dynamically

Reeza — Mon, 06 Jun 2022 21:00:04 GMT

@amar961 wrote:

Since I can't alter the data on the external server, I can't set them into one data set.

Not sure what makes you think that, but yes you can.

@amar961 wrote:

Hence, I want to read them one-by-one (I don't know how), do some operations to get what I need (don't need help with that), and how to save them by giving them dynamic names (I don't know).

Then you use the approach I outlined in my second link. You first create a working program. Then you convert it to a macro - think R function/package. Then call the macro/package for each data set listed.

The data set loops itself automatically so as you run through the data set, it'll call the function once for each data set using CALL EXECUTE.

It starts with working code first for the process then determine how to generalize it.

https://github.com/statgeek/SAS-Tutorials/blob/master/Turning%20a%20program%20into%20a%20macro.md