Data reading problem

[Chung-Li]

Hi all, 

I face a problem, and I couldn't find better way to solve it.

Hope I can learn someting useful from you guys.

My problem is that:

In my libname folder, I have thounds of data.

Name of thses datas is basically composed by number, like

 "10002.MON", "10003.MON", ...,"90036.MON"

As you can imagine, if these names are continuous, then I will have 80035 (90036-10002+1) datas.

However, because these names are not continuous, I have only 2042 datas.

That is, "10002.MON", "10003.MON",(no 10004)"10005.MON"...

Given that there are some steps need to be done within these datas, I need to read them all.

But when I use "do loop" to read them all, it is just too inefficient!

So I want to ask if there is more efficient way to catch all the data names in a specefic libname file.

Last, here is my code:

libname DB "F:\RunningDB\Chung-Li Wu\ABPM_subgroup analysis";

%macro Read_CleanData;
    %DO i=10002 %TO 90036;
    data raw&i;
        infile "F:\BackUp\CL-Wu\Morning BP surge\raw data\ABPMAGE\MON\&i..MON"
        firstobs=9 delimiter="," truncover;
        input SBP_night mean_BP_night DBP_night HR Hour Min;
    data raw&i(drop=mean_BP_night DBP_night HR Min);set raw&i;
    data DB.raw&i;set raw&i;run;
        proc printto log=myalfa new;

    %end;
%MEND;
%Read_CleanData;

Best, 

Bill

[PGStats · Accepted solution]

Look into the possibility of using wildcard characters in the file name to read multiple files, as in

infile "F:\BackUp\CL-Wu\Morning BP surge\raw data\ABPMAGE\*.MON" filename=currentFile ... ;

source = currentFile;

[Chung-Li]

PG, 

Thank you for this!

Though I'm still trying how to apply this after I read all datas, it is really helpful!

Bill

[Reeza]

Read all at one:

https://communities.sas.com/t5/SAS-Communities-Library/How-do-I-write-a-macro-to-import-multiple-tex...

[Chung-Li]

Reeza, 

I have read your article.

The coding skill you mention is oustanding, and thank you for sharing it with us!

But somehow I'm wondering that after I input all these datas, can I still manipulate data without "do loop"?

I curious about this, because I still have tons of procedure need to do with my data.

Bill 

[Reeza]

It depends on what you're doing, but in general BY group processing is very powerful and you should look into it, if you have repetitive tasks.

http://support.sas.com/documentation/cdl/en/lrcon/69852/HTML/default/viewer.htm#n138da4gme3zb7n1nifp...

Intro to macro if you want to go down that route

http://www.ats.ucla.edu/stat/sas/seminars/sas_macros_introduction/

The last Appendix has a set of examples...Check out Example 1

http://support.sas.com/documentation/cdl/en/mcrolref/69726/HTML/default/viewer.htm#titlepage.htm