topic Re: Data reading problem in SAS Programming

Data reading problem

Chung-Li — Thu, 22 Dec 2016 03:10:35 GMT

Hi all,

I face a problem, and I couldn't find better way to solve it.

Hope I can learn someting useful from you guys.

My problem is that:

In my libname folder, I have thounds of data.

Name of thses datas is basically composed by number, like

"10002.MON", "10003.MON", ...,"90036.MON"

As you can imagine, if these names are continuous, then I will have 80035 (90036-10002+1) datas.

However, because these names are not continuous, I have only 2042 datas.

That is, "10002.MON", "10003.MON",(no 10004)"10005.MON"...

Given that there are some steps need to be done within these datas, I need to read them all.

But when I use "do loop" to read them all, it is just too inefficient!

So I want to ask if there is more efficient way to catch all the data names in a specefic libname file.

Last, here is my code:

libname DB "F:\RunningDB\Chung-Li Wu\ABPM_subgroup analysis";

%macro Read_CleanData;
%DO i=10002 %TO 90036;
data raw&i;
infile "F:\BackUp\CL-Wu\Morning BP surge\raw data\ABPMAGE\MON\&i..MON"
firstobs=9 delimiter="," truncover;
input SBP_night mean_BP_night DBP_night HR Hour Min;
data raw&i(drop=mean_BP_night DBP_night HR Min);set raw&i;
data DB.raw&i;set raw&i;run;
proc printto log=myalfa new;

%end;
%MEND;
%Read_CleanData;

Best,

Bill

Re: Data reading problem

PGStats — Thu, 22 Dec 2016 04:11:48 GMT

Look into the possibility of using wildcard characters in the file name to read multiple files, as in

infile "F:\BackUp\CL-Wu\Morning BP surge\raw data\ABPMAGE\*.MON" filename=currentFile ... ;

source = currentFile;

Re: Data reading problem

Reeza — Thu, 22 Dec 2016 04:24:31 GMT

Read all at one:

https://communities.sas.com/t5/SAS-Communities-Library/How-do-I-write-a-macro-to-import-multiple-text-files-that-have/ta-p/223627

Re: Data reading problem

Chung-Li — Thu, 22 Dec 2016 06:50:34 GMT

PG,

Thank you for this!

Though I'm still trying how to apply this after I read all datas, it is really helpful!

Bill

Re: Data reading problem

Chung-Li — Thu, 22 Dec 2016 06:57:49 GMT

Reeza,

I have read your article.

The coding skill you mention is oustanding, and thank you for sharing it with us!

But somehow I'm wondering that after I input all these datas, can I still manipulate data without "do loop"?

I curious about this, because I still have tons of procedure need to do with my data.

Bill

Re: Data reading problem

Reeza — Thu, 22 Dec 2016 07:51:01 GMT

It depends on what you're doing, but in general BY group processing is very powerful and you should look into it, if you have repetitive tasks.

http://support.sas.com/documentation/cdl/en/lrcon/69852/HTML/default/viewer.htm#n138da4gme3zb7n1nifpfhqv7clq.htm

Intro to macro if you want to go down that route

http://www.ats.ucla.edu/stat/sas/seminars/sas_macros_introduction/

The last Appendix has a set of examples...Check out Example 1

http://support.sas.com/documentation/cdl/en/mcrolref/69726/HTML/default/viewer.htm#titlepage.htm