Data step on many files with unique names and different variables

[NewUsrStat]

Hi guys, 

I have a folder where many files *.sas7bdat are stored. The files have different content in terms of variables and not a common prefix or suffix for mapping. So, they have unique names. 

I have to perform the following in order to apply formats from a catalogue. 

%let path=\...\;
libname mylib "&path.\myfiles";

options fmtsearch = (mylib .formats work);

data mydata; 
set mylib.mydatainput;run;

Formats and data are in the same folder (mylib). 

Since there are many files I would like not to do this job by reading file by file in data step. 

I tried using filename pipe ... but without success because files were not find. 

Can anyone help me please? 

[Tom · Accepted solution]

I don't see what it is you are trying to accomplish.

Changing the FMTSEARCH option will tell SAS how to find the format definitions.  That should be all you need to do. 

But if you might need to do this for multiple libraries it might be best to remember the original setting of the option so you can reset it afterwards;

%let old_fmtsearch=%sysfunc(getoption(fmtsearch));
libname mylib "mypath";
options insert=fmtsearch=(mylib.formats);
....
options fmtsearch=&old_fmtsearch;

The data step you posted is just making a copy of the dataset without doing anything at all about formats. 

You could do that with PROC COPY without needing to specify the datasets.

proc copy inlib=mylib outlib=work mt=data;
run;

[NewUsrStat]

Thank you very much for your suggestions. I did not use proc copy because, contrary to the data step, it does not perform any data modification, which in this case is to apply data formats.

[Tom]

How do you know what formats to apply?

You did not include any example code that was applying formats to variables in your original post.

[NewUsrStat]

I really do not know because only the catalogue was passed to me and the instructions you read were given. I have no access to other codes.

[Tom]

Use PROC CONTENTS to look at the datasets.

Do the variables have any user defined formats attached to them?

Use PROC FORMAT to look at the format catalog.

Are the formats that exist int eh format catalog the same as those attached to the vairables?

If both are YES then all you need to do is add the format catalog to the FMTSEARCH option and SAS will use those formats when displaying the variable's values.

If both (or either) is NO then you will need to figure it out yourself.

You might look at the set of values the formats can decode and then compare those to the set of values that appear in the dataset for the different variables.  For example say you have format named YESNO that displays 1 as "YES" and 0 as "NO".  And you have variable named PRESENT that only has the values 0 and 1.  In that case you might deduce (quess) that you should attach the YESNO format to PRESENT.

[ballardw]

---

@NewUsrStat wrote:
I really do not know because only the catalogue was passed to me and the instructions you read were given. I have no access to other codes.

---

There is a step that you may want to add to this process. SAS Catalogs are usually version dependent. That means that it is possible that an upgrade to SAS will make the current catalog unreadable. Once you have that catalog responding to the FMTSEARCH option you may want to us Proc format to create a data set of the format definitions using the CNTLOUT option. Then at a later date you can rebuild the catalog for a different version using Proc Format and the CNTLIN option pointing to that data set. Having the data set may also be helpful to answer questions about the actual behavior of the formats.

Proc format library=somelib cntlout=alib.formatcontrol;
run;

For example will read the catalog named FORMATS, the default name for a format catalog, in the library that has been defined on your system as Somelib and write the contents to a data set named Formatcontrol in the library defined on your system named Alib.  You likely want to write the data set into library other than work for later use. The exact library is up to you.

To recreate the formats use:

proc format library=thatlib cntlin=alib.formatcontrol;
run;

The above code will create the formats in the library named Thatlib, which would need to be added to the FMTSEARCH path to be usable.

Suggestion: I would provide such a control data set and the proc format code instead of catalogs. That way things like operation system dependencies and versions are minimized.