topic Re: How to edit a format file in SAS Programming

How to edit a format file

RobertWF1 — Thu, 06 Oct 2022 15:41:48 GMT

I'd like to edit an ICD-10 diagnosis category format file by adding new categories.

I'm running code written by a former coworker and was unable to find the original proc format code that created the format catalog.

However, running the following code:

libname fmt "D:\Project_Files\395_Matched_Cohort_Study\engaged_performance";
options obs=max compress=yes fmtsearch=(fmt);

proc format library=fmt cntlout=FormatDef;
run;

outputted a copy of the format dataset FormatDef into my work folder:

Can I edit this file by adding new lines to the dataset, then save as fmt.FormatDef (or fmt.FormatDef_new)?

Re: How to edit a format file

Tom — Thu, 06 Oct 2022 15:59:38 GMT

Yes. But I wouldn't "edit" it. I would make a new version that has changes.

EDIT implies that you somehow open the file like a spreadsheet or a Word processor file and type in new things. UPDATE implies a piece of code that you run. For example you might have the new code/decode pairs in another dataset and so the update process will be something like a merge or concatenation of the two datasets.

Re: How to edit a format file

ballardw — Thu, 06 Oct 2022 16:07:17 GMT

Snarky short answer: Yes.

The approach I would take would be to extract the format(s) of interest into a new data set to reduce issues.

You do not describe what changes you need. If you are ADDING new values then you would provide such things as the FMTNAME, Type, as a minimum new Start and Label values. Best would likely be to copy the other variables as well.

These new values could be done with data step code then appended to the extract for the format data above.

I would test the format by using Library=work when using that data set with the Cntlin option to create the format. That way you don't overwrite what is currently working until actually ready to. Create a test data set with a combination of values to test some of the old format elements and ALL of the new ones and then display with data step put statements or Proc Print to see if they work as expected.

Re: How to edit a format file

Reeza — Thu, 06 Oct 2022 16:19:36 GMT

libname fmt "D:\Project_Files\395_Matched_Cohort_Study\engaged_performance";
options obs=max compress=yes fmtsearch=(fmt);

proc format library=fmt cntlout=FormatDef;
run;

*change records;
data formatDef_modified;
set formatDef;
if start=7804 then do;
start=7806;
end=start;
label='New Label';
end;
run;

*add records;
data formatDef_added;
set formatDef_modified newFormats;
run;

*create new format;
proc format cntlin=formatDef_added;
run;

*save fmt for later usage;
data fmt.ICD10fmt_v02;
set formatDef_added;
run;

Probably something along the lines of what I'd do.

@RobertWF1 wrote:

I'd like to edit an ICD-10 diagnosis category format file by adding new categories.

I'm running code written by a former coworker and was unable to find the original proc format code that created the format catalog.

However, running the following code:
libname fmt "D:\Project_Files\395_Matched_Cohort_Study\engaged_performance";
options obs=max compress=yes fmtsearch=(fmt);

proc format library=fmt cntlout=FormatDef;
run;
outputted a copy of the format dataset FormatDef into my work folder:

Can I edit this file by adding new lines to the dataset, then save as fmt.FormatDef (or fmt.FormatDef_new)?

Re: How to edit a format file

RobertWF1 — Fri, 07 Oct 2022 19:17:05 GMT

Thanks Reeza!

Follow-up question - I'm not certain how I should be saving my edited format file.

Say I want to save the format file to a new library named out. Should I code as:

*create new format;
proc format cntlin=formatDef_added;
run;

*create new format;
proc format library=out cntlin=formatDef_added;
run;

*create new format;
proc format cntlin=out.formatDef_added;
run;

Then when I attempt to retrieve my new format file to make certain the updates were added, I'm still seeing the old format file.

/* Output formats to check updates */
proc format library=out cntlout=FormatDef_added;
run;

and the new diagnosis format categories aren't being applied when I run a test:

options obs=max compress=yes fmtsearch=(out);
data test;
input icd_version diagnosis $;
cards;
10 F18280
10 F1927
10 I69169
10 C8376
;
run;
data test;
set test;
if icd_version=9 then elix=put(diagnosis,$RCOM9FMT.);
else if icd_version=10 then elix=put(diagnosis,$RCOM10FMT.);
if elix ne '';
cnt=1;
run;

Re: How to edit a format file

Reeza — Fri, 07 Oct 2022 20:01:46 GMT

You can have formats with the same name in different libraries.

The FMTSEARCH option tells SAS what order to search the libraries.

Work is default, ahead of your created library.

If you want to specify the formats in your library ahead of work, you need to tell SAS.

Here's an example of how that works. Note the default behaviour and how it changes with the different FMTSEARCH options.


*create library to store format catalog;
libname demo '/home/fkhurshed/Demo1';

proc format;
value age_fmt 
  low - 12 = 'Child'
  12-13 = 'Pre-Teen'
  13-16 = 'Teen'
  16-high = 'Young Adult';
run;

proc format lib=demo;
value age_fmt 
  low - 12 = '0 to 12 years old'
  13-15 = '13 to 15 years old'
  16-high = '16+ years old';
run;

title 'Default work library format';
proc print data=sashelp.class;
var age;
format age age_fmt.;
run;

title 'Using format from demo library';
*specify to use the format from the demo library first then check work;
options fmtsearch=(demo work);
proc print data=sashelp.class;
var age;
format age age_fmt.;
run;

title 'Using format from work library';
*specify to use the work library;
options fmtsearch = (work);
proc print data=sashelp.class;
var age;
format age age_fmt.;
run;

Re: How to edit a format file

RobertWF1 — Mon, 10 Oct 2022 15:20:35 GMT

Thanks Reeza, understood - but what am I doing wrong here when I try to edit an existing format and then save it as a new library catalog? My new format should have 4,448 records but when I output the new format file it only contains 43 records:

/* Output old formats to work folder */
options fmtsearch=(fmt);
proc format library=fmt cntlout=FormatDef;
run;
/*
NOTE: The data set WORK.FORMATDEF has 3728 observations and 21 variables.
*/

/* Add new format records to new library (fmt2) */
data fmt2.FormatDef_BH;
set FormatDef out.format_new;
run;
/*
NOTE: There were 3728 observations read from the data set WORK.FORMATDEF.
NOTE: There were 720 observations read from the data set OUT.FORMAT_NEW.
NOTE: The data set FMT2.FORMATDEF_BH has 4448 observations and 22 variables.
*/

/* Assign new format library (to prevent stacking old & new formats). */
options fmtsearch=(fmt2);

/* Create new format */
proc format cntlin=fmt2.FormatDef_BH;
run;
/*
24         proc format cntlin=fmt2.FormatDef_BH;
NOTE: Format $RCOM10FMT is already on the library WORK.FORMATS.
NOTE: Format $RCOM10FMT has been output.
NOTE: Format $RCOM9FMT is already on the library WORK.FORMATS.
NOTE: Format $RCOM9FMT has been output.
25         run;

NOTE: PROCEDURE FORMAT used (Total process time):
      real time           0.00 seconds
      cpu time            0.00 seconds
      
NOTE: There were 43 observations read from the data set FMT2.FORMATDEF_BH.
*/

/* Options statement for library with new format */
options fmtsearch=(fmt2);

/* Output formats to check updates */
proc format cntlout=FormatDef_BH;
run;
/*
NOTE: PROCEDURE FORMAT used (Total process time):
      real time           0.03 seconds
      cpu time            0.01 seconds
      
NOTE: The data set WORK.FORMATDEF_BH has 43 observations and 21 variables.
*/

Re: How to edit a format file

Tom — Mon, 10 Oct 2022 17:14:58 GMT

You seem to have pulled some lines from the SAS log and pasted them into your program. Can you instead show the actual SAS log?

NOTE that If you don't tell PROC FORMAT what catalog to use it will default to WORK.FORMATS.

So you build FormatDef from FMT.FORMATS catalog.

You then build the dataset fmt2.FormatDef_BH from that and out.format_new.

You then try to convert that to catalog entries in WORK.FORMATS catalog.

It seems to have read only 43 observations instead of all of them.

Is perhaps the data not sorted properly?

Add a proc sort step after the concatenation step.

Re: How to edit a format file

Reeza — Mon, 10 Oct 2022 18:33:13 GMT

NOTE: Format $RCOM10FMT is already on the library WORK.FORMATS.

You're still creating the formats in the WORK library because the PROC FORMAT doesn't have a lib name as to where you're creating the formats.

But I'm not sure I understand why only 43 records are being noted as read....

FMTSEARCH affects the formats when you use them, see the proc print steps in the example.

LIB on the PROC FORMAT controls where the formats are created. With your code you're still creating them in the work library.

As Tom said, please post the full log, that's a modified version, perhaps the full log has the data needed to debug the issue.

/* Output old formats to work folder */
options fmtsearch=(fmt);
proc format library=fmt cntlout=FormatDef;
run;


/* Add new format records to new library (fmt2) */
data fmt2.FormatDef_BH;
set FormatDef out.format_new;
run;


/* Assign new format library (to prevent stacking old & new formats). */
options fmtsearch=(fmt2);

/* Create new format */
proc format library =fmt2 cntlin=fmt2.FormatDef_BH;
run;


/* Options statement for library with new format */
options fmtsearch=(fmt2);

/* Output formats to check updates */
proc format library = fmt2 cntlout=FormatDef_BH;
run;

Re: How to edit a format file

Reeza — Mon, 10 Oct 2022 18:47:55 GMT

*create library to store format, catalog is formats;
libname demo '/home/fkhurshed/Demo1';

*create format;
proc format lib=work;
value age_fmt 
  low - 12 = 'Child'
  12-13 = 'Pre-Teen'
  13-16 = 'Teen'
  16-18 = 'Young Adult';
run;

*output back to data set;
proc format cntlout=age_fmt_original;
run;

*create a new data set with an adult age group;
data add_adult;
fmtname = 'AGE_FMT';
type = 'N';
start = '18';
end = '99';
label= 'Adult';
output;
run;

*add adult to the age category;
data new_fmt_dset;
set age_fmt_original add_adult;
run;

*create new age format in different library;
proc format lib=demo cntlin=new_fmt_dset;
run;

*output back to verify it;
proc format lib=demo cntlout=age_fmt_new;
select AGE_FMT;
run;

*create sample data set to test application;
data ages;
do age=0 to 45 by 5;
output;
end;
run;

*test application;
options fmtsearch=(work);
proc print data=ages;
format age age_fmt.;
run;

options fmtsearch=(demo work);
proc print data=ages;
format age age_fmt.;
run;

This is analogous to your example and shows it works. You may need more details to help debug your issue.

Re: How to edit a format file

AllanBowe — Wed, 12 Oct 2022 08:10:43 GMT

If you'd like the "point and click" ability to edit formats in a catalog, you are welcome to try out this tool (built by my team): https://docs.datacontroller.io/formats/

It's free for up to 5 users, you can edit formats directly, upload from excel, download in various formats, run filters, and even configure jobs to run after a format is updated.