topic reading data file with formats defined but not format file in SAS Programming

reading data file with formats defined but not format file

emaguin — Fri, 03 Jun 2022 18:10:58 GMT

A co-worker read in multiple csv files using a data-infile command and defined an informat and a format statement for each variable, e.g., best12., best32. for numerics. Each file included numeric, string, and date-time variables. Files were concantenated and an spss data file was read and matched to the csv files. Processing ensued and a data file was saved. Now, i wish to read that created file but i get errors:

ERROR: The format $MS_CLI was not found or could not be loaded.
ERROR: The format $MS_CL1A was not found or could not be loaded.
ERROR: The format $MS_CL2A was not found or could not be loaded.
ERROR: The format $MS_CL3A was not found or could not be loaded.
ERROR: The format $MS_CL4A was not found or could not be loaded.
ERROR: The format $MS_CL5A was not found or could not be loaded.
ERROR: The format MS_STAT was not found or could not be loaded.
ERROR: The format EMASTUD was not found or could not be loaded.
ERROR: The format EMAPHON was not found or could not be loaded.

From reading Cody's book, i believe that a format file is missing/was not saved.

I'd like to know several things.

1) Is there a method/scheme to get sas to read the current data file? If yes, how.

2) Did the co-worker, an experienced, long-time sas user need to specify formats and informats?

3) The above error messages refer to specific formats; however, when i search for that format in the syntax file, i do not find those names. Conversely, why are no error messages noted for the Bestw., $w, and date-timew formats? How do i understand this?

Thanks, Gene Maguin

Re: reading data file with formats defined but not format file

Kurt_Bremser — Fri, 03 Jun 2022 18:39:38 GMT

The missing formats are custom formats created by SAS user. Formats provided by the SAS software (like BEST) are always available.

Run this:

options nofmterr;

so you can view the dataset to see the raw, unformatted values.

Re: reading data file with formats defined but not format file

Tom — Fri, 03 Jun 2022 18:51:16 GMT

BEST, $, DATE etc formats are part of the BASE SAS installation. So you do not need to define them to use them.

First thing to try is to tell SAS that it is not an ERROR if the format cannot be found, then you can at least look at the data.

options nofmterr;

Do you have the formats saved somewhere? Perhaps you just need to tell SAS where to find them. Modify the FMTSEARCH option. Either directly set it or use the APPEND= or INSERT= option to add a new catalog to the current list of catalogs to search.

The most likely source of those formats was the conversion of the SPSS file into a SAS dataset. The SPSS file probably had "data labels" and those were translated into user defined formats and the formats were attached to the variables. When you later combined the data from SPSS with the data from the CSV file you did not bother to remove the format from the variables. You probably do not need all of those formats. If the variables MS_CLI... use the same set of codes then you can probably use one format to decode all of them. Or perhaps the coded values are meaningful enough that you do not need to see the decoded values to work with the data.

Most likely if you re-convert the SPSS file into a SAS dataset you will get the formats defined again. This time either change the options used so that the formats are written to a permanent catalog instead of a WORK catalog or copy the formats from the WORK.FORMATS catalog into a permanent catalog that you can use in the future.

Note there is normally no reason to attach formats or informats to variables read from a CSV file. You do not need them with normal numbers or strings, so not need for the BEST or $ formats to be attached. (NOTE A lot of novice coders think that attaching a FORMAT or an INFORMAT to a variable somehow is defining the variable. Those statements will only define the variable as side effect when the variable was not previously defined. Use the LENGTH statement to DEFINE the variables).

Attach formats to variables whose values you want to display in a special way. Like DATE, TIME or DATETIME values that just look like random numbers to humans if not formatted.

Similarly for reading the text in the CSV into values you do not usually need to specify any informat. (Again DATE, TIME and DATETIME values are normally the exception). And even if you did need to use an informat when reading the value you could include it in the INPUT statement instead of permanently attaching it to the variable.