topic Re: SAS large format in SAS Programming

SAS large format

GeorgeSAS — Wed, 09 Aug 2017 16:39:35 GMT

Hello All,

How to conver a large format into SAS dataset? the SAS memsize is not enough to do this.

May I ask is there any tricky method to split the format and convert them to SAS small SAS dataset one by one?

Any suggestion will be appreciated!(without change memory size)

proc format cntout=newdataset  lib=alargefmt;
run;



*******error* insufficient memory........;

Thanks!

Re: SAS large format

Tom — Wed, 09 Aug 2017 16:46:28 GMT

@GeorgeSAS wrote:

Hello All,

How to conver a large format into SAS dataset? the SAS memsize is not enough to do this.

May I ask is there any tricky method to split the format and convert them to SAS small SAS dataset one by one?

Any suggestion will be appreciated!(without change memory size)
proc format cntout=newdataset  lib=alargefmt;
run;

*******error* insufficient memory........;
Thanks!

Does the catalog have more than one format? Does it help to use the SELECT statement to export them one by one?

Re: SAS large format

GeorgeSAS — Wed, 09 Aug 2017 16:48:28 GMT

Thank you,but only one format ,that single one is too large to be operated

Re: SAS large format

RW9 — Wed, 09 Aug 2017 16:57:35 GMT

How large are we talking here? A format is supposed to be a distinct grouping, so how has it got so big? Maybe consider using datasets rather than formats.

Re: SAS large format

Tom — Wed, 09 Aug 2017 17:00:08 GMT

Try using FMTLIB option and see if you can scrape the information from the listing output. But I doubt it will help since it will still need to load the file.
You need to figure out how to start SAS with a larger MEMSIZE setting.
- What version of SAS are you using?
- What operating system.
You might be able to contact SAS technical support and get help with converting the file, but I am not sure if they provide that type of service.

Re: SAS large format

GeorgeSAS — Wed, 09 Aug 2017 17:24:08 GMT

Thank you,
This is on mainframe ,the format is 1 GB size.and consider using datasets rather than formats is a good option but this format is from an old file.

Re: SAS large format

DaveHorne — Wed, 09 Aug 2017 18:15:52 GMT

I can't tell if that error message relates to the output data set or not, so you can limit the variables sent to cntlout to see if that helps:

options compress=yes;                                                                                                                   
proc format library=alargefmt                                                                                                           
  cntlout=newdataset(keep=start end label);                                                                                             
run;

If you don't use any ranges (i.e start and end are the same) then you could just keep start, label.

Re: SAS large format

ballardw — Wed, 09 Aug 2017 18:16:21 GMT

Are you running the format code on the mainframe?

You may be running into your available space settings for users and not a SAS limit. Disc space would be a common element limited to most users and if the catalog is thal large.

By any chance do you have the Code that created the format? It may be that running that code with cntlout would be a better option.

Re: SAS large format

GeorgeSAS — Wed, 09 Aug 2017 18:38:11 GMT

Thank you Dave,
that is a good idea. but it not work, because the keep statement happen only after all file already load into memory.
there would be a solution if can find how to use the 'keep' before all file load into memory.

And ballardw, thank you,but I don't have the code that created the format

Re: SAS large format

ballardw — Wed, 09 Aug 2017 19:28:29 GMT

Have you tried something like this:

proc format cntout=formatone lib=alargefmt;

select formatone;
run;

Where formatone is the name of one of the formats in the library?

If you also have informats you would use @informatname since it is very common to have an informat and format with the same name.

Re: SAS large format

GeorgeSAS — Wed, 09 Aug 2017 22:23:43 GMT

I tried, but just the single format is too large to be loaded.

Thanks

Re: SAS large format

Kurt_Bremser — Thu, 10 Aug 2017 06:11:14 GMT

Then you either need increased memory, or go back one further step and recreate the data that was originally used to create the format (nobody types a 1GB format by hand). With data like that, there MUST be documentation about its creation.

Re: SAS large format

Tom — Thu, 10 Aug 2017 11:38:20 GMT

Get help from your local systems team. If you really are running MVS then change the memory settings in the JCL.

Are you sure the file is really a format catalog? Can you see the list of formats in it using PROC CATALOG? Or using the DICTIONARY views?

Are you sure the file is a format catalog for the version of SAS that you are running? Format catalogs a normally not compatible across releases or operating systems.