BookmarkSubscribeRSS Feed
ehsanmath
Obsidian | Level 7

Hi,

I have the following question from all experts here and will be thankful for answer.

If you get the following error in SAS Log,

"ERROR: File MYFILES.BIGFILE contains 2G -1 observations and cannot hold more because it contains an index or an Integrity Constraint that uses an index"

then you need to set EXTENDOBSCOUNTER=YES to get rid of above error. Such an error happens when you are working with very big data sets(36112 - Managing large SAS® data sets that exceed the maximum number of observations). Actullay, sas documentation says, by specifying the EXTENDOBSCOUNTER= option when you create an output SAS data file, the resulting 32-bit file behaves like a 64-bit file regarding counters.

I have observed after setting the option "EXTENDOBSCOUNTER=YES" the performance of sas decreses i.e. SAS takes more time for the same task.

My question is:  does anybody know about this option and does it really effect the performance of SAS?

regards

ehsan

3 REPLIES 3
FriedEgg
SAS Employee

Why are you interested in using the option?

The option is really only useful in two situations:

1. You are purposely creating a sas dataset that exceed's the observation count of you current system ( setting the eoc option to yes enables the file to be output/read )

2. You are migrating a dataset that exceeds the observation limit of one of your systems that you are migrating between.  This process impacts performance itself, but not as a result of using the eoc option, but because of the CEDA processing to transcode between to two environments.

When you are using a 32 bit systems and exceeding a observation counter, it is better to switch the data engine you are using to SPDE.

I have never seen a performance impact by setting this option.

ehsanmath
Obsidian | Level 7

I think I have situation 1. (explained in your earliear post)

I am setting EXTENDOBSCOUNTER=YES in the LIBNAME statement option,  so newly created output files in the SAS library with an extended observation count. But it seems to me that this costs performance. Is it so? I am using 64-bit SAS 9.3 version on a 64-bit plateform.

FriedEgg
SAS Employee

I think that the biggest concern about performance would be to question the need to create a single file containing more that 9.2 quadrillion records.

sas-innovate-2024.png

Join us for SAS Innovate April 16-19 at the Aria in Las Vegas. Bring the team and save big with our group pricing for a limited time only.

Pre-conference courses and tutorials are filling up fast and are always a sellout. Register today to reserve your seat.

 

Register now!

How to Concatenate Values

Learn how use the CAT functions in SAS to join values from multiple variables into a single value.

Find more tutorials on the SAS Users YouTube channel.

Click image to register for webinarClick image to register for webinar

Classroom Training Available!

Select SAS Training centers are offering in-person courses. View upcoming courses for:

View all other training opportunities.

Discussion stats
  • 3 replies
  • 518 views
  • 0 likes
  • 2 in conversation