topic Does EXTENDOBSCOUNTER= Option Effect Performance (speed efficiency) of SAS in SAS Programming

Does EXTENDOBSCOUNTER= Option Effect Performance (speed efficiency) of SAS

ehsanmath — Wed, 10 Jul 2013 07:03:11 GMT

Hi,

I have the following question from all experts here and will be thankful for answer.

If you get the following error in SAS Log,

"ERROR: File MYFILES.BIGFILE contains 2G -1 observations and cannot hold more because it contains an index or an Integrity Constraint that uses an index"

then you need to set EXTENDOBSCOUNTER=YES to get rid of above error. Such an error happens when you are working with very big data sets(36112 - Managing large SAS® data sets that exceed the maximum number of observations). Actullay, sas documentation says, by specifying the EXTENDOBSCOUNTER= option when you create an output SAS data file, the resulting 32-bit file behaves like a 64-bit file regarding counters.

I have observed after setting the option "EXTENDOBSCOUNTER=YES" the performance of sas decreses i.e. SAS takes more time for the same task.

My question is: does anybody know about this option and does it really effect the performance of SAS?

regards

ehsan

Re: Does EXTENDOBSCOUNTER= Option Effect Performance (speed efficiency) of SAS

FriedEgg — Wed, 10 Jul 2013 15:31:37 GMT

Why are you interested in using the option?

The option is really only useful in two situations:

1. You are purposely creating a sas dataset that exceed's the observation count of you current system ( setting the eoc option to yes enables the file to be output/read )

2. You are migrating a dataset that exceeds the observation limit of one of your systems that you are migrating between. This process impacts performance itself, but not as a result of using the eoc option, but because of the CEDA processing to transcode between to two environments.

When you are using a 32 bit systems and exceeding a observation counter, it is better to switch the data engine you are using to SPDE.

I have never seen a performance impact by setting this option.

Re: Does EXTENDOBSCOUNTER= Option Effect Performance (speed efficiency) of SAS

ehsanmath — Thu, 11 Jul 2013 07:58:02 GMT

I think I have situation 1. (explained in your earliear post)

I am setting EXTENDOBSCOUNTER=YES in the LIBNAME statement option, so newly created output files in the SAS library with an extended observation count. But it seems to me that this costs performance. Is it so? I am using 64-bit SAS 9.3 version on a 64-bit plateform.

Re: Does EXTENDOBSCOUNTER= Option Effect Performance (speed efficiency) of SAS

FriedEgg — Thu, 11 Jul 2013 12:02:45 GMT

I think that the biggest concern about performance would be to question the need to create a single file containing more that 9.2 quadrillion records.