sometimes limits observations in data view

[goat]

when no limit set. is this by design?

[Doc_Duke]

Please be more specific. What data source? What version of EGuide?

I have noticed that SAS datasets come into the data view in 'chunks'. I suspect that is both for performance (to view the beginning of the data quickly) and memory utilization (EGuide puts data into memory whereas SAS processes one record at a time).

[goat]

sure thing. I'm running eg 4.2 and these are native sas data sets on our unix server. sometimes, for no reason, sas decides to limit the number of observations to view. what's even more frustrating is that right clicking the data set to view properties -> rows shows the filtered count. I have to slog thru the log to find out the real number of rows

thanks for your help here!

[nrose]

Hi,

I have noticed the same problem. I am now using 4.3 and have not noticed this quirk so far.

Nick

[goat]

fingers crossed. thanks

[ChrisHemedinger]

The reason is that EG cannot determine the number of records that would be yielded by a VIEW. The information is not part of the VIEW definition (as it is for SAS data sets), and it could be quite expensive to count, since it requires a query to process it.

So the behavior you see is by design. Note that it affects just the data viewer and properties windows. When you run a task or query, the SAS session doing the work has access to the full set of records that are contained in the VIEW.

Chris

[goat]

allow me to clarify. sas will intermittently choose for me a row limit of 10,000. Checking the properties will show 10,000. I have no such limit set

thanks

[ChrisHemedinger]

Yes, EG does set that OBS= limit when viewing DBMS tables. It's done to improve performance, and it can make a big difference.

The OBS= is in effect only for the data viewer: any tasks or queries are run against the full data table.

There is an option in Tools->Options->Data Performance that you can adjust to change this behavior: specify a different limit or remove the limit altogether.

Chris

[goat]

by DBMS do you mean non SAS data sets?

[ChrisHemedinger]

Yes -- such as Oracle tables, DB2, Teradata, etc. Anything that you access via a libname with SAS/Access.

Chris

[goat]

Understood. Problem is that SAS exhibits said behavior on SAS data sets....not DBMS tables

[twocanbazza]

We also see the same behaviour on SAS data sets.

[ChrisHemedinger]

Do you know whether SAS is accessing the data via the META libname engine (instead of using the BASE or V9 engine directly)? That might have an effect on how EG interprets the style of access for this data source.

If you access the data via the META engine, that's a layer of indirection that sort of "hides" the location and data mechanism from your SAS process, and thus from EG.

See http://support.sas.com/techsup/technote/ts788.pdf for more info.

Chris

[goat]

sas is accessing the same library at all times. this behavior is intermittent. Sometimes sas exposes all 82 million records, sometimes it decides to limit to 10k. no changes in settings or libraries.

[twocanbazza]

In our case it is seen (but not limited to) some sas datasets in WORK so shouldn't be a meta lib. And as with other comments in this thread is intermittent.

Barry