BookmarkSubscribeRSS Feed
🔒 This topic is solved and locked. Need further help from the community? Please sign in and ask a new question.
Trent
Fluorite | Level 6

Hello folks,

I'm currently testing SAS EG7.1 before all my colleagues follow suit; we're upgrading from EG5.1.

System is Win7 64-bit, using EG7.1 32-bit and Base SAS 9.4 64-bit.

I have a source SAS dataset, which is compressed, of around 40 million records and 70 columns. Using BASE engine (not SPD).

In EG5.1, the table opens when double-clicked from the process flow almost instantly.  It can be navigated around rapidly with no noticeable delays, and when I hit Ctrl+End it jumps immediately to the last record (row 40 million etc).

In EG7.1, same file, same system, same specs, but the performance has dropped by orders of magnitude.

The dataset takes upwards of 30 seconds to open, seems to be constantly reading in as I scroll around, and there is a significant delay of up to 30+ seconds to jump to end of file.


Any ideas on where this performance drop stems form would be greatly appreciated.

PS - I notice a new entry in the EG Options menu under Data > Performance:  "Maximum dimensions to display in the data grid".  Could this be related?

[edited - oops, accidental premature post!]

1 ACCEPTED SOLUTION

Accepted Solutions
CaseySmith
SAS Employee

This performance regression in the data grid was unfortunately introduced in EG 7.1.  We are aware of the issue and actively working on a solution.  It is a top priority.

 

In the interim, here are a few workarounds:

-reduce the "Maximum dimensions to display in the data grid" option in EG's Tools->Options->Data->Performance (which, yes, were added as a result), which should provide better performance, but won't allow you to view all the data.

-use the Data Explorer (File->Open->Data Exploration) instead

-continue to use EG 6.1 or earlier for opening large data in the data grid (multiple versions of EG can be installed and used side-by-side on the same machine)

 

UPDATE: This issue was fixed in EG 7.12.


Register today and join us virtually on June 16!
sasglobalforum.com | #SASGF

View now: on-demand content for SAS users

View solution in original post

17 REPLIES 17
Trent
Fluorite | Level 6

Thanks for the quick response!

No,  most of the literature on the SAS website advises against 64-bit versoin of EG unless I misunderstand.  For example:

http://blogs.sas.com/content/sasdummy/2012/04/19/myths-about-64-bit-computing-on-windows/

At any rate, the old EG5.1 installation I used was 32-bit and worked much more quickly.

Is there advice to the effect that EG7.1 in particular works better as 64-bit?

Kurt_Bremser
Super User

Was just a quick shot from the hip.

Made a quick test here with our biggest datasets. With both datasets (compressed, both approx. 100 million datasets, 12 and 16 GB in size) I experienced no difference opening them in EG 4.3/32 bit or EG 6.1/64 bit. Same goes for navigating once open. (backend is SAS 9.2 on AIX).

Both EGs here connect to the same infrastructure. Are the appservers the same for 5.1 and 7.1 at your place?

Trent
Fluorite | Level 6

Yes, in fact I'm running on local SAS only - zero network connections.

Interestingly, I also tried on our single installation of EG6.1 64-bit, and like our older EG5.1 32-bit, the response was quick and seamless.

I still come back to the new Data > Performance:  "Maximum dimensions to display in the data grid" menu option, which was not in EG5.1.

I noted that by default, it wouldn't allow entire datatsets to open - it had a default setting that was fairly restrictive (if I recall correctly - can't remember what the default setting was, but it was certainly not in the tens of millions).

I also had issues with slow load times and so disabled the SAS Studio software option.

To me, all these point to some 7.1-specific issue. 

CaseySmith
SAS Employee

This performance regression in the data grid was unfortunately introduced in EG 7.1.  We are aware of the issue and actively working on a solution.  It is a top priority.

 

In the interim, here are a few workarounds:

-reduce the "Maximum dimensions to display in the data grid" option in EG's Tools->Options->Data->Performance (which, yes, were added as a result), which should provide better performance, but won't allow you to view all the data.

-use the Data Explorer (File->Open->Data Exploration) instead

-continue to use EG 6.1 or earlier for opening large data in the data grid (multiple versions of EG can be installed and used side-by-side on the same machine)

 

UPDATE: This issue was fixed in EG 7.12.


Register today and join us virtually on June 16!
sasglobalforum.com | #SASGF

View now: on-demand content for SAS users

Trent
Fluorite | Level 6

Thanks Casey,

Even reducing maximum dimensions results to 1 million or less results in poor performance.  In the context of rapidly checking task results in the process flow, the Data Exploration option is not really feasible from a productivity standpoint either, but appreciate the suggestions.

I note that EG 7.11 has just been released - has the 'performance regression' problem been fixed in that release?

SASKiwi
PROC Star

We are using EG 7.11 and haven't experienced the conditions you describe although we only use remote SAS servers. We love the new WHERE clause "command line" !  Smiley Happy

CaseySmith
SAS Employee

Trent, unfortunately no, it was not fixed in 7.11, as it requires pretty extensive changes.  We have made good progress and it *will* be fixed in the next release.

SASKiwi, glad you like the new WHERE clause feature in the data grid!


Register today and join us virtually on June 16!
sasglobalforum.com | #SASGF

View now: on-demand content for SAS users

jb_biyectivo
Calcite | Level 5

Hi,

 

Just checking to see if a fix for this issue is on the horizon. My organization has moved towards 7.11 but the performance of the data grid is terrible. Hope there are news of a fix soon!

 

Regards,

CaseySmith
SAS Employee

I am happy to report (though wish the issue had been avoided in the first place) the data grid performance bottleneck introduced in 7.1 has been fixed for our next release.  We are back to being able to open data with any number of records and will display and scroll through over 2 billion observations, with the performance only limited by the data access performance (as it should be).  This release will be available in the first quarter of 2016.


Register today and join us virtually on June 16!
sasglobalforum.com | #SASGF

View now: on-demand content for SAS users

jb_biyectivo
Calcite | Level 5
Great news, thanks! Will this be a 7.x release or an 8.0 release?
CaseySmith
SAS Employee

You are welcome!  Most likely a 7.x release.


Register today and join us virtually on June 16!
sasglobalforum.com | #SASGF

View now: on-demand content for SAS users

bini
Calcite | Level 5

Hi Casey, Is this issue fixed? I am using 7.11  and each minute it stops responding. Please advise.

ChrisHemedinger
Community Manager

The fix for this problem was part of v7.12, which was released in late February 2016.  If you have 7.11, check Help->Check For Updates to ensure you have the latest.

It's time to register for SAS Innovate! Join your SAS user peers in Las Vegas on April 16-19 2024.

sas-innovate-2024.png

Join us for SAS Innovate April 16-19 at the Aria in Las Vegas. Bring the team and save big with our group pricing for a limited time only.

Pre-conference courses and tutorials are filling up fast and are always a sellout. Register today to reserve your seat.

 

Register now!

SAS Enterprise Guide vs. SAS Studio

What’s the difference between SAS Enterprise Guide and SAS Studio? How are they similar? Just ask SAS’ Danny Modlin.

Find more tutorials on the SAS Users YouTube channel.

Click image to register for webinarClick image to register for webinar

Classroom Training Available!

Select SAS Training centers are offering in-person courses. View upcoming courses for:

View all other training opportunities.

Discussion stats
  • 17 replies
  • 7429 views
  • 1 like
  • 8 in conversation