topic Re: EG 7.1 very slow data view performance. in SAS Enterprise Guide

EG 7.1 very slow data view performance.

Trent — Wed, 10 Jun 2015 07:35:18 GMT

Hello folks,

I'm currently testing SAS EG7.1 before all my colleagues follow suit; we're upgrading from EG5.1.

System is Win7 64-bit, using EG7.1 32-bit and Base SAS 9.4 64-bit.

I have a source SAS dataset, which is compressed, of around 40 million records and 70 columns. Using BASE engine (not SPD).

In EG5.1, the table opens when double-clicked from the process flow almost instantly. It can be navigated around rapidly with no noticeable delays, and when I hit Ctrl+End it jumps immediately to the last record (row 40 million etc).

In EG7.1, same file, same system, same specs, but the performance has dropped by orders of magnitude.

The dataset takes upwards of 30 seconds to open, seems to be constantly reading in as I scroll around, and there is a significant delay of up to 30+ seconds to jump to end of file.

Any ideas on where this performance drop stems form would be greatly appreciated.

PS - I notice a new entry in the EG Options menu under Data > Performance: "Maximum dimensions to display in the data grid". Could this be related?

[edited - oops, accidental premature post!]

Re: EG 7.1 very slow Data Grid

Kurt_Bremser — Wed, 10 Jun 2015 07:45:24 GMT

Have you tried the 64-bit version of EG 7.1?

Re: EG 7.1 very slow Data Grid

Trent — Wed, 10 Jun 2015 08:06:35 GMT

Thanks for the quick response!

No, most of the literature on the SAS website advises against 64-bit versoin of EG unless I misunderstand. For example:

http://blogs.sas.com/content/sasdummy/2012/04/19/myths-about-64-bit-computing-on-windows/

At any rate, the old EG5.1 installation I used was 32-bit and worked much more quickly.

Is there advice to the effect that EG7.1 in particular works better as 64-bit?

Re: EG 7.1 very slow Data Grid

Kurt_Bremser — Wed, 10 Jun 2015 08:22:07 GMT

Was just a quick shot from the hip.

Made a quick test here with our biggest datasets. With both datasets (compressed, both approx. 100 million datasets, 12 and 16 GB in size) I experienced no difference opening them in EG 4.3/32 bit or EG 6.1/64 bit. Same goes for navigating once open. (backend is SAS 9.2 on AIX).

Both EGs here connect to the same infrastructure. Are the appservers the same for 5.1 and 7.1 at your place?

Re: EG 7.1 very slow Data Grid

Trent — Wed, 10 Jun 2015 10:24:24 GMT

Yes, in fact I'm running on local SAS only - zero network connections.

Interestingly, I also tried on our single installation of EG6.1 64-bit, and like our older EG5.1 32-bit, the response was quick and seamless.

I still come back to the new Data > Performance: "Maximum dimensions to display in the data grid" menu option, which was not in EG5.1.

I noted that by default, it wouldn't allow entire datatsets to open - it had a default setting that was fairly restrictive (if I recall correctly - can't remember what the default setting was, but it was certainly not in the tens of millions).

I also had issues with slow load times and so disabled the SAS Studio software option.

To me, all these point to some 7.1-specific issue.

Re: EG 7.1 very slow data view performance.

CaseySmith — Thu, 05 Oct 2017 14:56:59 GMT

This performance regression in the data grid was unfortunately introduced in EG 7.1. We are aware of the issue and actively working on a solution. It is a top priority.

In the interim, here are a few workarounds:

-reduce the "Maximum dimensions to display in the data grid" option in EG's Tools->Options->Data->Performance (which, yes, were added as a result), which should provide better performance, but won't allow you to view all the data.

-use the Data Explorer (File->Open->Data Exploration) instead

-continue to use EG 6.1 or earlier for opening large data in the data grid (multiple versions of EG can be installed and used side-by-side on the same machine)

UPDATE: This issue was fixed in EG 7.12.

Re: EG 7.1 very slow data view performance.

Trent — Thu, 11 Jun 2015 00:46:22 GMT

Thanks Casey,

Even reducing maximum dimensions results to 1 million or less results in poor performance. In the context of rapidly checking task results in the process flow, the Data Exploration option is not really feasible from a productivity standpoint either, but appreciate the suggestions.

I note that EG 7.11 has just been released - has the 'performance regression' problem been fixed in that release?

Re: EG 7.1 very slow data view performance.

SASKiwi — Thu, 11 Jun 2015 01:29:20 GMT

We are using EG 7.11 and haven't experienced the conditions you describe although we only use remote SAS servers. We love the new WHERE clause "command line" !

Re: EG 7.1 very slow data view performance.

CaseySmith — Thu, 11 Jun 2015 15:05:03 GMT

Trent, unfortunately no, it was not fixed in 7.11, as it requires pretty extensive changes. We have made good progress and it *will* be fixed in the next release.

SASKiwi, glad you like the new WHERE clause feature in the data grid!

Re: EG 7.1 very slow data view performance.

jb_biyectivo — Sat, 07 Nov 2015 04:14:20 GMT

Hi,

Just checking to see if a fix for this issue is on the horizon. My organization has moved towards 7.11 but the performance of the data grid is terrible. Hope there are news of a fix soon!

Regards,

Re: EG 7.1 very slow data view performance.

CaseySmith — Sat, 07 Nov 2015 22:41:59 GMT

I am happy to report (though wish the issue had been avoided in the first place) the data grid performance bottleneck introduced in 7.1 has been fixed for our next release. We are back to being able to open data with any number of records and will display and scroll through over 2 billion observations, with the performance only limited by the data access performance (as it should be). This release will be available in the first quarter of 2016.

Re: EG 7.1 very slow data view performance.

jb_biyectivo — Mon, 09 Nov 2015 16:53:43 GMT

Great news, thanks! Will this be a 7.x release or an 8.0 release?

Re: EG 7.1 very slow data view performance.

CaseySmith — Mon, 09 Nov 2015 18:52:20 GMT

You are welcome! Most likely a 7.x release.

Re: EG 7.1 very slow data view performance.

bini — Thu, 19 May 2016 19:27:59 GMT

Hi Casey, Is this issue fixed? I am using 7.11 and each minute it stops responding. Please advise.

Re: EG 7.1 very slow data view performance.

ChrisHemedinger — Thu, 19 May 2016 20:47:10 GMT

The fix for this problem was part of v7.12, which was released in late February 2016. If you have 7.11, check Help->Check For Updates to ensure you have the latest.

Re: EG 7.1 very slow data view performance.

bini — Fri, 20 May 2016 21:07:13 GMT

Thank You. I did update to 7.12. I still see it takes couples seconds to open the libaraies. Not very quick. Is there any other config changes I need to make to improve the EG performance?

Re: EG 7.1 very slow data view performance.

mbuchecker — Wed, 19 Apr 2017 17:45:46 GMT

Anyone still having this problem with EG 7.12? I have a customer with a 3.8 million obs data set that takes 84 seconds to open. EG is on Citrix machine, data is on a Linux server. And when she clicks "Filter and Sort" it takes "give up time" amount of time (we waited 5 minutes and gave up). Running 7.12 HF2.

Re: EG 7.1 very slow data view performance.

bini — Wed, 19 Apr 2017 17:58:02 GMT

I still see a dealay when it open the librarries for first time. After that it is a ok performance. Not too slow any way. Far better than7.11.