06-10-2016 06:45 AM
I had no idea what you were talking about, so I did a quick Google search and this turned up several topics on the matter:
Hope that helps.
06-10-2016 07:15 AM
Thanks for this - Yes I did a Google search as well before posting here and saw these results. But he was using EG 4.3 back in 2011, while we are on SAS9.4 with EG 7.1...
So I would need to do some kind of fine tuning I guess, and see why this is happening. It is everyone in our Estate using EG7.1 that have this issue.
06-10-2016 08:27 PM
Just to clarify - are you just running SAS programs, not EG projects?
Also do you have the EG project log switched on or not? If you do, turn it off as it adds to the collecting process.
Also try setting the maximum number of SAS datasets to add to a project from the default of 50 to 0.
We run EG 7.11 and don't experience any such problems.
06-23-2016 06:07 AM
Nope - just running SAS Progs.
I get the impression (from doing a few tests) that the collecting stage is much faster when you check the "Automatically open datasets when generated".
Even though it opens the dataset the actual collecting is much faster.
The Project Log is turned off.
Many thanks foir your help.
06-10-2016 06:58 AM
It seems that collecting the results implies a lot of back-and-forth of small packets over the network. Latencies therefore add up very quickly, as the connection keeps waiting for the ACK for each transfer.
While COM works quite well under ideal conditions (because HW and network performance "hide" the design problems), its shortcomings are quickly revealed under stress (=network connections over many hops).
06-10-2016 11:27 PM
@KurtBremser - the "chatty" network behavior used to be more of an issue in earlier days of EG, but the "collecting results" phase should not be too bad in modern versions of EG. However, the volume of ODS results and large numbers of output data sets can have an impact, especially over a slower network.
@Lenvdb - you might try the techniques in this blog post to turn off the default ODS output to see what happens. You can also try deleting unnecessary transient data sets at the end of your program to reduce the "checking" that EG performs to decide what data to add to the project output. As a trick, you can direct your output data to a libname that you CLEAR at the end of your program, effectively hiding them from EG. The data files still exist, but EG won't try to add them to the project since the libname isn't present.
If you decide to pursue with Tech Support, they will want you to turn on Application Logging (from Tools->Options). This will generate detailed application logs that Tech Support and R&D know how to read. You can have a look at these logs yourself and maybe glean an idea of what's slow...but the logs are not designed for a lay person to read.
06-24-2016 07:01 AM
Thank you Chris
I am recommending to our teams to introduce deleting transcient datasets, but I also find setting the Option in Tools-> Options-> Results : Automatically open data when generated, also somehow reduces the Collecting Stage.
06-24-2016 07:56 AM
I just tried your suggestion, and it seems to speed up the collecting phase quite a lot.
EG version is 4.3, will try later with 6.1.