Option to supress output data in EG 8.3

[jwilson]

Hi everyone,

We have a SAS GRID environment that is located in US West, and we have some users on the east coast who are experiencing slower "run times" than our west coast users.  I put "run time" in quotes because after a bunch of testing, it looks like the programs run in exactly the same time (which makes sense), but there is some extra overhead before and after the program actually runs.  It looks like the issue is caused by EG automatically loading every dataset that was created into the "output data" tab.  I've done some research and I tried the solution here) and it does not seem to help.  There seems to have been an option in EG 7 that I think is what I'm trying to do, referenced here, but I don't see a similar option in 8.3.  Does anyone know how I might accomplish this? 

Thanks for your help!

[jebjur]

Try this:

 In Tools - Options - Data - Performance, change the option to determine which tables are opened when generated to 'No Table'.

[jwilson]

I tried that, it did not make a difference.

[SASKiwi]

You can't suppress the listing of output datasets in the Output Data tab in EG 8.3. The option posted stops the first or last one being opened automatically. There is another option that may help performance and that is this one:

Change the Maximum number of output datasets to add to the project from 50 to 0. This will only help if the user is working on  an EG project.

[jwilson]

So there's no way to set this outside of working on an EG project?

[SASKiwi]

The option is only relevant if you are working on an EG Project. If you are only manually writing programs, the option doesn't apply, but in this case EG works faster anyway since it is not maintaining project artifacts. Also make sure Project logging for new projects is switched off (bottom checkbox in screenshot) - this can have a significant impact on performance:

In general, EG is sensitive to network latency. This can be an issue if working over long network distances and / or VPNs.

[Quentin]

Did you try @SASKiwi 's suggestion?

I know EG added the "project-less" option in 8.1 (https://go.documentation.sas.com/doc/en/egug/8.3/p10qv233h1du0un1vzhdxw320vqg.htm) but I've never really understood why people wouldn't want to use a project.  I only use projects as as a collection of links, but it's still handy.

Are you saying your users use EG without a project, open a program, run it, and EG still returns a bunch of output tables and adds them to a process flow or some-such?

Depending on network speed, EG can feel slow, especially when it has to return a big log file, big output file, etc.  Many companies find it faster to set up a terminal server that is near the SAS server, with EG running on the terminal server.  Then users can remote into the terminal server and run EG sessions there.  So you drastically reduce the distance between the EG client and the SAS server.

[jwilson]

---

@Quentin wrote:

Did you try @SASKiwi 's suggestion?

Yeah.  No luck.

I know EG added the "project-less" option in 8.1 (https://go.documentation.sas.com/doc/en/egug/8.3/p10qv233h1du0un1vzhdxw320vqg.htm) but I've never really understood why people wouldn't want to use a project.  I only use projects as as a collection of links, but it's still handy.

We are all very new to EG, so the concept of project feels foreign and scary to us.  And mostly unnecessary, since we very often just run individual programs anyway.

Are you saying your users use EG without a project, open a program, run it, and EG still returns a bunch of output tables and adds them to a process flow or some-such?

Yes, but I don't think it adds anything to a process flow or anything.  I don't even really know what that is.  It operates more or less like desktop SAS, except desktop SAS doesn't have an Output Data tab.

Depending on network speed, EG can feel slow, especially when it has to return a big log file, big output file, etc.  Many companies find it faster to set up a terminal server that is near the SAS server, with EG running on the terminal server.  Then users can remote into the terminal server and run EG sessions there.  So you drastically reduce the distance between the EG client and the SAS server.

---

Yeah, that's kind of the problem.  Our terminal servers were originally all located in US West, but our East Coast users were experiencing latency in general, so we added terminal servers closer to them, but now when running a program in EG, there seems to be additional fixed slowness at the beginning (not an issue) and a variable slowness after the log shows that the program is done running, and this slowness seems to scale with the number of output datasets.  My inference is that the slowness is caused by SAS having to download all the datasets in the "output data" tab, but I don't have a way to test that since I don't seem to be able to turn off the ability to download those datasets. 

[Quentin]

Hmm... if you work with technical support I think there are some hidden options which might help them diagnose network latencies.  But in my limited experience (just one employer but 2-3 servers), it's always felt a bit sluggish.

You might try using a project file with the max datasets set to 0, and see if that changes anything.  No matter what you do, EG has to push all the code up to the server, and pull back a log file.  So that alone can be slow.  If you add a big set of HTML/PDF output, it gets even slower.

EG project files don't have to feel scary.  Especially since you don't have to store your code in them.  In PC SAS, I would open a program in Display Manager, and run it (either as a whole or in pieces).

In EG, I have one project file (.egp) for each project that I work on.  When I open the project file, the left side shows all the programs for that project.  But these are just links / shortcuts.  The programs are still stored on the server.  It's convenient to have links to all the code for a project in one frame.  Beyond that, I don't use EG process flows or other EG artefacts.  So I basically use EG like display manager.  I used display manager for development, and production runs were done as batch submissions.  I now use EG for development (primarily), and production runs are done as batch jobs (scheduled outside of EG, since EG never got a real batch submit feature, unfortunately).

You might want to look into Studio and see if it's any faster.  I haven't used in for real work yet, but I've heard great things.

[jwilson]

Thank you! This is very helpful. I hadn't really considered HTML/PDF output, so the solution I had in mind probably would be pretty incomplete anyway.

[SASKiwi]

@jwilson  - Can you give some idea of how many datasets it takes before your EG users experience "unacceptable" slowness and how long do these programs run for overall? 

I'd suggest that if user jobs take over 30 minutes to run or output hundreds of datasets or are run on a regular basis then you should consider running these as batch jobs which will obviously remove any EG overheads entirely. In my experience, providing power users SAS Management Console so they can easily create and run their own batch jobs is well worth the effort.

Also it might be worth engaging SAS Tech Support to do more analysis on EG performance bottlenecks.