topic Re: Speed of EG Vs. Base...Nice Results on Export in SAS Enterprise Guide

Speed of EG Vs. Base...Nice Results on Export

BCNAV — Mon, 03 Jun 2019 16:30:37 GMT

I had posted about EG being slow in a project given to me and ideas of how to speed it up. The first one was to export all the code and run it in base SAS via:

"C:\Program Files\SASHome\SASFoundation\9.4\sas.exe" -SYSIN "R:\SAS Batch\SAS Programs\Partial V4.sas" -LOG "R:\SAS Batch\Logs\Partial\Partial2D_%today%.log" -PRINT "R:\SAS Batch\LST\Partial2D.lst"

The original EG run time is 70 minutes. Sending the code to Base SAS via above is 2 minutes....same results every time.

Why is this? Seems like this is they way to go without doing anything else. What is EG doing to make it so slow?

Re: Speed of EG Vs. Base...Nice Results on Export

Reeza — Mon, 03 Jun 2019 16:34:47 GMT

As someone mentioned in the previous thread - ODS output, HTML etc take up a lot of time to generate. Did you turn off outputs?

@BCNAV wrote:

I had posted about EG being slow in a project given to me and ideas of how to speed it up. The first one was to export all the code and run it in base SAS via:
"C:\Program Files\SASHome\SASFoundation\9.4\sas.exe" -SYSIN "R:\SAS Batch\SAS Programs\Partial V4.sas" -LOG "R:\SAS Batch\Logs\Partial\Partial2D_%today%.log" -PRINT "R:\SAS Batch\LST\Partial2D.lst"
The original EG run time is 70 minutes. Sending the code to Base SAS via above is 2 minutes....same results every time.

Why is this? Seems like this is they way to go without doing anything else. What is EG doing to make it so slow?

Re: Speed of EG Vs. Base...Nice Results on Export

BCNAV — Mon, 03 Jun 2019 16:48:58 GMT

There are no ODS or HTML commands used...or is EG doing something on its own? What ODS commands/HTML commands should be run in a code node first?

would it be:

ods graphics off;
ods exclude all;
ods noresults;
ods html close;

thx

Re: Speed of EG Vs. Base...Nice Results on Export

Reeza — Mon, 03 Jun 2019 17:10:17 GMT

ODS destinations are on by default, so if you didn't turn them off that's an issue.

ODS _all_ Close;
ods listing; *since that's the one you want;

It depends on the code you're running as well. If it's all data steps or SQL queries this wouldn't matter, unless it's trying to open results as well. Make sure show results as generated option is also turned off.

I'd try those for starters.

I would also consider not having any Listing output, unless you really need it and explicitly pipe those to tables and then export as needed.

Re: Speed of EG Vs. Base...Nice Results on Export

ballardw — Mon, 03 Jun 2019 17:30:54 GMT

If you can you might also post the code you have in "R:\SAS Batch\SAS Programs\Partial V4.sas"

That might give hints as to what is slower in EG.

Re: Speed of EG Vs. Base...Nice Results on Export

BCNAV — Mon, 03 Jun 2019 18:07:40 GMT

Well...it is a huge program, so I'm thinking I won't post.Very nice how fast it is when sent to base SAS.

So....is there anything in a code node that I can run before any of the other process flows that would speed up EG? In the end it doesn't matter, as I have a quick solution now. But it would be nice to figure out what EG is doing that makes it so slow.

Re: Speed of EG Vs. Base...Nice Results on Export

BCNAV — Mon, 03 Jun 2019 18:11:02 GMT

BTW, there is no ODS output, just SQL (joins/merges) and datasteps

Re: Speed of EG Vs. Base...Nice Results on Export

BCNAV — Mon, 03 Jun 2019 18:36:11 GMT

see attached....no so big after all

Re: Speed of EG Vs. Base...Nice Results on Export

TomKari — Mon, 03 Jun 2019 19:37:07 GMT

Very interesting; as you've found a workaround, I assume this has become an intellectual exercise. As such, it's an interesting one. Everything that you've said, and everything that the other posters have said, makes perfect sense in my opinion. I've taken a very quick look at the code, and I don't see anything that would intrinsically cause it to run significantly slower in EG than using your base SAS execution.

Here are a few things that occur to me:

When you run it outside of EG, is it running on the same machine as when you submit it through EG?
Is there something in the environment options that is different in EG and outside of it? Particularly the memory options.
I'm not quite sure what I'm asking here, but is there some way that there's a slow communication channel involved when you're running it in EG, as opposed to outside of EG? (You should see the difference when data is being slung around on 10Mb Ethernet as opposed to Gb).

If you have any energy for continuing to chase this down, you could put this before your first piece of SAS code, and after the last piece, and see if the times shown make sense in terms of what the elapsed time appears to be to you.

data _null_;
	file log;
	TimeStamp = put(datetime(), datetime.);
	put TimeStamp=;
run;

If you do chase this down, please let us know what the solution is. I'm quite intrigued!

Tom

Re: Speed of EG Vs. Base...Nice Results on Export

BCNAV — Mon, 03 Jun 2019 19:46:49 GMT

Same machine....and yes it has become an intellectual exercise to figure this out....EG is slow.

I do use the start end time you suggest. Here is my latest:

EG:

Time Partial Started	2019-06-03 9:17
Time Partial Ended	2019-06-03 10:45
Time Taken (Minutes)	88.45

Exported Code

Time Partial Started	2019-06-03 12:23
Time Partial Ended	2019-06-03 12:26
Time Taken (Minutes)	3.53

So bizarre!

Re: Speed of EG Vs. Base...Nice Results on Export

SASKiwi — Mon, 03 Jun 2019 21:29:30 GMT

I would compare the SAS logs of the two runs to identify which steps have the biggest time differences. It might also be worth using the SAS option FULLSTIMER and compare memory usage as well.

Our EG versus batch run times are pretty much the same, as they should be, except where ODS is involved.

Re: Speed of EG Vs. Base...Nice Results on Export

Reeza — Tue, 04 Jun 2019 03:30:07 GMT

Not sure if this was asked already but both are running the same version and hitting the same server?

Second, there was a post late last year that had a similar situation. I can't remember the exact details but I believe Rick Wicklin helped resolve that one.

Re: Speed of EG Vs. Base...Nice Results on Export

CaseySmith — Tue, 11 Jun 2019 04:52:08 GMT

It sounds like you exported all the code from your EG project to "Partial V4.sas" and then compared the time it takes the original EG project to run to the time it takes Base SAS to run the exported code "Partial V4.sas". How long does it take EG to run "Partial V4.sas" (run the exported code in a new EG session, rather than running the original EG project)? It should be very similar to the amount of time it takes Base SAS to run it, especially if EG isn't configured to generate ODS results (if you don't have any results formats checked in EG's Tools->Options->Results->Results General).

Casey

Re: Speed of EG Vs. Base...Nice Results on Export

BCNAV — Tue, 11 Jun 2019 16:19:13 GMT

My EG Results General Tab is attached

Re: Speed of EG Vs. Base...Nice Results on Export

CaseySmith — Thu, 13 Jun 2019 15:33:25 GMT

Ok, thanks. You could try unchecking the "SAS Report" results format (ODS results) and see if it makes a difference. It would also be helpful to know the answer to my previous question...

How long does it take EG to run "Partial V4.sas" (run the exported code in a new EG session, rather than running the original EG project)?

Were the earlier times you posted (~88 mins) for EG how long it took EG to run the original EG project or for EG to run the single program file "Partial V4.sas"? Assuming it was the amount of time it took EG to run the original EG project, I'd like to know how long it takes EG to run the single program file (Partial V4.sas), so we can then compare that to running the same program in Base SAS.

Casey