topic Disconnect? When running large processes or imports in SAS Enterprise Guide

Disconnect? When running large processes or imports

sasspan — Tue, 10 Sep 2019 13:40:25 GMT

Hello,

I have been struggling for some time with no help from my admins with SAS EG. Every time I run something "large" - for example, a import of a txt file containing 1.6m records by 80 columns or a program node containing thousands of lines of code, the process will run for several hours and then a box titled "Disconnect?" will pop up and ask if I want to continue. I'll click continue and will be disconnected from the server. I have no idea what to do and no one that seems to be able to help. I have several large deliverables due with no way to do it. HELP!

Thanks,

Andrew

Re: Disconnect? When running large processes or imports

jebjur — Tue, 10 Sep 2019 14:25:36 GMT

This sounds like a network firewall is causing the disconnect between Enterprise Guide and the SAS server. This SAS note should have more information. http://support.sas.com/kb/47825

Re: Disconnect? When running large processes or imports

TomKari — Tue, 10 Sep 2019 16:51:35 GMT

"An import of a txt file containing 1.6m records by 80 columns" doesn't sound particularly "large" to me. I'm wondering if you have some other issues in your environment. Of course, fix your immediate problem, but once that's done, how large (in megabytes) is the text file? I'm curious about why it's taking so long.

Tom

Re: Disconnect? When running large processes or imports

sasspan — Tue, 10 Sep 2019 17:05:05 GMT

Right, Tom, I get it is not large in the grand scheme of data mining, but comparatively to my other ad-hocs against 100k rows, it is a larger process. The file is 760MB.

Re: Disconnect? When running large processes or imports

sasspan — Tue, 10 Sep 2019 19:03:58 GMT

This is right. My SAS Admin agrees. We're in the middle of an upgrade to 9.4M5 so we will be able to control the keep alive then. Until then we're going to either get the firewall timeout changed (hard) or just start running SAS as batch jobs instead (easy). Looks like I have a work around until the upgrade. Thanks

Re: Disconnect? When running large processes or imports

TomKari — Tue, 10 Sep 2019 19:52:47 GMT

No, I wasn't meaning to denigrate your processing! 🙂

I simply meant that it sounds to me like your environment is taking a long time to process that volume of data. That's why I asked about the file size; if you had replied that it was gigabytes, I wouldn't have thought anything about it. Just to compare, I generated roughly the same data volume (1.6M records, 80 fields, 876MB). It only took five minutes to create the data, and less than 10 seconds to read it. This is on my PC. I'm wondering if you have a communications bottleneck somewhere that's slowing things down; it might be contributing to your "disconnect" issue as well (or maybe the "disconnect" issue is contributing to the slow processing). Anyhow, if you do decide to investigate, there are lots of experts on this board who would be happy to help you out!

Tom