@Tom I ran it again today.  This time, using just one remote signon.  That took about 25s to do the sign on.  For the 10m obs dataset written to sWork perm lib, it took 6s, rather than the 18s from yesterday.  Tried writing to WORK, and it matched the 0.2s runtime of the main server.   Not a big experiment, but it appears our perm server for small (80Mb) files is far slower than to WORK. If I run it to (work) in EG on Local (Windows), it took 0.3s,  and about 1.5s on our Unix box. The colleague who asked me about this is now running with it.  Hopefully they have a long running set of jobs that write to work and only have to put the final datasets to perm sWork to share and they will be more successful (or if there is less overhead with large files to perm) ... View more

I hadn't put a timer on it, as it was just seeing if it worked at all, just it seemed to take longer for the log to build, that it claimed in the log to have taken. and from the log   %put %sysfunc(time(), time8.); signon para1; %put %sysfunc(time(), time8.); signoff _all_; and from the log, the start time was 14:33:31 and the end time 14:34:13, so that was 42s.  The log showed initialisation of 5s real time and autoexec of 0. So I don't know what is going on!.  I use SignOn when I have to run BASE.  Without testing that, I can assure you it takes next to no time. 14:33:31 81 signon para1; NOTE: SAS initialization used: real time 5.08 seconds cpu time 0.05 seconds NOTE: AUTOEXEC processing beginning; file is NOTE: DATA statement used (Total process time): real time 0.00 seconds cpu time 0.01 seconds NOTE: AUTOEXEC processing completed. NOTE: Remote signon to PARA1 complete. 82 %put %sysfunc(time(), time8.); 14:34:13 83 signoff _all_;   ... View more

Trying to do this in EG, I got less far that I had 2-3 years ago.  If I run it on SASApp, the signon gives me a load of errors.  If I run it on Local, the signon works and I can run code in Rsubmit to 3 different servers; but then I had a few problems mixing code running on Local and code running on SASApp. Then I remembered, that the ask was "SAS Studio on Compute on Viya" I took some simple code into that environment and made some progress: Signon worked. Rsubmit para_i WAIT=YES; libname sWork "/sasvpath/.."; EndRsubmit; worked fine. Control was passed back to the main server and I created a 10m obs, dataset:   data series_1; do i = 10000001 to 20000000; output; end; run;  Fine. Ran the same code on each rSubmit; creating sWork.para_i;  this time with WAIT=NO (expecting they would run in parallel). A WAITFOR _ALL_; before running some more code on the main server to append the datasets:  series_1 and sWork.para_1, sWork.para_2 and sWork.para_3; using data and SET.   This all worked, but the remote code was ridiculously slow and it wasn't clear what if anything was running in parallel. From the log Signon seemed to take a good time longer than the log showed (perhaps 8s lag each) Creating work.series_1  took 0.27s. The FULLSTIMER log between the first and 2nd "main" steps was 38s; during which time the rsubmits were running. Only after the signoff did the log include the code that was in the RSubmit WAIT=NO steps.  They showed it took 12.6s, 18.4s and 18.6s to create the 10m obs para_i datasets.  This is a total of 48s, which is more than the 37s between the two "main" steps, but 37s is longer than the 19s of the longest remote.   From those times, I don't know what is running in parallel, or why it is a 100x slower to write a 82Mb file on the perm folder in Rsubmit than in Work on the main connection; unless it is 18s of overhead + 0.3s of work. I appreciate that I am trying to run something that is very quick on the Rsubmit, but when I accidentally ran it for 55m obs in the loop, it was taking forever, enough minutes for me to regret it and abort it. Thanks for everyone's suggestions, but I am going to close this thread.  I will pass my attempt on to the programmer who asked about it, as they most likely have a genuine use case.  It also looks like we have some setup that I don't fully understand or have access to, so any solution looks like it needs some systems support. I'll mark SASKiwi solution as best, even if I haven't been able to successfully produce useful results in my set up. ... View more

Thank you for your replies and ideas. Yes, we are on Viya, or at least some of our users are.  EG will be phased out, we are migrating existing projects. The organisation's priority was continuity, so lift and shift to compute, with access to CAS very limited at this point. It will come.  Anyway, we are seeing 8x speed improvement of Viya on new kit vs grid/Unix on 10 year old kit.  Whilst parallel processing in EG was a useful idea, most users will be happy with that upgrade.  There may be the odd code, especially the one I did get running in parallel, where we might want to keep it (hence I was asked about it). We are very much a community of users (500+ active programmers), but without a central programming support (we have systems support, obviously) users are left to their own imagination and what we can get from each other.  Hence, most of our code starts at the beginning, goes through the middle and gets to the end and tends to be bespoke for each challenge we meet. ... View more

Thanks for that reminder.  We do have SAS/Connect, we still have a huge number of user who continue to use BASE, even after we went belatedly to Unix in July 2015 and are now deep in the migration to Viya. I had tried that approach a couple of years ago in EG, but the performance wasn't doing it for me.  But that may have been due to testing the idea, rather than having a real problem.  Our system was (and still is) very slow to get past "preparing server", typically 2-3 mins and at some times of day 10 mins+, so I may have seen that as a problem, where in the real case I had, the savings of 5 x 50+mins (5x 100mins if run during daytime hours) outweighed any delay on connecting the servers. I'll pass that approach on and see it we can make it work. Thank you so much for your fast reply. I worked with a SAS star from New Zealand when I first started using SAS in a UK financial firm. He was my SAS hero.  I'm wondering if @SASKiwi is you SM? ... View more

Thank you for coming back so quickly. I haven't tried putting in a throw away CFT at the start, as now I've run it once, I can't replicate. Something for tomorrow and I will consider the Tech Support route (once I figure out how to save the Application Log). ... View more

As this seems to be successful the 2nd and subsequent time of asking, I'm wondering whether putting in a fake CFT that I want to fail earlier in the process will allow the one I really want to succeed? ... View more

I was just helping someone with a Copy Files Task and noticed two things: 1. Unlike a code code, there is no option to select Server on a CFT 2. Whilst moving a 230Mb file (which for some reason took 9mins - our server must be busy, or bad home Wifi or VPN), we ran another code node.  I was expecting this to be queued, but it was not. So this sort of confirms Chris's suggestion from 2015, that the CFT is not running in the same flow control as the normal code nodes on the Unix box.  As we can't select a server and can run code rather than it be queue, it looks like CFT is running in the Windows environment and is totally unrelated to what the SAS connections are doing. Doesn't solve my problem, but I feel it helps my understanding. ... View more