topic Re: Requesting advice/directions for copying libraries from one server to another? in SAS Data Management

Requesting advice/directions for copying libraries from one server to another?

novinosrin — Sun, 15 Sep 2019 17:49:40 GMT

At my work place, My team and I work with libraries that are in two different servers. There servers are completely independent and in essence do not talk to each other.

1. One is a SAS RISK Unix server- SYSHOSTINFOLONG IBM AIX AIX 64 1 7 00FA746A4C00

2. Two is a SAS Windows server that we call it BPT server. - SYSHOSTINFOLONG X64_ES08R2 WIN 6.1.7601 Service Pack 1 Server

We access datasets from libraries from both the servers and perform ad-hoc requests. Lately, there are certain requirements where the reports requires some logic to combine datasets from both independent servers and deliver it to end-user. Right now, how we go about it.

1. Login to Unix server, extract the subsets whatever is required. Then change the server from SASApp to Local. Create a libname connection to a folder that is accessible in BPT server. Manually copy the dataset/subset to the folder by using the mouse on the left hand side of EG. Log out /*While the subset program execution hardly takes few minutes for like 5 million records,the manual mouse point/click copy/paste takes ages*/

2. Log in to BPT server, assign the libname that very folder again and do whatever.

I am sure we can do better than that. I am certainly amazed by the execution speed in the UNIX server and how I wish i can use that as a platform to carry out all the development/test etc. The question is how, what and why? Thank you in advance for the solutions

PS I posted a similar question, but I didn't detail it like I have here. So kindly treat this as independent thread. Thanks!

Re: Requesting advice/directions for copying libraries from one server to another?

SASKiwi — Sun, 15 Sep 2019 20:21:14 GMT

Its a pity both of your SAS servers don't run on the same OS. If that were the case then simple folder shares between the two environments would enable you to establish data connections / SAS libraries between the two servers.

Is it a matter of policy or security that the two servers don't talk to each other, or is it simply nobody has put up a compelling case to do so?

If you have SAS/CONNECT installed and licensed on both servers then that would provide a much easier way of exchanging SAS data. From any server EG session you can PROC UPLOAD / DOWNLOAD data libraries and datasets between servers.

If you don't have SAS/CONNECT then CEDA together might be an alternative: https://documentation.sas.com/?docsetId=movefile&docsetTarget=p0c0l7xkprukh1n1ey1voicmgn6f.htm&docsetVersion=9.4&locale=en

In any case you would have to get agreement that the two SAS servers are allowed to communicate directly with each other.

Re: Requesting advice/directions for copying libraries from one server to another?

TomKari — Sun, 15 Sep 2019 20:28:49 GMT

This is a common issue in multi-server environments.

The best, and correct, answer is to get your administrators to set up a direct connection between the two servers, bypassing your workstation completely. If there's any possibility of this being done, it's the option to pursue. Bribery is acceptable!

If not, the one thing I can suggest is to try testing the copy speed from your local workstation to BPT server; just use a windows copy, or a shell script, to copy one of your extract files up to it. Find out what your communication link speed is supposed to be (1 gigabit per second, 10 gigabits per second), etc.

Then figure out (basic math):

1. Does your basic copy of the file execute in about the time that would seem reasonable (you'll never hit the exact speed; anything over 50% is good enough).

2. How does the copy speed in EG compare with the basic copy speed (should be similar, or there's something wrong in your EG configuration).

Hopefully someone with more expertise will have a better suggestion.

Tom

Re: Requesting advice/directions for copying libraries from one server to another?

Patrick — Sun, 15 Sep 2019 21:34:25 GMT

@novinosrin

Just adding/throwing ideas what others wrote already:

If there is no SAS/Connect and shared file system then you could consider to use the SAS EG copy task. This ties your code to EG but at least you wouldn't have to fully manually download/upload your file anymore when re-running your job.

If security at your site allows it then you could also try to use SCP or SFTP to connect from server A to Server B. Run a remote command on server B to execute your SAS script and then download to server A where you do the rest.

Re: Requesting advice/directions for copying libraries from one server to another?

novinosrin — Mon, 16 Sep 2019 11:46:07 GMT

Good morning from Bridgeport, CT to all who responded. I am hoping to take the case to my boss and to the Admin to enact upon the objective of making servers communicate. However, I am not sure what would they counter me with should they find that a big deal

Is it a matter of policy or security that the two servers don't talk to each other, or is it simply nobody has put up a compelling case to do so?

No idea, but I don't think so. My guess is our infrastructure isn't as well established/sophisticated in the 1st place. Also, majority are rather business users than tech folks. As a matter of fact, you wouldn't believe me, some here even prefer a hard coded solution as opposed to an automated solution 🙂

So in essence if it's a big deal that would require extensive knowledge to the extent seeking expertise of an external consultant to get this done, it's gonna be a non-starter. If not, there is a chance.

We do have SAS CONNECT though. Yes the DOWNLOAD/UPLOAD is possible. Anyways, is it possible to save results of execution of SASApp server on to my Desktop with some kind of Desktop as a service facility?

Re: Requesting advice/directions for copying libraries from one server to another?

Ksharp — Mon, 16 Sep 2019 12:59:14 GMT

Due to different OS between two server , better use PROC CPORT and PROC CIMPORT to avoid CEDA problem .

Re: Requesting advice/directions for copying libraries from one server to another?

Reeza — Mon, 16 Sep 2019 15:38:03 GMT

Also, in SAS Base you can connect and disconnect to multiple servers via code to keep you process still automated but that's not available in EG. I would also make an argument for an interim solution to have desktop SAS installed for some users to get around this for certain projects. That shouldn't be a heavy lift for your IT team.

Re: Requesting advice/directions for copying libraries from one server to another?

novinosrin — Mon, 16 Sep 2019 16:32:30 GMT

" in SAS Base you can connect and disconnect to multiple servers via code "- You have given me a reason to breathe now!

@Reeza wrote:
Also, in SAS Base you can connect and disconnect to multiple servers via code to keep you process still automated but that's not available in EG. I would also make an argument for an interim solution to have desktop SAS installed for some users to get around this for certain projects. That shouldn't be a heavy lift for your IT team.

Re: Requesting advice/directions for copying libraries from one server to another?

Reeza — Mon, 16 Sep 2019 17:16:54 GMT

Yeah, that was the main reason I didn't leave Base SAS to EG or Studio when working with a server. I hated having to switch or could only work locally or on server. On desktop that wasn't an issue. I should caveat this with that was now a few years ago...my current company doesn't have SAS.