Interact, learn and grow

Join Now
Reply
This is an open group. Sign in and click the "Join Group" button to become a group member and start posting.
Highlighted
Occasional Contributor
Posts: 17
Q: How to speedUp load to a remote work-lib?
[ Edited ]

Hi folks!

In traditional SAS/Connect

 

I need to copy a dataset from a client to a remote server using a remote libname

 

So far:

data remote.mydata;

set local.mydata;

run;

 

is slow, so is Proc SQL

 

Ihe datastep only runs on one core (out of 40 available) - right? But PROC SQL should be able to run on eg 4 cores. But the elapse time in the saslog is almost the same. THREADS is enabled and CPUCOUNT=4 in config.

 

Why is SQL not faster than Data?

Bandwith is about 10Gb/s

 

Any ideas?

Super Contributor
Posts: 259
Re: Q: How to speedUp load to a remote work-lib?
You should try proc cimport
Occasional Contributor
Posts: 17
Re: Q: How to speedUp load to a remote work-lib?

Hrmm...

according to the manual it

 

"Restores a transport file created by the CPORT procedure."

 

My data is not a transportfile, but a plain SAS dataset.

SAS Employee
Posts: 5
Re: Q: How to speedUp load to a remote work-lib?

Hi

 

According to this, it looks like the PROC SQL only uses multi-threading for group by and order by clauses:

 

http://support.sas.com/rnd/scalability/procs/index.html

 

Another issue could be the bandwidth not being 10Gb/s all the way between the two servers, but as the cpu time in the log is significant compared to real time in both cases, I guess it has to do with single-threading.

 

Not sure how to overcome the issue apart from hardcode the input into chunks, and transfer them seperately in parallel.

 

Christian

 

Occasional Contributor
Posts: 17
Re: Q: How to speedUp load to a remote work-lib?
Also my impression - tnx. What about a DS2 step?
SAS Employee
Posts: 2
Re: Q: How to speedUp load to a remote work-lib?

Have you tried proc upload?

Occasional Contributor
Posts: 17
Re: Q: How to speedUp load to a remote work-lib?

No - I suppose it also runs single threaded. But I will try - just in case :-)

SAS Employee
Posts: 170
Re: Q: How to speedUp load to a remote work-lib?

Hi @jmic_nyk

 

Have you tried to use proc copy ?

 

The data step and sql will perform 'twice' as much work, as they also put the data into the Program Data Vector.

 

Perhaps CLONE, BUFSIZE= or COMPRESS the data before Proc Copy.

 

/Gert

Occasional Contributor
Posts: 17
Re: Q: How to speedUp load to a remote work-lib?
Will try - tnx.
Occasional Contributor
Posts: 17
Re: Q: How to speedUp load to a remote work-lib?
All suggestions in a batch job - hopefully results on monday. TNX again
SAS Employee
Posts: 170
Re: Q: How to speedUp load to a remote work-lib?
[ Edited ]

Have a nice weekend - looking forward to read about your experience on performance. /Gert