topic Optimise Load via Proc Casutil in SAS Viya

Optimise Load via Proc Casutil

avinashjha1787 — Fri, 22 Mar 2024 06:56:03 GMT

I am trying to load a MS-SQL server table to CAS via Proc Casutil Load. The table has more than 100000 records with 2 columns of type NVARCHAR(MAX) which are really big string variables. The load table takes around 4hrs to complete. My environment has 5 worker nodes each of 1TB.

What are the options I have to speed up the load ?

Re: Optimise Load via Proc Casutil

Patrick — Fri, 22 Mar 2024 11:32:16 GMT

You could use numReadNodes=0 to ensure you're using all your workers for multi-node data transfer and then also look into sliceColumn= and sliceExpressions= in order to get evenly distributed data.

Re: Optimise Load via Proc Casutil

Patrick — Sat, 23 Mar 2024 00:39:14 GMT

And I forgot to mention readBuff= . You might have to do some testing to figure out the optimal number of rows to fetch in one go.

Re: Optimise Load via Proc Casutil

Mazi — Sat, 23 Mar 2024 04:38:24 GMT

Do you need all 100K+ rows?

Though this doesn’t seem much IMHO.

You might want to look at PROC CAS
Table.loadTable action as you can add a where clause to this and other parameters that are not available to PROC CASUTIL.