topic proc casuitil slow in SAS Programming

proc casuitil slow

dipand — Tue, 05 Jul 2022 14:19:30 GMT

Hello everybody,
the following proc casutil:

data hseq.HSEQ_INSP_FIND_TEMP1;
set staging.HSEQ_INSP_FIND_TEMP
;
run;

proc casutil incaslib="hseq" outcaslib="hseq";
droptable casdata="HSEQ_INSP_FIND_TEMP" quiet;
promote casdata="HSEQ_INSP_FIND_TEMP1" casout="HSEQ_INSP_FIND_TEMP";
save casdata="HSEQ_INSP_FIND_TEMP" casout="HSEQ_INSP_FIND_TEMP" replace;
quit;

deals with uploading a dataset of about 15 million observations and 240 columns to the CAS. The loading times are very long about 2 hours. Is there any option to improve this upload?

Thank you

Re: proc casuitil slow

sbxkoenk — Tue, 05 Jul 2022 18:17:16 GMT

Hello,

What if you use the Table action set in PROC CAS?

And also use DVR memory format?
DVR = duplicate value reduction
Take care : instead of the actions illustrated below, you will need to replace with the loadTable Action !

proc cas;
  t.name = "waterflow";
  tdvr.name = t.name || "_dvr";

  table.tableDetails / table=t.name;
  table.copyTable /    
    table=t,
    casOut=tdvr + {memoryFormat="DVR", replace=True};
  table.dropTable / table=t.name;

  table.tableDetails / table=tdvr.name;
  table.tableInfo / table=tdvr.name;
run;

Cheers,

Koen