I have a huge data set and trying to cut down the operation time. The code i usually using : DATA x; SET LIB.set1; /*set1 is 4.6GB for following example. For real work always > 20 sets or more*/
WHERE code1 in: ("sas","test"); RUN; I find that when using different storage device, the operation time (sec) is followin with this simple formula: Data size (GB) / Device speed (GB/s). When my LIB is setting on a traditional 2.5" HDD (~0.1g/s) the real time is 49.8 sec, and when LIB is setting on SATA SSD (~0.4GB/s) it takes 11.7 sec. I also try the ram disk and it only takes 2.1 sec (~2g/s). libname HDD 'E:/'; data H;set HDD.set1;where code1 in: ("401","402","403","404","405") or code2 in: ("401","402","403","404","405") or code3 in: ("401","402","403","404","405"); run; /*NOTE: DATA statement used (Total process time): real time 49.80 seconds user cpu time 3.43 seconds system cpu time 2.79 seconds memory 1014.28k OS Memory 18672.00k*/
libname SSD 'C:/test';
data S;set SSD.set1;where code1 in: ("401","402","403","404","405") or code2 in: ("401","402","403","404","405") or code3 in: ("401","402","403","404","405");
run;
/*NOTE: DATA statement used (Total process time):
real time 11.71 seconds
user cpu time 1.34 seconds
system cpu time 0.87 seconds
memory 1014.28k
OS Memory 18672.00k*/
libname RAM 'S:/';
data R;set RAM.set1;where code1 in: ("401","402","403","404","405") or code2 in: ("401","402","403","404","405") or code3 in: ("401","402","403","404","405");
run;
/*NOTE: DATA statement used (Total process time):
real time 2.13 seconds
user cpu time 1.45 seconds
system cpu time 0.64 seconds
memory 1285.28k
OS Memory 18416.00k*/ It is hard to have a bigger ram disk more than 20GB for me. But it's easy to reach 500GB by using PCIe SSD with a 1.5g/sec read speed. However, when i using a PCIe SSD with same code and same data set, it only runs 0.37G/sec and takes 12.4 sec. libname PCIE 'D:/cd';
data P;set PCIE.set1;where code1 in: ("401","402","403","404","405") or code2 in: ("401","402","403","404","405") or code3 in: ("401","402","403","404","405");
run;
/*NOTE: DATA statement used (Total process time):
real time 12.43 seconds
user cpu time 1.09 seconds
system cpu time 1.04 seconds
memory 1014.28k
OS Memory 18928.00k*/ So i wonder that whether SAS does NOT support PCIe 3.0*4, or it is lead by any other reason that i have to do some setting to reach the optimal read speed of PCIE SSD. Thanks! ps1. The read speed is cheecked by watching the Task Manager. ps2. WORK library is located at Ram Disk to increase the comparability and minimize the effect of the write speed bound of each storage device. ps3. The PCIe SSD can reach 1.5g/s on my laptop for normal seq-read work. ps4. SAS 9.4 on WIN10 home
... View more