topic Re: Help with saving large sas dataset in permanent library in SAS Procedures

Help with saving large sas dataset in permanent library

xliu1 — Sat, 26 Sep 2020 03:54:15 GMT

I created a sas dataset which has 9347777 rows and 37 columns and want to save it in a permanent library on my organization's network drive so that I can use it later in other programs. Here is the sas code. Looks like it takes forever to run. Is there a more efficient way to do this?

data dl.course_enrl_detail;
set course_enrl_detail;
run;

Re: Help with saving large sas dataset in permanent library

SASKiwi — Sat, 26 Sep 2020 03:58:31 GMT

If you are writing this from your PC to a network drive then you will be limited by the network connection speed.

Compressing the dataset so it takes up less space may help:

data dl.course_enrl_detail (compress = yes);
set course_enrl_detail;
run;

Re: Help with saving large sas dataset in permanent library

ballardw — Sat, 26 Sep 2020 14:00:31 GMT

@xliu1 wrote:

I created a sas dataset which has 9347777 rows and 37 columns and want to save it in a permanent library on my organization's network drive so that I can use it later in other programs. Here is the sas code. Looks like it takes forever to run. Is there a more efficient way to do this?
data dl.course_enrl_detail;
set course_enrl_detail;
run;

The data step also processes each record as if you are going to do some data manipulation.

If the idea is just to copy the data you might try either Proc Copy or Proc Datasets with the Copy option.

Re: Help with saving large sas dataset in permanent library

xliu1 — Sat, 26 Sep 2020 18:56:31 GMT

Thanks for suggestion! I tried proc copy with compress=yes statement and save the data on my desktop, it take 27 seconds.

libname DL1 'C:\DISTANCE LEARNING DASHBOARD';
OPTIONS COMPRESS=YES;
PROC COPY IN=WORK OUT=DL1 NOCLONE;
SELECT COURSE_ENRL_DETAIL;
RUN;