ERROR: Insufficient space in file WORK.NEISS_SUBSET_1.DATA.
ERROR: File WORK.NEISS_SUBSET_1.DATA is damaged. I/O processing did not complete.
I am trying to merge `14 datasets (2002 - 2015) w/ a combined number of observations that exceeds 5 million. I am using SAS University Edition on a Mac OS X w/ 70 GB of free storage.
Previous posts on this topic have not solved the problem.
How much storage space is needed? Is 70 GB not enough?!
Thanks.
Full log:
1 OPTIONS NONOTES NOSTIMER NOSOURCE NOSYNTAXCHECK;
55
56 DATA neiss_subset_1;
57 MERGE neiss_merge WORK.NEISS_2002_2015;
58 BY cpsc_case_no trmt_date;
59 RUN;
ERROR: Insufficient space in file WORK.NEISS_SUBSET_1.DATA.
ERROR: File WORK.NEISS_SUBSET_1.DATA is damaged. I/O processing did not complete.
NOTE: The SAS System stopped processing this step because of errors.
NOTE: There were 896761 observations read from the data set WORK.NEISS_MERGE.
NOTE: There were 1 observations read from the data set WORK.NEISS_2002_2015.
WARNING: The data set WORK.NEISS_SUBSET_1 may be incomplete. When this step was stopped there were 896760 observations and 28
variables.
WARNING: Data set WORK.NEISS_SUBSET_1 was not replaced because this step was stopped.
NOTE: DATA statement used (Total process time):
real time 0.90 seconds
cpu time 0.89 seconds
60
61 OPTIONS NONOTES NOSTIMER NOSOURCE NOSYNTAXCHECK;
73
You should not put those tables in WORK library, put them in Share Fold.And also make it as WORK library.
libname xx v9 '/folders/myfolders/';
options user=xx;
data NEISS_SUBSET_1;
merge NEISS_SUBSET_1 have;
.....
It depends on how you merge.
If you do a cartesian product on 14 tables, I suspect 70 *TB* would not be enough.
You should not put those tables in WORK library, put them in Share Fold.And also make it as WORK library.
libname xx v9 '/folders/myfolders/';
options user=xx;
data NEISS_SUBSET_1;
merge NEISS_SUBSET_1 have;
.....
3TB drives are running at about $90 which means that 70gb is costs about $2.5. You may want to increase tour investment in disk space.
However you do have to determine the nature of your joins, hopefully you are not doing I lot of cartesian joins.
@Ksharp I'm not sure I follow. Could you please elaborate? Thanks!
I mean take
'/folders/myfolders/'
as WORK library. All the table will appear in /folders/myfolders/
Please show the log for that code with the error messages.
The fewer the details you provide the more guesses or questions you get before any resolution.
1 OPTIONS NONOTES NOSTIMER NOSOURCE NOSYNTAXCHECK;
55
56 DATA neiss_subset_1;
57 MERGE neiss_merge WORK.NEISS_2002_2015;
58 BY cpsc_case_no trmt_date;
59 RUN;
ERROR: Insufficient space in file WORK.NEISS_SUBSET_1.DATA.
ERROR: File WORK.NEISS_SUBSET_1.DATA is damaged. I/O processing did not complete.
NOTE: The SAS System stopped processing this step because of errors.
NOTE: There were 896761 observations read from the data set WORK.NEISS_MERGE.
NOTE: There were 1 observations read from the data set WORK.NEISS_2002_2015.
WARNING: The data set WORK.NEISS_SUBSET_1 may be incomplete. When this step was stopped there were 896760 observations and 28
variables.
WARNING: Data set WORK.NEISS_SUBSET_1 was not replaced because this step was stopped.
NOTE: DATA statement used (Total process time):
real time 0.90 seconds
cpu time 0.89 seconds
60
61 OPTIONS NONOTES NOSTIMER NOSOURCE NOSYNTAXCHECK;
73
Join us for SAS Innovate April 16-19 at the Aria in Las Vegas. Bring the team and save big with our group pricing for a limited time only.
Pre-conference courses and tutorials are filling up fast and are always a sellout. Register today to reserve your seat.
Learn how use the CAT functions in SAS to join values from multiple variables into a single value.
Find more tutorials on the SAS Users YouTube channel.