DATA Step, Macro, Functions and more

ERROR: Insufficient space in file. is damaged. I/O processing did not complete.

Accepted Solution Solved
Reply
Regular Contributor
Posts: 199
Accepted Solution

ERROR: Insufficient space in file. is damaged. I/O processing did not complete.

[ Edited ]

ERROR: Insufficient space in file WORK.NEISS_SUBSET_1.DATA.
ERROR: File WORK.NEISS_SUBSET_1.DATA is damaged. I/O processing did not complete.

 

I am trying to merge `14 datasets (2002 - 2015) w/ a combined number of observations that exceeds 5 million. I am using SAS University Edition on a Mac OS X w/ 70 GB of free storage.

 

Previous posts on this topic have not solved the problem. 

 

How much storage space is needed? Is 70 GB not enough?!

 

Thanks.

 

Full log: 


1 OPTIONS NONOTES NOSTIMER NOSOURCE NOSYNTAXCHECK;
55
56 DATA neiss_subset_1;
57 MERGE neiss_merge WORK.NEISS_2002_2015;
58 BY cpsc_case_no trmt_date;
59 RUN;

ERROR: Insufficient space in file WORK.NEISS_SUBSET_1.DATA.
ERROR: File WORK.NEISS_SUBSET_1.DATA is damaged. I/O processing did not complete.
NOTE: The SAS System stopped processing this step because of errors.
NOTE: There were 896761 observations read from the data set WORK.NEISS_MERGE.
NOTE: There were 1 observations read from the data set WORK.NEISS_2002_2015.
WARNING: The data set WORK.NEISS_SUBSET_1 may be incomplete. When this step was stopped there were 896760 observations and 28
variables.
WARNING: Data set WORK.NEISS_SUBSET_1 was not replaced because this step was stopped.
NOTE: DATA statement used (Total process time):
real time 0.90 seconds
cpu time 0.89 seconds


60
61 OPTIONS NONOTES NOSTIMER NOSOURCE NOSYNTAXCHECK;
73

 

 


Accepted Solutions
Solution
‎10-25-2016 02:20 PM
Super User
Posts: 9,681

Re: ERROR: Insufficient space in file. is damaged. I/O processing did not complete.

You should not put those tables in WORK library, put them in Share Fold.And also make it as WORK library.

 

libname xx v9 '/folders/myfolders/';
options user=xx;
data NEISS_SUBSET_1;
merge NEISS_SUBSET_1 have;
.....

View solution in original post


All Replies
PROC Star
Posts: 1,561

Re: ERROR: Insufficient space in file. is damaged. I/O processing did not complete.

It depends on how you merge.

If you do a cartesian product on 14 tables, I suspect 70 *TB* would not be enough.

Solution
‎10-25-2016 02:20 PM
Super User
Posts: 9,681

Re: ERROR: Insufficient space in file. is damaged. I/O processing did not complete.

You should not put those tables in WORK library, put them in Share Fold.And also make it as WORK library.

 

libname xx v9 '/folders/myfolders/';
options user=xx;
data NEISS_SUBSET_1;
merge NEISS_SUBSET_1 have;
.....
Valued Guide
Posts: 505

Re: ERROR: Insufficient space in file. is damaged. I/O processing did not complete.

3TB drives are running at about $90 which means that 70gb is costs about $2.5. You may want to increase tour investment in disk space.

 

However you do have to determine the nature of your joins, hopefully you are not doing I lot of cartesian joins.

 

Regular Contributor
Posts: 199

Re: ERROR: Insufficient space in file. is damaged. I/O processing did not complete.

@Ksharp I'm not sure I follow. Could you please elaborate? Thanks!

Super User
Posts: 9,681

Re: ERROR: Insufficient space in file. is damaged. I/O processing did not complete.

I mean take 

'/folders/myfolders/'

as WORK library. All the table will appear in /folders/myfolders/ 

Super User
Posts: 10,500

Re: ERROR: Insufficient space in file. is damaged. I/O processing did not complete.

Please show the log for that code with the error messages.

The fewer the details you provide the more guesses or questions you get before any resolution.

Regular Contributor
Posts: 199

Re: ERROR: Insufficient space in file. is damaged. I/O processing did not complete.

 
1 OPTIONS NONOTES NOSTIMER NOSOURCE NOSYNTAXCHECK;
55
56 DATA neiss_subset_1;
57 MERGE neiss_merge WORK.NEISS_2002_2015;
58 BY cpsc_case_no trmt_date;
59 RUN;

ERROR: Insufficient space in file WORK.NEISS_SUBSET_1.DATA.
ERROR: File WORK.NEISS_SUBSET_1.DATA is damaged. I/O processing did not complete.
NOTE: The SAS System stopped processing this step because of errors.
NOTE: There were 896761 observations read from the data set WORK.NEISS_MERGE.
NOTE: There were 1 observations read from the data set WORK.NEISS_2002_2015.
WARNING: The data set WORK.NEISS_SUBSET_1 may be incomplete. When this step was stopped there were 896760 observations and 28
variables.
WARNING: Data set WORK.NEISS_SUBSET_1 was not replaced because this step was stopped.
NOTE: DATA statement used (Total process time):
real time 0.90 seconds
cpu time 0.89 seconds


60
61 OPTIONS NONOTES NOSTIMER NOSOURCE NOSYNTAXCHECK;
73

☑ This topic is SOLVED.

Need further help from the community? Please ask a new question.

Discussion stats
  • 7 replies
  • 1299 views
  • 4 likes
  • 5 in conversation