topic Re: ERROR: Insufficient space in file WORK. ERROR: WORK is damaged. I/O processing did not complete. in SAS Studio

ERROR: Insufficient space in file WORK. ERROR: WORK is damaged. I/O processing did not complete.

andreaat — Thu, 01 Aug 2019 20:57:43 GMT

I'm cleaning a dataset using SAS University Edition on a Mac OS with over 300 GB of storage.

My dataset has over 30,000 observations and about 3,000 variables.

I am creating new work libraries every time I create a new variable. Then I got this error message.

ERROR: Insufficient space in file WORK.CLSA6.DATA.

ERROR: File WORK.CLSA6.DATA is damaged. I/O processing did not complete.

PLEASE HELP 🙂

Re: ERROR: Insufficient space in file WORK. ERROR: WORK is damaged. I/O processing did not complete.

Kurt_Bremser — Thu, 01 Aug 2019 19:15:50 GMT

The first step is unnecessary, as AGE_DCS_GRP_COM does not exist anyway.

And you can do all the other actions in one step.

By the way, 3000 variables? This screams "design failure" at me.

Re: ERROR: Insufficient space in file WORK. ERROR: WORK is damaged. I/O processing did not complete.

Reeza — Thu, 01 Aug 2019 19:16:43 GMT

That's not nearly enough data to cause an issue. You say you're creating new 'work libraries' can you explain that?

At this point, you should restart the VM and see if the issue remains. If it does, reinstall the image (not Oracle, but readd the virtual image) and it should reset it for you. You won't lose files in myfolders, just don't delete them.

Re: ERROR: Insufficient space in file WORK. ERROR: WORK is damaged. I/O processing did not complete.

Kurt_Bremser — Thu, 01 Aug 2019 19:28:36 GMT

If this is a result of an Excel import, with lots of $254 variables, several runs recreating the datasets side-by-side can crack the limits of UE.

Re: ERROR: Insufficient space in file WORK. ERROR: WORK is damaged. I/O processing did not complete.

Reeza — Thu, 01 Aug 2019 19:31:56 GMT

Agreed. There's also the confusion with the code - all could be accomplished with formats that would make this very trivial so far.

Re: ERROR: Insufficient space in file WORK. ERROR: WORK is damaged. I/O processing did not complete.

andreaat — Thu, 01 Aug 2019 21:00:22 GMT

Thanks, I've figured it out. Instead of creating multiple work libraries, I'm just making all my new variables and data cleaning in the same work library. It seems that making multiple work libraries of 30,000 data points is too much. There is no error with the import or anything like that.

Re: ERROR: Insufficient space in file WORK. ERROR: WORK is damaged. I/O processing did not complete.

andreaat — Thu, 01 Aug 2019 21:01:43 GMT

Doing it all it one step fixes things. Thanks!

3000 variables is not 'design failure', but rather it is 'big data' lol

Re: ERROR: Insufficient space in file WORK. ERROR: WORK is damaged. I/O processing did not complete.

Reeza — Thu, 01 Aug 2019 22:18:09 GMT

I think you're using the term 'work library' wrong and I'm only pointing it out because it's a very commonly used term that will cause confusion down the line. I *think* you're referring to your temporary data sets instead, which are stored in the work library. You were creating a new temporary data set for each step rather than continuing to build on the old ones. You can still program in that manner, just add the new code in steps instead of new data sets each time. Or you can do drop variables that aren't needed ahead of time if that's an option.

Re: ERROR: Insufficient space in file WORK. ERROR: WORK is damaged. I/O processing did not complete.

mvniekerk — Sat, 18 Jan 2020 21:54:55 GMT

Reeza

I have the same problem. I am creating a LOT of datasets in my program and get the same error.

"... just add the new code in steps instead of new data sets each time ... "

How do you do this?

Even with your solution, I am beginning to wonder if SAS University Edition is powerful enough?

Regards

Michiel

Re: ERROR: Insufficient space in file WORK. ERROR: WORK is damaged. I/O processing did not complete.

Kurt_Bremser — Sat, 18 Jan 2020 22:15:48 GMT

University Edition is a learning tool (with specific applications in mind, like the free online SAS courses), not meant to do real-world analysis for free. Assigning a USER library under /folders/myfolders can help.

Re: ERROR: Insufficient space in file WORK. ERROR: WORK is damaged. I/O processing did not complete.

Reeza — Sun, 19 Jan 2020 03:08:55 GMT

SAS UE is not powerful at all, it's like the full version of SAS with it's hands tied behind it's back.

My comment below is intended to say, instead of coding like this:

data temp1;
set sashelp.class;

if age < 14 then desc = 'Pre-Teen';
else if age<16 then desc = 'Teen';
else if age < 21 then desc = 'Young Adult';
run;

data temp;
set temp1;

if sex = 'F' then sex_category = 1;
else sex_category=2;

run;

Instead do this:

data temp1;
set sashelp.class;

if age < 14 then desc = 'Pre-Teen';
else if age<16 then desc = 'Teen';
else if age < 21 then desc = 'Young Adult';

if sex = 'F' then sex_category = 1;
else sex_category=2;

run;

However, that was referring to the OP's specific situation, yours may differ.

@mvniekerk wrote:

Reeza

I have the same problem. I am creating a LOT of datasets in my program and get the same error.

"... just add the new code in steps instead of new data sets each time ... "

How do you do this?

Even with your solution, I am beginning to wonder if SAS University Edition is powerful enough?

Regards

Michiel

Re: ERROR: Insufficient space in file WORK. ERROR: WORK is damaged. I/O processing did not complete.

Reeza — Sun, 19 Jan 2020 03:10:19 GMT

https://github.com/statgeek/SAS-Tutorials/blob/master/SAS%20UE%20-%20How%20to%20work%20with%20big%20files.md