topic Re: Error on Insufficient space in Administration and Deployment

Error on Insufficient space

hanfei28 — Mon, 15 Jul 2019 21:03:38 GMT

Hi,

I am replicating code Program 2.10 in the material (https://support.sas.com/content/dam/SAS/support/en/books/free-books/exploring-sas-viya.pdf).

data bigcars;
set sashelp.cars;
do i=1 to 1000000;
output;
end;
run;

it shows

ERROR: Insufficient space in file WORK.BIGCARS.DATA.

ERROR: File WORK.BIGCARS.DATA is damaged. I/O processing did not complete.

I am using SAS Studio with Viya 3.3. I am wondering why I encounter insufficient space error.

Thanks,

Fei Han

Re: Error on Insufficient space

Reeza — Mon, 15 Jul 2019 21:10:10 GMT

Are you working on a company installation? If so, you'll need to talk to your IT team about space allotment.

Re: Error on Insufficient space

hanfei28 — Mon, 15 Jul 2019 21:23:40 GMT

Thanks @Reeza . I am running code on a company installation.

Re: Error on Insufficient space

Reeza — Tue, 16 Jul 2019 03:59:19 GMT

Then your IT has likely set up a limit per person that's too low or isn't running the clean work utility to clean up the libraries.

Re: Error on Insufficient space

Kurt_Bremser — Tue, 16 Jul 2019 08:14:36 GMT

Do you really want to multiply the observations by 1 million? On my installation, that would result in a 66 GB file, exceeding my (self imposed, as I am the admin) quota by a factor of >6.

Using a factor of 100,000 worked, resulting in a ~6.4 GB dataset file.

Re: Error on Insufficient space

Kurt_Bremser — Tue, 16 Jul 2019 08:24:12 GMT

PS my test run with a factor of 100,000 took 1 minute and 21 seconds on a 2-core pSeries. The log of the data step in the pdf shows a time of just 23 seconds for 10 times that amount, so SAS must have run that on a really big honking piece of iron.

This:

"We can see our CAS session has started and that we have 14
workers. Each worker has 16 CPU cores, which gives us a total of 224 threads."

(emphasis by me)

should give you a picture of the machine(s) used in that test.

You need to get a picture of the hardware available to you (both in SAS 9.4 and Viya) and adapt the codes accordingly.

Re: Error on Insufficient space

hanfei28 — Tue, 16 Jul 2019 12:24:32 GMT

Thanks for your answer @Kurt_Bremser @Reeza

Here are my system information. Viya system has one machine with 16 CPU cores, RAM=126GB.

Re: Error on Insufficient space

Kurt_Bremser — Tue, 16 Jul 2019 12:37:18 GMT

@hanfei28 wrote:

Thanks for your answer @Kurt_Bremser @Reeza

Here are my system information. Viya system has one machine with 16 CPU cores, RAM=126GB.

As long as you don't run it in CAS, the size of your WORK (on disk) is important. Have that checked.

After switching to CAS, since the projected size of the 1 million multiple dataset is 66 GB, that would eat more than half of your available RAM. As an admin, I would NOT let a single user occupy half my space.

Bottom line: reduce the multiply factor to a value that is sufficient to display the performance difference between SAS 9.4 and CAS/Viya.