topic Re: Could not open data file: ERROR: I/O processing did not complete in SAS Programming

Could not open data file: ERROR: I/O processing did not complete

danwarags — Tue, 31 May 2016 13:51:42 GMT

I am trying to open a large data set which is in a format .sas7bdat (1.2TB), but I am getting this error. It says ' Could not open the data file: File is damaged. I/O processing did not complete. How do I fix this error

Re: Could not open data file: ERROR: I/O processing did not complete

Reeza — Tue, 31 May 2016 14:26:53 GMT

What did you mean by open?

I'm not sure trying to view a 1.2TB file is a good idea. Take a subset and view that perhaps?

You can try repairing the dataset using proc datasets but if the file is damaged it may not be recoverable.

Proc datasets library=work nolist nodetails;
Repair brokenData;
Run;quit;

Re: Could not open data file: ERROR: I/O processing did not complete

rivieralad — Tue, 31 May 2016 14:34:58 GMT

I have seen similar when the machine on which I was working ran out of disk space when trying to open a large dataset.

You will need to find the drive on which your SAS work folder resides and check the available space.

To find out where it is check in your config file (SASV9.CFG) and look for the line starting -WORK.

If there is sufficient space then it could be insufficient memory.

HTH

Cheers

Chris

Re: Could not open data file: ERROR: I/O processing did not complete

danwarags — Thu, 02 Jun 2016 16:12:52 GMT

Hello Reeza,

Thanks for the reply. The SAS dataset (1.3TB) needs to be analyzed. So, working on the subset of the data might not help me. I wanted to check the number of variables inside the large dataset by running the CONTENTS procedure. As the dataset is large, there is an error showing that the 'File cannot be processed because its size is limited to 2G-1 observations on this host'. How do I fix this problem

Re: Could not open data file: ERROR: I/O processing did not complete

danwarags — Thu, 02 Jun 2016 16:20:38 GMT

Thanks for the reply Chris. How do I check config file (SASV9.CFG) you mentioned. Please guide me

Re: Could not open data file: ERROR: I/O processing did not complete

Reeza — Thu, 02 Jun 2016 17:44:06 GMT

I wasn't aware that SAS ever had an limit. What is your SAS setup?

Is this a SAS dataset or a dataset on a server?

Re: Could not open data file: ERROR: I/O processing did not complete

danwarags — Thu, 02 Jun 2016 20:58:40 GMT

Thanks reeza.The SAS dataset is on my local drive and not on a server. I tried searching this error on google and found that the SAS only supports 2G-1 observations. The link I found this information is below:

http://support.sas.com/kb/36/112.html

I dont know what to do to fix this problem. But I really need to get a variable list from this large dataset and do analysis on them.

Re: Could not open data file: ERROR: I/O processing did not complete

PoornimaRavishankar — Thu, 02 Jun 2016 21:21:58 GMT

First off,I didn't know you could get a local machine with >1.2 TB hard disk space 🙂

Second, to get just a variable list, can you try this?

*****************************************************

data varlist;

set <library.datasetname> obs=1;

run;

proc contents data=varlist;

run;

*****************************************************

As for the SASV9.CFG file, you should look for it in your SAS s/w folder. Try C:\Program Files (x86)\SASFoundation\9.2\sasv9.cfg .. or something similar, once you get to the path, it musn't be difficult to find. Unfortunately, I don't have a local installation.

Check this article - http://support.sas.com/documentation/cdl/en/hostwin/63285/HTML/default/viewer.htm#a000104286.htm#worklib and search for 'SAS Default Folder Structure'. The default path is listed there.

Now once you find it, how you determine 'sufficient' space, I cannot say.

Re: Could not open data file: ERROR: I/O processing did not complete

Reeza — Thu, 02 Jun 2016 22:40:26 GMT

What type of analysis are you planning to do. The link you provided has some workarounds listed, have you tried them and they don't work.

If you take a sample you'll still get the list of variables and can develop your code. Any type of processing is going to take time...and space. Will you have enough space to work with the data, since you usually need 2-3 times the space to process the data.

You may want to contact tech support to get some help specifically for your situation.

Re: Could not open data file: ERROR: I/O processing did not complete

rivieralad — Fri, 03 Jun 2016 12:49:30 GMT

Sorry for delay in replying, been away from office.

You can find this information in previous messages if you search Communities but to save time:

1. Run proc options:

proc options option = config;
run;

2. Should give you something similar to below:

1    proc options option = config;
2    run;

    SAS (r) Proprietary Software Release 9.1 TS1M3

CONFIG=C:\Program Files\SAS\SAS 9.1\nls\en\SASV9.CFG
                   Specifies a SAS configuration file.
NOTE: PROCEDURE OPTIONS used (Total process time):
      real time           0.24 seconds
      cpu time            0.00 seconds

3. Open up the config file specified.

4. Look for the line starting -WORK, as in the following example

/* DO NOT EDIT BELOW THIS LINE - INSTALL Application edits below this line */
/* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * */
-SET sasext0 "C:\Program Files\SAS\SAS 9.1"
-SET sasroot "C:\Program Files\SAS\SAS 9.1"
-SET sasext1 "C:\Program Files\SAS\SAS 9.1\nls"
/* Setup the MYSASFILES system variable                         */
-SET MYSASFILES "?CSIDL_PERSONAL\My SAS Files\9.1"
/* Setup the default SAS System user profile folder             */
-SASUSER "?CSIDL_PERSONAL\My SAS Files\9.1"
/* Setup the default SAS System user work folder                */
-WORK "!TEMP\SAS Temporary Files"

5. Navigate to the folder (if as above it shows !TEMP\... this will be your temporary files location e.g. C:\Documents and Settings\userid\Local Settings\Temp). The space available on the drive should be visible in the status bar at the bottom left of Windows Explorer. If there isn't enough space, you can specify a new work location following the instructions in SAS.

Cheers

Chris