topic Re: I/O Processing did not complete error in SAS Programming

I/O Processing did not complete error

dkanand86 — Wed, 07 Dec 2016 16:34:32 GMT

Hi Everyone,

Quick thing , when i execute following sas code in SAS 9.3 Windows, it gives the error, please advise for possible solution, thanks:

ERROR:

NOTE: A utility file had to be opened due to a lack of memory.
PROC TRANSPOSE might run much faster if 97511007600 more bytes of memory were available.
ERROR: File DD.TEMP6.DATA is damaged. I/O processing did not complete.

SAS CODE:

data dd.temp5;
set dd.temp4;
obs=compress('var_'||_n_);
run;

proc transpose data=dd.temp5 out=dd.temp6 prefix=var;
var &varlist;
run;

data dd.temp7;
set dd.temp6;
array vars[*] $ _all_;
do i=1 to dim(vars);
len=lengthn(vars[i]);
newval=compress(vars[i],".",'d');
len2=lengthn(newval);
if len2=0 then N_FLAG=1; ELSE N_FLAG=0;
end;
drop i;
run;

proc transpose data=dd.temp7 out=dd.temp8;
run;

Regards,

Re: I/O Processing did not complete error

Shmuel — Wed, 07 Dec 2016 17:09:14 GMT

can you run SAS session with system option memsize=0 ?

If yes - that will use maximum memsize the instalation enables.

You can't use this option in an open session, but only when you launch SAS.

Re: I/O Processing did not complete error

ballardw — Wed, 07 Dec 2016 18:25:17 GMT

Since you have multiple proc transpose statements it would help to post the log results for the entire program. It would also help to show the value of any macro variables.

It may be a good idea to describe what this process is meant to accomplish as well.

Re: I/O Processing did not complete error

mkeintz — Wed, 07 Dec 2016 21:14:02 GMT

You appear to be looking for character values which cannot be converted to numerics. You do this test over all vars in each row of the transposed data, setting NFLAG=1 if a convertable value occurs (else nflag=0). But the problem (well one of the problems) is that you do this in sequence for ALL the variables, resetting NFLAG each time. So NFLAG really only reflects the status of the last variable in each row, (i.e. the last row of the original untransposed data set).

And then you transpose back, which by default transposes only the numeric vars (LEN, LEN2 and NFLAG), yielding pretty useless information.

So I guess what you really want is to check each column of the original data set, to see if there are any non-convertable values. Consider this structure with ~~improved~~ improvised dataset HAVE. It will (1) report to the log any vars with any non-convertable values, and (2) generate dataset BADCOUNTS with one row containing number of non-convertable values for each of the original variables:

filename tmp temp;  /* Type is TEMP so it will automatically be deleted */

data _null_;
  set have end=eod;
  array chr {*} _character_;
  array nbad {100}  _temporary_ (100*0);
  do I=1 to dim(chr);
    nbad{I}=nbad{I} + NOT(compress(chr{I},'.','d')=' ');
  end;

  if eod then do;
    file log;
    put 'These vars have at least one instance that cant be converted to numeric:';
    do I=1 to dim(chr);
      chr{I}=put(nbad{I},3.);
      if nbad{I}>0 then put chr{I}=;
    end;
    file tmp;   /* Write the next PUT to this destination */
    put (chr{*}) (= ';')  ';';
  end;
run;

options source2;  /* to print out code retrieved via the %include below */

data badcounts;
  %include tmp;
run;