Special characters that stop the input processing

[oht]

Hi

We have a problem reading a huge file. The sas code only read part of the input records and stop without any error. Further examine the file where the input was stop, we find a symbol 'E S' that halted any further processing. The HEX value is '1A'.  We have tried:

data _null_;

infile fin lrecl=314 truncover;
input @;
_infile_=compress(_infile_,' ','NK');

input  ;
file tst;
put _infile_;
run;

But it does not work. Does anyone has this problem and how to resolve this issue ?

Thanks for all your help

[Linlin]

what is the purpose of   "compress(_infile_,' ','NK');"?

[oht]

Suppose to remove all the special char. But it didn't work in this case.

[Linlin]

if you want to only keep numbers then try:

_infile_=compress(_infile_,,'kd');

if you want to keep both numbers and characters then try:

_infile_=compress(_infile_,,'kda');

[oht]

Hi Linlin,

Your code elminated to many other symbol such as - , which we might need to retain. Besides, if the file come in as a fixed column, then the compress will shift that record left which can be a headache for the input statement.

Anyway, thanks for your input.

[art297]

Rather than trying to remove the characters, on your infile statement have you tried to include the ignoredoseof option?

[DanielSantos]

From the LRECL option you've specified I'm guessing that maybe this is a fixed rec size file, am I right?

If so, there's no CR/LF in the file and you have to specify the option recfm=F, then SAS will import everything to the end of each record (314 in length).

Then compress with modifier kpw (keep punction + printable chars) will remove the SUB code (1Ax).

data _null_;

infile fin lrecl=314 truncover recfm=F;

input @;

_infile_=compress(_infile_,'','kpw');

put _infile_;

run;

Cheers from Portugal.

Daniel Santos @ www.cgd.pt

[oht]

Thanks for your code. Our files are comma delimited.

[art297]

I still vote for trying the ignoredoseof option.  It was incorporated, specifically, to deal with the presence of 1A characters.  Take a look at: http://www.sascommunity.org/wiki/IGNOREDOSEOF_Option

[oht]

You are absolutely right. I tested the code and it works out beautifully. Thanks for your help.