topic Re: Special characters that stop the input processing in SAS Programming

Special characters that stop the input processing

oht — Thu, 06 Sep 2012 18:52:57 GMT

We have a problem reading a huge file. The sas code only read part of the input records and stop without any error. Further examine the file where the input was stop, we find a symbol 'E S' that halted any further processing. The HEX value is '1A'. We have tried:

data _null_;

infile fin lrecl=314 truncover;
input @;
_infile_=compress(_infile_,' ','NK');

input ;
file tst;
put _infile_;
run;

But it does not work. Does anyone has this problem and how to resolve this issue ?

Thanks for all your help

Re: Special characters that stop the input processing

Linlin — Thu, 06 Sep 2012 18:58:20 GMT

what is the purpose of "compress(_infile_,' ','NK');"?

Re: Special characters that stop the input processing

oht — Thu, 06 Sep 2012 19:01:26 GMT

Suppose to remove all the special char. But it didn't work in this case.

Re: Special characters that stop the input processing

art297 — Thu, 06 Sep 2012 19:04:17 GMT

Rather than trying to remove the characters, on your infile statement have you tried to include the ignoredoseof option?

Re: Special characters that stop the input processing

Linlin — Thu, 06 Sep 2012 19:07:09 GMT

if you want to only keep numbers then try:

_infile_=compress(_infile_,,'kd');

if you want to keep both numbers and characters then try:

_infile_=compress(_infile_,,'kda');

Re: Special characters that stop the input processing

DanielSantos — Fri, 07 Sep 2012 09:02:45 GMT

From the LRECL option you've specified I'm guessing that maybe this is a fixed rec size file, am I right?

If so, there's no CR/LF in the file and you have to specify the option recfm=F, then SAS will import everything to the end of each record (314 in length).

Then compress with modifier kpw (keep punction + printable chars) will remove the SUB code (1Ax).

data _null_;

infile fin lrecl=314 truncover recfm=F;

input @;

_infile_=compress(_infile_,'','kpw');

put _infile_;

run;

Cheers from Portugal.

Daniel Santos @ www.cgd.pt

Re: Special characters that stop the input processing

art297 — Fri, 07 Sep 2012 13:27:33 GMT

I still vote for trying the ignoredoseof option. It was incorporated, specifically, to deal with the presence of 1A characters. Take a look at: http://www.sascommunity.org/wiki/IGNOREDOSEOF_Option

Re: Special characters that stop the input processing

oht — Fri, 07 Sep 2012 13:44:47 GMT

You are absolutely right. I tested the code and it works out beautifully. Thanks for your help.

Re: Special characters that stop the input processing

oht — Fri, 07 Sep 2012 13:45:58 GMT

Thanks for your code. Our files are comma delimited.

Re: Special characters that stop the input processing

oht — Fri, 07 Sep 2012 13:50:09 GMT

Hi Linlin,

Your code elminated to many other symbol such as - , which we might need to retain. Besides, if the file come in as a fixed column, then the compress will shift that record left which can be a headache for the input statement.

Anyway, thanks for your input.