topic Re: Reading in a pipe delimited, unicode text file in SAS Procedures

Reading in a pipe delimited, unicode text file

Don — Tue, 26 May 2015 21:45:03 GMT

All,

I have a series of files that are pipe delimited text files. However sas is reading in some special characters at the beginning of the file that show up as "<U+EEFF>". I assume this means the files are unicode. I have tried a file name statement with encoding="unicode" and encoding="UTF-8". It works at some level, but the file now has a ^z at the beginning. I thought it may be the kind of unicode being used, so I tried all of the different codes listed in the sas help documentation. None work. I have searched though the discussion, but haven't been able to solve it from what I have found so far. Is it possibly some other version of encoding? Is there a way to strip all special characters? Any other suggestions?

Thanks in Advance!

Edit: Here is the code I am trying to use:

FILENAME ABC "OLDFILE.TXT" ENCODING="UTF-8"; ******HAVE TRIED MANY DIFFERENT ENCODING TYPES***********;

PROC IMPORT DATAFILE = ABC

OUT=NEWDATA

DBMS=DLM REPLACE;

DELIMITER='|';

GETNAMES=YES;

DATAROW=2;

RUN;

Message was edited by: Donald Hale

Re: Reading in a pipe delimited, unicode text file

ballardw — Tue, 26 May 2015 21:56:22 GMT

Please provide an example of the code you are running, especially the one that came closest to what you want.

If you are reading the files using a wildcard the ^z may be coming from the end of a previous file.

Re: Reading in a pipe delimited, unicode text file

Steelers_In_DC — Wed, 27 May 2015 11:05:20 GMT

This is a unix feature, it's been years since I was working with this frequently so I don't remember what it represents...a tab or enter I think. If you have putty, bluezone or any other unix access you want to vi into the file and copy paste into the vi editor, the Z is case sensitive:

:%s/^Z//g

That should do it.