topic Mask Unicode: 0x1a character. in SAS Programming

Mask Unicode: 0x1a character.

Yura2301 — Mon, 24 Mar 2014 13:47:19 GMT

Hi all,

Have issue with XML created from SAS , will be appreciate for any help.

So I create XML from SAS using standard libname engine:

filename &filLibName "...temp.xml" ;

LIBNAME &filLibName XML XMLTYPE=GENERIC ;

data &filLibName..TEST_XML;

set TEST_DATA;

run;

Then XML goes to validation on java side.For majority of data validation pass ok, without issue,but for some set of data issue occurs, and XML rejects with reason:

An invalid XML character (Unicode: 0x1a) was found in the element content of the document.

It looks like some invalid character come with SAS data and java validation tool reject XML due this character.

XML should be in UNICODE format so I tried to changed libname to :

LIBNAME &filLibName XML XMLTYPE=GENERIC XMLENCODING='utf8';

And this step changed XML header to:

.<?xml version="1.0" encoding="utf-8" ?>

Some special character occurs before first tag and it again crashed validation on java side, so I changed it in code(just hardcoding), so XML header after this looks like:

<?xml version="1.0" encoding="utf-8" ?>

But the problem still exists, XML message still rejects with same error.

I browsed and found that java developers fixed this issue simply by replacing this char to another allowed character:

http://stackoverflow.com/questions/8505816/streamexception-an-invalid-xml-character-unicode-0x1a

It's also ok for me, but I should realize it on SAS and I actually can't find this character in SAS table,so queston is if there are some smart way to search and replace this symbol in SAS data?

Maybe there are also some way to predict similar issues?I supposed enough should be set option XMLENCODING='utf8' to libname, but as I experienced it's not enough.

I'm using SAS 9.1.3.

Thanks!

Re: Mask Unicode: 0x1a character.

Tom — Mon, 24 Mar 2014 14:04:15 GMT

Since it only happens for some of the data I suspect that the value is actually in the data.

You could try check the character variables in your data sets and eliminate the characters it does not like before writing it to the XML file.

Re: Mask Unicode: 0x1a character.

Yura2301 — Mon, 24 Mar 2014 14:19:36 GMT

Hi Tom,

It was my initial idea, but the problem isn't so obvious probably.

I have small SAS table with ~20 rows.

When I send whole this table - issue occurs.

When I eliminate one row - issue doesn't occurs, but then, if I resend this one row - issue also doesn't occurs.

This character probably isn't visible:

http://www.utf8-chartable.de/unicode-utf8-table.pl?utf8=0x

And I can't just visually separate it in data.

I'm actually a little bit confused why the issue is happend and if this spec character at all is in SAS data, maybe it occurs on stage when SAS create XML from SAS table.

I also tried to find this char directly in XML file via n++, but can't see it also.

Thanks!