BookmarkSubscribeRSS Feed
🔒 This topic is solved and locked. Need further help from the community? Please sign in and ask a new question.
mouti
Fluorite | Level 6

Can someone help me with issue im facing ?

 

I'm woring with dataset from CRM  containing contacts. Names, surnames . dataset encoding is UTF-8, my sessions encoding is UTF-8. so up to my knowledge everything should work. But it's not. I'm recievieng such errror message (attached):

 

I need to remove diacritics etc.. so I'm using translate function to remove diacritics (Slovak)

%let TRANSLATE_FROM='ÁÄČĎÉÍĹĽŇÓÔŔŠŤÚÝŽ';
%let TRANSLATE_TO='AACDEILLNOORSTUYZ';

 

I'm also having list of polish letters I want to replace.. the thing is I can't

 

sas ,I 'dont know why, is trying to transcode data from utf-8 to utf 16-le ... 

 

i tried to parse string to find out what type of letters are present in string. Worked only with A='Tomas'

 

data A;
/*A = 'АлександраЪъ';*/
/*A= "Zażółć gęślą jaźń";*/
A='Tomas';
D=length(trim(a));
array parsed_vars(40) $ new_var1-new_var40;
Do i = 1 to D;
parsed_vars(i)=substr(a,i,1);
end;
run;

 

 

I'm running desktop version of SAS EG 9.4.

 

 


sas_error_encoding.jpg
1 ACCEPTED SOLUTION

Accepted Solutions
CaseySmith
SAS Employee

Have you run similar code in EG to confirm the session encoding?:

 

proc options group=languagecontrol;

run;

 

If you have confirmed it is UTF-8, I suspect you are running against your Local SAS server, rather than a remote SAS server?  Unlike the SAS Bridge protocol (remote server communication) which transmits in UTF-8, in COM (Local), strings have to be transmitted in UNICODE, UTF-16.  In this case, if there are characters that can't be transcoded from UTF-8 to UTF-16, you'll get the transcoding error.


Register today and join us virtually on June 16!
sasglobalforum.com | #SASGF

View now: on-demand content for SAS users

View solution in original post

5 REPLIES 5
Ksharp
Super User

It is famous error for EG. You have unprint or illegal character for EG, it is not problem about encoding.

Here are two option.

1) clean your sas table. to make all the character are legal for EG.

2) export this table into a CSV or TXT file ,then import it back by writing some sas code.

 

 

mouti
Fluorite | Level 6

What are illegal characters for EG? Do you mean reserved one like *&%$;... etc ? 

CaseySmith
SAS Employee

Have you run similar code in EG to confirm the session encoding?:

 

proc options group=languagecontrol;

run;

 

If you have confirmed it is UTF-8, I suspect you are running against your Local SAS server, rather than a remote SAS server?  Unlike the SAS Bridge protocol (remote server communication) which transmits in UTF-8, in COM (Local), strings have to be transmitted in UNICODE, UTF-16.  In this case, if there are characters that can't be transcoded from UTF-8 to UTF-16, you'll get the transcoding error.


Register today and join us virtually on June 16!
sasglobalforum.com | #SASGF

View now: on-demand content for SAS users

mouti
Fluorite | Level 6

please see log attached, Im running UTF8 on local machine (laptop)

 

Im working with data downloaded from PostrgeSQL DB. Customers can create their profile from web or they are imported from our internal systems. Is there way how to detect characters which can't be translated and remove them i.e. using compress function?

 

I don't have full IT background as I studied economics so this is beyond my knowledge 

 

Acording this paper http://unicode.org/faq/utf_bom.html  I suppose that UTF16 covers entire UTF8 and adds somthing more.. 

mouti
Fluorite | Level 6

i simply didn't use correct string functions able to work with double-byte character expressions (UTF8) 

 

http://morgan.dartmouth.edu/Docs/sas92/support.sas.com/documentation/cdl/en/nlsref/61893/HTML/defaul...

 

SAS could put such error message to log - use of inapropriate function

sas-innovate-2024.png

Join us for SAS Innovate April 16-19 at the Aria in Las Vegas. Bring the team and save big with our group pricing for a limited time only.

Pre-conference courses and tutorials are filling up fast and are always a sellout. Register today to reserve your seat.

 

Register now!

SAS Enterprise Guide vs. SAS Studio

What’s the difference between SAS Enterprise Guide and SAS Studio? How are they similar? Just ask SAS’ Danny Modlin.

Find more tutorials on the SAS Users YouTube channel.

Click image to register for webinarClick image to register for webinar

Classroom Training Available!

Select SAS Training centers are offering in-person courses. View upcoming courses for:

View all other training opportunities.

Discussion stats
  • 5 replies
  • 7098 views
  • 1 like
  • 3 in conversation