topic Re: NonEnglish Character in SAS Programming

NonEnglish Character

SASPhile — Wed, 01 Aug 2018 19:46:27 GMT

How to set a flag if a string has nonEnglish character?

Re: NonEnglish Character

PeterClemmensen — Wed, 01 Aug 2018 19:48:44 GMT

Is a non-english character a character not in a-z?

Re: NonEnglish Character

SASPhile — Wed, 01 Aug 2018 19:50:29 GMT

Yes. It has some chinese charcters

Re: NonEnglish Character

PeterClemmensen — Wed, 01 Aug 2018 20:01:40 GMT

Do something like this

data have;
string="abc";output;
string="ab人物";output;
string="xyz";output;
run;

data want;
   set have;
   flag=ifn(notalpha(string),1,0);
run;

Re: NonEnglish Character

PeterClemmensen — Wed, 01 Aug 2018 20:04:12 GMT

This solution however, relies on your OPTIONS LOCALE= System Option

Let me know if it does not meet your needs.

Re: NonEnglish Character

SASPhile — Wed, 01 Aug 2018 20:06:34 GMT

flag is set to 1 for all values

Re: NonEnglish Character

PeterClemmensen — Wed, 01 Aug 2018 20:19:42 GMT

Ok. Does this work for you?

data have;
string="abc";output;
string="ab人物";output;
string="xyz";output;
run;

data want;
   set have;
   flag=ifn(lengthn(compress(string, "abcdefghijklmnopqrstuvwxyz", "i")),1,0);
run;

EDIT: I added an IFN Function to the code.

Re: NonEnglish Character

ybolduc — Wed, 01 Aug 2018 20:23:49 GMT

I think regular expression would be the easiest way to do it. Here is how I would do this:

data want;
  length mytext $100.;
  input mytext $;
  flag = ifn(prxmatch('/[^a-zA-Z0-9 ]/', mytext) > 0, 1, 0);
datalines;
Hello
Sébastien
;
run;

Re: NonEnglish Character

PGStats — Wed, 01 Aug 2018 20:29:33 GMT

Try this if you have National Language Support

data want;
   set have;
   flag= string ne basechar(string);
run;

Try it on a real sample of your data, not datalines text.

Re: NonEnglish Character

SASPhile — Wed, 01 Aug 2018 20:47:03 GMT

I cannot extend it to below, it fails:

flag=lengthn(catx("-",string1, string2),"abcdefghijklmnopqrstuvwxyz", "i");

Re: NonEnglish Character

novinosrin — Wed, 01 Aug 2018 20:59:47 GMT

@PeterClemmensen , The code of yours can be tweaked to

data have;
string="abc";output;
string="ab人物";output;
string="xyz";output;
run;

data want;
   set have;
   flag=lengthn(compress(string, " ", "ai"))>0;
/*   flag=ifn(lengthn(compress(string, "abcdefghijklmnopqrstuvwxyz", "i")),1,0);*/
run;

Re: NonEnglish Character

ChrisNZ — Wed, 01 Aug 2018 23:55:06 GMT

This

if length(CHAR) ne klength(CHAR);

will detect any character that doesn't use single-byte encoding.

Re: NonEnglish Character

Patrick — Thu, 02 Aug 2018 05:22:58 GMT

@ybolduc

You need to be careful which string function you're using as soon as it comes to dealing with multi byte characters.

The PRX...() functions are unfortunately only good for single byte.

http://support.sas.com/documentation/cdl//en/nlsref/69741/HTML/default/viewer.htm#p1pca7vwjjwucin178l8qddjn0gi.htm