topic Re: Introduction of Special characters in a new database - Will SAS cope? in SAS Programming

Introduction of Special characters in a new database - Will SAS cope?

Brian_lewis67 — Tue, 04 Jul 2017 09:11:49 GMT

Hi,

A new database is being introduced to the business which which will start to include special characters (Welsh language accented). I've been informed that this caused issues previously so were removed from the original database.

We will need to read the fields in with the special characters included and also output them into an SQL dataset and an Excel spreadsheet. I've tried to find out if at any point this will become an issue but can't find a definitive answer.

I've read that SAS can't output special characters without using the BYTE function but that wouldn't be possible in this instance.

If someone can give me the reassurance that SAS will handle it all fine that would be great.

Thanks,

Brian

Re: Introduction of Special characters in a new database - Will SAS cope?

RW9 — Tue, 04 Jul 2017 09:39:45 GMT

So long as you setup your system for the correct character set it should work, UTF-8 apparently has Welsh characters in:

https://www.hesa.ac.uk/support/user-guides/xml-files/unicode

As for specific reassurance I would put into SAS helpdesk a request for information and examples from them directly about what they support.

Re: Introduction of Special characters in a new database - Will SAS cope?

Brian_lewis67 — Tue, 04 Jul 2017 11:53:01 GMT

Thanks for the response.

When you say system is that the SAS system, Windows or SQL environment?

I will be honest I'm clueless when it comes to Unicode, etc

Re: Introduction of Special characters in a new database - Will SAS cope?

RW9 — Tue, 04 Jul 2017 11:59:57 GMT

The SAS System, should be more or less the same across Windows/Unix. SQL system is outside the scope of this forum.

Re: Introduction of Special characters in a new database - Will SAS cope?

Ksharp — Tue, 04 Jul 2017 13:22:55 GMT

Change your SAS encoding as UTF8.

D:\SASHome\SASFoundation\9.4\sas.exe -CONFIG "D:\SASHome\SASFoundation\9.4\nls\u8\sasv9.cfg"

and using UNICODE() function. you can get any special character.

Here is an example.

data have;
x=5;y=15;star=unicode('\u2605');output;star=' ';
x=10;y=11;star1=unicode('\u2605');output;star1=' ';
x=10;y=18;star2=unicode('\u2605');output;star2=' ';
x=12;y=13;star3=unicode('\u2605');output;star3=' ';
x=12;y=16;star4=unicode('\u2605');output;
run;


proc sgplot data=have aspect=0.6 noautolegend noborder;
styleattrs wallcolor=red;
text x=x y=y text=star/ textattrs=(color=yellow size=80pt) ;
text x=x y=y text=star1/ textattrs=(color=yellow size=40pt) rotate=60;
text x=x y=y text=star2/ textattrs=(color=yellow size=40pt ) rotate=60;
text x=x y=y text=star3/ textattrs=(color=yellow size=40pt ) ;
text x=x y=y text=star4/ textattrs=(color=yellow size=40pt) rotate=20;

xaxis display=none offsetmin=0 offsetmax=0 values=(0 30);
yaxis display=none offsetmin=0 offsetmax=0 values=(0 20);
run;

Re: Introduction of Special characters in a new database - Will SAS cope?

Patrick — Tue, 04 Jul 2017 21:51:07 GMT

@Brian_lewis67

Using a multibyte character set like UTF8 will support almost all languages. Using multibyte poses also some challenges though:

1. You must have the SAS binaries installed for multibyte. If that's not already the case then you've got an installation task ahead.

2. SAS files will require more storage space (up to 4 bytes per character). You need to define the lengths of your characters accordingly as in SAS the length definition of a character variable defines the number of bytes and not the number of character you can store.

3. Some SAS string functions only work properly with single byte character sets and switching to multibyte means you'll have to revisit and eventually amend existing SAS code.

http://support.sas.com/documentation/cdl/en/nlsref/69741/HTML/default/viewer.htm#p1pca7vwjjwucin178l8qddjn0gi.htm

It appears there is a singlebyte character set (codepage) which could work as well: ISO-8859-14

http://www.comisiynyddygymraeg.cymru/English/Part%203/10%20Locales%20alphabets%20and%20character%20sets/10.3%20Character%20sets/Pages/10-3-Character-sets.aspx

In the end of the day:

You want to figure out what codepages gets used in the databases you're interfacing with and then use something which matches.

Re: Introduction of Special characters in a new database - Will SAS cope?

Brian_lewis67 — Thu, 06 Jul 2017 14:04:54 GMT

Thanks to everyone for their response.

I added the Encode element to the config file and SAS refused to initialise correctly due to it being set as the wrong environment. I looked for a few answers and apparently there are other settings that have to be set within the config file so have left it for now as the config file is on a drive (C) that I'm not supposed to be able to access.

Re: Introduction of Special characters in a new database - Will SAS cope?

Tom — Thu, 06 Jul 2017 14:38:45 GMT

When my IT support installs SAS on users PCs they pre-install multiple configuration files and associated Windows ICONS. So when I look in the START menu on Windows under SAS there are multiple icons. In particular there are two that I use most often:

SAS 9.4 (English)

SAS 9.4 (Unicode Support)

If you use the second one then you should be able to handle Welsch characters (or any other special characters).

If you do not have a configuration file with Unicode Support then ask your local support team to install one.

Also if you have access to SAS/Studio (which comes free with PC SAS) then that will normally also be available with Unicode support.

Re: Introduction of Special characters in a new database - Will SAS cope?

Brian_lewis67 — Thu, 06 Jul 2017 14:45:35 GMT

Thanks Tom. Funnily enough I do have an option that states SAS 9.4 (Unicode support) but get the error:

ERROR: Invalid physical name for library SASUSER.
NOTE: Unable to initialize the options subsystem.
ERROR: (SASXKINI): PHASE 3 KERNEL INITIALIZATION FAILED.
ERROR: Unable to initialize the SAS kernel.

This is where I am out of my depth so will have to rely on a theoretical, more knowledgeable IT department