Failed to transcode data from U_UTF8_CE to U_LATIN1_CE

[jackhost]

I have a big problem.When I use SAS EG to finish my model,there is always a mistaken like the picture.

How can I solve it?

Please give some details about that.thank you so much.

[jakarman]

Have your SAS session run a utf-8 one.  You will better use several connections (possible with server side) with utf8/latin-1 encodings.

Using a utf-8 sas session will require attention to character functions as not all of them are valid with utf8.

SAS(R) 9.4 National Language Support (NLS): Reference Guide, Third Edition

Would go for 9.4 with SAS/connect better supporting the interconnections.

[jackhost]

I appreciate your help! I realized the mistaken was something wrong with encoding.But how can I have my SAS session run a utf-8 one ? The encoding of my SAS server side is latin1 default.How can I change the configuration about encoding？

[Patrick]

Below two links should provide the information you're after.

SAS(R) 9.3 National Language Support (NLS): Reference Guide

SAS(R) 9.3 National Language Support (NLS): Reference Guide

[jackhost]

I found a solution on internet.The solution says adding the latin1 encoding to sasv9_usermods_cfg file.But I don't know how to do that. 

[Patrick]

You need to have admin privileges to change the .cfg. But have a look at the links I've posted. Some of the options work also as part of an options statement.

This bit in the docu for an option tells you where you can set/change it. If it says "configuration file, SAS invocation" then it's the .cfg or the autoexec, if it's "OPTIONS statement" then you can also set it in your code.

Valid in:

configuration file, SAS invocation, OPTIONS statement, SAS System Options window

[jackhost]

I reset the server side and deployed it as a unicode server.The error was solved.Maybe your answer also can solve this error,But the time is not enough for me.Thank you so much!

I also want to ask one question.

Now,I have a client,a sas application server and a database server(DB2),They are located in different computers.

I can use 'libname' statement to connect to the database.Can I directly analyse the data on the logic table from database?(the error :failed to I/O )

Should I move the data to the sas application server and then to analyse?

[jakarman]

Needed to process your DB2 question. When you are connecting that by a libname (db2 client configured and sas/access DB2)  working,  you can use that as input for analyses. just take care for the datatypes: SAS/ACCESS(R) 9.4 for Relational Databases: Reference, Fifth Edition

Wanting to do more with that interface you could use DB2 specific functions with explicit pass through or going for DS2.
DS2 is new world for accessing modifying data. SAS(R) 9.4 DS2 Language Reference, Third Edition (if you have time left to spend)    

[jackhost]

I have successfully connected the database using 'libname' statement like following picture(can see the tables):

then there are two errors when I use this code;

this is my database server code set(gbk):

So the reason why cause errors is the encoding of database is gbk and the application server is unicode？

[Patrick]

As points out you're not having enough disk space in your SAS work area.

To answer one of your other questions: Yes, it's a good idea to push processing to the data base. The normal approach is to first reduce volumes on the data base and then only move the data you actually need over the network into SAS.

For your code: If you can, select only the columns you need (so no Select *) and subset the rows if possible (where ....). Also should you only need summary values then push this bit as well to the data base (eg. using a group by statement in your SQL code).

Some SAS procedures allow you to push all processing to the data base (eg. Proc Freq and Proc Means). So if you can get your desired result with these procedures then use them.

Using below options shows you in the log where processing happens  SAS/ACCESS(R) 9.4 for Relational Databases: Reference, Fifth Edition

options sastrace=',,,d' sastraceloc=saslog nostsuffix;

There are also other techniques for improving performance when dealing with data stored in a data base. If you're working regularly with DB2 then it might be worth to spend some time and get up-to-speed with these techniques.

Here only one of many possible links: http://support.sas.com/documentation/cdl/en/acreldb/67473/HTML/default/viewer.htm#n13m5rcjajo3c5n12l...

[jackhost]

Thanks a lot.I learn much about SAS this time.

I hope you and  can help me when I face new problems.:smileylaugh:

[jakarman]

Multilanguage is becoming better.  http://support.sas.com/resources/papers/Multilingual_Computing_with_SAS_94.pdf

Still you have to deal with an en (English latin1 single byte) and u8 (utf8 multibyte) sas session.  To avoid compatiblity issue uses several appservers

http://support.sas.com/resources/papers/AddingAdditionalSASWorkspaceServerstoSupportMultipleEncoding...

Having done a serverside installation supporting Unicode and en language it is small configuration step to add an addtional server and change the sas-scriptname that will be used for the app-server (u8 or en) in the usermods-files.

Before 9.4 connections of sessions with different encoding is a disaster SAS/CONNECT(R) 9.2 User's Guide (Connect) and Proc cport / cimport also fails.

As you are mentioning DB2, that one is often limited to latin-1. When that is the case the DB2-client is ok.

Going for  DBMS supporting Unicode is posing an other issue. The RDBMS client will possible give back 4 bytes when in utf8 mode. With latin1 you need just one. As the DBMS client does this translation you possible need two configurations of the DBMS client.
Check this when you are getting into unexpected growth of char variables. (multiplied by 4). 
  

[jackhost]

It's me again.I have a new problem.

I want to test the highest pressure of my database server and SAS server(the maximum of SAS users) .

I have no idea about that.

Could you give me some advice or methods?

[jakarman]

You have a working connection nice.
It wil get the data to the sas-dataset work.test , then putting data there coming from db2 is running out of space of the saswork
SASwork is a standard name in SAS for temporary data. There is an other utilloc that is special for utilities  sort sql.

You analyses will probably run with small data (eg obs100).

I do not know how you configured that work, but it is running out of space.

Can you check the physical name and sizing on the server of that one?  To get the information on that:  

libname work list ;  /* shows all attributes */

proc options; run;  /* shows all std options   look for work   */

[jackhost]

yes.I checked the information about my SASWork.

it's too small to copy the whole data from database.How to enlarge the size of SASWork?