topic Re: Transcoding Problem in SAS Enterprise Guide

Transcoding Problem

agreekgod — Mon, 06 Apr 2015 09:09:08 GMT

Hello guys,

When I merge a particular dataset, Organization2014, in SAS with others I continually come up with the same error regarding transcoding. I have been able to find that this datase is in Wlatin1 format, but I don't know how I can convert it so that it could be merged with the other datasets. Below is a screenshot of the error if it is any use. Thank for taking your time to read the question, I really appreciate i

Re: Transcoding Problem

TomKari — Mon, 06 Apr 2015 16:07:32 GMT

I don't know if this will apply to you, but at my former organization I had a simple rule. When someone called, and mentioned an error that included the word "transcoding", I would reply that they had copied text out of Microsoft Word and tried to use it in SAS. There are a number of features, like smart quotes, where Word introduces characters that cause default SAS to burp.

If this applies to you, take a look at your data for things like slanted quotes, unusual dashes, and special characters. They could be the problem.

Tom

Re: Transcoding Problem

jakarman — Mon, 06 Apr 2015 16:36:57 GMT

Where is the screenshot?

I think the correct word is encoding when your are about NLS SAS(R) 9.4 National Language Support (NLS): Reference Guide, Third Edition

If you look at that you will find even Wlatin1 is used in different charactersets/encodings.
Your messages is indicating a difference has been marked so the process of transcoding will try to correct the differences

The most modern encoding used with internet (html) and office is utf8. This encoding is of variable length 1-4 bytes and you need a run-time version of SAS to handle that. Eguide however (.NET) is always utf8.

Another common message is foreign native format. That is about CEDA Moving and Accessing SAS(R) 9.4 Files, Second Edition.

Originally it was aprt of SAS/connect (SAS v6) but move to base as a generic approach. The cause of this is not having the SAS-datasets being processed by the current version. SAS is weird to common RDBMs approaches allowing cross ref versions machine layout to be read and created.

You should think of sas-datasets / tables as they are RDBMS like an not drag/drop using the files-tab of Eguide or Windows explorer.

Use the SAS explorer and libnames to access data.

Re: Transcoding Problem

agreekgod — Mon, 06 Apr 2015 20:09:36 GMT

Here is the screenshot. On the dataset I am using there are some special characters that must be the problem, I just don't know how to avoid the error though.

Re: Transcoding Problem

jakarman — Tue, 07 Apr 2015 05:27:00 GMT

That screen is showing a lot:

- You have Eguide that is UTF-8
- There is that CEDA message in the top, The data is coming from a different SAS releases/machines.
- The transcoding message is a result of having different encodings and different SAS releases/machines.

- It is a student/teachers Eguide approach that was the common approach SAS offered until a year ago. You are working on the Orion dataset.

What has happened is that your teacher did an upload from his system to the educations environment and it was a different version or afterwards the version/release did change. When your teacher wil do a fresh copy may be just using this environment the data is updated to the current native one and the message will disappear.

Re: Transcoding Problem

agreekgod — Tue, 07 Apr 2015 07:28:26 GMT

Thanks for your information Jaap.Unfortunately my teacher is not going to release another fresh copy of the data set. Is there anyway I could fix it manually? Thanks

Re: Transcoding Problem

jakarman — Tue, 07 Apr 2015 07:34:13 GMT

If you can copy the dataset to another location (thatn one is having the message) and use that copied version, the message will be gone.

Re: Transcoding Problem

agreekgod — Tue, 07 Apr 2015 09:38:18 GMT

The dataset with the error is connected to a lot of other further datasets that I have created and if I delete the original one then it will affect many other things in my process flow. Is there a way around this?

Re: Transcoding Problem

jakarman — Tue, 07 Apr 2015 09:41:04 GMT

Copy it back to your original location....

Re: Transcoding Problem

agreekgod — Tue, 07 Apr 2015 10:22:14 GMT

I have tried to copy the dataset out using this code:

proc append base=ORION.neworganization2014 data=ORION.organization2014;

run;

However, I am still receiving the same error:

ERROR: Some character data was lost during transcoding in the dataset ORION.ORGANIZATION2014. Either the data contains characters

that are not representable in the new encoding or truncation occurred during transcoding.

Re: Transcoding Problem

jakarman — Tue, 07 Apr 2015 10:29:31 GMT

Than rename those datasets and use the new one. Every time you are using the old one the message will pop-pup the new one is converted and should not give any conversions note anymore. Why not using proc copy with the noclone option?

Re: Transcoding Problem

agreekgod — Tue, 07 Apr 2015 11:03:38 GMT

How do I use proc copy exactly? I have looked online but it is hard to understand by me with very limited computer/coding skill.This is an example, is this the right code?

LIBNAME inlib cvp 'SAS-library';

LIBNAME outlib 'SAS-library' outencoding=”encoding value for output”;

proc copy noclone in=inlib out=outlib;

select car;

run;

Re: Transcoding Problem

jakarman — Tue, 07 Apr 2015 11:23:20 GMT

Base SAS(R) 9.4 Procedures Guide, Third Edition

LIBNAME inlib '/?/??? ' ; /* put in the physical location where the data is now */

LIBNAME outlib '/?/??? ' ; /* put in the physical location where the data should copy to */i

proc copy in=inlib out=outlib noclone;

/* selection is not needed when copying all */

run;

Use the new physical location in your old code using the old libname ref. See example 3 and the syntax.

Re: Transcoding Problem

agreekgod — Tue, 07 Apr 2015 11:43:43 GMT

I have inserted this code:

LIBNAME inlib '/home/zclchjh/sasuser.v94/all_SASDATA/';

LIBNAME outlib '/home/zclchjh/sasuser.v94/all_excel' ;

proc copy in=inlib out=outlib noclone;

select organization2014.sas7bdat

I am expecting that the datset organization2014 ends up all_excel but that hasn't happened

Re: Transcoding Problem

Kurt_Bremser — Tue, 07 Apr 2015 11:46:16 GMT

What does the log say?

Re: Transcoding Problem

agreekgod — Tue, 07 Apr 2015 11:51:56 GMT

This is the code I just ran:

;*';*";*/;quit;run;

2 OPTIONS PAGENO=MIN;

3 %LET _CLIENTTASKLABEL='Program';

4 %LET _CLIENTPROJECTPATH='C:\Users\Harry\Documents\University\3rd Year\Business Analytics\Orion Case Study SAS

4 ! Project6.egp';

5 %LET _CLIENTPROJECTNAME='Orion Case Study SAS Project6.egp';

6 %LET _SASPROGRAMFILE=;

8 ODS _ALL_ CLOSE;

9 OPTIONS DEV=JPEG;

10 GOPTIONS XPIXELS=0 YPIXELS=0;

11 FILENAME EGSR TEMP;

12 ODS tagsets.sasreport13(ID=EGSR) FILE=EGSR

13 STYLE=HtmlBlue

14 STYLESHEET=(URL="file:///C:/Program%20Files/SASHome2/SASEnterpriseGuide/7.1/Styles/HtmlBlue.css")

15 NOGTITLE

16 NOGFOOTNOTE

17 GPATH=&sasworklocation

18 ENCODING=UTF8

19 options(rolap="on")

20 ;

NOTE: Writing TAGSETS.SASREPORT13(EGSR) Body file: EGSR

22 GOPTIONS ACCESSIBLE;

24 LIBNAME inlib '/home/zclchjh/sasuser.v94/all_SASDATA/';

NOTE: Libref INLIB refers to the same physical library as EC100120.

NOTE: Libref INLIB was successfully assigned as follows:

Engine: V9

Physical Name: /home/zclchjh/sasuser.v94/all_SASDATA

24 ! /* put in the physical location where the data is now */

26 LIBNAME outlib '/home/zclchjh/sasuser.v94/all_excel' ;

NOTE: Libref OUTLIB was successfully assigned as follows:

Engine: V9

Physical Name: /home/zclchjh/sasuser.v94/all_excel

26 ! /* put in the physical location where the data should copy to

26 ! */i

28 proc copy in=inlib out=outlib noclone;

30 select organization2014.sas7bdat

32 run;

34 GOPTIONS NOACCESSIBLE;

35 %LET _CLIENTTASKLABEL=;

36 %LET _CLIENTPROJECTPATH=;

37 %LET _CLIENTPROJECTNAME=;

38 %LET _SASPROGRAMFILE=;

40 ;*';*";*/;quit;run;

41 ODS _ALL_ CLOSE;

44 QUIT; RUN;

Re: Transcoding Problem

jakarman — Tue, 07 Apr 2015 12:02:58 GMT

there is no semicolon (;) after the select statement.
You do not (should Not) code the sas7bdat. Within libnames it are sas-datasets and the suffix is only a part of the physical realization of the table. The idea is that libnames are behaving similar as schema/databases within a RDBMS. The physical layer is getting hidden.

Re: Transcoding Problem

Kurt_Bremser — Tue, 07 Apr 2015 12:10:37 GMT

There's several things:

! */i

That lone i should immediately lead to an ERROR message; that it does not suggests that something (unreadable character) causes SAS to not interpret the */ as end of comment, which also explains the missing ERROR messages for the syntactically incorrect proc copy.

Strip your code down to the bare (omit the .sas7bdat!), absolute necessary statements and run again.

You should get

NOTE: Copying INLIB.ORGANIZATION2014 to OUTLIB.ORGANIZATION2014 (memtype=DATA).

immediately after the run; in the log.

Re: Transcoding Problem

agreekgod — Tue, 07 Apr 2015 12:27:37 GMT

Thanks for your suggestions. I have now run this code

LIBNAME inlib '/home/zclchjh/sasuser.v94/all_SASDATA/';

LIBNAME outlib '/home/zclchjh/sasuser.v94/all_SASDATA/Organization';

proc copy in=inlib out=outlib noclone;

select organization2014;

And I get two errors. This is the program log:

1 ;*';*";*/;quit;run;

2 OPTIONS PAGENO=MIN;

3 %LET _CLIENTTASKLABEL='Program';

4 %LET _CLIENTPROJECTPATH='C:\Users\Harry\Documents\University\3rd Year\Business Analytics\Orion Case Study SAS

4 ! Project6.egp';

5 %LET _CLIENTPROJECTNAME='Orion Case Study SAS Project6.egp';

6 %LET _SASPROGRAMFILE=;

8 ODS _ALL_ CLOSE;

9 OPTIONS DEV=JPEG;

10 GOPTIONS XPIXELS=0 YPIXELS=0;

11 FILENAME EGSR TEMP;

12 ODS tagsets.sasreport13(ID=EGSR) FILE=EGSR

13 STYLE=HtmlBlue

14 STYLESHEET=(URL="file:///C:/Program%20Files/SASHome2/SASEnterpriseGuide/7.1/Styles/HtmlBlue.css")

15 NOGTITLE

16 NOGFOOTNOTE

17 GPATH=&sasworklocation

18 ENCODING=UTF8

19 options(rolap="on")

20 ;

NOTE: Writing TAGSETS.SASREPORT13(EGSR) Body file: EGSR

22 GOPTIONS ACCESSIBLE;

24 LIBNAME inlib '/home/zclchjh/sasuser.v94/all_SASDATA/';

NOTE: Libref INLIB refers to the same physical library as EC100120.

NOTE: Libref INLIB was successfully assigned as follows:

Engine: V9

Physical Name: /home/zclchjh/sasuser.v94/all_SASDATA

26 LIBNAME outlib '/home/zclchjh/sasuser.v94/all_SASDATA/Organization';

NOTE: Libref OUTLIB refers to the same physical library as ORION.

NOTE: Libref OUTLIB was successfully assigned as follows:

Engine: V9

Physical Name: /home/zclchjh/sasuser.v94/all_SASDATA/Organization

28 proc copy in=inlib out=outlib noclone;

30 select organization2014;

32 run;

NOTE: Copying INLIB.ORGANIZATION2014 to OUTLIB.ORGANIZATION2014 (memtype=DATA).

NOTE: Data file INLIB.ORGANIZATION2014.DATA is in a format that is native to another host, or the file encoding does not match the

session encoding. Cross Environment Data Access will be used, which might require additional CPU resources and might reduce

performance.

NOTE: System Options for BUFSIZE and REUSE were used at user's request.

NOTE: Libname and/or system options for compress, pointobs, data representation and encoding attributes were used at user's request.

NOTE: There were 733 observations read from the data set INLIB.ORGANIZATION2014.

ERROR: Some character data was lost during transcoding in the dataset INLIB.ORGANIZATION2014. Either the data contains characters

that are not representable in the new encoding or truncation occurred during transcoding.

ERROR: File OUTLIB.ORGANIZATION2014.DATA has not been saved because copy could not be completed.

NOTE: Statements not processed because of errors noted above.

NOTE: PROCEDURE COPY used (Total process time):

real time 0.12 seconds

2 The SAS System 03:29 Tuesday, April 7, 2015

user cpu time 0.02 seconds

system cpu time 0.01 seconds

memory 1226.18k

OS Memory 23740.00k

Timestamp 04/07/2015 08:24:44 AM

Step Count 158 Switch Count 60

Page Faults 0

Page Reclaims 177

Page Swaps 0

Voluntary Context Switches 789

Involuntary Context Switches 107

Block Input Operations 0

Block Output Operations 16

NOTE: The SAS System stopped processing this step because of errors.

34 GOPTIONS NOACCESSIBLE;

35 %LET _CLIENTTASKLABEL=;

36 %LET _CLIENTPROJECTPATH=;

37 %LET _CLIENTPROJECTNAME=;

38 %LET _SASPROGRAMFILE=;

40 ;*';*";*/;quit;run;

41 ODS _ALL_ CLOSE;

44 QUIT; RUN;

Re: Transcoding Problem

Ksharp — Tue, 07 Apr 2015 13:12:09 GMT

Change your dataset into a CSV file in UTF-8

filename ....... encoding='utf8';

proc export ...

and import it back in your SAS .

proc import ..........

Xia Keshan