Transcoding Problem

[Kurt_Bremser]

What does the log say?

[agreekgod]

This is the code I just ran:

  ;*';*";*/;quit;run;

2          OPTIONS PAGENO=MIN;

3          %LET _CLIENTTASKLABEL='Program';

4          %LET _CLIENTPROJECTPATH='C:\Users\Harry\Documents\University\3rd Year\Business Analytics\Orion Case Study SAS

4        ! Project6.egp';

5          %LET _CLIENTPROJECTNAME='Orion Case Study SAS Project6.egp';

6          %LET _SASPROGRAMFILE=;

7         

8          ODS _ALL_ CLOSE;

9          OPTIONS DEV=JPEG;

10         GOPTIONS XPIXELS=0 YPIXELS=0;

11         FILENAME EGSR TEMP;

12         ODS tagsets.sasreport13(ID=EGSR) FILE=EGSR

13             STYLE=HtmlBlue

14             STYLESHEET=(URL="file:///C:/Program%20Files/SASHome2/SASEnterpriseGuide/7.1/Styles/HtmlBlue.css")

15             NOGTITLE

16             NOGFOOTNOTE

17             GPATH=&sasworklocation

18             ENCODING=UTF8

19             options(rolap="on")

20         ;

NOTE: Writing TAGSETS.SASREPORT13(EGSR) Body file: EGSR

21        

22         GOPTIONS ACCESSIBLE;

23        

24         LIBNAME inlib   '/home/zclchjh/sasuser.v94/all_SASDATA/';

NOTE: Libref INLIB refers to the same physical library as EC100120.

NOTE: Libref INLIB was successfully assigned as follows:

      Engine:        V9

      Physical Name: /home/zclchjh/sasuser.v94/all_SASDATA

24       !                                                             /* put in the physical location where the data is now */

25        

26         LIBNAME outlib '/home/zclchjh/sasuser.v94/all_excel'    ;

NOTE: Libref OUTLIB was successfully assigned as follows:

      Engine:        V9

      Physical Name: /home/zclchjh/sasuser.v94/all_excel

26       !                                                             /* put in the physical location where the data should copy to

26       !  */i

27        

28         proc copy  in=inlib out=outlib   noclone;

29        

30         select organization2014.sas7bdat

31        

32         run;

33        

34         GOPTIONS NOACCESSIBLE;

35         %LET _CLIENTTASKLABEL=;

36         %LET _CLIENTPROJECTPATH=;

37         %LET _CLIENTPROJECTNAME=;

38         %LET _SASPROGRAMFILE=;

39        

40         ;*';*";*/;quit;run;

41         ODS _ALL_ CLOSE;

42        

43        

44         QUIT; RUN;

45        

[Kurt_Bremser]

There's several things:

26

!  */i

That lone i should immediately lead to an ERROR message; that it does not suggests that something (unreadable character) causes SAS to not interpret the */ as end of comment, which also explains the missing ERROR messages for the syntactically incorrect proc copy.

Strip your code down to the bare (omit the .sas7bdat!), absolute necessary statements and run again.

You should get

NOTE: Copying INLIB.ORGANIZATION2014 to OUTLIB.ORGANIZATION2014 (memtype=DATA).

immediately after the run; in the log.

[jakarman]

there is no semicolon (;) after the select statement.  
You do not (should Not) code the sas7bdat. Within libnames it are sas-datasets and the suffix is only a part of the physical realization of the table. The idea is that libnames are behaving similar as schema/databases within a RDBMS. The physical layer is getting hidden.  

[agreekgod]

Thanks for your suggestions. I have now run this code

LIBNAME inlib '/home/zclchjh/sasuser.v94/all_SASDATA/'; 

LIBNAME outlib '/home/zclchjh/sasuser.v94/all_SASDATA/Organization'; 

proc copy  in=inlib out=outlib   noclone;

select organization2014;

And I get two errors. This is the program log:

1          ;*';*";*/;quit;run;

2          OPTIONS PAGENO=MIN;

3          %LET _CLIENTTASKLABEL='Program';

4          %LET _CLIENTPROJECTPATH='C:\Users\Harry\Documents\University\3rd Year\Business Analytics\Orion Case Study SAS

4        ! Project6.egp';

5          %LET _CLIENTPROJECTNAME='Orion Case Study SAS Project6.egp';

6          %LET _SASPROGRAMFILE=;

7         

8          ODS _ALL_ CLOSE;

9          OPTIONS DEV=JPEG;

10         GOPTIONS XPIXELS=0 YPIXELS=0;

11         FILENAME EGSR TEMP;

12         ODS tagsets.sasreport13(ID=EGSR) FILE=EGSR

13             STYLE=HtmlBlue

14             STYLESHEET=(URL="file:///C:/Program%20Files/SASHome2/SASEnterpriseGuide/7.1/Styles/HtmlBlue.css")

15             NOGTITLE

16             NOGFOOTNOTE

17             GPATH=&sasworklocation

18             ENCODING=UTF8

19             options(rolap="on")

20         ;

NOTE: Writing TAGSETS.SASREPORT13(EGSR) Body file: EGSR

21        

22         GOPTIONS ACCESSIBLE;

23        

24         LIBNAME inlib '/home/zclchjh/sasuser.v94/all_SASDATA/';

NOTE: Libref INLIB refers to the same physical library as EC100120.

NOTE: Libref INLIB was successfully assigned as follows:

      Engine:        V9

      Physical Name: /home/zclchjh/sasuser.v94/all_SASDATA

25        

26         LIBNAME outlib '/home/zclchjh/sasuser.v94/all_SASDATA/Organization';

NOTE: Libref OUTLIB refers to the same physical library as ORION.

NOTE: Libref OUTLIB was successfully assigned as follows:

      Engine:        V9

      Physical Name: /home/zclchjh/sasuser.v94/all_SASDATA/Organization

27        

28         proc copy  in=inlib out=outlib   noclone;

29        

30         select organization2014;

31        

32         run;

NOTE: Copying INLIB.ORGANIZATION2014 to OUTLIB.ORGANIZATION2014 (memtype=DATA).

NOTE: Data file INLIB.ORGANIZATION2014.DATA is in a format that is native to another host, or the file encoding does not match the

      session encoding. Cross Environment Data Access will be used, which might require additional CPU resources and might reduce

      performance.

NOTE: System Options for BUFSIZE and REUSE were used at user's request.

NOTE: Libname and/or system options for compress, pointobs, data representation and encoding attributes were used at user's request.

NOTE: There were 733 observations read from the data set INLIB.ORGANIZATION2014.

ERROR: Some character data was lost during transcoding in the dataset INLIB.ORGANIZATION2014. Either the data contains characters

       that are not representable in the new encoding or truncation occurred during transcoding.

ERROR: File OUTLIB.ORGANIZATION2014.DATA has not been saved because copy could not be completed.

NOTE: Statements not processed because of errors noted above.

NOTE: PROCEDURE COPY used (Total process time):

      real time           0.12 seconds

2                                                          The SAS System                               03:29 Tuesday, April 7, 2015

      user cpu time       0.02 seconds

      system cpu time     0.01 seconds

      memory              1226.18k

      OS Memory           23740.00k

      Timestamp           04/07/2015 08:24:44 AM

      Step Count                        158  Switch Count  60

      Page Faults                       0

      Page Reclaims                     177

      Page Swaps                        0

      Voluntary Context Switches        789

      Involuntary Context Switches      107

      Block Input Operations            0

      Block Output Operations           16

NOTE: The SAS System stopped processing this step because of errors.

33        

34         GOPTIONS NOACCESSIBLE;

35         %LET _CLIENTTASKLABEL=;

36         %LET _CLIENTPROJECTPATH=;

37         %LET _CLIENTPROJECTNAME=;

38         %LET _SASPROGRAMFILE=;

39        

40         ;*';*";*/;quit;run;

41         ODS _ALL_ CLOSE;

42        

43        

44         QUIT; RUN;

45        

[Kurt_Bremser]

I suggest to set up a workspace server with encoding wlatin1 (this will most probably work with the old dataset) or use Base SAS from the commandline in wlatin1 and export the dataset(s) in question to csv file(s).

Then import the csv's into your new environment. If you encounter problems, you can use a text editor on the .csv to get rid of unwanted characters.

[agreekgod]

What would be the code for this?

[Kurt_Bremser]

No code. Define (or have defined) an additional logical Workspace Server in the Management Console that calls the same start script that your current server uses, with the addition of encoding=wlatin1 on the commandline. Add that server to the spawner definition for that machine and restart the spawner.

[Kurt_Bremser]

If your SAS is installed locally (no remote server), then you can copy the link from the Windows Start Menu to your desktop and add the encoding option in the "target" specification. Then run Base SAS in full interactive mode.

If SAS is installed on a remote server and you can get GUI access to that server, you can try the same thing there.

[jakarman]

52716 - "ERROR: Some character data was lost during transcoding in the data set" occurs when the dat... there are incompatible characters the transcoding is failing. The question now is how to solve that.

SAS(R) 9.4 National Language Support (NLS): Reference Guide, Third Edition  (NORSASIOTRANSERROR  just ignore them) Could be sufficient for learning but there is a risk of unexpected result (spaces/?).

Another would be setting the encoding of the sas.dataset different. Same idea accepting the fail of transcoding. http://support.sas.com/resources/papers/Multilingual_Computing_with_SAS_94.pdf
We should really not go that far for an educational environment. But this is also learning. 

[jakarman]

Yep Kurt that are the actions with a normal installation, this one is an educational university service.
I do not know code for converting the dataset in a normal way. Cannot test that with UE as it is running utf8 (never an encoding issue).

Only when it are some wrong chars and the original encoding is just some bytes off (French/Norway 850 437) the following copies from that pdf could be an option. But reading that one it looks not very sensible.  

When the program is working I would accept the messages, but it looks the process is blocking.

Able to make a physical copy (files-tab section eguide) and manipulate that in his environment  is a research whether it can do a circumvention.

I do not like circumventions as they mostly are come back with more hurting issues. When possible moving to an Utf8 environment will solve the encoding. As for a pitty you are getting a lot of utf8 issues (length and functions) with that.   

SAS(R) 9.4 National Language Support (NLS): Reference Guide, Third Edition
Base SAS(R) 9.4 Procedures Guide, Third Edition

proc datasets lib=&lib nolist;

    modify &mem / correctencoding=&to;

run; quit;

[Kurt_Bremser]

Gosh, I got foooled by the nice blue file:// URL to the css file. He's working in a UNIX environment, so he could try to ssh to the server and run an initial conversion in batch mode.

But I have the suspicion he won't be able to do this.

Well, it's the problem of trying to use out-of-date learning datasets.

[Cynthia_sas]

Hi:

  This location (and the fact that the OP mentioned a professor) implies that the OP might be using SAS OnDemand for Academics '/home/zclchjh/sasuser.v94/all_SASDATA/Organization' because on the OnDemand server, there is always a /home/<userid> folder that is used for the students data.

  In a class, the professor has an option to make a course directory. If this is an OnDemand for Academics account, then the student will not be able to use SSH or run other operating systems setup things, like changing to wlatin1, etc. Normally, the CEDA messages are not significant, but in your case, if your professor created the data on a different operating system, then he/she should be made aware of the error message you're getting in case it will impact your homework.

cynthia