BookmarkSubscribeRSS Feed
neethubabu
Calcite | Level 5

Hi, I am getting the below error when I am working on the SAS dataset. I think this is related to SAS UTF- 8 Unicode. Is there any way to deal with this? I need to do the same in multiple SAS datasets. Can anyone help me find a solution?

Untitled.png

2 REPLIES 2
MarkBodt_NZ
Obsidian | Level 7

Hi,

This is a common issue when using data that was created on one server on a different server. Often experienced when migrating from an older SAS environment to a new one, or a Windows server to a Unix server

 

Have a read of the documentation here:

https://go.documentation.sas.com/doc/en/pgmsascdc/9.4_3.5/movefile/n06qan4j3ffr6fn11bs4q11r8r56.htm

There's a section that discusses your error along with solutions.

 

You mention that you need to convert multiple datasets. I can't offer you code solutions, but I can reflect on how I went about a similar issue at a site where I carried out a migration:

They had I think about 100,000 or so datasets that needed converting. The datasets were strewn through out many directories, I think about 30,000 directories if I remember corrrectly.

The approach I took was that I wrote a SAS program that 'spidered' a data volume, that is, it examined each directory, starting at the highest level  and then each subdirectory recursively. It examined every SAS dataset as well as other SAS artifacts such as catalogs to see if they needed to be converted to UTF-8. It also checked for things like password protection, corrupt datasets etc.

Anyway, out of this, it produced a big 'shopping list' of SAS datasets that needed conversion, along with the path name

 

Part two of the process was a SAS program that used the 'shopping list' to assign libnames and execute code to read the dataset and convert it to UTF8. It did this using methods discussed in the documentation link above.

 

I hope that's helpful information.

ChrisNZ
Tourmaline | Level 20

If your title is correct, it looks like you are reading UTF-8 data in a session that does not support this encoding (probably some form of wlatin).

UTF-8 can encode many characters that do not exist in wlatin. So these characters are lost, as the message describes.

 

To avoid losing them, you should run your SAS session using UTF-8 encoding. That's a start-up option.

 

A similar issue arises when moving files to/from mainframes as some characters only exist in ASCII or EBCDIC representation.

sas-innovate-2024.png

Don't miss out on SAS Innovate - Register now for the FREE Livestream!

Can't make it to Vegas? No problem! Watch our general sessions LIVE or on-demand starting April 17th. Hear from SAS execs, best-selling author Adam Grant, Hot Ones host Sean Evans, top tech journalist Kara Swisher, AI expert Cassie Kozyrkov, and the mind-blowing dance crew iLuminate! Plus, get access to over 20 breakout sessions.

 

Register now!

How to Concatenate Values

Learn how use the CAT functions in SAS to join values from multiple variables into a single value.

Find more tutorials on the SAS Users YouTube channel.

Click image to register for webinarClick image to register for webinar

Classroom Training Available!

Select SAS Training centers are offering in-person courses. View upcoming courses for:

View all other training opportunities.

Discussion stats
  • 2 replies
  • 1202 views
  • 0 likes
  • 3 in conversation