Black Diamonds in my Dataset

Reply
Occasional Contributor
Posts: 13

Black Diamonds in my Dataset

Hi Guys,

I reading data from DB2 into my SAS environment. The dataset is UTF-8 encoded but I keep on getting what looks like a black diamond at the end of the values:

0101182358087�

0312026747785�

2111200044082�

2208274016188�

2501095657083�

Can you please help me in understanding why I am seeing this. The desired results will be like the below and I wish not to compress or do any form of transformation to remove them:

0101182358087

0312026747785

2111200044082

2208274016188

2501095657083

Super User
Super User
Posts: 6,502

Re: Black Diamonds in my Dataset

Looks like you are having transcoding issues.  Are you running SAS with UTF-8 Support?

Occasional Contributor
Posts: 13

Re: Black Diamonds in my Dataset

Yes

Valued Guide
Posts: 3,208

Re: Black Diamonds in my Dataset

DB2 is often related to mainframe usage Z/OS. Is this the case?

Mainframe usage is normally restricted to SBCS (Single Byte Character Set) using a code like 037 (similar to 437 850 latin-1)

An other issued is working with mainframe data on Z/os having zoned, binary, packed fields needing to be converted.

The SAS/Access DB2 is solving a lot (SBCS)  but binary filetransfers are different.

Please explain you environment (3270 terminal or transferred to eg WIndows) tooling (SAS version) and dataflow (DB/2 to SAS).

Using a SAS session with UTF-8 is related to running a multibyte approach. Utf8 is using characters varying between 1 and 4 bytes.
You must check the correct usage coding functions SAS(R) 9.4 National Language Support (NLS): Reference Guide, Third Edition

The advantage of utf8 is that it is supporting a lot of characters, like charmap windows.

Eguide is supporting UTF-8 but a session connected to EGUIDE can vary between lantin-1 and utf-8.

That diamond is an indication of some special character. It can be a tab or any non-displayable character.

---->-- ja karman --<-----
Ask a Question
Discussion stats
  • 3 replies
  • 244 views
  • 0 likes
  • 3 in conversation