topic French (Accent & Diacritic) Letter Encoding in MS SQL DB in SAS Data Management

French (Accent & Diacritic) Letter Encoding in MS SQL DB

manishiiita — Mon, 02 Aug 2021 15:12:19 GMT

We are facing issues with encoding for Accent & Diacritic characters, it is showing as Garbage Values.

We are using MS SQL Server DB (2013). Basically, we load data from other Data Warehouse through DI ETL Jobs. Within SAS Datasets, we are able to see these characters properly but post loading data into SQL Tables, it is showing as Garbage.

We are using UTF-8 encoding on SAS Server end. On SQL DB, the encoding is SQL_Latin1_General_CP850_CI_AS. Ideally, we should have UTF-8 encoding on DB side as well to resolve this issue and we tried using NVARCHAR column types for this but this didn't work.

Can you please assist to resolve this issue..

Re: French (Accent & Diacritic) Letter Encoding in MS SQL DB

jimbarbour — Tue, 03 Aug 2021 00:16:07 GMT

@manishiiita wrote:

We are using UTF-8 encoding on SAS Server end. On SQL DB, the encoding is SQL_Latin1_General_CP850_CI_AS. Ideally, we should have UTF-8 encoding on DB side as well to resolve this issue

I think you've probably identified the issue right there. I think you need to change the encoding of the database so that it matches, at least that's the first thing that I would try.

The alternative is to experiment with the various encoding options. Here's a recent paper that discusses some of the options: https://www.sas.com/content/dam/SAS/support/en/sas-global-forum-proceedings/2020/4530-2020.pdf

Jim