topic Re: Transcoding WARNING when importing a UTF-8 csv file in SAS Programming

Transcoding WARNING when importing a UTF-8 csv file

ewod — Mon, 19 Aug 2024 00:48:20 GMT

Hello SAS Community!

I'm having an issue importing a csv file with data from different sites around the world. If I don't set the encoding file to UTF-8 then my log is clean but the variable with the site name imports incorrectly. When I set encoding to UTF-8 my site names look good, but the investigator name causes a WARNING to my log, which gets caught in my companies log checker. I'm supposed to produce a clean log. I created a sample csv file with 3 rows and 3 variables of data.

In my sample csv file it's the data in TestField1 that is causing the warning, and the variable TestField2 contains the site names that I need to be imported correctly.

Reading file without UTF-8...

proc import file="U:\sas\encoding_test.csv"
    out=csv_1
    dbms=csv
    replace;
    GUESSINGROWS = MAX;
run;

results in a dataset that looks like this...

Importing it with encoding set to UTF-8...

filename fname "U:\sas\encoding_test.csv" encoding="utf-8"  ;
proc import file=fname
    out=csv_2
    dbms=csv
    replace;
    GUESSINGROWS = MAX;
run;

Results in a data set that looks like this (correct TextField2, TextField1 is still not quite correct)...

The warning in the log looks like this:

TextField1 is a variable I don't use, so I'm not worried about it being read in correctly, I would just like a clean log. I've attached my sample csv file (and this screenshot).

Any suggestions are appreciated.

Thank you

Re: Transcoding WARNING when importing a UTF-8 csv file

Ksharp — Mon, 19 Aug 2024 01:45:45 GMT

If you are using UTF-8 encoding SAS, that would not be a problem.

Re: Transcoding WARNING when importing a UTF-8 csv file

whymath — Mon, 19 Aug 2024 02:16:15 GMT

Your SAS session is not UTF-8 encoded, so the special characters "Çİ" can't be displayed normally even if you specify UTF-8 encoding to the file. That's why there is a transcoding——SAS can't display raw characters under the current session encoding.

You can import the file under UTF-8 encoded session, just like what @Ksharp does.

Re: Transcoding WARNING when importing a UTF-8 csv file

ewod — Mon, 19 Aug 2024 11:27:45 GMT

Thanks for the reply. I am choosing my companies "Unicode Support" SAS session...

I'll check with my IT department and ask them to check the SAS install since it doesn't appear to be working properly.

Thanks again