Export in Unicode

[STEDEMI]

Hi All,

I have set my EG as Unicode (U8) and I can import files that contain mixed languages (i.e. Chinese, arabic, mixed with English). No issues in working in EG however, I am unable to export TXT files in Unicode.

I can export in Excel and able to read the exported text but, if the Database that I need to export exceeds 1M records (limit set for Excel), I need to export in TXT. Files then contain question marks instead of Arabic or Chinese.

I have spoken to SAS support and it seems this is a problem with windows locale and I have been advised to change the Windows settings to reflect Arabic or Chinese. Needless to say this is not something I want to do.

Any thoughts?

Is there an Proc Sql script to extract in Unicode?

[Kurt_Bremser · Accepted solution]

Don't use EG as a conduit for exporting, as you have to deal with all the Microcrap (Windows) idiosyncrasies.

Create your text file purely in code, and use encoding='utf-8' in the file statement:

data _null_;
set sashelp.class;
file '$HOME/sascommunity/class.csv' dlm=',' encoding='utf-8';
put
  Name
  Sex
  Age
  Height
  Weight
;
run;

See this hexdump:

00000000   EF BB BF 41  6C 66 72 65  64 2C 4D 2C  31 34 2C 36  ...Alfred,M,14,6
00000000   39 2C 31 31  32 2E 35 0A  41 6C 69 63  65 2C 46 2C  9,112.5.Alice,F,
00000000   31 33 2C 35  36 2E 35 2C  38 34 0A 42  61 72 62 61  13,56.5,84.Barba
00000000   72 61 2C 46  2C 31 33 2C  36 35 2E 33  2C 39 38 0A  ra,F,13,65.3,98.
00000000   43 61 72 6F  6C 2C 46 2C  31 34 2C 36  32 2E 38 2C  Carol,F,14,62.8,
00000000   31 30 32 2E  35 0A 48 65  6E 72 79 2C  4D 2C 31 34  102.5.Henry,M,14
00000000   2C 36 33 2E  35 2C 31 30  32 2E 35 0A  4A 61 6D 65  ,63.5,102.5.Jame
00000000   73 2C 4D 2C  31 32 2C 35  37 2E 33 2C  38 33 0A 4A  s,M,12,57.3,83.J
00000000   61 6E 65 2C  46 2C 31 32  2C 35 39 2E  38 2C 38 34  ane,F,12,59.8,84
00000000   2E 35 0A 4A  61 6E 65 74  2C 46 2C 31  35 2C 36 32  .5.Janet,F,15,62
00000000   2E 35 2C 31  31 32 2E 35  0A 4A 65 66  66 72 65 79  .5,112.5.Jeffrey
00000000   2C 4D 2C 31  33 2C 36 32  2E 35 2C 38  34 0A 4A 6F  ,M,13,62.5,84.Jo
00000000   68 6E 2C 4D  2C 31 32 2C  35 39 2C 39  39 2E 35 0A  hn,M,12,59,99.5.
00000000   4A 6F 79 63  65 2C 46 2C  31 31 2C 35  31 2E 33 2C  Joyce,F,11,51.3,
00000000   35 30 2E 35  0A 4A 75 64  79 2C 46 2C  31 34 2C 36  50.5.Judy,F,14,6
00000000   34 2E 33 2C  39 30 0A 4C  6F 75 69 73  65 2C 46 2C  4.3,90.Louise,F,
00000000   31 32 2C 35  36 2E 33 2C  37 37 0A 4D  61 72 79 2C  12,56.3,77.Mary,
00000000   46 2C 31 35  2C 36 36 2E  35 2C 31 31  32 0A 50 68  F,15,66.5,112.Ph
00000000   69 6C 69 70  2C 4D 2C 31  36 2C 37 32  2C 31 35 30  ilip,M,16,72,150
00000000   0A 52 6F 62  65 72 74 2C  4D 2C 31 32  2C 36 34 2E  .Robert,M,12,64.
00000000   38 2C 31 32  38 0A 52 6F  6E 61 6C 64  2C 4D 2C 31  8,128.Ronald,M,1
00000000   35 2C 36 37  2C 31 33 33  0A 54 68 6F  6D 61 73 2C  5,67,133.Thomas,
00000000   4D 2C 31 31  2C 35 37 2E  35 2C 38 35  0A 57 69 6C  M,11,57.5,85.Wil
00000000   6C 69 61 6D  2C 4D 2C 31  35 2C 36 36  2E 35 2C 31  liam,M,15,66.5,1
00000000   31 32 0A                                            12.

you can see the UTF BOM at the beginning.

[STEDEMI]

Many thanks!!