topic Re: Compress array is adding weird characters in SAS Procedures

Compress array is adding weird characters

ProcWes — Mon, 28 Oct 2024 21:02:48 GMT

I run this block before I send data to GCP but for some reason, it is adding weird characters to my strings that aren't there before the compress.

array Chars[*] _character_;
   do i = 1 to dim(Chars);
      Chars[i] = compress(Chars[i],,'kw');
   end;
   drop i;
array Nums[*] _numeric_;
   do i = 1 to dim(Nums);
      Nums[i] = round(Nums[i],0.01);
   end;
   drop i;

These weird characters, the @ etc. Is there a better way to go about this? Where are these coming from??

Re: Compress array is adding weird characters

data_null__ — Mon, 28 Oct 2024 21:15:07 GMT

Are you using DBCS or MBCS. You might try KCOMPRESS to resolve issue.

@ProcWes wrote:

I run this block before I send data to GCP but for some reason, it is adding weird characters to my strings that aren't there before the compress.
array Chars[*] _character_;
   do i = 1 to dim(Chars);
      Chars[i] = compress(Chars[i],,'kw');
   end;
   drop i;
array Nums[*] _numeric_;
   do i = 1 to dim(Nums);
      Nums[i] = round(Nums[i],0.01);
   end;
   drop i;
These weird characters, the @ etc. Is there a better way to go about this? Where are these coming from??

Re: Compress array is adding weird characters

Ksharp — Tue, 29 Oct 2024 01:52:59 GMT

compress(Chars[i],,'kw') would also keep the Chinese,Japanese ..... charaters as a-zA-Z .
Therefore you need to use 'kad' if you only want keep alphabeta（a-zA-Z) and digit(0-9) characters.

Chars[i] = compress(Chars[i],,'kad');

Re: Compress array is adding weird characters

Patrick — Tue, 29 Oct 2024 03:32:59 GMT

@Ksharp wrote:
compress(Chars[i],,'kw') would also keep the Chinese,Japanese ..... charaters as a-zA-Z .
Therefore you need to use 'kad' if you only want keep alphabeta（a-zA-Z) and digit(0-9) characters.

Chars[i] = compress(Chars[i],,'kad');

@Ksharp Wouldn't it need to be Kcompress() ?

Re: Compress array is adding weird characters

Ksharp — Tue, 29 Oct 2024 06:12:45 GMT

Patrick,
If OP only need to keep characters (a-zA-Z0-9) , then COMPRESS() and KCOMPRESS() is the same. KCOMPRESS() has the ability to process Chinese,Japanes ....... DBMS characters.

Re: Compress array is adding weird characters

ProcWes — Tue, 29 Oct 2024 14:04:35 GMT

Hi, these replies haven't been solutions to this point. Compress isn't removing these: ÀõÖ@ because they're showing up as letters and punctuation. What's troubling to me is that these characters do not exist on the input, even if I convert the string to hex format, they're not there. The compress is creating them.

Controllable Fraud - Other 436F6E74726F6C6C61626C65204672617564202D204F7468657220202020202020202020202020202020202020202020

Re: Compress array is adding weird characters

yabwon — Tue, 29 Oct 2024 15:18:19 GMT

could you share results of this code with us:

data test;
  set <your Data Set> (keep= <one variable that makes problems>);
  test1=<one variable that makes problems>;
  test2 = compress(test1,,'kw');
  a=put(test1,$hex128.);
  b=put(test2,$hex128.);
run;
proc print data = test;
run;

Bart