topic Remove hidden characters from a variable in SAS Programming

Remove hidden characters from a variable

AshleyBright — Tue, 22 Jan 2019 19:43:29 GMT

Hi All,

I have a text from a variable that has some repeated quotes and I want to remove those. I tried dequote, compress functions it didn't work. Any help is much appreciated.

When I copy the text to notepad it's like this:

"""test:{""""text"""":""""test/5.0 (werrwe VV 1.0; Qwe77; y99) TestTestTest/123.89 (ABCD"""

But when I see it in the dataset it's like this:

"test:{""text"":""test/5.0 (werrwe VV 1.0; Qwe77; y99) TestTestTest/123.89 (ABCD"

Re: Remove hidden characters from a variable

ErikLund_Jensen — Tue, 22 Jan 2019 20:07:11 GMT

Try to see if the quotes are actually the same character:

data _null_; set have (obs=1); put var $hex6. run;

If you get 222222, I have no suggestions based on the information so far. otherwise the unwantet can be renoved with

var = compress(var,'nn'x),

where nn is the 2 chars that is not 22.

Re: Remove hidden characters from a variable

Cynthia_sas — Tue, 22 Jan 2019 20:16:43 GMT

Hi:

COMPRESS worked for me when I did this:

newvar1=compress(var1,'"');

that is single quote-double quote-single quote inside the Compress function.

Here's the code I ran -- first I made the variables, with both types of multiple quotes and then I used COMPRESS.

data with_qt;
length var1 var2 $125;
var1='"test:{""text"":""test/5.0 (werrwe VV 1.0; Qwe77; y99) TestTestTest/123.89 (ABCD"';
var2='"""test:{""""text"""":""""test/5.0 (werrwe VV 1.0; Qwe77; y99) TestTestTest/123.89 (ABCD"""';
run;

data without_qt;
  set with_qt;
  newvar1 = compress(var1,'"');
  newvar2 = compress(var2,'"');
run;

** proc print steps;

Results:

(I did a separate PROC PRINT for every variable so they'd line up one under the other for comparison.)

Cynthia

Re: Remove hidden characters from a variable

ballardw — Tue, 22 Jan 2019 21:05:22 GMT

@AshleyBright wrote:

Hi All,

I have a text from a variable that has some repeated quotes and I want to remove those. I tried dequote, compress functions it didn't work. Any help is much appreciated.

When I copy the text to notepad it's like this:

"""test:{""""text"""":""""test/5.0 (werrwe VV 1.0; Qwe77; y99) TestTestTest/123.89 (ABCD"""

But when I see it in the dataset it's like this:

"test:{""text"":""test/5.0 (werrwe VV 1.0; Qwe77; y99) TestTestTest/123.89 (ABCD"

Again I ask, with some more details, what do you copy from? The Viewtable when opening a dataset in the SAS explorer? or something else?

When I create a string like what you see in the "dataset" and copy from Viewtable to Notepad I do not get any additional quotes in the string. And please show using the create data step from the data macro, some of the values involved. Instructions here: https://communities.sas.com/t5/SAS-Communities-Library/How-to-create-a-data-step-version-of-your-data-AKA-generate/ta-p/258712 will show how to turn an existing SAS data set into data step code that can be pasted into a forum code box using the {i} icon or attached as text to show exactly what you have and that we can test code against.

Or paste some lines from the raw data file and the SAS program to read it into a code box opened using the forum's {I} icon.

Re: Remove hidden characters from a variable

Astounding — Tue, 22 Jan 2019 21:54:57 GMT

If you suspect there is some funky character there, remember you can use an expression as the second parameter to COMPRESS:

textvar = compress(textvar, substr(textvar, 1, 1));

You don't necessarily need to find out what the character is, in order to remove it.