topic Re: Why "TRIM" and "VERIFY" functions fail to solve my problem? in SAS Programming

Why "TRIM" and "VERIFY" functions fail to solve my problem?

Zhanxiong — Sun, 02 Jul 2017 20:00:09 GMT

I have a data set good_bad created by the following data step:

data good_bad;
    input answer $40.;
    datalines;
1324AcB876acccCCC
123 456
aabbccAABBCC123123
abcde12345
invalid
;
run;

Now I want to create a new data set valid which consists of the observations from the good_bad whose characters are from 'ABC', 'abc', and '0123456789'. While the trailing blanks are ignored, the embedded blank should be considered invalid. Based on these requirements, I wrote a data step as follows:

data valid;
    set good_bad;
    answer = trim(upcase(answer));
    if verify(answer, 'ABC0123456789') = 0; 
run;

Unluckily, this program doesn't produce the output data set which consists of line 1 and line 3 of good_bad. Instead, it contains 0 rows. It seems that the TRIM function didn't remove the trailing blanks, as it was supposed to do. Can anyone help me solve this problem (by only using TRIM and VERIFY functions)? Thanks in advance.

Re: Why "TRIM" and "VERIFY" functions fail to solve my problem?

art297 — Sun, 02 Jul 2017 01:18:58 GMT

Trim only removes spaces to the right, not the left. However, verify identifies the first invalid character, so you could use:

data good_bad;
    input answer $40.;
    datalines;
1324AcB876acccCCC
123 456
aabbccAABBCC123123
abcde12345
invalid
;
data valid;
    set good_bad;
    answer = trim(upcase(answer));
    if length(answer) le verify(answer, 'ABC0123456789');
run;

Art, CEO, AnalystFinder.com

Re: Why "TRIM" and "VERIFY" functions fail to solve my problem?

Zhanxiong — Sun, 02 Jul 2017 01:45:19 GMT

Thanks. But I don't think I have leading blanks in the original ANSWER variable. Even so, I can use STRIP in place of TRIM to remove all leading and trailing blanks as follows:

data valid;
    set good_bad;
    temp = strip(upcase(answer));
    if verify(temp, 'ABC0123456789') = 0; 
run;

However, this still gives an empty data set.

Your suggested program works correctly, I am just curious about why (my simple) VERIFY statement doesn's solve the problem, even the blanks problem is properly handled in advance?

Re: Why "TRIM" and "VERIFY" functions fail to solve my problem?

art297 — Sun, 02 Jul 2017 02:01:55 GMT

Depends upon where you use the trim or strip function. e.g., the following does do what you expect:

data valid3;
    set good_bad;
    if verify(trim(upcase(answer)), 'ABC0123456789') =0;
run;

Art, CEO, AnalystFinder.com

Re: Why "TRIM" and "VERIFY" functions fail to solve my problem?

Zhanxiong — Sun, 02 Jul 2017 02:16:22 GMT

Yes, it does!

On the other hand, why does the additional assignment statement spoil the program? To me, the working program and the previous one do not have an essential difference. Could you please explain more to me why these two are different? Thanks again.

Re: Why "TRIM" and "VERIFY" functions fail to solve my problem?

Reeza — Sun, 02 Jul 2017 02:17:52 GMT

What's the length of the two variables?
How does SAS treat a variable that has a longer length than then the actual values?

Re: Why "TRIM" and "VERIFY" functions fail to solve my problem?

art297 — Sun, 02 Jul 2017 02:20:43 GMT

My programs uses the trim function within the verify function .. your's didn't!. As such, you trimmed the variable, but then SAS right padded it before it got to the function.

Art, CEO, AnalystFinder.com

Re: Why "TRIM" and "VERIFY" functions fail to solve my problem?

Zhanxiong — Sun, 02 Jul 2017 02:24:50 GMT

Understood now!

Re: Why "TRIM" and "VERIFY" functions fail to solve my problem?

Tom — Sun, 02 Jul 2017 13:27:36 GMT

SAS stores character variables as fixed length. When your actual value is shorter than the storage length it is padded on the right with spaces. So this type of a statement does nothing since any spaces removed by the trim function are just added back when the value is saved back into the variable.

charvar = trim(charvar);

Re: Why "TRIM" and "VERIFY" functions fail to solve my problem?

Zhanxiong — Sun, 02 Jul 2017 14:22:32 GMT

Thanks for your crystal-clear explanation, appreciate it!

Re: Why "TRIM" and "VERIFY" functions fail to solve my problem?

Zhanxiong — Sun, 02 Jul 2017 14:23:20 GMT

Thanks for the helpful hint.

Re: Why "TRIM" and "VERIFY" functions fail to solve my problem?

Zhanxiong — Sun, 02 Jul 2017 14:29:33 GMT

By the way, may I ask how do you highlight the SAS keyword in the code chunk?

Re: Why "TRIM" and "VERIFY" functions fail to solve my problem?

Tom — Sun, 02 Jul 2017 19:51:27 GMT

If you click on Insert SAS Code icon (looks like the SAS run icon) in the Rich Text editor on this site it will make a pop-up window were you can place your SAS code. In addition to attempting to highlight the SAS syntax it will also preserve your lines of text as-is instead of treating them as paragraphs like the normal edit box does. To edit the text you have created this way put the cursor anywhere in the code block and click the icon. Do not try to edit in the simple edit box or it will mess up the formatting.

@Zhanxiong wrote:
By the way, may I ask how do you highlight the SAS keyword in the code chunk?

Re: Why "TRIM" and "VERIFY" functions fail to solve my problem?

Zhanxiong — Sun, 02 Jul 2017 20:01:01 GMT

Thanks, it works. I edited my original post.