Why "TRIM" and "VERIFY" functions fail to solve my problem?

[Zhanxiong]

I have a data set good_bad created by the following data step:

data good_bad;
    input answer $40.;
    datalines;
1324AcB876acccCCC
123 456
aabbccAABBCC123123
abcde12345
invalid
;
run;

Now I want to create a new data set valid which consists of the observations from the good_bad whose characters are from 'ABC', 'abc', and '0123456789'. While the trailing blanks are ignored, the embedded blank should be considered invalid. Based on these requirements, I wrote a data step as follows:

data valid;
    set good_bad;
    answer = trim(upcase(answer));
    if verify(answer, 'ABC0123456789') = 0; 
run;

Unluckily, this program doesn't produce the output data set which consists of line 1 and line 3 of good_bad. Instead, it contains 0 rows. It seems that the TRIM function didn't remove the trailing blanks, as it was supposed to do. Can anyone help me solve this problem (by only using TRIM and VERIFY functions)? Thanks in advance.

[art297]

Trim only removes spaces to the right, not the left. However, verify identifies the first invalid character, so you could use:

data good_bad;
    input answer $40.;
    datalines;
1324AcB876acccCCC
123 456
aabbccAABBCC123123
abcde12345
invalid
;
data valid;
    set good_bad;
    answer = trim(upcase(answer));
    if length(answer) le verify(answer, 'ABC0123456789');
run;

Art, CEO, AnalystFinder.com

[Zhanxiong]

Thanks. But I don't think I have leading blanks in the original ANSWER variable. Even so, I can use STRIP in place of TRIM to remove all leading and trailing blanks as follows:

data valid;
    set good_bad;
    temp = strip(upcase(answer));
    if verify(temp, 'ABC0123456789') = 0; 
run;

However, this still gives an empty data set.

Your suggested program works correctly, I am just curious about why (my simple) VERIFY statement doesn's solve the problem, even the blanks problem is properly handled in advance?

[art297 · Accepted solution]

Depends upon where you use the trim or strip function. e.g., the following does do what you expect:

data valid3;
    set good_bad;
    if verify(trim(upcase(answer)), 'ABC0123456789') =0;
run;

Art, CEO, AnalystFinder.com

[Zhanxiong]

Yes, it does! 

On the other hand, why does the additional assignment statement spoil the program? To me, the working program and the previous one do not have an essential difference. Could you please explain more to me why these two are different? Thanks again.

[Reeza]

What's the length of the two variables?
How does SAS treat a variable that has a longer length than then the actual values?

[Zhanxiong]

Thanks for the helpful hint.

[art297]

My programs uses the trim function within the verify function .. your's didn't!. As such, you trimmed the variable, but then SAS right padded it before it got to the function.

Art, CEO, AnalystFinder.com

[Zhanxiong]

Understood now!

[Tom]

SAS stores character variables as fixed length. When your actual value is shorter than the storage length it is padded on the right with spaces.  So this type of a statement does nothing since any spaces removed by the trim function are just added back when the value is saved back into the variable.

charvar = trim(charvar);

[Zhanxiong]

Thanks for your crystal-clear explanation, appreciate it!

[Zhanxiong]

By the way, may I ask how do you highlight the SAS keyword in the code chunk?

[Tom]

If you click on Insert SAS Code icon (looks like the SAS run icon) in the Rich Text editor on this site it will make a pop-up window were you can place your SAS code. In addition to attempting to highlight the SAS syntax it will also preserve your lines of text as-is instead of treating them as paragraphs like the normal edit box does.  To edit the text you have created this way put the cursor anywhere in the code block and click the icon.  Do not try to edit in the simple edit box or it will mess up the formatting.

---

@Zhanxiong wrote:
By the way, may I ask how do you highlight the SAS keyword in the code chunk?

---

[Zhanxiong]

Thanks, it works. I edited my original post.