topic Re: SUPRESS IF THE ENTIRE OBS IS SAME in SAS Programming

SUPRESS IF THE ENTIRE OBS IS SAME

R_Win — Thu, 28 Jun 2012 14:14:07 GMT

HI i am having data in some obs the data will be repetative i want to supress that records how can i do i.i dont know the length as it may be 3 or 100 how can i do.I want to supress them ex data Test; input id$ 1-100; cards; AAA VVVVVVVVVVVVVVVVVVVVVV EEEEEEEEEEEEEEEEEEEEEEEEEE RTYUY QWEPO ZZZZZZZZZZZZZ KKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKK KOOL GONE RUN; OUTPUT SHOULD BE: RTYUY QWEPO KOOL GONE

Re: SUPRESS IF THE ENTIRE OBS IS SAME

R_Win — Thu, 28 Jun 2012 14:15:32 GMT

Re: SUPRESS IF THE ENTIRE OBS IS SAME

ballardw — Thu, 28 Jun 2012 14:21:55 GMT

Are the unwanted strings of repeated characters ALWAYS separated by spaces?

Will the repeated characters ALWAYS be the same character within a repeat group? (Will never have a group like AAAAAZZZZZ that is unwanted.)

Re: SUPRESS IF THE ENTIRE OBS IS SAME

Astounding — Thu, 28 Jun 2012 14:25:23 GMT

Re: SUPRESS IF THE ENTIRE OBS IS SAME

data_null__ — Thu, 28 Jun 2012 14:32:04 GMT

if missing(compress(id,first(id))) then delete;

Re: SUPRESS IF THE ENTIRE OBS IS SAME

Haikuo — Thu, 28 Jun 2012 14:32:48 GMT

Try this:

data Test;

input id:$100.;

cards;

AAA

VVVVVVVVVVVVVVVVVVVVVV

EEEEEEEEEEEEEEEEEEEEEEEEEE

RTYUY

QWEPO

ZZZZZZZZZZZZZ

KKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKK

KOOL

GONE

;

data want;

set test;

if lengthn(compress(id,first(id)))=0 then delete;

run;

proc print;RUN;

Haikuo

Re: SUPRESS IF THE ENTIRE OBS IS SAME

ballardw — Thu, 28 Jun 2012 14:48:44 GMT

Your example input does not make the string long enough to read the example data.

Re: SUPRESS IF THE ENTIRE OBS IS SAME

ballardw — Thu, 28 Jun 2012 15:44:26 GMT

This is a brute force method that works for your example. Caveats: Single characters will be eliminated. A check for length could be added to the execute the line with compress only if length is greater than minimum acceptable duplication. Also, case is not taken into account. If AaA is supposed to be removed it won't unless UPCASE is applied.

The array size is arbitrary but needs to 1) have enough elements to catch all of your repeat strings, 2) each element needs to long enough to contain the longest repeat string.

data Test;

input id$ 1-127;

array t {100} $ 100 _t1 -_t100;

do i=1 to (countw(ID));

t= scan(id,i);

if compress(t,first(t)) = '' then t=compress(t,first(t));

end;

outstr = catx(' ', of _t1 - _t100); /* this is the hopefully desired output string*/

drop _t: i;

cards;

AAA VVVVVVVVVVVVVVVVVVVVVV EEEEEEEEEEEEEEEEEEEEEEEEEE RTYUY QWEPO ZZZZZZZZZZZZZ KKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKK KOOL GONE

;

RUN ;

Re: SUPRESS IF THE ENTIRE OBS IS SAME

tfrerichs — Thu, 28 Jun 2012 23:31:53 GMT

I would use Perl Regular Expressions and do it like this:

This code will eliminate characters a-z and A-Z (ASCII codes 65 to 90) (others can be added of course!) if they appear at least two times in sequence and are surrounded by so called word boundaries.

data Test;

infile cards;

input;

s = _infile_;

do i=65 to 90;

s = prxchange(cats("s/\b", byte(i),"{2,}\b//i"), -1, s);

end;

s = strip(compbl(s));

cards;

AAA VVVVVVVVVVVVVVVVVVVVVV EEEEEEEEEEEEEEEEEEEEEEEEEE RTYUY QWEPO ZZZZZZZZZZZZZ KKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKK KOOL GONE RUN

;

RUN ;

Kind Regards

Thomas

Re: SUPRESS IF THE ENTIRE OBS IS SAME

PGStats — Fri, 29 Jun 2012 01:52:34 GMT

How about :

data Test;

input;

s = compbl(prxchange("s/\b(\w)\1{2,}\b//o", -1, _infile_));

put s;

datalines;

AAA VVVVVVVVVVVVVVVVVVVVVV EEEEEEEEEEEEEEEEEEEEEEEEEE RTYUY QWEPO ZZZZZZZZZZZZZ KKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKK KOOL GONE

;

It removes any mono-character word of length 3 or more. You could use the pattern "s/\b([[:alpha:]])\1{2,}\b//o" to remove only alphabetic mono character words.

Re: SUPRESS IF THE ENTIRE OBS IS SAME

R_Win — Fri, 29 Jun 2012 08:23:41 GMT

Thqs its working

Re: SUPRESS IF THE ENTIRE OBS IS SAME

R_Win — Fri, 29 Jun 2012 09:37:23 GMT

thqs it worked