DATA Step, Macro, Functions and more

What is the fastest way to search text for matching value

Reply
SAS Employee
Posts: 1

What is the fastest way to search text for matching value

I want to search for a list of words in a long text in a column (. If any word in the list exists in the column, the row be in the output dataset.

 

Please see my partial code. 

I want to search for dog,cat and boat in a column called "Column"

 

 

 

WL = PRXPARSE("/dog|cat|boat/");

do i=1 to N;

    position = PRXMATCH( WL ,Column(i));
    if position ge 1 then do;

           ALERT_TRIGGER_TXT= SUBSTR(Column(i),max(1,position-200),position-1)|| '*' ||                          

          SUBSTR(A_TSC_SUSP_TXT(i),max(1,position),position+200) ;
          output;
    end;
end;

 

Is there other text searching function that would make the program faster? 

 

Super User
Super User
Posts: 7,942

Re: What is the fastest way to search text for matching value

I would do - without test data (form of a datastep) and required output - :

data want;
  set have;
  do i="DOG","CAT","BOAT";
    if index(upcase(<your string variable>),i) > 0 then do;
      found=1;
      leave;
    end;
  end;
run;
Super User
Posts: 7,760

Re: What is the fastest way to search text for matching value

I guess that you will be mostly I/O bound, but you could try

 

%let list=dog,cat,boat;

data want;
set have;
indicator = 0;
do i = 1 to countw("&list",',');
  if indexw(column,scan("&list",i,',')) > 0 then indicator = 1;
end;
if indicator then output;
drop indicator i;
run;

just to compare runtimes.

---------------------------------------------------------------------------------------------
Maxims of Maximally Efficient SAS Programmers
Respected Advisor
Posts: 3,799

Re: What is the fastest way to search text for matching value

You need to supply data and a complete example program.

 

PRXPARSE is slow and should only be done one time and retain the RC for use in PRXMATCH.  See the exampes in the documentation where they use the RETAIN statement and IF _N_ eq 1 THEN ....

Ask a Question
Discussion stats
  • 3 replies
  • 209 views
  • 0 likes
  • 4 in conversation