topic Re: data search engine (based on text) in SAS Data Management

data search engine (based on text)

ilikesas — Sat, 28 Nov 2015 01:19:11 GMT

Hi,

suppsoe that i ave the following data&colon;

description

A car

1 car 23j @

Car

cars

2 trucks

One truck

One truck and one car

What I would like to have is to create a new data set where in the description either "car" or "truck" is present, so the new data table is:

description

A car

1 car 23j @

Car

One truck

One truck and one car

Thnak you!

Re: data search engine (based on text)

PGStats — Sat, 28 Nov 2015 02:18:20 GMT

Use IF FINDW(description, "car", " ", "i") > 0 OR FINDW(description, "truck", " ", "i") > 0;

Re: data search engine (based on text)

Jagadishkatam — Sat, 28 Nov 2015 02:36:27 GMT

Alternatively you could use the perl regular expression like

prxmatch('m/(car|truck)\b/i',description)>0

Re: data search engine (based on text)

PGStats — Sat, 28 Nov 2015 03:07:59 GMT

@Jagadishkatam you would need a word boundary before the words too 'm/\b(car|truck)\b/i', otherwise you will match anything like "I was struck by lightning"

Re: data search engine (based on text)

Jagadishkatam — Sat, 28 Nov 2015 03:20:19 GMT

@PGStats thank you for the correction

Re: data search engine (based on text)

ilikesas — Sat, 28 Nov 2015 06:08:01 GMT

Hi,

thank you for replying!!!

Just another small extension of the question, if I have a list of words, say 10, is it possible to put the list instead of each word individually?

Thnka you!

Re: data search engine (based on text)

pearsoninst — Sat, 28 Nov 2015 09:18:53 GMT

/*
Yes, You can in this code (word,"car","truck","ANYTHINHYOUWANT); you specify which word you need to find
find 
dataiwant created a new dataset with the result;

*/

data have dataiwant; 
input word$ 30.;
matchingword = find(word,"car","truck");
if matchingword > 0 then output dataiwant;
drop  matchingword;
datalines; 
A car
1 car 23j @
Car
cars
2 trucks
One truck
One truck and one car
;
run;
Proc Print data = dataiwant;
run;

Re: data search engine (based on text)

Jagadishkatam — Sat, 28 Nov 2015 15:36:26 GMT

Yes there is way to put the list into the function directly rather than passing each word, for this you need to create a macro variable which carries these words. Also it depends on the function which you are going to use.

For example, as i mentioned in the my previous reply about the prxmatch which could be used for checking the words, this function requires the words to be checked separated by pipe '|' so when you create the macro variables you need to have the words separated by pipe.

please try something like below

proc sql;
select distinct words into : word separated by '|' from have;
quit;

data have;
set want;
if prxmatch('m/\b(&word)\b/i',description)>0;
run;

Re: data search engine (based on text)

Tom — Sat, 28 Nov 2015 15:52:09 GMT

That is not the right syntax for the FIND() function.

Re: data search engine (based on text)

Tom — Sat, 28 Nov 2015 15:52:47 GMT

Don't you need to use double quotes to have the macro variable reference resolve?

Re: data search engine (based on text)

Jagadishkatam — Sun, 29 Nov 2015 01:55:28 GMT

Thank you @Tom for the correction, It should be prxmatch("m/\b(&word)\b/i",description)>0;
replaced the single quotes with double quotes