topic Re: substract text from another text in SAS Programming

substract text from another text

JoYZ — Wed, 16 Dec 2015 15:11:42 GMT

Hi SAS support community,

I am rather new to sas and would need your help on subtracting text from another text. Below is a similar example of the database I have. Basically, I would like to create a new variable which equals

newvar = Text_A - Text_B

Text_ID	Text_A	Text_B
1	A philosopher, in a wide sense, is someone who studies philosophy. The term "philosopher" comes from the Ancient Greek φιλόσοφος (philosophos), which means "lover of wisdom". The introduction of the terms "philosopher" and "philosophy" has been ascribed to the Greek thinker Pythagoras	A philosopher, in a wide sense, is someone who studies philosophy.
2	A philosopher, in the more narrow and common usage, is any intellectual who has made contributions in one or more current fields of philosophy, such as aesthetics, ethics, epistemology, logic, metaphysics, social theory, and political philosophy.	is any intellectual who has made contributions in one or more current fields of philosophy, such as
3	A philosopher may also be one who worked in the humanities or other sciences which have since split from philosophy proper over the centuries, such as the arts, history, economics, sociology, psychology, linguistics, anthropology, theology, and politics	centuries, such as the arts, history, economics, sociology, psychology, linguistics, anthropology, theology, and politics

Is there any simple way to do it? Tranwrd seems not to work.

Thanks a lot for your help!

Re: substract text from another text

ballardw — Wed, 16 Dec 2015 15:24:48 GMT

Please provide an example dataset in the form of a datastep so we have something to work with. Also show code and describe how TRANWRD "seems not work". Did you get no results? unexpected results? Too short of a result?

Note: the greek letters aren't likely going to be handled nicely due to encoding and fonts.

Re: substract text from another text

Steelers_In_DC — Wed, 16 Dec 2015 15:40:03 GMT

Are you working with this within SAS or are you passing through to somewhere else, Oracle, Teradata etc?

Re: substract text from another text

JoYZ — Wed, 16 Dec 2015 17:13:16 GMT

Hi, I am working with SAS...

I provided a xls based (small) datafile as example.

This is what I got so far:

proc import datafile='/folders/myfolders/Philosophers.xls' out=philosophers dbms=xls replace;
   getnames=yes;
Run;

*first attempt to do text_a - text_b via tranwrd;
data philosophers_newvar;
            Set philosophers;
             newvar = tranwrd(Text_A,Text_B,' ');
Run;
*Problems with tranwrd:
Problem Nr1: newvar gets cut off after 200 characters
Problem Nr2: it only works if text_b is positioned at the end of the text;

*Then I tried substr but I fail to determine the exact position of Text_B within Text_A;
data philosophers_newvar;
            Set philopshers;
           c=length(Text_A);
           d=length(Text_B);
           e=d+1;
           if d>1 then i=substr(Text_A,e,c); *in case there is a text in Text_B;
           if d=<1 then i=Text_A; *in case there is no text in Text_B;
           put i;

*for now substr only works if Text_B is positioned in the beginning of Text_A...

I would need to find a reliable way to detect the exact position (start and end) of Text_B within Text_A...

Thanks for your help!!

Re: substract text from another text

ballardw — Wed, 16 Dec 2015 17:57:19 GMT

JoYZ wrote:

*Problems with tranwrd:
Problem Nr1: newvar gets cut off after 200 characters
Problem Nr2: it only works if text_b is positioned at the end of the text;

Problem 1 is in documentation for tranwrd, if the length of the new variable is not declared prior to use the length with be 200.

Solution: Before using NewVAR have a line like Lenght Newvar $ 1500 ; The number has to be basically the length of the longest line you may encounter. Note there is a limit of 32767 characters.

Prolem Nr2: not in my experience though the single space replacement may not be quite what you want.

You can use INDEX to find the position in the search string to find the postion of the target string. Then using the length of the target you use substr. But there are 3 cases to address, target at front of string, in middle and at the end. So you need to use different lines of code to get possibly what you want.

Pos = index (Text_a,Text_b);

(If POS = 1 then your current substr works.)

if pos > 1 then Newvar = cat(substr(text_a,1, Pos-1),substr(text_a, Pos + d+1);

Re: substract text from another text

Ksharp — Thu, 17 Dec 2015 03:22:33 GMT

You didn't post the result you want yet ?

data have;
length Text_A Text_B $ 2000;
Text_ID=1;
Text_A='A philosopher, in a wide sense, is someone who studies philosophy. The term "philosopher" comes from the Ancient Greek φιλόσοφος (philosophos), which means "lover of wisdom". The introduction of the terms "philosopher" and "philosophy" has been ascribed to the Greek thinker Pythagoras';
Text_B="A philosopher, in a wide sense, is someone who studies philosophy.";
output;
Text_ID=2;
Text_A="A philosopher, in the more narrow and common usage, is any intellectual who has made contributions in one or more current fields of philosophy, such as aesthetics, ethics, epistemology, logic, metaphysics, social theory, and political philosophy.";
Text_B="is any intellectual who has made contributions in one or more current fields of philosophy, such as";
output;
run;
data want;
 set have;
 length pid new $ 32767;
 pid=cats('s/',Text_B,'//');;
 new=prxchange(pid,1,Text_A);
 drop pid;
run;