topic Re: Reading Text in SAS Programming

Reading Text

afs — Sat, 05 Nov 2016 17:42:10 GMT

i want to read the last sentence in a long text having length 350 .it can be separated by marks, or ! or ? or period. I don't want to use No leading blanks be there. I don't know how to develop a code for this .

Re: Reading Text

PGStats — Sat, 05 Nov 2016 21:17:32 GMT

You can use either SCAN() or regular expression matching. Here is how to do the later:

data have;
infile datalines truncover;
input text $char350.;
datalines;
i want to read the last sentence in a long text having length 350 .it can be separated by marks, or ! or ? or period. I don't want to use  No leading blanks be there.   I don't know how to develop a code for this .
;

data lastSentence;
set have;
length sentence $350;
if not prx then prx + prxParse("/[^.?!]*[.?!]/");
start = 1;
stop=length(text);
call prxNext(prx, start, stop, text, pos, len);
do while(pos>0);
    sentence = substr(text, pos, len);
    call prxNext(prx, start, stop, text, pos, len);
    end;
keep text sentence;
run;

proc print; run;

Re: Reading Text

Shmuel — Sat, 05 Nov 2016 21:40:34 GMT

Function INDEXC enables look forward for first occurence of a character from a given string characters.

As I couldn't find similar function to look backwards, I shall use a loop searching the end of a last-previous sentence:

length paragraph $350; /* contains your long string */

do i = length(trim(paragraph)) to 1 by -1;

if substr(paragraph,i,1) in ('.' , '!' , '?') /* you may add any other characters you want */

then leave; /* get out of the loop with i points to end of last_previous sentence */

end;

last_sentence = substr(paragraph, i+1, length(trim(paragraph)) - i );

Re: Reading Text

Patrick — Sun, 06 Nov 2016 00:59:37 GMT

@Shmuel

You can search from right to left with findc()

http://support.sas.com/documentation/cdl/en/lefunctionsref/67960/HTML/default/viewer.htm#n1mdh2gvd5potjn14jipysvzn4o7.htm

Re: Reading Text

afs — Sun, 06 Nov 2016 02:12:35 GMT

It worked -Thanks But the result is showing both the text and the sentence..Is it possible just to see the result ?

Re: Reading Text

afs — Sun, 06 Nov 2016 02:13:49 GMT

Dear Patrick-Thanks - I will study the link

Re: Reading Text

PGStats — Sun, 06 Nov 2016 02:28:12 GMT

keep sentence;

instead of

keep text sentence;

Re: Reading Text

Shmuel — Sun, 06 Nov 2016 02:46:39 GMT

Thank you @Patrick, I have skiped the right modifier reading the FINDC documentation.

this function shorts the code into:

i = findc(paragraph , '.?!' , B); /* B modifies to search Backwards */

sentence = substr(paragraph, i+1, length(trim(paragraph)) - i );

put sentence=; /* write the sentence to the log */

Re: Reading Text

Ksharp — Sun, 06 Nov 2016 04:48:13 GMT

Can't use scan() ?

 want = scan(paragraph ,-1, '.?!' );

Re: Reading Text

PGStats — Sun, 06 Nov 2016 05:03:31 GMT

@Ksharp, you would have to use something like

sentence = coalescec(scan(text ,-1, '.?!' ), scan(text ,-2, '.?!' ));

but you would still miss the last punctuation character.

Re: Reading Text

Shmuel — Sun, 06 Nov 2016 06:40:02 GMT

As any of the functions: INDEXC , FINDC, SCAN - will miss the delimiter character (. ? ! )

then maybe a slight change to the loop code will do the work:

do i = length(trim(paragraph)) - 1 to 1 by -1;

if substr(paragraph,i,1) in ('.' , '!' , '?') /* you may add any other characters you want */

then leave; /* get out of the loop with i points to end of last_previous sentence */

end;

last_sentence = substr(paragraph, i+1, length(trim(paragraph)) - i );

Re: Reading Text

PGStats — Sun, 06 Nov 2016 18:53:06 GMT

Same, but slightly improved and better tested:

data have;
infile datalines truncover;
input text $char350.;
datalines;
i want to read the last sentence in a long text having length 350 .it can be separated by marks, or ! or ? or period. I don't want to use  No leading blanks be there.   I don't know how to develop a code for this .
First sentence. Last sentence with missing punctuation
First sentence!!! Last sentence, with emphatic punctuation ?!

First sentence is also the last.
;

data lastSentence;
set have;
length sentence $350;
if not prx then prx + prxParse("/[^.?!]+([.?!]+|$)/");
start = 1;
stop = length(text);
call prxNext(prx, start, stop, text, pos, len);
do while(pos>0);
    sentence = left(substr(text, pos, len));
    call prxNext(prx, start, stop, text, pos, len);
    end;
keep sentence;
run;

proc print; run;