topic extract contents within html tags in SAS Programming

extract contents within html tags

kaziumair — Tue, 13 Apr 2021 12:29:48 GMT

Hi , everyone I am trying to extract an article from a website , the web page has the paragraphs in the following structure:

Our

universal abuse of natural resources has created an imbalance in nature, contributing to the beginning of extinction.

Over the last 100 years over 500 species have already gone extinct.

If we do not act now many more animals face extinction over the next 30 years including Orangutans, Rhinos, Polar Bears, Gorillas, Gibbons, Chimpanzees to name a few.

How do I extract the content within the html tags?

Re: extract contents within html tags

Kurt_Bremser — Tue, 13 Apr 2021 12:23:10 GMT

If the tags are always on an individual line, it's easy:

data want;
infile datalines truncover;
retain flag 0;
input line $200.;
if line = "<p>" then flag = 1;
else do;
  if line = "</p>" then flag = 0;
  if flag then output;
end;
drop flag;
datalines;
<p>
Our universal abuse of natural resources has created an imbalance in nature, contributing to the beginning of extinction. Over the last 100 years over 500 species have already gone extinct.
If we do not act now many more animals face extinction over the next 30 years including Orangutans, Rhinos, Polar Bears, Gorillas, Gibbons, Chimpanzees to name a few.
</p>
;

Re: extract contents within html tags

kaziumair — Tue, 13 Apr 2021 12:28:28 GMT

Hi , sorry I made a mistake in the example , the tags are not on individual lines, the paragraphs are similar to the following :
Our
universal abuse of natural resources has created an imbalance in nature, contributing to the beginning of extinction. Over the last 100 years over 500 species have already gone extinct.
If we do not act now many more animals face extinction over the next 30 years including Orangutans, Rhinos, Polar Bears, Gorillas, Gibbons, Chimpanzees to name a few.

Re: extract contents within html tags

Kurt_Bremser — Tue, 13 Apr 2021 12:51:12 GMT

I modified the code, now it can also deal with lines where is not at position 1 and not at the end of the line, or where both appear on one line (it does not handle cases where more than one paragraph appears on one input line):

data want;
infile datalines truncover;
retain flag 0;
input line $200.;
pos = index(line,'<p>');
if pos
then do;
  flag = 1;
  line = substr(line,pos + 3);
end;
pos = index(line,'</p');
if pos
then do;
  line = substr(line,1,pos - 1);
  output;
  flag = 0;
end;
else if flag then output;
drop flag pos;
datalines;
some uninteresting text
<p>Our
universal abuse of natural resources has created an imbalance in nature, contributing to the beginning of extinction. Over the last 100 years over 500 species have already gone extinct.
If we do not act now many more animals face extinction over the next 30 years including Orangutans, Rhinos, Polar Bears, Gorillas, Gibbons, Chimpanzees to name a few.</p>
xxx<p>this is a test</p>yyy
more uninteresting text
xxx<p>another
test</p>zzz
even more uninteresting text
;

Re: extract contents within html tags

Ksharp — Tue, 13 Apr 2021 12:45:06 GMT

Then remove these html tags .

data want;
infile cards truncover;
input line $200.;
want=prxchange('s/<.+?>//o',-1,line);
datalines;
<p>Our
universal abuse of natural resources has created an imbalance in nature, contributing to the beginning of extinction. Over the last 100 years over 500 species have already gone extinct.
If we do not act now many more animals face extinction over the next 30 years including Orangutans, Rhinos, Polar Bears, Gorillas, Gibbons, Chimpanzees to name a few.</p>
;

proc print;run;

Re: extract contents within html tags

kaziumair — Tue, 13 Apr 2021 12:59:17 GMT

Thanks a lot, it worked

Re: extract contents within html tags

kaziumair — Tue, 13 Apr 2021 13:08:23 GMT

Hi , thanks for taking out the time to help .