topic How to Extract information between line(s) from HTML in SAS Programming

How to Extract information between line(s) from HTML

may0423 — Mon, 26 Dec 2016 22:00:38 GMT

Hello everybody,

I am trying to extract infomation from a website. As I am very new to SAS, I don't know how to get information/paragraph between lines. I've been thinking of this for sever days already T.T . Please help!!!

Question 1: to get usertitle information between and , and also to specifiy if such information is missing when there is nothing between and< span style="font-weight: ).

Member

......

Junior Member

......

Not your guy, fwiend...

Question 2: to extract replied contents between <blockquote class="postcontent restore "> and </blockquote>, and to delete in the output.

<blockquote class="postcontent restore ">
replied content line 1 -- omit details here for brevity 
 
replied content line 2 -- omit details here for brevity. 
 
replied content line 3 -- omit details here for brevity.
</blockquote>

......

Thank you very much in advance!

Re: How to Extract information between line(s) from HTML

mkeintz — Mon, 26 Dec 2016 22:41:49 GMT

You could read line-by-line and look for '', or you could use the INPUT statement to do it for you as in

input @ '' / _line_ :&$200. ;

The @ '' says to look for the specified string, even if it goes over serveral lines.
The '/' means skip to next line.
The remainder says to read in a character variable named _LINE_ of up to 200 chacracters (and the '&' means don't stop before 200 characters if you encounter interior single blanks in the line - so you get "Junior Member" instead of just "Junior").

Then all you have to do is check the contents of the _LINE_ variable for the unwanted markup, and assign USERTITLE accordingly, as below (note the "=:" relation is different from the ordinary "=". It means compare only the first X characters, where X is the length of the shorter character value)

Also you'll be reading from an external file, so use the INFILE statement to point the INPUT operation to the right source.

data want;

length usertitle $20 ;

input @ '' / _line_ :&$200. ;

if not (_line_ =: '<span style="font-weight:') then usertitle=_line_;

drop _line_;

datalines4;

Member

Junior Member

Not your guy, fwiend...

;;;;

run;

That solves your first request. And you can use the same tools to begin solving the second.

Re: How to Extract information between line(s) from HTML

may0423 — Mon, 26 Dec 2016 23:27:28 GMT

Hi mkeintz,

Thank you for the quick response. What if my data is from an url where there are 25 usertitles? The datalines4; seems not work for me.

Re: How to Extract information between line(s) from HTML

may0423 — Tue, 27 Dec 2016 00:21:56 GMT

I adjusted a little bit. Works now!!!!!

Thank you very much mkeintz

Re: How to Extract information between line(s) from HTML

mkeintz — Tue, 27 Dec 2016 01:33:24 GMT

When SAS reads data from a series of lines directly following the data step program (rather than from an external file), the DATALINES statement is needed to tell SAS that the program code is ended and the data is about to start. I should have told you that when you read from an external file, the datalines statement is not needed. The reason it's DATALINES4 rather then DATALINES is because otherwise SAS will take the first semicolon in the data to indicate end-of-data. DATALINES4 tells SAS that 4 consecutive semicolons are required to indicate end of data. (So you can drop the line of 4 semicolons also).