topic Re: How to parse html? in SAS Programming

How to parse html?

piton — Wed, 25 Jun 2014 18:04:34 GMT

My sas table contains two columns: id and my_text . Each observation of my_text variable is a complete html string, something like this

<html> . . .

<head> . . .

<table style="WIDTH: 360.0pt;BORDER-COLLAPSE: collapse;"

border="0" cellspacing="0" cellpadding="0" width="480">

<tr style="HEIGHT: 15.0pt;"> <td style="BORDER-BOTTOM: rgb(236,233,216);

BORDER-LEFT: rgb(236,233,216); BACKGROUND-COLOR: transparent;

WIDTH: 360.0pt;HEIGHT: 15.0pt; " width="480">

So it contains <table> <tr <td . . . and other complex html structures. How to parse this kind of htmlinto plain text ?

____________________________________________________________________________________________________________________

I need to get a SAS table like this

id	my_text	plain_text
1	<html> . . . <head> . . . <meta blah ... blah ... blah ...	ONLY the "blah ... blah ... blah ... " part of my_text
2	...	...

my_text

plain_text

<html> . . .

<head> . . .

<meta

blah ... blah ... blah ...

ONLY the "blah ... blah

... blah ... " part of my_text

...

____________________________________________________________________________________________________________________

PS I was looking everywhere online for a good parsing code, however all the example are very trivial. The following PERL expression works fine only for 5 bytes. So this approach is okay for very simple tags .In my case it is useless. Please help

rx1=prxparse("s/<.*?>//");

call prxchange(rx1,99,my_text);

Re: How to parse html?

Ksharp — Thu, 26 Jun 2014 12:27:22 GMT

"PERL expression works fine only for 5 bytes."

What do you mean can't work ? prxchange can't do that ? or did you try call prxnext() ?

Re: How to parse html?

RW9 — Thu, 26 Jun 2014 12:36:25 GMT

Have you tried opening with Excel and saving the output. Excel has a pretty decent XML/HTML parser present, and then you could save the output in delimited format to read into SAS. Its dependant on your HTML data of course.

Re: How to parse html?

piton — Thu, 26 Jun 2014 19:34:00 GMT

Thank you. I have a sample program . Here is what I tried

data t; text1='<html> <head> <meta name=''generator'' content=''HTML Tidy, see www.w3.org'' /> <title></title> </head> <body> <p>Test</p> <p></p> <table style=''WIDTH: 360.0pt;BORDER-COLLAPSE: collapse;'' border=''0'' cellspacing=''0'' cellpadding=''0'' width=''480''>' ; regex = prxparse('s/<\s+.*?>/ /'); call prxchange(regex,-1,text1); put text1; run;

But it did not work

Re: How to parse html?

Ksharp — Fri, 27 Jun 2014 12:17:51 GMT

How about this :

data t;

text1='<html> <head> <meta name=''generator'' content=''HTML Tidy, see www.w3.org'' />

<title></title> </head> <body> <p>Test</p> <p></p> <table style=''WIDTH:

360.0pt;BORDER-COLLAPSE: collapse;'' border=''0'' cellspacing=''0'' cellpadding=''0'' width=''480''>'

;

text2=prxchange('s/<[^<>]*>//',-1,text1);

run;

Xia Keshan