DATA Step, Macro, Functions and more

Pulling website data

Reply
Occasional Contributor
Posts: 9

Pulling website data

I am struggling in my attempt to pull data into a table from a website.  Here is the code that I have thus far, but with this method, I have to parse the data (which I am not exactly sure how).  I also see other approaches where I could use the XML Mapper, but my mapper references keep failing (libname sec xmlv2 xmlfileref= test xmlmap=tempmap automap=replace)  Error:  The creation of the XML Mapper file failed.

 

Any thoughts are appreciated.  Thanks.  

 

 

filename src temp;

proc http

method="GET"

url="http://www.barrons.com/public/page/majormarket-nasdaqnational-A.html"

out=src;

run;

 

data rep;

infile src length=len lrecl=32767;

input line $varying32767. len;

line = strip(line);

if len>0;

run;

Respected Advisor
Posts: 4,678

Re: Pulling website data

[ Edited ]
Posted in reply to Lost_Gary

@Lost_Gary

The page is in HTML so I fear you'll have to parse the result.

 

On the positive side: The links to all the different pages (A, B, C,...) are easily accessible in the code so once you've got the parsing of the first A page working, you should be able to just iterate over all the links using the same HTTP request and parsing code.

 

And just as a thought: If that's a one off then it's eventually much easier to just copy/paste the tables into a text editor and then use a simple data step infile/input statement reading in orthogonal tab delimited data.

Ask a Question
Discussion stats
  • 1 reply
  • 107 views
  • 0 likes
  • 2 in conversation