DATA Step, Macro, Functions and more

Pulling website data

Occasional Contributor
Posts: 9

Pulling website data

I am struggling in my attempt to pull data into a table from a website.  Here is the code that I have thus far, but with this method, I have to parse the data (which I am not exactly sure how).  I also see other approaches where I could use the XML Mapper, but my mapper references keep failing (libname sec xmlv2 xmlfileref= test xmlmap=tempmap automap=replace)  Error:  The creation of the XML Mapper file failed.


Any thoughts are appreciated.  Thanks.  



filename src temp;

proc http






data rep;

infile src length=len lrecl=32767;

input line $varying32767. len;

line = strip(line);

if len>0;


Respected Advisor
Posts: 4,678

Re: Pulling website data

[ Edited ]
Posted in reply to Lost_Gary


The page is in HTML so I fear you'll have to parse the result.


On the positive side: The links to all the different pages (A, B, C,...) are easily accessible in the code so once you've got the parsing of the first A page working, you should be able to just iterate over all the links using the same HTTP request and parsing code.


And just as a thought: If that's a one off then it's eventually much easier to just copy/paste the tables into a text editor and then use a simple data step infile/input statement reading in orthogonal tab delimited data.

Ask a Question
Discussion stats
  • 1 reply
  • 2 in conversation