How To Fetch Data From Website Through SAS Code

[kaushalsolanki]

Hello All,

I want to know that how we can fetch particuler information from the website through SAS Code.

As an example i want to fetch all values of particuler class from website how i can get that?

Regards,

Kaushal Solanki

[RW9]

You will need to be more specific in your request, what data are you getting, is it a web page, a json stream, a database etc.  There is proc http for instance:

http://support.sas.com/documentation/cdl/en/proc/65145/HTML/default/viewer.htm#n0t7v16eitluu2n15ffpf...

[kaushalsolanki]

Hi RW9,

Thank you for the reply, My data is web page.

As an example we are having all state values on one website and it consist same class=state at source code level

so i want to fetch all state names.

Regards,

kaushal Solanki

[RW9]

Hi,

Sorry, that doesn't help.  A web page can contain any number of items ranging from HTML, pictures, through flash objects, javascript, database connections etc.  Maybe post the web link?

[kaushalsolanki]

Hi,

As an example link is as given below.

Link: http://www.naukri.com/sas-jobs 

On this webpage there are number of different sas jobs so i want fetch all jobs designation and location,

If you will go through source code of this web page all designation comes under class="desig"

and locations will comes under class="loc".

So how we can fetch the required data?

[RW9]

Ah, what you mean is you have saved that web page to your local drive and you are looking at the text saved.  You can parse that file then for the tag and pull out the required information.  A simple:

data want;

  file "<yourfile>.html";

  input;

  if index(_infile_,"class=") > 0 then output;

run;

This should give you all the information, but each run you will need to save the document.

The web page itself is actually javascript pulling data from a database or json stream and rendering it.  So if you want to  access it programmatically you would need the API to access their DB directly

[Reeza]

Scraping a website is possible with SAS but I would recommend a third party software. 

Import.io is my favourite tool since it's free. Python has many packages that do this well. 

You could write a script you then call from SAS. 

Also, make sure you're not violating the terms of service for the particular website your scraping. 

[kaushalsolanki]

Hi Reeza,

Can you guide me how we can call python script from SAS.

Regards,

Kaushal Solanki

[Reeza · Accepted solution]

Assuming you have X command enabled you can use the system call. 

You would need to write the python script. 

Format would be path to Python application followed by path to Python script. 

X 'c:\program files\Python27\python.exe scrape.py';

Heres an example of a Python scraping script

http://docs.python-guide.org/en/latest/scenarios/scrape/