topic Extracting Data on Website in SAS Procedures

Extracting Data on Website

deleted_user — Wed, 27 May 2009 14:46:09 GMT

I currently access data via an external website. The current process is to log on to the website and run queries on the databases held within the website. This is a very manual process, and I am sure there is a quicker way to do this via SAS. However, I am not sure where to start with this. I hope you can explain to me how I would go about doing this.

I have already asked this question on the SAS helpdesk, and it had them pretty stumped. They originally suggested the URL filename engine, but as the URL did not change when the query changed, it was not possible to use this.

Would welcome any suggestions.

Re: Extracting Data on Website

sbb — Wed, 27 May 2009 20:19:33 GMT

It's unclear whether you want to automate the query execution process itself (the keystrokes normally entered in the browser at the website) or if you are more interested in accessing the website's back-end server database resources for generating some type of outputs/reports? I would say that a SAS application program developed to access the database is definitely possible, after determining the details about the server environment, authentication requirements, and the query code requirements, to start with.

Scott Barry
SBBWorks, Inc.

Re: Extracting Data on Website

deleted_user — Thu, 28 May 2009 07:21:53 GMT

Hi Scott.

I would be looking to automate the access of the backend databases rather than automate the keystrokes. The databases we access contain all company account info posted at company house. We need to run queries on the backend databases to extract all <£5m turnover businesses and group them by industry code. There is an initial log on to the website so this would need to be built into the code. Not sure about server environment. I may need to contact Experian to get this info as it is their website that we access.

Let me know what your thoughts are, and please give me a shout if you need any more info.

Appreciate any help you can provide. At the moment it takes us 2-3 days to manually extract this, but Im sure it can be done in a matter of hours!

Re: Extracting Data on Website

LinusH — Thu, 28 May 2009 09:29:02 GMT

Yes, you have to find out alternative ways to get to the data. It could be done via direct DBMS access, or using some other API:s. Since you are an external client, you also have to consider firewalls and other limitations due to company policy and security issues.

/Linus

Re: Extracting Data on Website

deleted_user — Thu, 28 May 2009 10:29:37 GMT

Thanks Linus.

What information do i need to request from Experian to access this data directly via DBMS?

Re: Extracting Data on Website

sbb — Thu, 28 May 2009 12:11:26 GMT

From the SAS support http://support.sas.com/ - start here:

“How Do I Love Thee? Let Me Count The Ways.”
SAS Software as a Part of the Corporate Information Factory
John E. Bentley, Wachovia Bank, Charlotte, North Carolina

http://www2.sas.com/proceedings/sugi28/163-28.pdf

Then inquire with Experian about exactly what data is accessible to your enterprise, and then discuss what's already been developed with your colleagues and peers. Then begin to "spec" out a SAS application suitable to your data mining needs. Then....go forward to design, develop, test, document, and implement a strategy.

Scott Barry
SBBWorks, Inc.