topic Re: Using SAS to automate login and data download from web portal in SAS Programming

Using SAS to automate login and data download from web portal

Ryanb2 — Fri, 22 Dec 2017 18:15:20 GMT

I'm trying to find a way to use SAS to automatically login to a front-end web portal to then enter filter criteria (e.g., date range) and select download. I would also have to deal with an "Are you sure you want to download this file..." popup window. Can you direct me to an article or post that addresses how this can be done in SAS? My hope is that I can automate this process so it can be run overnight.

I tried googling this topic but it was difficult to narrow down the search parameters to get what I needed. Any information you can provide is appreciated.

Thanks,

Ryan

Re: Using SAS to automate login and data download from web portal

ChrisHemedinger — Fri, 22 Dec 2017 18:30:38 GMT

Hi @Ryanb2,

Any chance that the web portal you're using also offers an API? If so, it would be much easier to use PROC HTTP and REST methods to drive this.

If not, then I think PROC HTTP is still your best hope. If the site uses Basic Auth then the signin could be simple with options on the proc. If not, then a more complex authentication might require that you add an auth key into the HEADERS statement.

The confirmation dialog is a special problem though. Maybe there is a URL parameter supported by the site that would bypass this? You might need to spy on the http traffic using your browser's debugging features to see what's going on.

Aside from looking for an API method, I'd start by looking to see of another person has automated this using a tool like cURL. If so, it's pretty straightforward to translate those commands to PROC HTTP.

Re: Using SAS to automate login and data download from web portal

Ryanb2 — Fri, 22 Dec 2017 22:50:13 GMT

Thanks for the response Chris. Unfortunately this web portal doesn't offer an API.

I checked out the link in your reply but I'm not familiar enough with web syntax or language to know even what post and get mean. I also googled proc http but didn't find anything that was more user-friendly. I was hoping to find an example that showed a web page and the code used to login.

Re: Using SAS to automate login and data download from web portal

ChrisHemedinger — Sat, 23 Dec 2017 15:37:07 GMT

Is this a web portal that is open to the internet -- something we can look at? Or is it internal to your organization?

Re: Using SAS to automate login and data download from web portal

Ryanb2 — Tue, 26 Dec 2017 16:43:25 GMT

I appreciate your willingness to look at the website. It is a web portal open to the Internet, but due to the sensitive nature of the data and the restrictions required to access them I don't feel comfortable posting the link here.

Re: Using SAS to automate login and data download from web portal

Ryanb2 — Tue, 26 Dec 2017 16:52:13 GMT

I don't know if this is helpful but I pasted the html associated with the user name and password below:

<div>

</div>

<div>

<label>Password:</label>