Execute steps I do in SAS Studio programmatically

[the_yeti]

Hello,

As part of a workflow, I routinely need to upload some input data and run some SAS code to process that data, which outputs some result files that I download to consume these results.

I do all that "manually" via SAS Studio, using the web interface to upload an input file, then copy/paste some code into a new tab, run it, check for errors, then ultimately download the output files via the web interface.

In a simplified way, these would be the steps I do in SAS Studio:

1.) Upload a file, say "input.csv"

2.) Run a code that perform some calculations using the "input.csv" data, and generates a second file "output.csv"

3.) Download the "output.csv" to my machine

Except that I do that dozens of times every month... It's tedious and unnecessary to have a human doing this when it could be automated by some software that can orchestrate these activities... Could be a simple Python or PowerShell script, or a more robust .NET application built in C# or Java, etc. The tech doesn't really matter.

My question is: How can I automate these steps? Is there any kind of API for SAS Studio that I can use? I'd hope there is some sort of web api/REST that I could use to perform the same operations I can do in SAS Studio, but from code.

Thank you in advance for any pointers!

[ballardw]

Are these files you "upload" in the same format or does the format change?

Does the input file(s) name change?

Are the calculations or steps exactly the same, mostly the same or do they change every time?

Do the output file names change?

Ideally you could show the code you are using. If you can get a FILENAME to point to the file location and there are not lots of changes then this is fairly routine:

1: Have a filename statement to point to the csv file location. Note that FILENAME can access many different file locations using things such as FTP, URL but specifics depend on the file locations.

2: Have a data step to read the data from that file

   (note if you have been using proc import to bring in the data your log likely shows the data step code generated by SAS to read the file,  copy the code from the log and clean it up. Make sure that the variable types/lengths make sense)

3: run the remaining code

4: Use an appropriate output method for the results (could be Proc Export, ODS destination with multiple procs depends)

There could very well be absolutely no need for external programs depending on exactly what you mean by "using the web interface to upload an input file".

If you have different file formats repeat 1 through 4 for each.

[the_yeti]

Thank you for your reply ballardw!

The input files are always different, their names are different, their contents are different, and the SAS code can be different depending on the contents of the file. The SAS code I need to run is being automatically generated by an application (running on my desktop) that analyzes the input file and generates the necessary SAS code to process this file. The output file is always different as well, based on the contents of the input file. The name is dynamically determined based on conventions and the input file.

I need the ability to (from my desktop) upload a file to the same folders that SAS Studio can see. I don't have another way of accessing those folders other than via SAS Studio. I don't have an FTP I can connect to, or similar. All I have is SAS Studio.

For instance, on step #1, I want to run a script on my machine, that can (somehow) upload a CSV file to some folder in SAS Studio, the same way that I can right click on a folder and select "Upload".

How can I do that, using a programming language?

Basically I want to use SAS Studio in a "headless" mode...

[Reeza]

Where’s is SAS installed, I’m assuming not locally, so on your company server somewhere? Easiest way is to get your company to set up a shared space where the files can be stored to avoid the upload/download. 

[the_yeti]

We use SAS OnDemand, so we don't have a "server" that we own. It's the cloud service.

My only interface with SAS is via HTTP/HTTPS through SAS Studio. I don't have a way to have my company setup a shared space. I need to perform the upload through HTTP or HTTPS

Uploading data is just one part of the issue. I also need to execute code from a .sas file that exists only on my machine. How can I - remotely - tell SAS Studio to execute a particular SAS code or .sas file?

[ my computer ]    ------->    [ SAS OnDemand]

SAS code           ------->      run

[Reeza]

I'm assuming this isn't Academics on Demand, because otherwise that sounds like a violation of ToS unless you're possibly a researcher, but basically you're working with a system not designed to do what you want. And if it's OnDemand, I doubt you can even automate your connection.

You can look into Selenium if you want full automation. 

---

@the_yeti wrote:

We use SAS OnDemand, so we don't have a "server" that we own. It's the cloud service.

My only interface with SAS is via HTTP/HTTPS through SAS Studio. I don't have a way to have my company setup a shared space. I need to perform the upload through HTTP or HTTPS

Uploading data is just one part of the issue. I also need to execute code from a .sas file that exists only on my machine. How can I - remotely - tell SAS Studio to execute a particular SAS code or .sas file?

[ my computer ]    ------->    [ SAS OnDemand]

SAS code           ------->      run

---

[the_yeti]

Correct. This is a regular production licensed on demand instance licensed by the company I work for.

I considered Selenium and similar tools, but it's 2018 so it's hard to imagine that there's not an API that I can use to execute the same actions I can do on SAS Studio.

Any ideas?

[ballardw]

---

@the_yeti wrote:

Thank you for your reply ballardw!

The input files are always different, their names are different, their contents are different, and the SAS code can be different depending on the contents of the file. The SAS code I need to run is being automatically generated by an application (running on my desktop) that analyzes the input file and generates the necessary SAS code to process this file. The output file is always different as well, based on the contents of the input file. The name is dynamically determined based on conventions and the input file.

---

So are you asking if you can have SAS run that other application (or is it a SAS application) to generate the code?

Does the code generated by that application get written to a location? If so does it have a "pattern" you can use to determine which file to run. If this is the case you should be able to generate the "pattern" from some information and build an %include statement to bring the code into a session and execute.

I have no clue what "headless" means.

I know that I have written code that parsed a data storage location and then called the appropriate SAS code files to read the data and then do a number of diagnostics and send tables and graphs to a report location. I actually did not move any of the text files read I created filename statements to read them from their storage location on our network.

Where are these files actually located? On your local network or a remote site? The Filename with the URL access method might work but some details about where the files are would be needed to tell.

[the_yeti]

Hi ballardw,

No, I just need a way to tell the SAS engine that is running behind SAS Studio, to execute SAS code that I will give to it.

Imagine that I have a MyProgram.sas file on my local machine at my company.

I want to give this file to the OnDemand SAS Server (remotely), tell it to run it, and get results back.

[JohnJPS]

Any headway on this? At SASGF 2018, there were several sessions where people were doing some pretty cool stuff via REST API. At that time, I had no specific need, but I've come to a point where remote submission of something to run server side as a long-running background process would be pretty handy.

[the_yeti]

I ended up building a faux API layer on top of Selenium using Headless Chrome, and it does the job well.

AFAIK currently there are no documented REST APIs that we can call *into* SAS / SSOD to execute commands or read data. All the documentation I found on REST APIs are about calling *out* of SAS/SSOD to send/get data to/from external sources.

In other words - AFAIK - SAS/SSOD only provides a REST API **client** not a REST API **server**.