topic Re: Viya, scheduling and automation in Administration and Deployment

Viya, scheduling and automation

tisbam — Mon, 07 Sep 2020 14:40:35 GMT

Hi all,

I am looking to have some feedbacks on practical implementation of scheduling on Viya.

For now we have a few reload jobs for data generated from a Base SAS server but we would like to move our existing code base to run on Viya, and thus also move the scheduling (which for now is done with Windows Task Scheduler).

Users (myself included) are not really into the Studio V, and we still use EG to develop SAS code. I like the recent integration with Git, and would rather store our codebase in git, rather than in Sas Drive.

So far I haven't found a comprehensive way of scheduling simple execution of SAS code stored in git. I have started looking into alternative such as Jenkins (which I think I saw in a SAS blog post), but am not fond of having an additional server and tool to maintain beside Viya.

Do you have similar requirements on your side? How are you handling them?

Looking forward hearing from all of you!

Thanks

Re: Viya, scheduling and automation

SASKiwi — Mon, 07 Sep 2020 20:03:45 GMT

An interesting question and something I'd like to know more about before migrating to Viya.

I do know Viya Environment Manager provides scheduling capabilities: https://documentation.sas.com/?docsetId=calscheduling&docsetTarget=p0be15ptyesy4bn0ze76u3gd87mo.htm&docsetVersion=3.3&locale=en

Re: Viya, scheduling and automation

JuanS_OCS — Wed, 09 Sep 2020 11:30:18 GMT

Hello @tisbam ,

if I would be you, I would be looking at google searches such as "mount git as permanent volume" or similar.

There are some projects out there, such as GitFS, which should allow you to mount your git repo as a OS volume. Once it is locally accessible by your server, you should be able to run sas code stored in a Git server, but calling it as it was local.

https://www.presslabs.com/docs/code/gitfs/

https://github.com/PressLabs/gitfs

If you work with Docket containers this might be of your interest: https://docs.microsoft.com/en-us/azure/container-instances/container-instances-volume-gitrepo

Unfortunately, it only works with linux containers, for now.

Of course you can have intermediate solutions, such as to create your own macros that would temporary download the sas code from git, before running it ... but... it seems as an overkilling solution to me.

Re: Viya, scheduling and automation

tisbam — Thu, 10 Sep 2020 16:24:18 GMT

Thanks for the reply.

I just tried something similar:

Job created with SASJobExecution to regularly pull new changes to a directory on the fileserver
Fileref in autoexec referencing to the git directory on the fileserver
Using this fileref to manage include (since SAS has no better way for modular code)
Creating new jobs with SASJobExecution (things like %include root("src/project1/main.sas");

It works ok, but once the scheduling done is not bound anymore to the file (hence the use of %include so that if source code change i don't need t update the job

This also means no branch whatsoever in git, and that we have to push and wait for the job to be bale to run code while testing...

Not so bad but not entirely convinced either !

Really wondering how people are managing this. I feel like people either go the SAS Studio way, or use external CI/CD tools.

Cheers