Hi experts:
We have SAS Viya 3.5 and need to deploy projects following DevOps practices for ML. We have a folder structure with .sas scripts that do the following tasks:
- Data Validation
- Feature Engineering: for now is pure CAS tasks, but in the future it will call a databricks data pipeline
- Model Scoring: it call a SAS Model Manager's published model
- Reporting: table for reporting in Visual Analytics
- Other steps (email/teams notification, etc.): call outook or teams API
I need to schedule these steps.
I learnt how to execute scripts from command line and through APIs (from MAS in SAS Model Manager and SAS Intelligent Decision), but I need a way to orchestrate these .sas files in order. I see that in SAS 9 we have SAS Management Console, jobs dependencies and flow scheduling. Is there a way to do the same in SAS Viya through SAS Workflow Manager and schedule the whole workflow? Or should I use other tools like Airflow?
The only way I see is to create a main .sas script that includes other scripts and schedule it throught jobs in Environment Manager, but I would like to deacople the steps.
Sorry if the question is basic and general. I am open to all opinions.
Best regards,
Javier
