topic Migration LSF to apache Airflow. in Moving to SAS Viya

Migration LSF to apache Airflow.

Narendra_P — Fri, 09 Feb 2024 14:12:48 GMT

Hi Team,

Is there a method to migrate from LSF to Apache Airflow? Are there any accelerators or automated conversion tools/scripts available for this migration? Additionally, could you provide any documentation, or steps outlining the approach or solution for this migration?

Thanks.

Re: Migration LSF to apache Airflow.

SASKiwi — Thu, 15 Feb 2024 23:57:40 GMT

If you search the Apache Airflow documentation you will find no reference to LSF, PPM and no mention that I could find about importing content from other schedulers so I'm pretty confident the answer is no.

Re: Migration LSF to apache Airflow.

Patrick — Sat, 17 Feb 2024 02:46:39 GMT

@Narendra_P

I don't know if such scripted migration is possible for any lsf scenario but I do know of a site where such a migration has been done using a scripted approach.

I can't share the details because they are confidential.

Given the importance of such a migration path I would expect though that it's only a question of time until there will also be some public information available.

The approach I know of in a nutshell:
1. Export the lsf flow definitions to an xml file using jexport

2. Create an xmlmap to convert the xml to a format (SAS tables) that's easier to work with.

3. Use the exported data to generate Python scripts with the Airflow DAG definitions to which the lsf flow definitions can get mapped.

Re: Migration LSF to apache Airflow.

SASKiwi — Sat, 17 Feb 2024 19:47:49 GMT

@Patrick - The XML Export function is also available as a menu option in PPM.