SAS DataFlux documentation on error handling

[mahadeep2627]

Hi everyone,

I would like to know if there is any SAS DataFlux documentation on error handling of various kinds. Failing documentation, if anyone is willing to share their experiences, I'd be very grateful.

The applications: SAS 9.4, DataFlux Data Management Server and Studio 2.6

What are the processes?: DF Jobs and DF web services (based on jobs)

Types of errors:

1) License shortage (what is the behavior of DF if this occurs? any way to handle gracefully?)

2) Database connection failure or interruption (In a job or web service, any way to detect this and handle gracefully -- i.e., stop the job, log the error, alert operations to failure?

3) Any other job/web service failure of an unexpected kind?

4) Is there any mechanism or node that can detect all DF failures of any kind? Kind of like a java try/catch exception block, but applied to the entire job?

Thank you in advance!

-Dipak

[SASKiwi]

@mahadeep2627 - IMHO I think you are overthinking all of the possible things that could go wrong with your DataFlux jobs.

I, myself, am in the process of productionising a reasonably complicated DataFlux job. I have found that converting all DF processes into SAS code gives you great control over identifying errors and ensuring that all DF processes run error-free. I've also found that "normal" SAS code is great for extracting and preparing data to be processed and also for building the required data outputs at the end. I highly recommend this approach.

With regard to error handling, if my SAS program calling all DF processes returns a non-zero return code then I know something went wrong and I can investigate the cause. It is impossible to trap every possible thing that might go wrong, but it is easy to ensure via non-zero return codes and SAS log warnings and errors that you know there has been a problem. To me this is a lot easier than trying to programmatically check every step in your job. 

[Rama_V]

Licence shortage usually ends with errors in jobs. Few thoughts on this, you can also read Studio application startup logs located at C:\Users\userid\AppData\Roaming\DataFlux\DMStudio\studio1 or you can use the batch cmd to start the data flux server and read the log or read the job log file using other programs or create program which captures error using SAS program.

In the past organisation,we used to scan all the batch job executions logs and send an email with failures.

To your 2,3,4 error question: In Expression language, there are logging functions you can use to handle the error. You need to change the job settings to continue to run the jobs even when there is error in the nodes or job. Then catch Data flux job node status using source binding and Expression language functions to send the error and terminate the job gracefully.

Hope this helps.

[VincentRejany]

The document attached is 8 years old but what is described still applies.

[mahadeep2627]

Thanks for sharing this document.