Broken process flow after node failure

[RyanK]

I cannot be the first person to ask this question, but my searching has not turned up any results.  I have run into the scenario a few times that a process flow becomes unusable, or very difficult to fix, after a fairly minor failure.  In this case, I have a somewhat complex process flow with a combination of task nodes and code nodes.

After the failure of a node, the link from the node to the output is removed.  The subsequent node is then linked to the output, not to the node that created it.  When the issue is corrected, the process flow cannot be executed again because a node attempts to access data that has not yet been created.

The only method to avoid this I have found is frequent save points, but it very easy to forget.

In this screenshot the top is the working process flow.  The bottom is the result after a failure step 9.  When it is re-run, step 10 kicks off without the input dataset being created.

EG 7.13 HF1 against SAS 9.4

[Kurt_Bremser]

This looks very much like an issue for SAS technical support. It could be a real bug.

[LinusH]

Not sure if that's a bug, perhaps just lacking functionality. I have experienced the same behaviour. But if there's a trick tech support might be the ones to know about it.
Remember that EG is primarily an adhoc tool. If you want to develop more robust applications I suggest that you use other tools.

[RyanK]

I tend to agree with you @LinusH, but it is being pushed more and more as a development tool, and we all know how easy it is for something to start adhoc, and turn into something more.

I have had some success with linking directly between the "work" nodes instead of linking from an output to the next node.  Unfortunately, that is not default behavior, making it quite tedious.

This is another situation where it would be helpful to have a process flow stop in its tracks on a failure.  That is currently also tedious to implement.

[CaseySmith]

---

@RyanK wrote:
This is another situation where it would be helpful to have a process flow stop in its tracks on a failure.  That is currently also tedious to implement.

---

We added a feature in EG 7.13, which does exactly this.  You can now control the action EG takes when an error occurs during execution.  You can set the option ("Action to take on errors during execution") for the whole project via File->Project Properties->Code Submission.  Or you can override it individually on each node, by right-clicking the node, selecting Properties, Code Submission.  The default option value is Stop current branch.  The pre-7.13 behavior was to continue execution.

Here is a screenshot of the option and the three choices.

Note: The Query Builder node with the error (ex. a Query Builder) will still break its link, since the process flow attempts to reflect the results of the most recent run.  However, at least all subsequent nodes are halted, limiting downstream break-up.

Note 2: Turning off Auto Arrange on the process flow (by right-clicking inside the process flow and toggling Auto Arrange) will help preserve the process flow layout when links are unintentionally broken.

[RyanK]

Thanks the tips!  I will certainly try these out.  When I looked at my project settings, I found that the code submission action was set to "Continue execution".  I do not recall ever seeing that setting before, but the project was initially created in EG 7.100.  Perhaps that setting persisted from the prior version.

When I created a new project in 7.13, it did default to "Stop current brach".

[CaseySmith]

You are welcome!  Yes, that is correct...  for existing projects (created in versions prior to 7.13), the option will be Continue execution, so the previous behavior is preserved by default.  We didn't want to change the behavior of an existing project unbeknownst to the user.  As you noted, new projects will default to Stop current branch.