Loop's Parallel Processing Configuration in Data Integration Studio

[gkopanidis]

Hello,

There is a strange error I receive when I execute a loop with parallel processing into Data Integration Studio 4.906.

Specifically, I have an If Condition to specify the tables' technology among SQL, Oracle and DB2. After the successful execution of the If Condition, there are three different jobs, one for each technology, that manipulate properly the tables. Inside each job there is a loop node in which each table is transformed with some code. Here comes the error. When I try to execute this loop node giving as loop options the execution iteration in parallel with number 4, I receive the following error:   "ERROR 180-322: Statement is not valid or it is used out of proper order". But when I run the loop with no execution iteration in parallel then it runs successfully.

Is there anything that I can change in the loop options that is responsible for this error?

[JosvanderVelden]

Are there iterations that are dependent on results (tables/views) or macro variables set in previous iterations?

[gkopanidis]

Hello,

I read a txt file and I write it in a work table. The work table contains only one column with distinct tables' names that I want to manipulate later. Then I execute the loop in which I create sample Data Sets based on the context of the previous work table. The parallel processing takes each time 4 rows from the work table. SAS code receives as parameter the name of each table.

 

[Patrick]

It's been a while that I've used DIS but from memory the main difference in the generated code when using parallel execution is that the code in the inner node (your Create Sample Data Sets) gets executed in an rsubmit block. 

My first test would be to set the number of parallel jobs to 1 and see what happens.

If this still fails (what I'd expect) then my next test would be to replace your code in the inner node with a simple hello world program. 

This lets you test if there is something going wrong with creating the rsubmit block with parameter passing or if it's with your actual code.

If it's the parameter passing then you would need to look into your parameter table if there is something that potentially could break the generated code (like unbalance quotes, or macro tokens). 

If the hello world program works then you would need to dig into the log to actually understand what exactly creates the error message - like if your program uses some macro variable that gets created in the parent process but doesn't get passed to the child process.

[gkopanidis]

Hello,

Thank you for your answer.

The first test ( parallel jobs to 1) failed. Second test (hello world program) also failed.

Let me remind you that if I disable the parallel execution, the job will run successfully.

How can I check what's going wrong with the rsubmit block?

[Patrick]

---

@gkopanidis wrote:

Hello,

....

How can I check what's going wrong with the rsubmit block?

---

You need to look into the DIS generated code and log. 

You can also grab the generated code for the whole job and run it out of SAS EG/SAS Studio. Make sure that set options mprint mlogic symbolgen to get sufficient log messaging.