topic Re: Parallel Iterator in SAS Data Management

Parallel Iterator in DM Studio

Romix — Fri, 05 Sep 2014 08:00:30 GMT

Hi!

How get Instance_id like variable in expression within Data Job inside Parallel Iterator process job?

Re: Parallel Iterator

MikeFrost — Mon, 08 Sep 2014 18:57:07 GMT

Hi Romix,

It's not obvious to me what your question is. Can you explain in more detail what you are trying to do within the Parallel Iterator node, perhaps with an example?

Regards,

Mike F.

Re: Parallel Iterator

Romix — Tue, 09 Sep 2014 10:10:46 GMT

Hi Mike,

May be this screenshot will help us.

I would like to recieve and to use Instance_Id inside Data job node.

For example, I want to cut the table in to 4 parts and do parallel processing with index.

Re: Parallel Iterator

skillman — Tue, 09 Sep 2014 20:33:44 GMT

I believe this is just a counter to let you know which iteration of the iterator is running. Depending on what exactly you are doing, you could get a row count and divide the table into 4 equal parts using row counts.

Re: Parallel Iterator

MikeFrost — Tue, 09 Sep 2014 21:02:49 GMT

Hi Romix,

I have attached an example job, though I am not sure if it does exactly what you are looking for.

The job creates a NEW_PRODUCTS table, then, for each Product Category, reads the data from the PRODUCTS table, reformats the PRODUCT CODE and writes the values to the NEW_PRODUCTS table. After all Product Category records have been processed by the Parallel Iterator, indexes are then created on the NEW_PRODUCTS table.

Have a look and see if it helps illustrate how to use this feature.

Regards,

Mike F.

Re: Parallel Iterator

kevind21 — Tue, 08 Sep 2015 17:27:11 GMT

Hi Mike,

I know there has been some time since your last post to this particular thread. The topic is relevant for what I'm working on now so I'm hoping you can help me out. Do you have the referenced data job that the process job uses so I can see the job in it's entirety? The zip file only has the process job, but in the parallel iterator node it references a data job that isn't there. I just want to see how the data job is pulling the value from the process table reader. Also, based on the naming convention of the file (CH04_parallel_iterator_example), I assume the CH04 means Chapter 4, and that there are other files available. Any chance that you could provide these as well?