Paralle Iterator or Fork for Parallel Processing

[Shakti_Sourav]

Dear Team,

I have created a Data job having 30 nodes and it's taking 16 hours to complete. I want to run the job in Parallel. How to do that ? 

I have attached one data job which is conatins 30 nodes.

I have created New process job and then what will I do ? 

Thanks and regrads,

Shakti

[JosvanderVelden]

Can you provide some more details?
For instance:
Is the job created with data management studio or another SAS product/solution?
Where will the job run? A single server? A cluster? How many sas workspaces do you have?
Is all data in SAS or do you have a RDBMS or cloud storage involved?

[Shakti_Sourav]

Hi
1. Job is created in the Data management Studio.
2. Job run in DM server
3. I have connected to the Oracle.

[JosvanderVelden]

Have you seen this thread? https://communities.sas.com/t5/SAS-Data-Management/Parallel-Data-Jobs-in-a-Process-Job/td-p/148824

Can all the flows you have run in parallel or are there steps that are dependent on other steps?

[SASKiwi]

The way I run SAS DQ jobs in parallel is to split the DQ job into separate jobs, one for each parallel stream. Then I run the DQ jobs from a SAS program which uses SAS/CONNECT to run the jobs in separate SAS sessions. If you don't have SAS/CONNECT you could always just run separate SAS programs as batch jobs at the same time. The job originally took over 12 hours and now runs in less than 4 hours.

[Shakti_Sourav]

I have created Process job. Then I took Parallel Iterator node for parallel processing. Under Parallel processing, I referred my batch job based on Path.

Is it way to parallel processing or what ?

[SASKiwi]

I've never tried it the way you are doing it. I call a DQ job from a SAS program like this:

data _null_;
  rc        = dmsrvauthdom("DMServerAuth");
  JobName   = 'My DQ Job Name.ddf';
  jobid     = dmsrvBatchJob (JobName, 'http://MyDQAppServer.com', 21036);
  jobstatus = dmsrvJobStatus(jobid, 'http://MyDQAppServer.com', 21036, -1, 5);
  put _all_;
run;

If you are not familiar with this way and you don't know how to code SAS programs then I suggest that you continue with your way and open a Tech Support track to get help.

[SASKiwi]

After a bit of research I think you need to use the Fork node. The Parallel Iterator is for when you want to loop multiple times in parallel.