SAS Data Integration Studio, DataFlux Data Management Studio, SAS/ACCESS, SAS Data Loader for Hadoop and others

one Job which load automate 5 jobs with validation

Reply
Contributor
Posts: 68

one Job which load automate 5 jobs with validation

Hello SAS DI experts,

I need a help in SAS DI Job related to seneario based Job.

Suppose I have a layer name Raw Jobs which contain 5 jobs, and it is quire simple One to one.
to execute those 5 jobs, I want one master job which should trigger those 5 jobs and as well as
validate that which job data is available or not?


Can any body have experience on this mixing kind of Scenario, pelase let me know the procedure.,How we can do it.
please help me.

 

SAS Employee
Posts: 12

Re: one Job which load automate 5 jobs with validation

Posted in reply to Ritesh_dellvostro

hi Ritesh_dellvostro,

 

The typical scenario that I've seen for this is done by deploying the 5 jobs to the scheduler and creating a scheduling flow that runs the jobs conditionally. A 6th job could be created to validate the results, or they could be validated after each of the 5 jobs is run. 

 

But you could also create a "master" DI Studio flow by creating a new flow and dragging the other 5 flows onto it. You can implement Conditional Start and Conditional End transformations to control execution and write/drop transformations to evaluate results.  I'm not exactly sure what you mean by "validate that which job data is available or not" (there are several interpretations of this) so I can't give a specific answer, but it is possible.

 

Steve

Contributor
Posts: 68

Re: one Job which load automate 5 jobs with validation

[ Edited ]

Thanks @SteveED@Patrick for your input. it was good suggestion given by you, I will try to implement it.

 

For more information asked by @Patrick and @SteveED is given below:

 

1)-  For all 5 job, Source file is available or not?

2)-  If 4 Files are available and 1 is not updated with new file, what kind of validation we can keep for schduling and How?

3)-  @SteveED-  Can you Give more information on this statement - "You can implement Conditional Start and Conditional End transformations to control execution and write/drop transformations to evaluate results."

 

 

Thanks for your help.

 

Regards,

Ritesh

SAS Employee
Posts: 12

Re: one Job which load automate 5 jobs with validation

Posted in reply to Ritesh_dellvostro

Ritesh,

 

from the users guide:

 

Running Conditional Processing

 

Managing the status of jobs and transformations

 

You can use these to selectively execute transformations (or nested flows) and evaluate the results to determine if processing should continue. Investigate deploying jobs and using the schedule manager first though. It's a better solution.

 

Steve

Respected Advisor
Posts: 4,173

Re: one Job which load automate 5 jobs with validation

Posted in reply to Ritesh_dellvostro

I'm very much with @SteveED on the same page here. If you have to run 5 jobs in parallel, sequentially or with some dependencies then you'd typically use a scheduler.

Logically you can also go for a master DIS job by dragging all the other jobs into the master DIS job canvas - BUT: What this does is actually creating a single big job which runs in a single workspace session. Such an approach works but I'd consider it from an operational point of view as inferior to proper scheduling.

 

As for your data availability question:The answer depends very much of what kind of control/validation you're after.

- Source files available? Source files not empty? Data loaded into target? Data in source was valid? Extract job run successfully and loaded data?

- What should happen if a condition is not met?

Ask a Question
Discussion stats
  • 4 replies
  • 175 views
  • 0 likes
  • 3 in conversation