Output of scheduled SAS DI Job is different from manually run DI job

[noobs]

Hello SAS EG Users,

I did not see category for SAS Data Integration Studio and hence I had to post this thread here.

I have DI job that is created by exporting code in SAS EG. This job is meant to mine all the data and create one observation and update six different tables. Attached is the code for the DI Job.

When this job is run manually, it gives correct output and it matches perfectly with that of SAS EG.

However when this job is scheduled to run on two dependencies, which is basically the source tables are ready and at 5:01 am it generates slightly different data. However it should match exactly with that of output of SAS EG.

Can you explain this behavior? What might be resulting in this discrepency.

I am also attaching excel workbook that pretty much explains the monitoring that I have been doing for last one week, when there are some days when the output
of scheduled DI job matches exactly the output of SAS EG project and some days when there is slight variance. Please check the pink worksheets that are named as EM_*_F. Let me also mention that the source tables for this job is snapshot and not live production data that changes from time to time.

Thanks,

Dhanashree

[SASKiwi]

Your worksheet doesn't appear to be attached. What are the differences you are seeing? Could the source data be different between 5.01am and when you run your EG job?

[Quentin]

I think that "Data Management" community is the main home for DIS questions.

One possibility is that the scheduled job is probably running on a different logical server (probably a batch server) than the EG or manually run DI job (probably a workspace server).  It's possible there are different system options on the different servers, etc.

Assuming the data isn't huge, maybe have your batch job make a permanent copy of the source data and the final output data.  Then run your code in EG, and PROC COMPARE both the source data to the source data from the batch run, and the output data to the output data from the batch run.  If you've got the same source data, same code, and different output data, that would suggest a difference in the server environment / system options.

-Q.