11-14-2016 01:23 PM
I have a etl job running on spds server. The scheduled job on lsf is recently going into long run. The etl does not get completed in a days time and when server restarts automatically, job gets killed.
There are 32 lakhs of input records are present in the source table, that needs to be loaded into target table.
Job goes in hung state at scd type2 loader. No new record gets inserted in the spds target table.
Kindly help me with the solution here.
11-14-2016 03:22 PM
If it doesn't abend, it's probably lack of resources, or some kind of really inefficient "programming"/ETL logic.
Try to work on a small subset from the sources, then carefully follow the log on the timing in each step.
Temporarily change from view to tables as transformation outputs.
If you are executing in DI Studio you could get help via the "Collect runtime statistics" option which present the results as a graph or table.
11-14-2016 05:32 PM
Some other questions/suggestions:
11-14-2016 05:49 PM
A couple more:
11-14-2016 10:07 PM
11-15-2016 11:19 AM
11-15-2016 12:58 PM
11-16-2016 01:09 AM