hi,
I have a etl job running on spds server. The scheduled job on lsf is recently going into long run. The etl does not get completed in a days time and when server restarts automatically, job gets killed.
There are 32 lakhs of input records are present in the source table, that needs to be loaded into target table.
Job goes in hung state at scd type2 loader. No new record gets inserted in the spds target table.
Kindly help me with the solution here.
Thanks
Neha
If it doesn't abend, it's probably lack of resources, or some kind of really inefficient "programming"/ETL logic.
Try to work on a small subset from the sources, then carefully follow the log on the timing in each step.
Temporarily change from view to tables as transformation outputs.
If you are executing in DI Studio you could get help via the "Collect runtime statistics" option which present the results as a graph or table.
Some other questions/suggestions:
A couple more:
Don't miss out on SAS Innovate - Register now for the FREE Livestream!
Can't make it to Vegas? No problem! Watch our general sessions LIVE or on-demand starting April 17th. Hear from SAS execs, best-selling author Adam Grant, Hot Ones host Sean Evans, top tech journalist Kara Swisher, AI expert Cassie Kozyrkov, and the mind-blowing dance crew iLuminate! Plus, get access to over 20 breakout sessions.
Need to connect to databases in SAS Viya? SAS’ David Ghan shows you two methods – via SAS/ACCESS LIBNAME and SAS Data Connector SASLIBS – in this video.
Find more tutorials on the SAS Users YouTube channel.