Flexibility in Data Pull for Execution of Production Model

[kahuso]

I am attempting to change the production execution of a model from 3am to 11pm the previous day. The data pull requires completion of a data load for execution that is tied to a daily date column and the model is executed daily. Do you have any suggestions on how to create flexibility in scripting to account for pulling data after midnight, in the event the data load is incomplete by that time?

[SASKiwi]

What is the earliest time your source data will be complete and ready to be extracted? There is no point doing a data pull until you are sure that your source data is complete - how do you tell if it is complete or not?

I have a similar daily extraction process. To confirm the source data is complete I run a query process that loops with a time delay. Once the process confirms the row count and the latest load time, the looping stops and the extraction starts. This is one way you could approach your problem.