Table loader append to exsting

[89sankar]

When I use Table loader append to existing data, I get the latest data appended to old data. While doing this, incase of error of space issue, how to avoid appending again the latest data two times?

[Reeza]

There's no automatic way, assuming you're in base SAS. You need to define some rules on how to check if the process should proceed. 

[89sankar]

Do we have any option to do this setting in table loader? to make sure that I am not appending the latest data again?

[Reeza]

Are you referring to Data Integration Studio? Or DaraFlux!

[89sankar]

Data integration studio - Table loader transformation

[Patrick · Accepted solution]

You could use Update/Insert - but this will slow down performance.

One of the advantages in using DIS and scheduling is more control over your processes. Executing the same job twice on the same data shouldn't happen (eg. once data loaded copy the data source to an archive directory as part of your process).

You could also maintain some sort of control table where you keep the max. date of data loaded; and you then use this max date in a where clause to only select newer dates from your source. Then have a post process which updates the control table with the latest max. date.

[89sankar]

Thanks!! It helps.