topic Re: Table loader append to exsting in SAS Data Management

Table loader append to exsting

89sankar — Sun, 27 Nov 2016 06:19:16 GMT

When I use Table loader append to existing data, I get the latest data appended to old data. While doing this, incase of error of space issue, how to avoid appending again the latest data two times?

Re: Table loader append to exsting

Reeza — Sun, 27 Nov 2016 07:13:40 GMT

There's no automatic way, assuming you're in base SAS. You need to define some rules on how to check if the process should proceed.

Re: Table loader append to exsting

89sankar — Sun, 27 Nov 2016 09:37:26 GMT

Do we have any option to do this setting in table loader? to make sure that I am not appending the latest data again?

Re: Table loader append to exsting

Reeza — Sun, 27 Nov 2016 09:40:32 GMT

Are you referring to Data Integration Studio? Or DaraFlux!

Re: Table loader append to exsting

89sankar — Sun, 27 Nov 2016 09:41:36 GMT

Data integration studio - Table loader transformation

Re: Table loader append to exsting

Patrick — Mon, 28 Nov 2016 03:48:32 GMT

You could use Update/Insert - but this will slow down performance.

One of the advantages in using DIS and scheduling is more control over your processes. Executing the same job twice on the same data shouldn't happen (eg. once data loaded copy the data source to an archive directory as part of your process).

You could also maintain some sort of control table where you keep the max. date of data loaded; and you then use this max date in a where clause to only select newer dates from your source. Then have a post process which updates the control table with the latest max. date.

Re: Table loader append to exsting

89sankar — Mon, 28 Nov 2016 04:16:51 GMT

Thanks!! It helps.