Append datasets with various variables

[TashaBee]

I have created a basic dataset. In the future, the files will be modified to create new fields. How can I append the dataset to include the new files?

This is what I am using:

append base=Cumulative data=Current force;

;

Of course it works, but the new variables are not being added. I won't know what the new variables are.

I guess I should also include that there is a possibility of an nonexistent dataset. If I use the SET, it will not create because the dataset does not exist. Is there a possible work around for this?

[ballardw]

Force option only forces the append procedure to continue if it finds variables 1) not in the base data, 2) do not have the same type or 3 are longer than in the base data.

No new variables are added though.

You might find a datastep and SET statement to be more amenable if the contents change frequently:

Data cumulative;

     set cumulative

           current;

run;

##WARNING: Even though the construct is legal very great care must be used when using the source and output data set as the same. I might recommend adding a variable for records from current to indicate date appended or some such.

ALSO, with this approach if any variable of the same name are of a different type you'll get errors and fail to append. If any character variables are longer in the Current set they may get truncated to the length in Cumulative.

If the only issue is new variables than you should be good;

To mark the "new" records you could use:

Data cumulative;

     set cumulative

           current (in=in2);

     if in2 then DateAdded=today();

run;

[TashaBee]

Thank you. The second solution helped. The formats should not change.

[RW9]

Hi,

It would sound to me like you haven't put enough thought into the dataset intially.  If you have a base dataset to which further data is added, then it should be robust enough to cover further data.  For instance, it sounds like your dealing with transposed datasets, i.e. something like this:

ID        JAN15     FEB15...

A          10           20

Each time new data is recieved then new columns would be present.  In this situation I tend to advise against having columns with data.  Parameterise your data and normalise it:

ID     PARAMETER     RESULT

A      JAN15                 10

A      FEB15                 20

...

This way your base dataset does not need to change each time, observations can be appended with no problems.  If later on you really need a transposed dataset then proc tranpose the base dataset to your requirement.

Know your data and what you want to do with it, thats the key point.  Trying to fight that, e.g. by using force, or allowing SAS to guess what you want to do will just give you problems further down the line, say for instance you allow more columns, any program down the line then breaks.

[TashaBee]

There is no transpose.

One dataset:

ID DueDate AmountPaid  DatePaid

Next dataset:

ID DueDate AmountPaid AmountDue DatePaid

[RW9]

Transfer 1

ID     PARAM          RESULT

A      DueDate         xyz

A      AmountPaid    xyz

A      Datepaid         xyz

Transfer 2

ID     PARAM          RESULT

B      DueDate         xyz

B      AmountPaid    xyz

B      AmountDue    xyz

B      Datepaid         xyz

Set the two together.  However if its only one additional variable, then just set the two datasets together.  This should all be in your transfer specifications document which documents what is to be transfered, the structure, schedule of transfer etc.  I.e you should know what is to be received and when, so just setup from there.