New columns and changed data types in source table. How to do same for Hadoop and LASR?

[rgreen33]

My SQL DBA added a few columns and changed a few data types in some of my "source" tables for SAS.  Updating the registered tables in Management Console (library).  I ended up recreating the tables in DI Studio for HDFS and LARS.  Is there a better/easier way to accomplish this?

Thanks,

Ricky

[LinusH]

When you say recreating in LASR, do you mean physically or in meta data?
You should avoid recreating meta data objects.
Avoiding recreation of physical SAS tables when the structure changes is hard, even impossible.

[rgreen33]

Recreating the tables for HDFS and LASR in DI Studio, so this would be in metadata.  I update the metadata for the registered table via Management Console.  But, then I have the metadata for HDFS and LASR that needs to be updated.  If only new columns are added, it appears that the easiest way to import the columns vid DI Studio.  However, if the data type changes, the change is not picked up on the import.  So, what is the best/most efficient way for handeling this?  That's what I am trying to figure out.  I am new to SAS, so I have a lot to learn.

Thanks,

Ricky

[LinusH]

I don't have DIS at my fingertips right now so I can test myself.
But perhaps you need to do this manual. Which is not only bad since you sure need to change the format throughout your flow(s) and potentially need to specify data type conversion logic.

[rgreen33]

@LinusH,

Thank you for the reply.  I agree that in some cases, manually would be the better scenario, as other things would also need to change (as you mentioned).  However, in my case, most of the "transformations" of the data are occuring in Oracle or SQL, before the data reaches SAS.  Thus, when it gets to SAS, it is a straight load to HDFS and then to LASR.  So, the code/logic remains consistent.  So, the "manual" way in this scenario is a bit of a pain.  Maybe "manual" is the only way to do this.  Like I said, I am new to SAS and I am just trying to make my processes more efficient...and, making sure that I am doing things correctly.

Thanks,

Ricky