Hi,
Can a merge transformation be applied to a hive table using a sas dataset?
Appreciate any help.
Regards,
Sheeba
Hi Linush,
Thanks for the reply.
I believe it wouldnt impact the data right.
I am looking for inserting new rows and updating existing rows which is same as what merge transformation does. i Have used it to update sas datasets but i have not tried this to update hive tables.. Is there any other way to do an insert and update to a hive table using sas DI studio?
Regards,
Sheeba
data step merge does not update/insert data at rest, it's a type of query that results in a new data set.
So from a data management/ETL perspective I would say this is a no-no.
There are ready made transformations for dealing with target table updates, select the one appropriate to your logic.
Sounds like that you need help from an ETL designer...?
Hi Linush,
tnx a lot for the details.
I will look into the transformations which does the target table updates.
Regards,
Sheeba
Bare in mind that Hive have limited set of methods for update (none until recently). So be sure to align your ETL with Hive capabilities.
Hi Linush,
Thanks. Sure I will check if it is supported by SAS DI.
Regards,
Sheeba
Given that Hadoop is only good at Insert (Append) and Read you might also consider a strategy as outlined here:
https://hortonworks.com/blog/four-step-strategy-incremental-updates-hive/
Hi Patrick,
Thanks a lot for the link. It is really helpful.
I will try to deep dive into it and follow it.
Regards,
Sheeba
Don't miss out on SAS Innovate - Register now for the FREE Livestream!
Can't make it to Vegas? No problem! Watch our general sessions LIVE or on-demand starting April 17th. Hear from SAS execs, best-selling author Adam Grant, Hot Ones host Sean Evans, top tech journalist Kara Swisher, AI expert Cassie Kozyrkov, and the mind-blowing dance crew iLuminate! Plus, get access to over 20 breakout sessions.
Need to connect to databases in SAS Viya? SAS’ David Ghan shows you two methods – via SAS/ACCESS LIBNAME and SAS Data Connector SASLIBS – in this video.
Find more tutorials on the SAS Users YouTube channel.