Does anyone know if SAS/ACCESS to Hadoop in 9.4 M3 supports creating partition files in Hive in a non-text format i.e. ORCFile?
I've tried to specify creating a table as an ORCFile usign the LIBNAME option in SAS 9.4 M2:
DBCREATE_TABLE_OPTS="stored as ORCFile"
However, when I add the data set option to create a partition file:
DBCREATE_TABLE_OPTS="PARTITIONED BY (x_facility_offer_cd VARCHAR(4))"
The resulting HiveQL is showing the table is being created as a TEXTFILE:
PARTITIONED BY (x_calling_system_cd VARCHAR(4)) ROW FORMAT DELIMITED FIELDS TERMINATED BY '\001' LINES TERMINATED BY '\012' STORED AS TEXTFILE
Also, it seems that indexes on Hive tables can't be created using the SAS/ACCESS to Hadoop in SAS 9.4 M2, is this fixed in M3?
The indexing is an issue, as we can't partition ORCFile(which are pre-otimised with an internal index), so we're left with un-optimised TEXTFILE formats.
Any help / info would be appreciated.
Thanks
David
"In the third maintenance release for SAS 9.4, these features are new or enhanced.
It's finally time to hack! Remember to visit the SAS Hacker's Hub regularly for news and updates.
Need to connect to databases in SAS Viya? SAS’ David Ghan shows you two methods – via SAS/ACCESS LIBNAME and SAS Data Connector SASLIBS – in this video.
Find more tutorials on the SAS Users YouTube channel.