03-18-2016 08:22 AM
Does anyone know if SAS/ACCESS to Hadoop in 9.4 M3 supports creating partition files in Hive in a non-text format i.e. ORCFile?
I've tried to specify creating a table as an ORCFile usign the LIBNAME option in SAS 9.4 M2:
DBCREATE_TABLE_OPTS="stored as ORCFile"
However, when I add the data set option to create a partition file:
DBCREATE_TABLE_OPTS="PARTITIONED BY (x_facility_offer_cd VARCHAR(4))"
The resulting HiveQL is showing the table is being created as a TEXTFILE:
PARTITIONED BY (x_calling_system_cd VARCHAR(4)) ROW FORMAT DELIMITED FIELDS TERMINATED BY '\001' LINES TERMINATED BY '\012' STORED AS TEXTFILE
Also, it seems that indexes on Hive tables can't be created using the SAS/ACCESS to Hadoop in SAS 9.4 M2, is this fixed in M3?
The indexing is an issue, as we can't partition ORCFile(which are pre-otimised with an internal index), so we're left with un-optimised TEXTFILE formats.
Any help / info would be appreciated.
03-18-2016 08:39 AM
"In the third maintenance release for SAS 9.4, these features are new or enhanced.