topic Re: Compute column statistics for HIVE in SAS Programming

Compute column statistics for HIVE

dkassis — Fri, 16 Aug 2019 21:25:45 GMT

Hi,

I am trying to help query performance in HIVE by computing the column statistics within SAS. I am having trouble getting the syntax correct. I was reading this paper http://support.sas.com/resources/papers/proceedings17/SAS0190-2017.pdf and trying the method in tip 7 but I can't get the following to run Analyze table xxxxxxxxxxx compute statistics;

Any advice?

Thanks,

Dan

Re: Compute column statistics for HIVE

r_behata — Fri, 16 Aug 2019 22:27:41 GMT

The method you are referring in the paper runs the query optimization statistics from within Hive, which queries the hive's native metastore . You may not be able to able to run the same command within sas.

Try running the command from Hive prompt or HUE (cloudera).

Re: Compute column statistics for HIVE

LinusH — Mon, 19 Aug 2019 09:30:07 GMT

But if you for some reason wnat to call thsi fram SAS, what have you tried so far?

Not sure, but I guess that you need to put this inside an EXECUTE ( ) BY HADOOP; block.

Re: Compute column statistics for HIVE

yevgeniyelbert — Wed, 14 Oct 2020 19:53:24 GMT

I am looking whether this is provided as part of implicit functionality as well, but you can surely do this with explicit pass-thru adding hive.stats.column.autogather = true