topic Re: Using SAS to read Parquet File in Hadoop without hive table in SAS Programming

Using SAS to read Parquet File in Hadoop without hive table

cstegeman — Thu, 16 Aug 2018 19:49:28 GMT

There is a parquet file in our hadoop cluster without a hive table built on top of it.

- Is there something I can do to read it into SAS without requiring someone to build a hive table on top of it?

- Is there some way to access the table with Spark through SAS (kind of like pyspark)?

Re: Using SAS to read Parquet File in Hadoop without hive table

ChrisNZ — Fri, 17 Aug 2018 02:47:00 GMT

1. There is no native way to read the parquet format in SAS as far as I know.

2. Access to Spark is supported with SAS Viya. See https://support.sas.com/en/documentation/third-party-software-reference/viya/34/support-for-databases.html

Re: Using SAS to read Parquet File in Hadoop without hive table

LinusH — Fri, 17 Aug 2018 06:40:19 GMT

Why not building Hive metadata on top of it. Seems easiest if you wish to use it using relational language like SAS.
With Hive, you can access the file using Impala (which uses Spark).

Re: Using SAS to read Parquet File in Hadoop without hive table

cstegeman — Mon, 05 Nov 2018 21:26:16 GMT

I was hoping for an alternative to hive - I didn't have permissions to create a hive table in that environment. Also Hive is slower than PySpark.

Re: Using SAS to read Parquet File in Hadoop without hive table

ChrisNZ — Mon, 05 Nov 2018 21:28:31 GMT

Impala can be much faster than Hive and supports parquet.