BookmarkSubscribeRSS Feed
🔒 This topic is solved and locked. Need further help from the community? Please sign in and ask a new question.
cstegeman
Fluorite | Level 6

There is a parquet file in our hadoop cluster without a hive table built on top of it. 

 - Is there something I can do to read it into SAS without requiring someone to build a hive table on top of it?

 - Is there some way to access the table with Spark through SAS (kind of like pyspark)?

 

1 ACCEPTED SOLUTION

Accepted Solutions
ChrisNZ
Tourmaline | Level 20

1. There is no native way to read the parquet format in SAS as far as I know.

 

2. Access to Spark is supported with SAS Viya. See https://support.sas.com/en/documentation/third-party-software-reference/viya/34/support-for-database...

 

View solution in original post

4 REPLIES 4
ChrisNZ
Tourmaline | Level 20

1. There is no native way to read the parquet format in SAS as far as I know.

 

2. Access to Spark is supported with SAS Viya. See https://support.sas.com/en/documentation/third-party-software-reference/viya/34/support-for-database...

 

LinusH
Tourmaline | Level 20
Why not building Hive metadata on top of it. Seems easiest if you wish to use it using relational language like SAS.
With Hive, you can access the file using Impala (which uses Spark).
Data never sleeps
cstegeman
Fluorite | Level 6
I was hoping for an alternative to hive - I didn't have permissions to create a hive table in that environment. Also Hive is slower than PySpark.
ChrisNZ
Tourmaline | Level 20

Impala can be much faster than Hive and supports parquet.

sas-innovate-2024.png

Join us for SAS Innovate April 16-19 at the Aria in Las Vegas. Bring the team and save big with our group pricing for a limited time only.

Pre-conference courses and tutorials are filling up fast and are always a sellout. Register today to reserve your seat.

 

Register now!

How to Concatenate Values

Learn how use the CAT functions in SAS to join values from multiple variables into a single value.

Find more tutorials on the SAS Users YouTube channel.

Click image to register for webinarClick image to register for webinar

Classroom Training Available!

Select SAS Training centers are offering in-person courses. View upcoming courses for:

View all other training opportunities.

Discussion stats
  • 4 replies
  • 4941 views
  • 0 likes
  • 3 in conversation