topic SAS/ACCESS for Hadoop Error in SAS Programming

SAS/ACCESS for Hadoop Error

rameshs — Thu, 17 Sep 2015 15:51:41 GMT

Hi,

While executing the below code, iam getting error like "A common cause of this issue is conflicting HDFS permissions between the data file and the Hive warehouse directory for the table."

Also "ERROR: org.apache.hive.service.cli.HiveSQLException: Error while compiling statement: FAILED: SemanticException No valid
privileges".

Whether anyone encountered this issue and resolved?? please advice us.

====================

46 libname hdplib hadoop server=xxxxxxxx port=10000 schema=default subprotocol=hive2
47 hdfs_tempdir="/user/rseelam";
NOTE: Libref HDPLIB was successfully assigned as follows:
Engine: HADOOP
Physical Name: jdbc:hive2://xxxxxxxx:10000/default
48 data hdplib.testop;
49 x=1; output;
50 x=2; output;
51 x=3; output;
52 x=4; output;
53 x=5; output;
54 run;

NOTE: The data set HDPLIB.TESTOP has 5 observations and 1 variables.
ERROR: org.apache.hive.service.cli.HiveSQLException: Error while compiling statement: FAILED: SemanticException No valid
privileges
Required privileges for this query:
Server=server1->URI=hdfs://nameservice1/user/rseelam/sasdata-2015-09-16-20-21-26-684-e-00002.dlv->action=*;
ERROR: Unable to execute Hadoop query.
ERROR: Execute error on statement: LOAD DATA INPATH '/user/rseelam/sasdata-2015-09-16-20-21-26-684-e-00002.dlv' OVERWRITE
INTO TABLE `TESTOP`. Could not load /user/rseelam/sasdata-2015-09-16-20-21-26-684-e-00002.dlv into table TESTOP in
schema DEFAULT. A common cause of this issue is conflicting HDFS permissions between the data file and the Hive
warehouse directory for the table.
NOTE: DATA statement used (Total process time):
real time 4.91 seconds
cpu time 0.10 seconds

Re: SAS/ACCESS for Hadoop Error

rameshs — Mon, 28 Sep 2015 14:49:03 GMT

Issue got resolved after setting the permissions at sentry level.

Re: SAS/ACCESS for Hadoop Error

Ram-N — Tue, 08 Mar 2016 21:40:15 GMT

Hello,

I am having similar issue on our hadoop cluster with kerberos and sentry enabled. What is the modification you did on sentr to get this working? I tested the same statement hadoop cluster with no KDC/Sentry and it works but it fails with below error on sentry enabled clusetr.

ERROR: Execute error on statement: LOAD DATA INPATH '/tmp/sasdata-2016-03-07-14-34-12-770-e-00001.dlv' OVERWRITE INTO TABLE

`FULLTEST`. Could not load /tmp/sasdata-2016-03-07-14-34-12-770-e-00001.dlv into table FULLTEST2 in schema

EBI. A common cause of this issue is conflicting HDFS permissions between the data file and the Hive warehouse directory for

the table. Another possible cause is the "sticky" bit set on HDFS directory /tmp.

Re: SAS/ACCESS for Hadoop Error

Ram-N — Thu, 21 Apr 2016 01:10:44 GMT

Hello All,

This problem was resolved by working with hadoop admin team to create a HDFS-TEMPDIR=/sas-tmp and giving the ownership to account which is doing the hadoop operations from SAS. Please do note that HDFS-TEMPDIR= needs be on libname statement and it can not be options set.

Thanks,

Ram