redirecting HDFS TEMP FILES

[vorkady]

Hi

We are encountering issue in redirecting  HDFS temp Directory using option   hdfs_tempdir in the libname syntax. My id has full privilege on the temp directory path  but getting access error , which is misleading. Please advise if any community members encountered similar issues and workaround used to resolve the issue. I have included below code and SAS logs for your ready reference. Appreciate if you could help.

libname x3 hadoop AUTHDOMAIN=&auth_domain. server="xxxxt"
uri="jdbc:hive2://xxxx3/;ssl=true;transportMode=http;httpPath=gateway/default/hive2" schema="sbx_cr_mdl__user_xx"
DBMAX_TEXT=255 READ_METHOD=HDFS hdfs_tempdir="/hdp-staging/home/xxx";

Logs:

! hdfs_tempdir="/hdp-staging/home/svorkad";
ERROR: HDFS_TEMPDIR directory /hdp-staging/home/xxx does
5 The SAS System
14:31 Wednesday, September 5, 2018

not exist or is inaccessible.
ERROR: Error in the LIBNAME statement.
35

[JBailey]

Hi @vorkady,

If Kerberos is being used I would bet that your Kerberos ticket will not allow writing to that location.

Try writing a file to the HDFS_TEMPDIR= location using PROC HADOOP. 

PROC HADOOP displays better (more descriptive) error messages because it goes through fewer Hadoop layers when writing the file.

There may be an example in my SGF workshop materials.

https://github.com/Jeff-Bailey/SGF2016_SAS3880_Insiders_Guide_Hadoop_HOW

If you have something like HUE available it is worth a try too.

 Best wishes,
Jeff

[vorkady]

Thanks Jeff,
We also encountering issues displaying migrated Netezza table data in EG , which is visible in non sas applications such as Aginity hadoop work bench but not in EG . Further it is observed that data in table is visible in EG if column attributes set to string. I did open track with SAS support . Appreciate if you could help.

[vorkady]

Hi,

I am able to resolve the issue by uploading the required  hive site xml file . We are now able to redirect temp direct in libname and also able to view table data in EG.

Note from Jeff and other contributors immensely helped us to resolve our Hadoop SAS config issues.