Hi SAS Friends,
Platform_long=Linux for X64
We are connecting SAS to Hadoop with HDFS Engine from cloudera Manager and ServiceDiscoveryMode is zooKeeper and we used kerberos secuity authentication(without username and password)
We have not found any libname for connecting HDFS, but by using Proc Hadoop we are connected to WebHDFS.
Issue:By using Proc Hadoop we need to read the data from HDFS Server, but to read HDFS commands are not there.
By using copy to local command we are downloading the data from HDFS Server, from this Process we are facing Server Storage utilization issue.
Summary is that we want to read data directly from HDFS Server, instead of downloading data from HDFS to SAS Server.
Suggest us if it possible with Libname for connecting to HDFS Server from cloudera Manager and ServiceDiscoveryMode is zooKeeper and kerberos secuity authentication
Please suggest and support us to resolve.
check out the following on-line doc SAS Help Center: SAS 9.4 Hadoop Configuration Guide for Base SAS and SAS/ACCESS, Fifth Edition and see if you are missing anything!?
Once you ensured your configuration is set correctly, you can checkout this on-line doc SAS Help Center: SAS/ACCESS Interface to Hadoop for information about setting up Hadoop Libname statement.
Hope this helps
Hi @AhmedAl_Attar ,
Thank you for response.
From the following links we already done the pre-requisite and by using Proc Hadoop for HDFS we are able to connected to HDFS Server. PF attached doc.
And you shared the link for Libnames but in our process we don't have username and password by using kerberos authetication we are working and to mention the Hadoop server and port we are processing through Zookeper ServiceDiscoveryMode. And we observe that in libname statement link it is showing for Hive Server connection but here we are connecting to HDFS Server.
Please support to resolve this.
Hi @AhmedAl_Attar ,
From the below libname we have tried to connect to HDFS.
libname HD Hadoop server='masternode' port=9870(webHDFS port);HDFS_PRINCIPAL="hdfs/_HOST@CDPDEVMASTER01";
From the attached zip file you can see the screenshot, we use that libname for connecting to HDFS with kerberos security authentication..
And find the HDFS error log file shared in zip file.
We added in server as name node with Web HDFS Port
Please suggest us to resolve this.
The SAS Users Group for Administrators (SUGA) is open to all SAS administrators and architects who install, update, manage or maintain a SAS deployment.
Learn how to install the SAS Viya CLI and a few commands you may find useful in this video by SAS’ Darrell Barton.
Find more tutorials on the SAS Users YouTube channel.