sas hadoop connectivity issue

[woo]

Hello,

I am trying connecting to hadoop using Kerberos with below proc sql code, and getting error.

any parameter is being missing or any configuration needs to be done from sas end?

Please advise,,,

2          proc sql;
3          connect to hadoop (HIVE_PRINCIPAL="hive/_HOST@xyz.company.com"
3        ! SERVER="abc.company.com");
ERROR: java.sql.SQLException: Could not open client transport with JDBC Uri:
jdbc:hive2://abc.company.com:10000/default;principal=hive/_HOST@xyz.company.com: GSS
       initiate failed

ERROR: Error trying to establish connection.
NOTE: PROC SQL set option NOEXEC and will continue to check the syntax of statements.
4          quit;
NOTE: The SAS System stopped processing this step because of errors.
NOTE: PROCEDURE SQL used (Total process time):

[LinusH]

If you feel that you followed every inch of the SAS-Hadoop configuration, and use a supported distribution, contact SAS Tech support.

[woo]

any other libname or proc sql statement that I can test it (with may be different parameter)?

[JuanS_OCS]

Hello @woo,

To be honest, I think this problems is out of the boundaries of the SAS system, but on your OS + 3rd party configuration.

So, i would recommend, as already suggested, to share this with SAS Technical Support, yes, but please keep in mind they might be supporting you on best effort basis.

Instead of that, I would reach a good Hive/Hadoop expert and external tool, and try the connection out from SAS, ensure that it works, then come back to SAS and fix it.

Also, I think here you would get great results just google-ing. I did a try and some interesting answers are in the net: https://www.google.nl/search?q=GSS+initiate+failed&oq=GSS+initiate+failed&aqs=chrome..69i57.863j0j7&...

[woo]

Do we need to install Beeline utility to test hiveserver2 connection? OR it needs to be enabled from hadoop/hive side and we can test it out?

[nhvdwalt]

Have a look at this thread:

https://communities.sas.com/t5/Administration-and-Deployment/Slow-logins-to-Hadoop/m-p/426047#M11809

@alexal pointed me to this independent tool. There are some additional params that you need to set for Kerberos, but I did make a note in the thread.

The message that you are getting means something went wrong with the TGT. First place to check is to see if a TGT is being generated for you. Assuming Linux, the default location is /tmp. If not, then manually generate a TGT with the kinit command. All of this needs to be working ok before you start using SAS.

[woo]

Thanks nhvdwalt 

Appreciate your input,

I am generating Kerberos ticket fine and it is generating right under my home directory from where I run sas test job to connect to hive.

I was able to export both environment variable but failed to run that "SASJAVA HiveCheck" command. It says SASHIVE command not found.

Additionally, I get below error

libname hdp hadoop uri="jdbc:hive2://server-name.com:10000/;ssl=true"
20       ! server="server-name.com" port=10000 SUBPROTOCOL=hive2;

ERROR: java.sql.SQLException: Could not open client transport with JDBC Uri:
       jdbc:hive2://server-name.com:10000/;ssl=true: Peer indicated failure: PLAIN auth failed: LDAP
       Authentication failed for user
ERROR: Error trying to establish connection.

[nhvdwalt]

I think let's just get the HiveCheck working before trying in SAS. Hive connectivity must work outside of SAS before you try the libname.

Can you send me the full command you run on the command line for the HiveCheck program and the values of the environment variables you exported ?

[woo]

when I run "hivecheck.sh" on sas host I use only two below environment variable.

export SASJAVA=/sas-install-dir/SASPrivateJavaRuntimeEnvironment/9.4/jre/bin/java

export CLASSPATH="/test/jars/*:/test/config/:$CLASSPATH"

however, I get error saying,

$./hivecheck.sh

Error: Could not find or load main class HiveCheck

Thank You.

[nhvdwalt]

You are missing your current directory where HiveCheck is located in your CLASSPATH. Change it to the below:

export CLASSPATH=.:/some_directory/hadoop/lib/*:some_directory/hadoop/conf/*:

Notice the "   .:   " at the start of CLASSPATH

[woo]

Not sure but may be something else,

$ echo $CLASSPATH
/Hadoopjarsdir/2.6.0-cdh5.8.3/jars/*:/hadoopconfigdir:/dirwherehivecheck.sh:.

$ echo $SASJAVA
/sas-install-dir/SASPrivateJavaRuntimeEnvironment/9.4/jre/bin/java

$ ./HiveCheck.sh
Error: Could not find or load main class HiveCheck

[nhvdwalt]

Set your variables inside the HiveCheck.sh script. Variables set on the command line will not be visible to the script unless you export them. Best way always is to set all required variables needed by the script, inside the script so that it becomes self-contained.

Put this in your script....

export SASJAVA=<<SASHome_dir>>/SASPrivateJavaRuntimeEnvironment/9.4/jre/bin/java
export CLASSPATH=.:/Hadoopjarsdir/2.6.0-cdh5.8.3/jars/*:/hadoopconfigdir/*:

If you use Kerberos, there are some additional settings.....