topic Re: sas hadoop connectivity issue in SAS Enterprise Guide

sas hadoop connectivity issue

woo — Wed, 22 Nov 2017 15:57:14 GMT

Hello,

I am trying connecting to hadoop using Kerberos with below proc sql code, and getting error.

any parameter is being missing or any configuration needs to be done from sas end?

Please advise,,,

2          proc sql;
3          connect to hadoop (HIVE_PRINCIPAL="hive/_HOST@xyz.company.com"
3        ! SERVER="abc.company.com");
ERROR: java.sql.SQLException: Could not open client transport with JDBC Uri:
jdbc:hive2://abc.company.com:10000/default;principal=hive/_HOST@xyz.company.com: GSS
       initiate failed

ERROR: Error trying to establish connection.
NOTE: PROC SQL set option NOEXEC and will continue to check the syntax of statements.
4 quit;
NOTE: The SAS System stopped processing this step because of errors.
NOTE: PROCEDURE SQL used (Total process time):

Re: sas hadoop connectivity issue

LinusH — Wed, 22 Nov 2017 18:45:41 GMT

If you feel that you followed every inch of the SAS-Hadoop configuration, and use a supported distribution, contact SAS Tech support.

Re: sas hadoop connectivity issue

woo — Tue, 05 Dec 2017 18:18:31 GMT

any other libname or proc sql statement that I can test it (with may be different parameter)?

Re: sas hadoop connectivity issue

JuanS_OCS — Wed, 06 Dec 2017 08:18:15 GMT

Hello @woo,

To be honest, I think this problems is out of the boundaries of the SAS system, but on your OS + 3rd party configuration.

So, i would recommend, as already suggested, to share this with SAS Technical Support, yes, but please keep in mind they might be supporting you on best effort basis.

Instead of that, I would reach a good Hive/Hadoop expert and external tool, and try the connection out from SAS, ensure that it works, then come back to SAS and fix it.

Also, I think here you would get great results just google-ing. I did a try and some interesting answers are in the net: https://www.google.nl/search?q=GSS+initiate+failed&oq=GSS+initiate+failed&aqs=chrome..69i57.863j0j7&sourceid=chrome&ie=UTF-8

Re: sas hadoop connectivity issue

woo — Wed, 10 Jan 2018 05:15:45 GMT

Do we need to install Beeline utility to test hiveserver2 connection? OR it needs to be enabled from hadoop/hive side and we can test it out?

Re: sas hadoop connectivity issue

nhvdwalt — Wed, 10 Jan 2018 05:59:10 GMT

Have a look at this thread:

https://communities.sas.com/t5/Administration-and-Deployment/Slow-logins-to-Hadoop/m-p/426047#M11809

@alexal pointed me to this independent tool. There are some additional params that you need to set for Kerberos, but I did make a note in the thread.

The message that you are getting means something went wrong with the TGT. First place to check is to see if a TGT is being generated for you. Assuming Linux, the default location is /tmp. If not, then manually generate a TGT with the kinit command. All of this needs to be working ok before you start using SAS.

Re: sas hadoop connectivity issue

woo — Wed, 10 Jan 2018 17:06:08 GMT

Thanks nhvdwalt

Appreciate your input,

I am generating Kerberos ticket fine and it is generating right under my home directory from where I run sas test job to connect to hive.

I was able to export both environment variable but failed to run that "SASJAVA HiveCheck" command. It says SASHIVE command not found.

Additionally, I get below error

libname hdp hadoop uri="jdbc:hive2://server-name.com:10000/;ssl=true"
20 ! server="server-name.com" port=10000 SUBPROTOCOL=hive2;

ERROR: java.sql.SQLException: Could not open client transport with JDBC Uri:
jdbc:hive2://server-name.com:10000/;ssl=true: Peer indicated failure: PLAIN auth failed: LDAP
Authentication failed for user
ERROR: Error trying to establish connection.

Re: sas hadoop connectivity issue

nhvdwalt — Thu, 11 Jan 2018 05:02:16 GMT

I think let's just get the HiveCheck working before trying in SAS. Hive connectivity must work outside of SAS before you try the libname.

Can you send me the full command you run on the command line for the HiveCheck program and the values of the environment variables you exported ?

Re: sas hadoop connectivity issue

woo — Tue, 16 Jan 2018 22:38:01 GMT

when I run "hivecheck.sh" on sas host I use only two below environment variable.

export SASJAVA=/sas-install-dir/SASPrivateJavaRuntimeEnvironment/9.4/jre/bin/java

export CLASSPATH="/test/jars/*:/test/config/:$CLASSPATH"

however, I get error saying,

$./hivecheck.sh

Error: Could not find or load main class HiveCheck

Thank You.

Re: sas hadoop connectivity issue

nhvdwalt — Wed, 17 Jan 2018 03:42:16 GMT

You are missing your current directory where HiveCheck is located in your CLASSPATH. Change it to the below:

export CLASSPATH=.:/some_directory/hadoop/lib/*:some_directory/hadoop/conf/*:

Notice the " .: " at the start of CLASSPATH

Re: sas hadoop connectivity issue

woo — Fri, 19 Jan 2018 19:15:01 GMT

Not sure but may be something else,

$ echo $CLASSPATH
/Hadoopjarsdir/2.6.0-cdh5.8.3/jars/*:/hadoopconfigdir:/dirwherehivecheck.sh:.

$ echo $SASJAVA
/sas-install-dir/SASPrivateJavaRuntimeEnvironment/9.4/jre/bin/java

$ ./HiveCheck.sh
Error: Could not find or load main class HiveCheck

Re: sas hadoop connectivity issue

nhvdwalt — Mon, 22 Jan 2018 05:10:43 GMT

Set your variables inside the HiveCheck.sh script. Variables set on the command line will not be visible to the script unless you export them. Best way always is to set all required variables needed by the script, inside the script so that it becomes self-contained.

Put this in your script....

export SASJAVA=<<SASHome_dir>>/SASPrivateJavaRuntimeEnvironment/9.4/jre/bin/java
export CLASSPATH=.:/Hadoopjarsdir/2.6.0-cdh5.8.3/jars/*:/hadoopconfigdir/*:

If you use Kerberos, there are some additional settings.....