LSF scheduled flow will not be able to connect do HADOOP but works when trigger manually

WorkingMan — Fri, 14 Jun 2019 09:16:35 GMT

Topic: LSF scheduled flow will not be able to connect do HADOOP but works when trigger manually

Hi everyone. I have a monthly job that will first connect to HADOOP database and perform a series of SAS scripts. However, when trying to connect to HADOOP via LSF flow which scheduled to run once a month at midnight, it will fail. When i rerun from LSF Flow form the failed flow process, it will work, during daytime office hour. I have been suspecting that this could be due to network or infra issue during midnight for the past few months until today when i schedule a dummy flow that connects to the same HADOOP that runs 24 hours in 30mins interval. It will FAIL to connect to HADOOP.

The error from log:

ERROR: Error trying to establish connection: Could not open client transport with JDBC Uri: jdbc:hive2://<removed the actual URL>/netcap_planning;principal=hive/_HOST@<removed>.MY: GSS initiate failed
ERROR: Error in the LIBNAME statement.
ERROR: Error trying to establish connection: Could not open client transport with JDBC Uri: jdbc:hive2://<removed the actual URL>/asset_dev_dn;principal=hive/_HOST@<removed>.MY: GSS initiate failed
ERROR: Error in the LIBNAME statement

If i trigger the flow manually, it will be able to connect to HADOOP. Why is this so strange? Doesn't make any sense to me at all. Why will a flow fail to connect to HADOOP but works when i trigger manually?

Here is the log when i trigger in LSF manually:

NOTE: Libref HPSNCP was successfully assigned as follows: 
      Engine:        HADOOP 
      Physical Name: jdbc:hive2://<removed>/netcap_planning
NOTE: Libref HPSASTD was successfully assigned as follows: 
      Engine:        HADOOP 
      Physical Name: jdbc:hive2://<removed>/asset_dev_dn

This is just not normal.

Re: LSF scheduled flow will not be able to connect do HADOOP but works when trigger manually

Criptic — Fri, 14 Jun 2019 09:36:10 GMT

Are the two flows running with the same user? Maybe your user as more rights then the batch user?

Re: LSF scheduled flow will not be able to connect do HADOOP but works when trigger manually

VDD — Fri, 14 Jun 2019 14:31:43 GMT

good point @Criptic some user accounts can be setup to allow the user some admin permissions to some process's while batching process's might not be allowed those permissions.

topic Re: LSF scheduled flow will not be able to connect do HADOOP but works when trigger manually in Administration and Deployment

LSF scheduled flow will not be able to connect do HADOOP but works when trigger manually

Re: LSF scheduled flow will not be able to connect do HADOOP but works when trigger manually

Re: LSF scheduled flow will not be able to connect do HADOOP but works when trigger manually