Topic: LSF scheduled flow will not be able to connect do HADOOP but works when trigger manually
Hi everyone. I have a monthly job that will first connect to HADOOP database and perform a series of SAS scripts. However, when trying to connect to HADOOP via LSF flow which scheduled to run once a month at midnight, it will fail. When i rerun from LSF Flow form the failed flow process, it will work, during daytime office hour. I have been suspecting that this could be due to network or infra issue during midnight for the past few months until today when i schedule a dummy flow that connects to the same HADOOP that runs 24 hours in 30mins interval. It will FAIL to connect to HADOOP.
The error from log:
<code></code>
ERROR: Error trying to establish connection: Could not open client transport with JDBC Uri: jdbc:hive2://<removed the actual URL>/netcap_planning;principal=hive/_HOST@<removed>.MY: GSS initiate failed
ERROR: Error in the LIBNAME statement.
ERROR: Error trying to establish connection: Could not open client transport with JDBC Uri: jdbc:hive2://<removed the actual URL>/asset_dev_dn;principal=hive/_HOST@<removed>.MY: GSS initiate failed
ERROR: Error in the LIBNAME statement
<br /><code></code><code></code>
If i trigger the flow manually, it will be able to connect to HADOOP. Why is this so strange? Doesn't make any sense to me at all. Why will a flow fail to connect to HADOOP but works when i trigger manually?
Here is the log when i trigger in LSF manually:
<code></code>
NOTE: Libref HPSNCP was successfully assigned as follows:
Engine: HADOOP
Physical Name: jdbc:hive2://<removed>/netcap_planning
NOTE: Libref HPSASTD was successfully assigned as follows:
Engine: HADOOP
Physical Name: jdbc:hive2://<removed>/asset_dev_dn
<br /><code></code><code></code>
This is just not normal.