BookmarkSubscribeRSS Feed
WorkingMan
Calcite | Level 5

Topic: LSF scheduled flow will not be able to connect do HADOOP but works when trigger manually

 

Hi everyone. I have a monthly job that will first connect to HADOOP database and perform a series of SAS scripts. However, when trying to connect to HADOOP via LSF flow which scheduled to run once a month at midnight, it will fail. When i rerun from LSF Flow form the failed flow process, it will work, during daytime office hour. I have been suspecting that this could be due to network or infra issue during midnight for the past few months until today when i schedule a dummy flow that connects to the same HADOOP that runs 24 hours in 30mins interval. It will FAIL to connect to HADOOP.

 

The error from log:

 

 

 

 

<code></code>

 

ERROR: Error trying to establish connection: Could not open client transport with JDBC Uri: jdbc:hive2://<removed the actual URL>/netcap_planning;principal=hive/_HOST@<removed>.MY: GSS initiate failed
ERROR: Error in the LIBNAME statement.
ERROR: Error trying to establish connection: Could not open client transport with JDBC Uri: jdbc:hive2://<removed the actual URL>/asset_dev_dn;principal=hive/_HOST@<removed>.MY: GSS initiate failed
ERROR: Error in the LIBNAME statement

 

<br /><code></code><code></code>

 

 

 

If i trigger the flow manually, it will be able to connect to HADOOP. Why is this so strange? Doesn't make any sense to me at all. Why will a flow fail to connect to HADOOP but works when i trigger manually?

 

Here is the log when i trigger in LSF manually:

 

 

 

<code></code>

 

NOTE: Libref HPSNCP was successfully assigned as follows: 
      Engine:        HADOOP 
      Physical Name: jdbc:hive2://<removed>/netcap_planning
NOTE: Libref HPSASTD was successfully assigned as follows: 
      Engine:        HADOOP 
      Physical Name: jdbc:hive2://<removed>/asset_dev_dn

 

<br /><code></code><code></code>

 

 

 

 

This is just not normal.

2 REPLIES 2
Criptic
Lapis Lazuli | Level 10
Are the two flows running with the same user? Maybe your user as more rights then the batch user?
VDD
Ammonite | Level 13 VDD
Ammonite | Level 13

good point @Criptic some user accounts can be setup to allow the user some admin permissions to some process's while batching process's might not be allowed those permissions.   

suga badge.PNGThe SAS Users Group for Administrators (SUGA) is open to all SAS administrators and architects who install, update, manage or maintain a SAS deployment. 

Join SUGA 

Get Started with SAS Information Catalog in SAS Viya

SAS technical trainer Erin Winters shows you how to explore assets, create new data discovery agents, schedule data discovery agents, and much more.

Find more tutorials on the SAS Users YouTube channel.

Discussion stats
  • 2 replies
  • 854 views
  • 0 likes
  • 3 in conversation