SAS Object spawner throws an error when the user home directory created using PAM modules(automatic)

[anguraj1]

Hi All,

Used the below link posted by Paul for automatic creation of home directory for New users. 

https://platformadmin.com/blogs/paul/2017/04/sas-user-linux-home-dir-auto-creation/#comment-14502

After creating the pam module for auto creation /home/directory - when i login to the SAS EG i am getting the below error

Objspawn was unable to launch the server SASApp – Workspace Server because the Grid provider has exceeded the specified wait time.

Please find below the object spawner error message:
2020-06-17T13:28:29,525 ERROR [00008418] :azsj – The specified uuid DA816073-34D2-5F4E-B85A-EFDD03AC30F6 did not match any process managed by this spawner.
2020-06-17T13:28:29,526 ERROR [00008418] :azsj – Objspawn was unable to launch the server SASApp – Workspace Server (A5NYPY3W.AY00000C) because the Grid provider has exceeded the specified wait time.

Note: Other users who logged in the server directly not thru PAM Automatic home directory is not facing any issue.
Thanks,
Anguraj S

[alexal]

@anguraj1 ,

Are you sure oddjob-mkhomedir module did create the home directory for the connecting user? What are you using for managing the SAS jobs on the grid? LSF or SAS Workload Orchestrator?

[anguraj1]

Yes, its creating the home directory for the user, we are using LSF. SAS9.4 M5

[alexal]

@anguraj1 ,

Did you try to check the status of your LSF jobs using bhist command? What do you see there when the user is trying to connect?

[anguraj1]

Hi,
When its trying to connect SASAPP its throws the above timeout error. I dont know will that produce any error

[AnandVyas]

Is the job submitted to GRID and then timeout or it is not? What tool are you using to connect server to AD, Centrify, Quest?

[anguraj1]

we are connecting thru SAS EG, its AD protocol

[jcFranklin]

Hello,

In the object spawner log, find the "Created grid job xxxx" line that is related to thread [00008418]:

2020-06-17T13:28:29,525 ERROR [00008418] :azsj – The specified uuid DA816073-34D2-5F4E-B85A-EFDD03AC30F6 did not match any process managed by this spawner.
2020-06-17T13:28:29,526 ERROR [00008418] :azsj – Objspawn was unable to launch the server SASApp – Workspace Server (A5NYPY3W.AY00000C) because the Grid provider has exceeded the specified wait time.

Once you have the grid job #, you can run a bhist to see what happened once it went into grid:

bhist -u all -n 0 -l <grid job #>

Also, compare the timestamps from the object spawner log and from the bhist. Did the job immediately get created and go into grid, then the (probably 60 second) timeout occur once it was in grid? Are there errors in the bhist output at all? 

[gwootton · Accepted solution]

I suspect the problem has to do with PAM authentication via sasauth occurring on the Object Spawner host rather than the LSF execution host. 

You could try configuring an LSF PAM file to call the home directory creation script.

https://www.ibm.com/support/pages/configure-pam-resource-limits