BookmarkSubscribeRSS Feed
DHS_SASADM
Fluorite | Level 6

I have a user who has been having his account locked out at the domain level and has happened numerous times since Friday. This user is high profile. This started on Friday and has been continuing for the last few days. I have my server and security teams working the issue with no results. The user gets his account unlocked and within the hour it is getting locked. The security monitor SPLUNK reports that my SAS server is the one locking him out. Over the last three days we have rebooted the server, deleted and re-added the account, and had him try to access production, backup and devel servers. Today we opened a SAS Track and are waiting for a response. Since this user supports higher management here, I used a "SERVICE" account that I had in reserve and logged him in as that from his PC through to the server. He has been stable all afternoon via that method. I, in turn is masquerading as the user to see if I while impersonating him get locked out.

 

I'm runnning Office Analytics single machine 9.4 M2. The user is running EG 6.1 and I am running EG 7.1.

 

I'm open to any and all suggestions.

 

Thanks,

Jerry Coppa SAS Admin at PA DHS

19 REPLIES 19
SASKiwi
PROC Star

I had a problem very similar to this and it was caused by old remote login sessions to our SAS server that were just disconnected but not signed out. In the meantime I had changed my password but the old sessions kept trying to authenticate and kept locking me out.

 

Does your user have remote login access? If not another possibility is having his old password stored in SAS metadata or in scheduled jobs. Use SAS Management Console to update the metadata-stored password or to re-schedule the jobs.

twocanbazza
Quartz | Level 8

Have seen this before also but can't recall the exact problem...  

 

I gather the user can't log onto the server directly

 

Given you have rebooted the server etc, a rouge session doesn't seem to be the issue...

 

do you have credentials stored in the metadata? or hardcoded in SAS code?  are there any SAS jobs running on the server when the account is locked?  Is it after they do something in SAS or is it even when they don't do anything?

 

Although it is the SAS server that is seen to be be doing this it may not be SAS that is the issue....

 

Barry

jakarman
Barite | Level 11

There are a lot of possible causes.

- The existance of some user ghostprocesses still running and connecting at intervals

- The coding of the old/wrong password in a connection profile. (eguide amo/ .net    DI SMC /java)

- the caching (eg conncet) in a SAS metadatadata autentication location

- the usage hard coded of user/password combination.
Finding and seeing these user errors can be hard.

Than you can ahve problems in the SAS system itself.

- The login can be delayed by failed logins. In those cases the metadata login can get delayed in a unusable way.

When the user does a login and wille retry with different password thinking it are typos it can cause a lock.
- After changing the password the sasauthentication can be delayed separtely  of your OS setttings.

Getting a new password after an unlock can cause the marvelous sitaution you can loging at the OS level but using SAS for that will fail. After several retries it can get locked. 

   

---->-- ja karman --<-----
mfab
Quartz | Level 8

Hello @jakarman,

 

I have stumbled upon a problem that might be related to what you were mentioning.

"the caching (eg conncet) in a SAS metadatadata autentication location"

and

"After changing the password the sasauthentication can be delayed separtely  of your OS setttings"

 

Situation is as follows:

Host is unix and the password is changed.

User tries to login from EG/WebReportStudio/Excel Add-In.

User receives promt to enter valid username and password

User enters new information.

SAS says that the information is still invalid.

I then deleted the user account, created it as new account - still no success.

After a while (15-30 minutes) the login suddenly works.

 

I have been trying to figure out if any setting in the omaconfig.xml (hints) could have had any effect, but no clue was found.

After 3 unsuccessful attempts, the account should have been "locked", but what does this mean? How long is it locked and why did it suddenly work out?

Also, where would the metadata-server or connect spawner (?) cache any information? Are there any hints to how frequently cashing will take place?

Also log files did not provide any useful information aside from the fact, that the user login was denied.

 

Thank you for any hints and suggestions.

 

Cheers,

Michael

Kurt_Bremser
Super User

This is something that I have noticed repeatedly. Once the metadata server has received a "failed" message from the OS for an authentication attempt, it takes some time before it re-checks the authentication.

eg

- user attempts connection, is rejected because password is expired in OS

- user logs in via ssh, changes password as required

- user tries with new password, but is rejected; metadata log reports "Access denied"

- user waits a given amount of time and tries again - voila, works

JuanS_OCS
Amethyst | Level 16

Hello @mfab,

 

this is a behaviour quite old ( I know it since back 9.1.3 ) and it remains the same, with some improvements, now it happens less often.

 

General solution is to refresh the SAS Application Server, or even the Object Spawner.

 

If you have a Grid environment, this might happen on the LSF level as well.

 

mfab
Quartz | Level 8

Thanks for the answers so far!

 

@Kurt_Bremser: do you see any chance, that SAS will document the default behaviour anywhere? It would be pleasant to at least know how the system works, even if there is no chance in changing things. Would you open a support ticket for this or does anyone from SAS have a look into the communities from time to time?

 

@JuanS_OCS: what do you mean by "refresh" - are you thinking of restart? This would not be applicable for our purposes.

We want to prevent user logins by using the "passwd --lock <username>" command in Cronjobs. So users can't login during given times. We would not want to restart any services because some users did try to login and have their login in SAS ... ahm ... let's say deactivated or not refreshed.

A solution from within SAS to deactivate accounts or prevent login at certain times would also be desirable, but there does not seem to be any option like this (and I wonder why, since the rest of the metadata management in SAS is quite nice).

 

Cheers,

Michael

alexal
SAS Employee

@mfab,

 

I support authentication on UNIX/Linux.

 

Are you guys talking about PAM or Host authentication?

 

@Kurt_Bremser

user tries with new password, but is rejected; metadata log reports "Access denied"

 

Please enable sasauth-debug. Repeat the problem and show me:

 

  • The metadata server log
  • sasauth-debug log
  • An output from this command (must be started as root): grep sasauth /var/log/secure
Kurt_Bremser
Super User

Hi @alexal. It's not really a problem, as it can be solved by just giving the metadata server time to "drop the cache", or whatever it is.

I just had a case right now, where a user changed the password after failing to log in to the metadata server (because of password expiration), and could not log in to the metadata server for about half an hour. Then the EG connection profile sudddenly worked (with the already entered new password) without asking for credentials again.

 

We are using host authentication on AIX, the phenomenon has been there since at least 9.2 on AIX 5.3, and it persists with 9.4 TS1M2 on AIX 7.1

 

I can find no sasauth-debug in my !SASROOT/utilities/bin

alexal
SAS Employee

@Kurt_Bremser,

 

SAS relies on the underlying operating system and APIs to handle authentication request, so most likely the cache was somewhere on the system. Without an additional debug I cannot help. If you wish, open a track and ask to assign it to me, I'm sure we will find the root cause.

Kurt_Bremser
Super User

Is KB

15231

really the one that you wanted to direct me to? It's the one that deals with the necessary setuid bits, but that is surely not a problem here.

I think you meant http://support.sas.com/kb/39/891.html

I have enabled the logging, will try if I can glean something from the logs if the effect happens again.

alexal
SAS Employee

@Kurt_Bremser,

 

You are right, sorry about that, I've copied the wrong link 🙂

mfab
Quartz | Level 8

@alexal and @Kurt_Bremser: I really appreciate all your effort! I hope, you will track this thing down.

 

I can add, that we use Host authentication as well (SLES11.4 / SAS9.4M1).

Unfortunately I cannot enable sasauth-debug as of now and as quickly as Kurt.

Our timeline was roughly as follows:

t0 - "passwd --lock ..." on commandline

t1 - first login attempt: not possible (as expected)

t5 - "passwd --unlock ..." on commandline

t6 - login attempt: not possible (not as expected)

t7 - login via SSH: possible (as expected)

t8 - setting user passwort in credentials via Management Console + login attempt: no effect

t11 - removing user login credentials in Management Console + login attempt: no effect

t12 - removed user completely in Management Console, added user again + login attempt: no effect

t30 - randomly tried another login attempt: possible

 

So I would also think that there might be some sort of caching on the host side. Especially since I removed the login credentials and even removed and added the complete user (I would strongly hope that a new user has his credentials checked against the host, not against some cache).

However, I am wondering that SSH-Login was working properly way earlier than via SAS Metadata.

 

Hope this helps!

Michael

alexal
SAS Employee

@mfab,

...

I can add, that we use Host authentication as well (SLES11.4 / SAS9.4M1).

...

I am wondering that SSH-Login was working properly way earlier than via SAS Metadata.

 

SSHD uses PAM for the authentication. You can also switch SAS to use PAM for the authentication, even for local users:

 

http://support.sas.com/kb/49/432.html

suga badge.PNGThe SAS Users Group for Administrators (SUGA) is open to all SAS administrators and architects who install, update, manage or maintain a SAS deployment. 

Join SUGA 

CLI in SAS Viya

Learn how to install the SAS Viya CLI and a few commands you may find useful in this video by SAS’ Darrell Barton.

Find more tutorials on the SAS Users YouTube channel.

Discussion stats
  • 19 replies
  • 5305 views
  • 3 likes
  • 9 in conversation