Shadow Groups for LDAP Synchronisation

7 Likes

SAS 9.4 comes with macros for synchronising users and groups from LDAP into metadata. When everything works, they are great. But if you schedule a user synchronisation job to run regularly in batch, and something goes wrong, it could make a real mess of your implemented security model.

Let's look at what can go wrong, and three options for things you can implement to stay safe!

The User Import Macros

SAS provides a set of User Import Macros, with a sample program to call them. You can adapt the sample program, first to bulk load selected groups and users from Active Directory or another LDAP server into SAS Metadata Server. Then afterwards, you can create a scheduled batch job to regularly run the code, to synchronise your metadata users and groups from the LDAP service. Many SAS customers schedule such a job so that it runs nightly.

See the SAS® 9.4 Intelligence Platform: Security Administration Guide, especially the Appendix on User Import Macros, at http://support.sas.com/documentation/cdl/en/bisecag/67045/HTML/default/p1ar98lajfgm4jn1wa1h6e19jjre.... (or search the web for “SAS 9.4 User Import Macros”, and similar sections can be found for earlier versions of SAS 9).

In this post, I will call the groups created by code which synchronises them in from LDAP dynamic groups, and the non-LDAP ones static groups, if I need to be clear which sort I mean. They're exactly the same type of metadata object, there's no difference at all; this is just a label to help us distinguish where they came from in the explanation which follows.

Ok. So you created some dynamic groups, and they have users in them who originated in an LDAP directory service. Now that you have your users and groups in metadata, you can start using those groups in ACTs, and assign those ACTs to folders and other objects, right?

Well, no, I would not advise you do that, unless you take steps to protect the batch synchronisation process from potential problems.

What could possibly go wrong?

Several things, but it all comes down to the dynamic groups and using them in ACTs.

While you are developing and debugging your SAS code to synchronise users from an LDAP provider (such as Active Directory) into SAS Metadata, and even after that code is in production and has been working well for some time, it is possible that something may go wrong with the LDAP->Metadata syncronization. For example:

A network issue, changed password or LDAP outage could result in the query your SAS program runs against the LDAP provider returning no groups.
A well-meaning administrator may delete or rename your SAS groups in Active Directory without telling the SAS administrator who maintains the syncronization code first.

If the sync code decides that your groups have been deleted, and you don't build in some protection against this risk, the group deletion will be propagated through to SAS Metadata, deleting some or all of your dynamic groups (and users) in SAS metadata.

If you delete a group, the metadata server also removes that group from any ACTs in which it featured. Now all the ACTs which featured those groups, no longer do, and may even have nothing in them. As a result, your SAS users (including interactive and batch/service users) will most likely lose access the metadata they need to do their work, and your SAS applications and services will stop working.

Oops, I'll just fix that! Oh...

You can fix the issue (or get someone else to fix it), and then re-run the synchronisation to create the groups again. But doing this won't add the new, replacement groups to all your existing ACTs. Those aren't the old groups. They are new ones of the same name. How well did you record the design of the ACTs? Is that record up to date? It's certainly an interesting way to resign. You had better go get a box for your stuff.

When we've heard about this happening at real customer sites, the first that the SAS administrator knows about the problem is usually that users cannot log in, or that they have lost access to pretty much everything in metadata. When the SAS Administrator opens SAS Management Console to investigate, his or her first indication that something is wrong is that the dynamic groups are no longer there.

But by the time this happens to you, the damage to those ACTs has already been done. The groups they used to contain have disappeared. Even if you re-create the groups, they have to be laboriously added back in to the ACTs, and the appropriate grants and denies have to be set (by hand or by a script if you have one), or a metadata backup has to be restored to recover them, losing work done in SAS metadata since the backup was taken.

Whatever you do, don't delete the damaged ACTs, and attempt to re-create them. That would make the problem even worse! When you delete an ACT, you remove it from any objects it was applied to. Re-creating the ACT does not re-apply it to those objects. How well did you document the current state of which ACTs were applied to which objects?! And is that documentation up to date? Never mind your box of personal stuff, we'll send it to you.

If you are in this unfortunate position, your best remaining options are to either:

restore a metadata backup and lose whatever other changes were made since the backup was taken, or
re-promote the ACTs from another environment in the same ecosystem (e.g. from Test or Staging back into Prod) if they were originally promoted in from that other environment, or
add the groups and their grants back into the existing ACTs by whatever means you can - manually, or by running a script.

None of these is a very simple or painless solution.

So, prevention of this problem is going to be better than a cure. What are your options?

Option 1: The Metacoda Identity Sync Plug-Ins

Metacoda Pty Ltd in Australia sell a set of Metacoda Security Plug-ins for SAS Management Console, which can help a SAS administrator with many tasks relating to metadata security. Personally, I think they are good value, and for the avoidance of doubt I have no vested interest in the company.

Since version 5.0, they have included an Identity Sync Plug-In, which can “import, check and synchronize identity-related SAS metadata with enterprise directories without the need to edit SAS code”. It has a graphical user interface, and the ability to "check" what would change before you commit the change to metadata is the big benefit here, if you are running your synchronisation of users and groups interactively. At the time of writing, The Metacoda Identity Sync Plug-In interfaces with Active Directory, but not with other LDAP services. It also uses SAS’s User Import Macros behind the scenes, so it updates metadata via a supported method.

You can read more about it at, www.metacoda.com.

Option 2: Shadow Groups

Another good solution to this problem is to set up something called shadow groups, and use those in your ACTs instead. Again the term 'shadow groups' is just a label. They are normal, static metadata groups. Create them in a 1:1 relationship with your dynamic groups, make each dynamic group a member of its corresponding shadow group, and use the shadow group in your ACTs.

I'm sure you can picture the arrangement if you set your groups up like that. Here's how things would stand if you then lost your dynamic groups:

Your users have still lost access to the system, so you still have an outage in this situation. But once you fix the issue that caused the loss of the dynamic groups, and re-run the synchronisation job to re-create the users and groups who were lost, it's much easier to make each dynamic group a member of its corresponding shadow group again. Once you've done that, users have the correct permissions again.

Option 3: Sanity checks in the synchronization code

Another good solution is to introduce some logic into the LDAP user and group synchronization code, which can recognise when there are a lot of groups being added or deleted, and will stop and alert a SAS Administrator to the high volume of additions or deletions. For example, if more than 10% of your existing groups are being deleted, or certain named groups you know should never be deleted are in the 'groups to delete' dataset in the synchronisation program, have the program alert the SAS Administrator and stop.

The SAS administrator can then investigate, and may decide to run the synchonization code interactively, to force the mass creation or deletion of users and groups if that is what is supposed to happen. But if there is a problem, he has time to figure out the root cause and fix it, and no harm was done to the existing users and groups.

Conclusion

The options are not mutually exclusive - you could use all of them. For a larger site with more users (and a bigger budget), I think Option 1 is a strong choice. For a smaller site, or one where there is no realistic chance of buying a third-party tool to help with user synchronisation (and other security-related administration tasks!), a combination of both Option 2 and Option 3 is ideal.

PaulHomes · ‎07-06-2017

Hi David,

Thanks for sharing this. In addition to your 5 GEL security papers, this is another very useful best-practice post that I think many SAS administrators will benefit from. It's also something I can very much relate to for a couple of reasons:

Firstly, because you mention our Metacoda Identity Sync plug-in - thank you! 🙂

Secondly, because I have personal experience with this whilst an adminstrator of a SAS 9.1.3 platform many years ago... For some reason, on a Saturday night, AD returned no users/groups and no errors. It looked like all AD-sourced users and groups had been removed and so the sync process dutifully deleted them all from metadata. On Sunday nobody was working so nobody noticed. On that Sunday night AD then returned all the users and groups as normal. This was processed by adding everything back in as NEW users and groups in metadata. On Monday, at first glance, it looked like nothing had changed. All of the users and groups appeared to still be there but, of course, all of the carefully crafted access controls had disappeared! This event is partly the source of my preference for 1) Tag-Deletion instead of actual deletion, and 2) a history of audit reports for every sync operation (both of which are also features in our Identity Sync plug-in).

In Tag Deletion, we intercept the user and group deletes and turn them into updates instead: by adding a prefix in front of the display name, removing group and role memberships and removing logins. An administrator can review the tag-deleted identities from time to time and then explicitly delete them if appropriate. The benefit of tag-deletes are they are automatically reversed when the users and groups are synced again (as would have happened in the story above) and crucially, because the users and groups are not deleted, their metadata relationships with things like access controls are never lost.

Before following the tag-deletion method I did try the shadow groups method but was never entirely comfortable with it. It potentially doubled the number of groups in metadata. It added a manual burden in terms of creating shadow groups for the dynamic groups, which syncronization was intended to alleviate (this may not be much of an issue if there are a small number of groups with low frequency of changes). It also made the identity hierarchies for users more complex than necessary. It was this uncomfortableness that led me to settle on tag-deletion as my preferred option.

Some other areas where the automatic removal of dynamic groups (and dynamic users) can cause problems include:

The automatic removal of explicit permissions (ACEs). Whilst a good security implementation will limit the use of ACEs, they are still used automatically/necessarily in several key areas including:
1. Portal permission trees.
2. Private user folders.
3. Permission conditions in OLAP cubes for member-level security.
4. Permission conditions in Information Maps for row-level security.
5. Permission conditions in LASR Tables for row-level security.
The automatic removal of dynamic users/groups as members of roles.
The loss of the association of dynamic users with their old private user folders and the automatic creation of new ones.

I hope these anecdotes are also of use to the community.

Cheers
Paul

DavidStern · ‎07-06-2017

Excellent points Paul - I'd forgotten your use of tag-deletion, which I think you may have explained to me a year or two ago. Very sensible suggestions. This is exactly the sort of reply which makes this such a great community - thank you!

Best wishes,

David

MichelleHomes · ‎07-31-2017

Further to @PaulHomes' comment above, we've created a Metacoda Identity Sync utilities repository, available on GitHub, to help Metacoda customers and partners, and potentially other SAS customers. The repository contains extra samples and SAS macros that can be used with the Metacoda Identity Sync Plug-in. The utilities may be useful when setting up an identity synchronisation process between a SAS Metadata Server and an external identity provider, such as Microsoft Active Directory.

Please have a look at the documentation and utilities and let us know if you have any questions/feedback - we'd love to hear from you!

Kind Regards,

Michelle