SAS response/recommendations for zero-day log4j2 CVE-2021-44228 vulnerabilities

[chawkins]

I actually am about to employ SAS follow-the-sun support on this in about 20 minutes but was wondering if there's anything I missed. Our server admins and information security folks are asking us to define potential exposure to the zero-day log4j2 vulnerabilities discovered recently. We've shut down anything on our SAS 9.4 mid-tiers. We think we're ok on Viya. But we utilize CI360 as SaaS and are uncertain if there is anything on SAS's end that has been done or anything that CI360 might be able to do via the direct agent to put itself in harm's way. Didn't see anything on the security bulletins page about this, so I thought I'd ask here. 

I do have an open track on this: SAS 7613479191, but we didn't hear anything before the end of the day so I'm going to wade into after hours support, because security is still waiting for our answer.

It looks to me like Environment Manager and a couple of mid-tier products might need some adjustments. But I only play a SAS Admin on TV. If anyone knows anything or has done anything, I'm all ears. Our 9.4 M6 environments are on Linux. Obviously I can get more product specific but I have no idea if this is even a point of concern and thus no idea how to guide our admins who are usually all over things at the OS level but often unsure of when SAS is using it's own version of things.

I work at a non-profit so we're often wearing many hats.

Chris

[SASKiwi]

SAS 9.4 runs Java (Java Run-time Environment) on all server tiers: metadata, compute and mid-tier. Have you run Java vulnerability tests on all SAS servers? If your SAS servers are only available via your company intranet then I'd say your security risk is pretty low and immediately shutting them down seems like a bit of an over-reaction. The answer would be different for public-internet available servers.

We went through a similar exercise ourselves recently and our IT management were quite happy to go through a normal change control process to delete vulnerable Java versions from SAS servers. This took around two weeks.

[chawkins]

Thanks Kiwi. To be clear, I know that the mid-tier shutdown is overkill, but bringing it down over the weekend to bring the temperature down as we figure things out was an easy thing to do. SAS has told us they are preparing a response so we'll see where we go from there.

Chris

[jchlebowy]

I am interested in knowing more about how this log4j vulnerability impacts SAS/VIYA as well and any necessary mitigation steps.

[BrianK]

Hi all,

Please see the following URL for status on the CVE-2021-44228 vulnerability issue:

Remote Code Execution Vulnerability (CVE 2021-44228) | SAS Support

This URL will continue to be updated with the latest information as and when available.

Thank you,

Brian Knoll

SAS Product Security Incident Response Team

[Hrry]

I checked VIYA3.5 which seems not affected. Other versions I do not know.
*Update: SAS Viya 3.5 is also affected. Confirmed with https://github.com/anchore/grype and also SAS publised information. Correcting this.

Different story for some SAS9.4 versions, where I found log4j2 versions that are affected.

This is not something that 'can wait till monday' in my opinion. The amount of possible attack vectors to drop a payload onto a server is high in my opinion. It is very useful to secure your network and scan logfiles for unwanted 'jndi' strings. I am eagerly awaiting a response from SAS.

[BrianK · Accepted solution]

Thank you @Hrry and all!

I do need to say that SAS has been working this issue very actively, all weekend long.  We will provide more information once we have it available, but I just want everyone to know we are aware of the issue and taking steps to respond.

The official location for this issue is the one mentioned in my comment above, and that will continue to be updated and will be the authoritative source for information in this issue as related to SAS:

Remote Code Execution Vulnerability (CVE 2021-44228) | SAS Support

Finally, I also want to mention that security bulletins can always be found at the following location:

Security Bulletins | SAS Support

Thank you,

Brian Knoll

SAS Product Security Incident Response Team

[andreas_lds]

Still no updates regarding bi-servers and client installations.

Out it-sec-folks are somewhat upset, that the Deployment Manager (and many other components) still use log4j 1.2.x.

Are there any plans to provide updates for those problematic version of log4j in the near future?

[PriitL]

In a scope of CVE-2021-44228 the 1.x shouldn't be affected according to http://slf4j.org/log4shell.html
There is a clause "As log4j 1.x does NOT offer a look up mechanism, it does NOT suffer from CVE-2021-44228."

Of course, there should be no excuse not to update the packages as support for versions 1.x ended 2015. (https://logging.apache.org/log4j/1.2/)

Regards,


PL

[Hrry]

Some information I gathered. Providing as information.

Information

-official statement from the maintainer: https://logging.apache.org/log4j/2.x/
-official SAS statement: https://support.sas.com/content/support/en/security-bulletins/remote-code-execution-vulnerability-cv...
-there is log4j -> not affected (important to note that there is version 1 and version 2).
-there is Log4j2 2.0-beta9 <= 2.14.1 which are vulnerable
-Search for ‘log4j2’ jar files in your sas config and sas binary folder to identify threaths
-I identified log4j2 versions that are affected in SAS components on SAS9.4M6 (gemfire, SAS Environment Manager)

How do we mitigate?
-Update log4j2 to version 2.15, it can be as simple as replacing the *.jar files. This is not an easy task, as SAS uses different components that use different versions of log4j2
-For those who cannot upgrade to 2.15.0, in releases >=2.10, this behaviour can be mitigated by setting either the system property log4j2.formatMsgNoLookups or the environment variable LOG4J_FORMAT_MSG_NO_LOOKUPS to true.
-For releases >=2.7 and <=2.14.1, all PatternLayout patterns can be modified to specify the message converter as %m{nolookups} instead of just %m.
-For versions >=2.0-beta9 and <=2.10.0 the mitigation is to remove the JndiLookup class from the classpath: zip -q -d log4j-core-*.jar org/apache/logging/log4j/core/lookup/JndiLookup.class For windows this can be as easy as opening the jarfile with 7zip and deleting the class.

In my opinion, if you have the possibility, waiting for a SAS Hotfix as solution is optimal. It is not entirely clear what is the impact at this very moment.
Making changes to your environment is always risky, know what you are doing (!) and have a rollback scenario.
-Setting the environment should be a safe mitigation, this will not provide a solution to all affected versions however.
-Replacing the jarfile with version 2.15 is not always possible due different java versions: Log4j 2.13.0 and greater require Java 8. Version 2.4 through 2.12.1 required Java 7 (the Log4j team no longer supports Java 7). Some features require optional dependencies; the documentation for these features will specify the required dependencies.
-Removing the class from the jarfile should be safe, I have not tried and verified this.
-When you perform these actions, a restart is required.
-In my opinion, if you have the possibility, waiting for a SAS Hotfix as solution is optimal.

[andreas_lds]

---

@Hrry wrote:

Some information I gathered. Providing as information.

Information

-official statement from the maintainer: https://logging.apache.org/log4j/2.x/
-official SAS statement: https://support.sas.com/content/support/en/security-bulletins/remote-code-execution-vulnerability-cv...
-there is log4j ->not affected (important to note that there is version 1 and version 2).
-there is Log4j2 2.0-beta9 <= 2.14.1 which are vulnerable
-Search for ‘log4j2’ jar files in your sas config and sas binary folder to identify threaths
-I identified log4j2 versions that are affected in SAS components on SAS9.4M6 (gemfire, SAS Environment Manager)

SAS has added CVE-2021-4104 to the security bulletin, implying that log4j 1.x is vulnerable to a problem similar to the one affecting 2.x. Unfortunately log4j 1.2.15 is a part of the Deployment Manager, automatically installed along with sas clients. Still to be checked is, if it can be used as easy as cve-2021-45046.

---

[PriitL]

I know this probably isn't a good idea to give here even temporary solutions which are untested and unconfirmed but maybe adding "-Dlog4j2.formatMsgNoLookups=true" to $JVM_OPTS variable in setenv.sh files reduces the risks?

For our system the files would be

./config/Lev1/Web/SASEnvironmentManager/server-5.8.0-EE/hq-engine/hq-server/bin/setenv.sh
./config/Lev1/Web/WebAppServer/SASServer1_1/bin/setenv.sh
./config/Lev1/Web/WebAppServer/SASServer12_1/bin/setenv.sh
./config/Lev1/Web/WebAppServer/SASServer2_1/bin/setenv.sh
./config/Lev1/Web/WebAppServer/SASServer9_1/bin/setenv.sh

Sure there are more processes which use log4j but as a quick solution at least these visible services should be fine with this fix? Until SAS gives us a confirmed solution.

Anyone can confirm?

Thanks!

PL

[Hrry]

This will provide a solution for versions >=2.10 <=2.14.1. I really cannot advise on this, but my personal opinion is that the risk is low.

[BrianVarney]

Hello,

Does anyone know a way to determine the version of log4j one is running? Is there a command to run or can it be determined by the SAS release? We are running SAS 9.4M6.

Thanks,

 Brian

[Kurt_Bremser]

I found this on stackoverflow.