Reclaim Disk Space After a SAS 9.4 Upgrade-in-Place

3 Likes

After performing an upgrade-in-place to SAS 9.4, most likely to remove Adobe Flash dependencies, you may have noticed that your SAS deployment is consuming an increased amount of disk space. After, and only after, you complete all of your validation tests to ensure that your upgraded environment is acceptable, you can take steps to reclaim fairly significant disk space.

How much space you are able to reclaim depends on your product mix and how many updates has your environment been through. The test deployment I am using is a basic single machine Visual Analytics system that has been updated several times so it is likely to show much higher storage recovery numbers than you will see for individual machines in a distributed deployment or more carefully maintained systems.

As we go through these steps, let's keep an eye on how much space we reclaim. After applying an UIP to SAS 9.4M7, the deployment I have shows disk usage of 35G in SASHome and 34G in vaconfg, which is the configuration directory.

[sas@sasserver sasinside]$ du -sh ./SASHome
35G ./SASHome
[sas@sasserver sasinside]$ du -sh ./vaconfig/
34G ./vaconfig/

Remove Old Products

An upgrade-in-place installs new products alongside existing products to provide a way to recover to an earlier point should the need arise. Therefore, after you have validated the resulting upgrade, you can safely remove older versions of products.

I mentioned earlier that my test deployment has been allowed to accumulate products through multiple rounds of upgrades. When I look at many of my web applications in SASHome, I see this pattern for many of them.

/opt/sasinside/SASHome/SASVisualAnalyticsHub
[sas@sasserver SASVisualAnalyticsHub]$ ll
drwxr-xr-x. 6  sas sas 4096 Sep 18 15:52 ./
drwxr-xr-x. 94 sas sas 4096 Sep 18 16:05 ../
drwxr-xr-x. 8  sas sas 4096 Sep 18 16:08 7.2/
drwxr-xr-x. 8  sas sas 4096 Sep 18 16:08 7.3/
drwxr-xr-x. 8  sas sas 4096 Sep 18 16:08 7.4/
drwxr-xr-x. 8  sas sas 4096 Sep 18 16:08 7.51/

The UIP to M7 installed Visual Analytics Hub 7.51 but I still have three earlier versions of VA Hub taking up space in SASHome.

Removing these old products is easy to do.

Run the SAS Deployment Manager
Select Uninstall SAS Software
Specify you want Earlier SAS software versions uninstalled.

Select any image to see a larger version.
Mobile users: To view the images, select the "Full" version at the bottom of the page.

That's it. Notice that the SAS Deployment Manager automatically selects all earlier versions of all products so all you have to do is click Next and complete the process.

So how much space did this quick step reclaim? Roughly 40% of SASHome's original 35G.

[sas@sasserver sasinside]$ du -sh ./SASHome
21G ./SASHome

Unless you have, or are, a lazy administrator, you probably will not see a 40% reduction in your deployment. As a rough rule of thumb, you will likely see reduction of a little more than 4G per old version (e.g., Visual Analytics 7.2) you have lying around.

Remove files from the Web configuration directory

To protect the administrator, the UIP process makes backup copies of the web applications that are rebuilt during configuration processing. After validating the upgrade, you can go through and remove these backups because they contain previous versions of configured products and you will not be able to restore them even if you wanted to having just removed the associated products from SASHome.

The first place we will clean up is the Levn/Web/Staging where we are starting out with 5.8G of content.

[sas@sasserver Web]$ du -sh ./Staging
5.8G ./Staging

Delete files in ./Web/Staging/Backup

When new versions of the web application are built, the .war/.ear files being replaced are copied to ./Web/Staging/Backup. Because the files copied Backup during an upgrade are for earlier product versions and are not any use to us after the upgrade, we can delete them. The ./Staging/Backup directory in my deployment has almost 1G to remove.

[sas@sasserver Web]$ du -sh ./Staging/Backup
996M ./Staging/Backup/
[sas@sasserver Web]$ rm ./Staging/Backup/*

Delete leftover apps in ./Web/Staging

Now let's look at the Staging directory itself. We currently show 4.9G of content but interestingly, there are three large web applications related to Flex theming and documentation that did not get rebuilt but did not get removed (marked with blue bubbles below).

The theming applications have been replaced with non-Flash theming applications so we do not need them any longer, nor do we need the earlier webdoc application. Let's remove those .ear files and check our savings.

[sas@sasserver Staging]$ rm sas.flexthemes4.7.2.ear
[sas@sasserver Staging]$ rm sas.themedesigner4.7.2.ear
[sas@sasserver Staging]$ rm sas.webdocmd9.4.ear
[sas@sasserver Staging]$ du -sh .
2.6G .

So those two easy steps recouped 3.2G of space.

Delete ./Web/WebAppServer/Backup files

Another easy place to look for space is in the WebAppServer/Backup directory. The UIP makes a complete copy of the WebAppServer configurations for every managed server in case you need to reference earlier configurations during validation. After your system validation tests are complete, it is reasonable to delete these backups as your new managed server configurations will incorporate any earlier configuration information.

My deployment is starting with 16G in the WebAppServer directory.

[sas@sasserver Web]$ du -sh ./WebAppServer/
16G ./WebAppServer/

Having been through an earlier upgrade, my deployment actually has two backups of the SASServer1_1 managed server. Let's remove both.

[sas@sasserver WebAppServer]$ ll Backup/
drwxr-xr-x. 4 sas sas 4096 Sep 18 16:20 ./
drwxr-xr-x. 4 sas sas 4096 Sep 18 16:22 ../
drwx------. 10 sas sas 4096 Sep 18 17:32 SASServer1_1.2017-11-13-17.03.59/
drwx------. 9 sas sas 4096 Sep 18 17:32 SASServer1_1.2020-09-18-16.20.07/
[sas@sasserver WebAppServer]$ rm -rf ./Backup/*

That saved 9.5G and the WebAppServer directory has shrunk from 16G to 6.5 G.

[sas@sasserver Web]$ du -sh ./WebAppServer/
6.5G ./WebAppServer/

Total space savings for the Configuration directory

So how much space have we reclaimed from our configuration directory by cleaning up the Web area? Starting with 34G, it looks like we have reclaimed 13G.

[sas@sasserver sasinside]$ du -sh ./vaconfig
21G ./vaconfig

Final Tally

So with just these few easy steps, we have reclaimed a significant amount of disk storage.

Location	Before	After	Savings
SASHome	35G	21G	14G
Config Directory	34G	21G	13G

Keep in mind that the space you reclaim will definitely differ from the results I show here but that should not deter you from taking steps to conserve the disk storage consumed by your SAS 9.4 deployment.

If you know of other space-saving techniques, please share them.

JuanS_OCS · ‎10-26-2020

Hello @ScottMcCauley ,

thank you for this article, I feel confident this will help and support many.

I would like to raise a few questions:

1- why is this left to the user / after upgrade? Is this by design?

One could argue that the UIP process should already take care of this removal of previous versions, automatically. In the software market this is generally done by the UIP software, not left to the user. If you want to keep something from previous versions, you move it to a backup, not a leftover in the binaries and configuration.

2- I would imagine that perhaps it is a concern for products that still would connect, such as Enterprise Guide, therefore even if you install the 8.x public types you don't want to remove automatically the older versions, such as 7.x, in the case a user plans to continue using the SAS EG 7.5, for instance.

However, what about the other products? I would assume that there is no benefit at all on having SAS VA Builder 7.2, 7.3 and 7.4 in addition to the newly installed 7.5. Correct? Therefore, this could be easily integrated in the UIP process. Otherwise, I would be interested in hearing the reason this is implemented this way, by design.

3- Is the UIP process from SAS Viya leaving older products in the binaries or configuration of the products?

Thank you in advance!

Best regards,

Juan

ScottMcCauley · ‎10-26-2020

Hi Juan, thanks for your questions.
The UIP process for both SAS 9 and Viya leaves binaries for both the old and new versions by design. This is so rolling back to the older version is possible if something goes amiss during the UIP. Removing the previous versioned product is something to be done post-validation so for the admin's protection, both products need to be retained until that time. Having the uninstall of previous versions as a separate step allows the admin to decide when it is safe to fully commit to the upgraded version.
I see your point that an admin may want the option to go ahead and automatically remove all previous versions of products during the UIP process. While that will incur a certain amount of risk that may not be suitable for everyone, I'll pass your suggestion along to the product management team.

JuanS_OCS · ‎10-26-2020

Hi Scott, thank you for your reply, appreciated.

I understand well the design consideration. However, in my humble opinion I think it might need to be refreshed/updated.

Perhaps the main consideration would be the fact that there is no documented procedure to rollback the changes, and there is no option in SDW/SDW to automatically rollback the changes. It is left to infrastructure.

It makes sense to me that the backup and rollback procedures are left to the underlying infrastructure, specially nowadays that most of the server components are virtual machines or containers. Specially due to this fact, it does not make much sense to me that, if this is left to infrastructure and there is no procedure documented and no procedure automated in SDW/SDM/Ansible playbooks, that we leave binaries and configuration behind the scenes, just leaving more manual work (and hence human errors) to the SAS user/administrator.

Indeed, there might be certain risks for certain scenarios. I suppose the exceptions and risks they need to be assessed with certain recommendations, as usual. The current scenario it has also certain risks, such as the human errors, or the unknowingly clean up.

Thank you very much for passing these through. Would it help you if I register this as a SAS Ballot Idea?

ScottMcCauley · ‎10-26-2020

Hi Juan, points well taken. The entire UIP process could use many improvements and an automated recovery mechanism would be welcomed by all admins. I think opening a SAS Ballot item is a great idea as others interested in the same improvements could chime in.

ABV · ‎08-14-2021

Thank you for the article!

The update to M7 normally goes along with the update of PostgreSQL to 12.5. When upgrading PostgreSQL's content the data backup directory is created (\data_backup_95) which seems to be left unattended by the upgrading process.

Should it be deleted manually also ?

Would appreciate your reply.

BR,

ABV

ScottMcCauley · ‎08-16-2021

Yes, you can delete that directory as well. Thanks for adding another space-saver to the list.