I am not suggesting removing the datasets, only the references to them. For instance, if I point a libname to abc and in there there is 30 datasets, and a libname to def which has another 40 datasets, sashelp.vtable will have standard ones + these 70 observations - SAS keeps metadata about all items in the references. I am only working on data from def, it makes no sense to have a libname pointing to abc, just a waste of resources. This is called your personal setup. There are many ways to do it, in one case, if I am working on one study I will have a setup file to setup libanes for that one study - I need not references to any study every made.
Condense the user work setup to what is needed by that user at that time, rather than just having one setup which points to everything on your system.