Issues with E-Miner Project Size

[AvinashGinjupalli]

Hi All,

Recently we started using SAS E-Miner 6.2 and every one is new to it. Each E-Miner project is utilizing 10GB - 15 GB of space and filling up the disk.

I am  trying to understand why Enterprise Miner is storing data in the project folders when they already exist in the Business library XXXXXXX, and if there’s a way to move this without corrupting the Miner projects.

I am not ready to delete any Miner files until I got confidence that it will not corrupt the project.

Can anyone let me know whether we can delete the files without affecting the projects?

Quick help will be appreciated.

Thanks,

Avinash G

[jakarman]

Find the miner documentation how project are organized.   There is no good single point of that, but see:

SAS(R) Enterprise Miner(TM) 12.1: Administration and Configuration (macros migration)

43711 - Unlocking a SAS® Enterprise Miner(tm) diagram

32904 - Migrating SAS® Enterprise Miner(tm) projects to a new server on the same operating system

What you are seeing is the Miner projects are represented by a lot of OS files in that miner-project structure

All relations on diagrams transformations is kept at that location.

Miner projects are starting with ETL Transformations data conversions and missing imputations before te modelling takes place.

Sounds confusing as most people think I have already done that....  There is the buseness Library.  Wrong these steps  are needed again and again.

It is aligning the data to be able to use them by the analytics programs. Analyzing combing some predictors that are highly correlated etc.

So where must these data of newly created dataset go?....

The only option is investigating whether intermediate tables resulting of the business data could be relocated to another location.

The is goingt into the miner detailed working settings like a data-scientist.

[Reeza]

Jaap Karman wrote:


Miner projects are starting with ETL Transformations data conversions and missing imputations before te modelling takes place.
Sounds confusing as most people think I have already done that....  There is the buseness Library.  Wrong these steps  are needed again and again.
It is aligning the data to be able to use them by the analytics programs. Analyzing combing some predictors that are highly correlated etc.

That strongly depends on where you work. In my case no, the logic would need to be applied and an edited version of the data is stored in the Business Library.

Some models need imputations (regression) whereas others don't (tree models) so some data manipulation has to happen in the project and if the original data is big then the project will be big. Additionally you'll have copies of scored files which may also be big. 

[jakarman · Accepted solution]

There is nice sample of working with miner still in used with 12.3 (open that intro on the support site)

Follow it with the donor dataset as being the business library 
Getting Started with SAS(R) Enterprise Miner(TM) 5.3 (missings imputations/transformations)

Getting Started with SAS(R) Enterprise Miner(TM) 5.3 (Develop Other Competitor Models)

a snapshot how to position Miner (source SAS) is: