topic Re: Data Mining without using EM in SAS Data Science

Data Mining without using EM

aha123 — Tue, 07 Sep 2010 18:16:33 GMT

Called SAS and found out EM is prohibitively expensive for individual users. I wonder if I can produce the same results using SAS/STAT for data mining problems.

Re: Data Mining without using EM

WayneThompson — Tue, 07 Sep 2010 18:38:07 GMT

Hello Aha,

Please contact me at wayne.thompson@sas.com to discuss EM pricing relative to your budget and application needs. I am here to listen to your concerns at a minimum.

STAT provides a broad range of predictive/classifciation methods along with clustering and many other techniques. EM provide some algorithms like decision trees, gradient boosting, neural networks, memory based reasoning , etc not found in STAT along with the GUI for developing repeatable self documenting analysis very quickly with an emphasis on complete score code generation for deployment. Others from the list may have more practical reasons -- I work at SAS. Anway we hope you are using SAS and STAT is a great product --- EM requires it and includes the SAS code node for embeding STAT procedures into your EM analysis.

Wayne

Re: Data Mining without using EM

oloolo — Fri, 17 Sep 2010 15:21:43 GMT

you can find some popular Data Mining algorithms in STAT, such as KNN, LDA, CDA, various regression models, (regularized) Logistic, LASSO (v9.2), disjoint partition clustering and hierarchical clustering, SVD&PCA, Bagging, etc.

With some tweak, you can also implement algorithms such as AdaBoost, LSI & vector space model in TextMining, the efficiency is not on par with EM, though. consult my blog for more:
http://www.sas-programming.com

You get extra that is not available in STAT, such as Neural Network, CART, etc

Re: Data Mining without using EM

deleted_user — Fri, 24 Sep 2010 13:37:54 GMT

Do you know if EM can do the Random Forest method for decision trees? I've studied it a little, and it's kind of cool.

Re: Data Mining without using EM

goladin — Tue, 23 Nov 2010 01:59:48 GMT

Hi,

@OP: I believe that there are many of us who have written routines and sub-routines which can be used to produce similar results to the ones in Enterprise Miner. However, the computational time will become prohibitively high as the number of observations and variables increases. A suggestion is that to check whether your data is big enough to justify the use of EM.

@JQ: I believe you can do it via PROC SURVEYSELECT mixed with recursive partitioning algorithms.

Regards,
Murphy