About aha123

aha123 · ‎02-20-2017

This paper is excellent. Do you have any paper about how much information is lost after doing SVD? PCA can tell you how much variance is kept in first x numbers of PCs. Wonder if SVD has such measurement. Also why not use PCA for text clustering purpose? I read online many articles and still can't get any clear answer.

aha123 · ‎02-19-2017

Text Cluster node in Text Miner peforms SVD before clustering. Can anyone tell me the advantage of SVD here?

aha123 · ‎07-28-2016

For Text Topic node, the default setting for # of topics is 25. I would like to know how to determine the right number. For example, can I base on the # of clusters I get from Text Cluster node to decide how many topics I should set for Text Topic node?

aha123 · ‎07-27-2016

For my dataset, I run Regression, Neural Network, Decision Trees & Gradient Boosting Nodes. GB node always has the highest MSE, even higher than Regression. Even after I changed N interations to 200, it is still the worst one. Could you suggest what other parameters I can try out to improve the outcome? I run my dataset on R & Python, GB from both give very good results. Don't know what is wrong with SAS's GB node. My problem is of regression, not of classification.

aha123 · ‎06-28-2016

Russ, Thanks for the reply! More questions here: 1. In TextCluster output dataset, SVD columnds contain document coordinates projected on SVD dimensions. Is this different from rotation? Also a bit confused with orthogonal rotation you mentioned here. 2. For TextCluster node, how are discriptive terms assigned to each cluster generated? Any research paper describing the algorithm? 3. For TextTopic node, how are discriptive terms assigned to each topic generated? Any research paper describing the algorithm? 4. For TextCluster node, if I set number of clusters to Max., how does SAS TM determine the number of clusters to create?

aha123 · ‎06-25-2016

DoesTextCluster node also rotate the SVD dimensions before doing clustering?

aha123 · ‎06-24-2016

Russ, For my dataset, TextCluster_SVD has 18 columns and TextTopic_raw has 25 columns. Then I set # of topics for TextTopic Node to be 18. The output now has 18 TextTopic_raw columns.. But I compare these 18 TextTopic_raw columns to 18 TextCluster_SVD columns and they are different. Why is so? After obtaining SVD columns, does TextTopic node do any further processing to produce TextTopic_raw columns? It seems for TextTopic node, TextTopic_raw coulmns are not the same as its SVD columns. If they are the same, then these 18 TextTopic_raw columns should be the same as 18 TextCluster_SVD columns.

aha123 · ‎06-14-2016

I forgot to mention this is a question for regression problem. As far as I know, ROC is for classifier problems.

aha123 · ‎06-13-2016

Lift curve allows modelers to compare models. But this is done visually. Can lift chart be translated into a single metric so that comparison can be made even easier? Here is a lift curve example online: http://www.kdnuggets.com/gpspubs/kdd99-est-ben-lift/sld022.htm

aha123 · ‎05-26-2016

Topic node use the same heuristic used by Cluster node? What resolution does Topic node use for generating SVD?

aha123 · ‎05-26-2016

How does EM handle categorical predictor variables for all modeling nodes? Does it do encoding transforming like Linear Regression? If this is the case, will encoding affect prediction power for Neural Network, Decision Trees, Gradient Boosting etc.?

aha123 · ‎05-26-2016

I use the following flow with all default settings for my dataset. Input Data Source->Text Parsing->Text Filter->Text Cluster->Text Topic->SAS Code The final output dataset has the following columns: TextCluster_SVD:1~18 ===> indicating 16 SVD dimensions are created TextCluster_prob:1~10 ===> indicating 10 clusters are created TextTopic_raw:1~25 ===> indicating 25 topics are created because the default is 25 TextTopic_:1~25 Since Text Cluster Node precedes Text Topic Node here, are SVD coordinates created in Text Cluster being used by Text Topic Node? Is each TextTopic_raw column actually a SVD coordinate? If this is the case, Text Topic node just creates its own SVD coordinates and doesn't use 16 SVD coordinates created by Text Cluster Node.

aha123 · ‎02-17-2016

For my dataset, Text Cluster Node produces 10 clusters, 16 SVD and 16 Prob columns/variables in the output dataset. Questions: 1) How are these 10 clusters related 16 SVD variables? 2) Do 16 SVD variables represent "concepts" which are different from clusters? 3) How are Prob variables computed?

aha123 · ‎03-15-2015

Binary response variable has two values - 0 & 1. 98% of obs have 0 while 2% have 1. I ran Decision Trees node and got no split.Any suggestions for how to produce a decision trees model for such dataset.

aha123 · ‎07-29-2014

If SAS is installed in a Linux box, can client EG be installed in PC Windows to access SAS in Linux?

Online Status	Offline
Date Last Visited	‎02-23-2017 07:04 PM

Re: Why Do We Need SVD Before Clustering?

Why Do We Need SVD Before Clustering?

Determine # of Text Topics

Tune Gradient Boosting Node

Re: SVD for Text Topic & Text Cluster Nodes

Re: SVD for Text Topic & Text Cluster Nodes

Re: SVD for Text Topic & Text Cluster Nodes

Re: Lift Curve

Lift Curve

Re: SVD for Text Topic & Text Cluster Nodes

Re: Why Do We Need SVD Before Clustering?

Re: Why Do We Need SVD Before Clustering?

Re: SVD for Text Topic & Text Cluster Nodes

A Principal Components Analysis question

Re: EM Gradient Boosting unable to produce a model

proc transpose ID column sorting

Re: Why Do We Need SVD Before Clustering?

Why Do We Need SVD Before Clustering?

Determine # of Text Topics

Tune Gradient Boosting Node

Re: SVD for Text Topic & Text Cluster Nodes

Re: SVD for Text Topic & Text Cluster Nodes

Re: SVD for Text Topic & Text Cluster Nodes

Re: Lift Curve

Lift Curve

Re: SVD for Text Topic & Text Cluster Nodes

How EM Handle Categorical Predictor Variables?

SVD for Text Topic & Text Cluster Nodes

SVD and Prob columns Output by Text Cluster Node

Binary Response Variable No Split for Decision Trees

Client EG to Linux box