Determine # of Text Topics

aha123 — Thu, 28 Jul 2016 14:32:11 GMT

For Text Topic node, the default setting for # of topics is 25. I would like to know how to determine the right number. For example, can I base on the # of clusters I get from Text Cluster node to decide how many topics I should set for Text Topic node?

Re: Determine # of Text Topics

rayIII — Thu, 28 Jul 2016 16:57:42 GMT

Hi.

I don't think the TT node provides much in the way of guidance, but you might have a look at the HP Text Miner node (HPDM tab) if you have access to it. It gives various options for selecting the number of topics based on percentage of total variance accounted for.

Here's a snippet from Help:

"Suppose that the maximum number of SVD dimensions that you specify for the Max SVD Dimensions property is maxdim, and these maxdim SVD dimensions account for p% of the total variance. High resolution always generates the maximum number of SVD dimensions (maxdim). For medium resolution, the recommended number of SVD dimensions accounts for 5/6*(p% of the total variance). For low resolution, the recommended number of SVD dimensions accounts for 2/3*(p% of the total variance)."

(You could also try posting to the Text Analytics Community as they are the text mining experts)

Ray

topic Determine # of Text Topics in SAS Data Science

Determine # of Text Topics

Re: Determine # of Text Topics