Building models with SAS Enterprise Miner, SAS Factory Miner, SAS Visual Data Mining and Machine Learning or just with programming

Optimal number of clusters

Accepted Solution Solved
Reply
Occasional Contributor
Posts: 8
Accepted Solution

Optimal number of clusters

Hi,

How can we determine the number of Optimal cluster in cluster analysis?

Thanks,

Nikhil


Accepted Solutions
Solution
‎07-07-2017 01:16 PM
Frequent Contributor
Posts: 82

Optimal number of clusters

I think there are no strict rules for optimal number of clusters and as in all cluster analysis – there is a lot of room for variations and interpretation.

Maybe someone can give more specific criteria, but the ones I would consider:

* Use of graphical analysis to understand if your clusters are well separated, maybe some are very close and can be joined. I think also a tree (PROC TREE) is a very useful tool. There you can see how many groups (more separated tree branches) you have.

* Most likely you wouldn’t like to have clusters with just 1 or few observations.

* In some cases your data or task can give hint about number of clusters (e.g. maybe you want to separate items with high, low and middle level of something).

View solution in original post


All Replies
Solution
‎07-07-2017 01:16 PM
Frequent Contributor
Posts: 82

Optimal number of clusters

I think there are no strict rules for optimal number of clusters and as in all cluster analysis – there is a lot of room for variations and interpretation.

Maybe someone can give more specific criteria, but the ones I would consider:

* Use of graphical analysis to understand if your clusters are well separated, maybe some are very close and can be joined. I think also a tree (PROC TREE) is a very useful tool. There you can see how many groups (more separated tree branches) you have.

* Most likely you wouldn’t like to have clusters with just 1 or few observations.

* In some cases your data or task can give hint about number of clusters (e.g. maybe you want to separate items with high, low and middle level of something).

Occasional Contributor
Posts: 8

Optimal number of clusters

For hierarchical clustering try the Sarle's Cubic Clustering Criterion in PROC CLUSTER:

plot _CCC_ versus the number of clusters and look for peaks where _ccc_ > 3 or look for local peaks of pseudo-F statistic (_PSF_) combined with a small value of the pseudo-t^2 statistic (_PST2_) and a larger pseudo t^2 for the next cluster fusion

(see http://support.sas.com/documentation/cdl/en/statug/63033/HTML/default/viewer.htm#statug_introclus_se... ).

For K-Means clustering use this approach on a sample of your data to determine the max limit for k and assign it to the maxc= option in PROC FASTCLUS on the complete data.                   

☑ This topic is solved.

Need further help from the community? Please ask a new question.

Discussion stats
  • 2 replies
  • 2369 views
  • 0 likes
  • 3 in conversation