topic Re: How do i compare results of k-means, agglomerative heirarchical clustering & kohnen SOM ? in SAS Data Science

How do I compare results of k-means, agglomerative hierarchical clustering & Kohonen SOM?

saadgill — Thu, 03 May 2018 20:30:06 GMT

Hello,

I have a transnational data from which Im making customer segments through k-means clustering, agglomerative hierarchical clustering and Kohnen Self Organising Map on SAS Enterprise Miner 14.2 . My questions is i want to compare these methods as which one has produced the best segments. Can somebody suggest me some measures available on SAS Miner through which i can compare these methods in terms of performance, distance measures or segment results ?

Thank you.

Re: How do i compare results of k-means, agglomerative heirarchical clustering & kohnen SOM ?

Reeza — Mon, 30 Apr 2018 17:04:08 GMT

Do you have some metric that tells you what the 'correct segments' are?

Re: How do i compare results of k-means, agglomerative heirarchical clustering

saadgill — Mon, 30 Apr 2018 19:07:18 GMT

No, actually this is what i really want to know that on what metrics should i measure my segments for each method ?

Re: How do i compare results of k-means, agglomerative heirarchical clustering & kohnen SOM ?

Ksharp — Tue, 01 May 2018 10:39:53 GMT

You can do Analysis of Variance .

In SAS/STAT, PROC GLM can do that. I am not sure which node in EM you can refer to .

Re: How do i compare results of k-means, agglomerative heirarchical clustering & kohnen SOM ?

RalphAbbey — Wed, 02 May 2018 14:35:48 GMT

As mentioned previously, you can do an analysis of variance. I don't know if there is an Enterprise Miner node that does this either, but you can use the SAS code node to run the PROC GLM.

In general it can be hard to determine "best" clusters/segments. There are multiple measures, but some of them don't compare across types of clustering methods (centroid based vs hierarchical based). Ultimately it may be worth trying all the clustering methods, and then computing your analysis on each set. If your analysis is better on one set of segments/clusters than another, then that could be one way to determine "best."

If you're looking at the results of a final modeling, then you might need to consider a holdout set so that you're not biased in determining your best clusters. Ultimately approaching the definition of "best" in this way ties the definition of best clusters to the ultimate modeling results that you're looking for.