turn on suggestions

Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.

Showing results for

Find a Community

- Home
- /
- SAS Programming
- /
- SAS Procedures
- /
- choose between Proc cluster methods

Topic Options

- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

08-04-2013 09:03 PM

Hello!

I have a pretty large data set and have to cluster it. I have no logical assumption to prefer a specific number of clusters neither have a preference over the clustering method.

Considering the fact that each of the many available SAS clustering methods provides different clusters which may affect my subsequent analysis significantly, is there any method to evaluate the clustering methods to select the most appropriate one? There is highly cited paper, which discusses this, but I can not find any SAS procedures or macros to perform the algorithms introduced in the paper

Accepted Solutions

Solution

08-05-2013
10:09 AM

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

Posted in reply to niam

08-05-2013 10:09 AM

The "Introduction to Clustering Procedures" and many of the clustering proc's Reference sections cite Milligan (although not this particular paper) quite regularly. Comparison of algorithms has been done extensively on simulated data, but not so much on real data as you don't have prior knowledge of the number of clusters.

Of the algorithms in the Milligan and Cooper paper, the FASTCLUS procedure implements the Calinski and Harabasz method, while the CLUSTER procedure implements an improved formula for the Duda and Hart method.

Good luck.

Steve Denham

All Replies

Solution

08-05-2013
10:09 AM

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

Posted in reply to niam

08-05-2013 10:09 AM

The "Introduction to Clustering Procedures" and many of the clustering proc's Reference sections cite Milligan (although not this particular paper) quite regularly. Comparison of algorithms has been done extensively on simulated data, but not so much on real data as you don't have prior knowledge of the number of clusters.

Of the algorithms in the Milligan and Cooper paper, the FASTCLUS procedure implements the Calinski and Harabasz method, while the CLUSTER procedure implements an improved formula for the Duda and Hart method.

Good luck.

Steve Denham

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

Posted in reply to SteveDenham

09-30-2013 12:29 AM

Thank you very much for the info!

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

Posted in reply to SteveDenham

06-19-2015 10:46 AM

Steve,

You mention that FASTCLUS procedure implements the Calinski and Harabasz method, but how to get it's value (is that readily available in the output). How to get value of c(g) =trace(B)/(g-1)/trace(W)/(n-g), where n and g are the total number of observation and number of clusters manually. Yyour help will be pivotal for me.

Thanks!

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

Posted in reply to niam

10-02-2013 03:12 AM

The goodness of the coefficients depend on your data, the distance measure you use (euclidiean, city-block, correlation etc. ) and the kind of algorithmus that you choose. If you have no idea, how the data is going to look like, most of the people start with hierarchical clustering, especially average-linkage methode. the problem is the speed. it takes relativly longer time. if you have a reasonable guess, then k-means or K-medoids will be better, because they are pretty fast for large data sets. if you want more detailed information about the data sets, then try soft-clustering methode, like Fanny algo. In fast there is no the best methode. All the steps, like how you choose your distance measure, are not trivial.