turn on suggestions

Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.

Showing results for

Find a Community

- Home
- /
- Analytics
- /
- Data Mining
- /
- Improve K-Means results

Topic Options

- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

07-26-2016 05:08 PM

Hi, I've run a k-means clustering in Enterprise Miner, but I get a giant cluster with 99% of the registers.... Any idea to get clusters more equitable? Thanks!

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

Posted in reply to fri0

07-27-2016 10:01 AM

The HP Cluster node by default uses the Aligned Box Criterion method to pick the best K (number of clusters). But you can override this by choosing Number of Clusters: User Specify in node properties, then use the Segment Profiler node to characterize solutions with different Ks.

Also keep in mind that the HP Cluster node uses only interval inputs, ignoring any nominal/binary inputs. So, depending on your dataset, some of the information that could potentially help separate observations into clusters might be ignored. One way to deal with this is to binary-encode nominal inputs that you want to use in clustering.

Ray

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

Posted in reply to rayIII

07-27-2016 10:41 AM

Hi ryall! thanks for your answers. I used the default options in Cluster node, the result it's a ward clustering with abouth 10 clusters and also it has a giant cluster... I used the CCC plot to identify a possible optimal number of cllusters, with the graph I identified between 4 or 5 clusters. That's why I used a 4 and 5 k-means clustering, but the result it's almost the same with one cluster taking almost all the registers. I only have 2 intervals variables: total amount and number of operations. Any other idea? I found that sometimes use replacement and filter node could help.

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

Posted in reply to fri0

07-27-2016 11:04 AM

Hi. If you only have two clustering inputs, the first thing I would do is plot them as a scatterplot. Do you see distinct clusters in the 2 dimensions or is it basically one big blob?

Ray

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

Posted in reply to rayIII

07-27-2016 12:05 PM - edited 07-27-2016 12:08 PM

Yes, I ran a scatterplot and I could see about 3-4 groups, but, I just ran a density graph because I think the scatterplot could be a little tricky. The groups that I saw don't have son many observations as I thought... =(

Please, could you check the attached images and give me your opinion please?

Edit.PS. Do you know about Range standarization? When is a good idea to use it?

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

Posted in reply to fri0

07-27-2016 03:17 PM

Hi.

K-means tends to work best with well-separated spherical (or in your case, circular) groups. I'm not seeing that in your plots.

It might help to know a bit more about how you plan to use the clusters. Are you trying to characterize a sample of observations? Derive an input for a predictive model?

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

Posted in reply to rayIII

07-27-2016 03:43 PM

Nop, the objective is identify group of clients based in their money movement and number of operations. Using the replacement node, I could find 3 clusters... well at least is something jejejej

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

Posted in reply to fri0

08-02-2016 05:20 PM

Hi,
One key aspect is the variables you list to cluster the records. How many continuous variables you have? How many categorical variables you have (you should somehow quantify them to be interval scale, right?)? Have you standardized variables?
If you have concern you have too many variables to enter the Cluster node, you may consider variable clustering to 'select' variables. Generally speaking, more variables more likely you will break open big clusters. On the other hand, you need to order variables according to variable clustering. If you don't do that and just add more variables, either you find yourself having to add many variables to break up clusters, or the ending results, broken up albeit, are not meaningful.
Hope this helps? Thanks for the questions. Jason Xin