turn on suggestions

Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.

Showing results for

Find a Community

- Home
- /
- Analytics
- /
- Stat Procs
- /
- Process New Data Set with Clustering Model

Topic Options

- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

06-08-2016 12:46 AM

I've created some clusters using proc fastclus, and I would like to apply the same clustering rulesets to a new dataset.

I'm assuming that this can be done by measuring the distance of each observation to the cluster centroids, and classifying each observation into it's nearest cluster.

Is there a built in function that allows for this?

Thanks in advance.

Accepted Solutions

Solution

06-09-2016
01:06 AM

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

06-08-2016 09:40 AM

In PROC FASTCLUS you can use the OUTSTAT= option to create an output data set that contains the centers of each cluster:

```
proc fastclus data=sashelp.iris maxclusters=3 outstat=OutClus;
VAR SepalLength SepalWidth PetalLength PetalWidth;
run;
```

If you have some new data, you can merge the new data with the centers and use PROC DISTANCE to compute the distance between the new obs and the centers. the followin uses Euclidean distance, but other distances are available:

```
data NewData;
input SepalLength SepalWidth PetalLength PetalWidth;
datalines;
50 33 14 2
62 22 45 15
59 32 48 18
64 28 56 22
67 31 56 24
;
data ALL;
set OutClus(where=(_TYPE_="CENTER")) NewData;
run;
proc distance data=All prefix=DistFromClus_ out=distances;
var interval(SepalLength SepalWidth PetalLength PetalWidth);
copy Cluster;
run;
/* we only want distances between centers */
data distances;
set distances;
where cluster = .;
array arr[*] DistFromClus_1-DistFromClus_3;
ClusterID = whichn(min(of arr[*]), of arr[*]); /* find index of min value in row */
keep DistFromClus_1-DistFromClus_3 ClusterID;
run;
proc print; run;
```

All Replies

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

06-08-2016 02:52 AM

No. you can't . It lead you to another Statistical Analysis Approach -- DISCRIM Procedure .

Why not use Logistic Regression or Decision Tree ? both could handle character and numeric variables , while proc fastclus can't .

Solution

06-09-2016
01:06 AM

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

06-08-2016 09:40 AM

In PROC FASTCLUS you can use the OUTSTAT= option to create an output data set that contains the centers of each cluster:

```
proc fastclus data=sashelp.iris maxclusters=3 outstat=OutClus;
VAR SepalLength SepalWidth PetalLength PetalWidth;
run;
```

If you have some new data, you can merge the new data with the centers and use PROC DISTANCE to compute the distance between the new obs and the centers. the followin uses Euclidean distance, but other distances are available:

```
data NewData;
input SepalLength SepalWidth PetalLength PetalWidth;
datalines;
50 33 14 2
62 22 45 15
59 32 48 18
64 28 56 22
67 31 56 24
;
data ALL;
set OutClus(where=(_TYPE_="CENTER")) NewData;
run;
proc distance data=All prefix=DistFromClus_ out=distances;
var interval(SepalLength SepalWidth PetalLength PetalWidth);
copy Cluster;
run;
/* we only want distances between centers */
data distances;
set distances;
where cluster = .;
array arr[*] DistFromClus_1-DistFromClus_3;
ClusterID = whichn(min(of arr[*]), of arr[*]); /* find index of min value in row */
keep DistFromClus_1-DistFromClus_3 ClusterID;
run;
proc print; run;
```

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

06-09-2016 01:05 AM

Thanks Rick_SAS, that answers my question!

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

06-09-2016 08:03 AM

Rick, If that should be , Why not combine them together and then run proc fastclus ? data want; set sashelp.iris new_date; run; proc fastclus data=want .....

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

06-09-2016 08:22 AM

I interpret the question as "I want to build a model on Data A and then score the model on Data B." If you merge the data, then you are using the second set of obs to build the model, which is not the same thing.

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

06-09-2016 08:27 AM

Yes that's right. Using your example: Data A is consumer sales for the past month that will be used to cluster consumers. The client will compile a number of strategies around these clusters. At the end of the following month, Data B will be scored with the clusters developed on Data A. So we'll need to understand if there were any shifts in the clusters that were developed on Data A. Combining Data A and Data B and reclustering could result in totally different clusters and mess with the client's strategies....

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

06-09-2016 08:41 AM

Rick,
Discrim Analysis do the exact same thing as "I want to build a model on Data A and then score the model on Data B."
So I suggest OP to use PROC DISCRM .

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

06-09-2016 08:49 AM

Sorry, KSharp, but I disagree. Discriminant analysis is an example of supervised learning. You need a nominal target variable Y with k levels and the goal is to group the explanatory variables X into k groups so that most of Group1 has Y=1, most of Group2 has Y=2, etc.

Clustering is a form of unsupervised learning. The OP did not mention a target variable. In unsupervised learning you only have X and you want to group the observations together, often by using some distance metric.

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

06-09-2016 08:52 AM

Correct, this is an exploratory analysis. The client simply wants to know who their clients are, there is no target to be modelled/predicted.....

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

06-09-2016 09:03 AM

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

06-09-2016 09:16 AM

If OP make sure which obs belong to which Y , Why not use Discriminant analysis?