Silhouette Analysis in SAS for unsupervised learning

[thistleandtweed]

hi! I wanted to ask if anyone has managed to perform silhouette analysis in SAS for clustering results before? I had to group emails together based on some predefined conditions on SAS code, and I did that by conducting dimensionality reduction with LDA and then clustering the topic vectors together with Kmeans clustering 

I wish to evaluate the accuracy of the clustering through silhouette analysis so that I can tweak any hyperparameters if needed. However, I can't find much SAS code online about the implementation of it in SAS. Was wondering if anyone had any links or possible guidance on how I can go about tackling it? 

Thanks in advance!

[Rick_SAS · Accepted solution]

For those who do not know about silhouette analysis, see "What is the silhouette statistic in cluster analysis?"

As of May 2023, there was not a built-in procedure in SAS that computed the silhouette statistic. I wrote some SAS IML functions to compute the silhouette statistic and to create silhouette plots. You can see my blog post for details and examples. The code is freely available on GitHub, I don't know whether it will serve your needs, but you are welcome to it.