Multivariate Gaussian Mixture Model in SAS/STAT or SAS/IML ???

[ODENWALD]

Hi all :

it appears to be a Proc GMM  in

SAS Visual Data Mining and Machine Learning 8.3: Procedures

Is something similar also available via  SAS/STAT  or  SAS/IML  ??

If not what would be the most direct way to copy a table of values into Proc GMM  and have it perform the GMM-Fit ?

Is there a very rapid way to handle the technicalities ?

Have only licenses and experience with  SAS/ STAT    /IML   /OR  etc

Odenwald

[Ksharp]

It looks like PROC MIXED or PROC GLIMMIX .

Anyway, calling @Rick_SAS 

[Rick_SAS]

Depends on what you are trying to accomplish. For regression, you might use GLMMIX or FMM. If you have SAS Viya, the "SAS/STAT license" is called "Visual Statistics," which supports the MBC procedure for model-based clustering.

I have written about efficient ways to work with radial basis functions and Gaussian kernels in SAS/IML, but I have not published any articles about how to fit models that involve those features. If you can write down a likelihood function, you can use standard MLE.

[ODENWALD]

I'm looking for SAS functionality  just as in  SAS/STAT  Proc MI  (which is limited to the univariate case) but for multivariate distributions, i.e. fitting the mixture probabilities and the components' parameters at best via  EM  or  MCMC  (because missing values are possible).  The (new)  Proc   GMM  in  SAS Visual Data Mining and Machine Learning 8.3: Procedures  - as is being written in a summary - does clustering based on Multivariate Gaussian Mixture Models, hence I would need and want just the first part, the model fitting. If possible in SAS/STAT  or /IML. Proc FMM  seems to work only for univariate cases, too.

[Rick_SAS]

Please post some data and give an example that shows what you are trying to accomplish. PROC MI, PROC MCMC, and PROC FMM all handle multivariate data, so I do not understand what you mean when you say they are "limited to the univariate case." I think an example would be helpful.

[ODENWALD]

I have several problems at hand, from 2-variate (== bivariate)  to 10-variate. The easiest is the following :

data MIX_of_Four_NORMALS ;
input X Y ;
datalines ;
5.8236317e-01 3.2144090e+00
-2.3075091e-01 5.7222812e-02
2.1169226e+00 3.2754264e+00
-4.3229182e-02 -3.7732497e-01
5.2826550e-01 1.3233708e+00
3.1512625e-01 6.0427425e-01

...................................................

;

where I want to fit a mixture with  4 components of bivariate normals, if possible internally using some kind of EM algo.

I might have overlooked something, but terms as  <multivariate normals> for starters (then maybe using non-normals) I have only detected with  Proc  GMM  and not with  FMM  . The latter seems to be using univariate distributions and co-variates, but no truly multivariate distributions. It should be clarified.

I think I can techn. use all SAS/Stat Procedures but I do not know how to access Proc GMM.

Any ideas ?

Odenwald

[Rick_SAS]

Thanks for the clarification. It sounds like you want to perform an analysis that will output the following 

• four mean vectors. Each is the center of a bivariate cluster
• four covariance matrices. Each is the covariance of a bivariate cluster.

Do you need anything else? For example, the mixing probabilities that define the probability of an observation belonging to each cluster?

IMHO, this sounds like model-based clustering, as implemented in the MBC procedure in SAS Viya. I had linked to this paper in an earlier response. 

[ODENWALD]

Yes, the mixture probabilities are needed, too.

There seems to be at least some overlap between Procs  MBC  and  GMM , both being  Viya only.

The basic question was : Is such an approach available via  SAS/STAT or  /IML, too ?

The new question would be : Where can I find a step-by-step recipe on how to solve it via Viya ?

Where the point is how to access  Viya  faster than R: mvnormalmixEM  - for an old sas-ler.

[Rick_SAS]

> The basic question was : Is such an approach available via  SAS/STAT or  /IML, too ?

I do not think it is available in SAS/STAT. It is certainly possible to implement EM in SAS/IML (and others have done so), but I have not done it myself.

> The new question would be : Where can I find a step-by-step recipe on how to solve it via Viya ?

If you have a license for Viya, I would start with the Kessler (2019) paper. The documentation for PROC MBC in the SAS Visual Statistics Procedures has other examples. The doc also has sections such as "Quick Start for SAS Viya" and "An Introduction to SASViya Programming" for learning how to interact with Viya procedures.

[ODENWALD · Accepted solution]

Thanks for your tips.

Problem solved.

It turned out to be the most straightforward way to use  R's  mvnormalmixEM  package. That did the jobs.