topic Multivariate Gaussian Mixture Model in SAS/STAT or SAS/IML ??? in Statistical Procedures

Multivariate Gaussian Mixture Model in SAS/STAT or SAS/IML ???

ODENWALD — Tue, 16 Jun 2020 12:39:53 GMT

Hi all :

it appears to be a Proc GMM in

SAS Visual Data Mining and Machine Learning 8.3: Procedures

Is something similar also available via SAS/STAT or SAS/IML ??

If not what would be the most direct way to copy a table of values into Proc GMM and have it perform the GMM-Fit ?

Is there a very rapid way to handle the technicalities ?

Have only licenses and experience with SAS/ STAT /IML /OR etc

Odenwald

Re: Multivariate Gaussian Mixture Model in SAS/STAT or SAS/IML ???

Ksharp — Tue, 16 Jun 2020 13:09:49 GMT

It looks like PROC MIXED or PROC GLIMMIX .

Anyway, calling @Rick_SAS

Re: Multivariate Gaussian Mixture Model in SAS/STAT or SAS/IML ???

Rick_SAS — Tue, 16 Jun 2020 13:40:43 GMT

Depends on what you are trying to accomplish. For regression, you might use GLMMIX or FMM. If you have SAS Viya, the "SAS/STAT license" is called "Visual Statistics," which supports the MBC procedure for model-based clustering.

I have written about efficient ways to work with radial basis functions and Gaussian kernels in SAS/IML, but I have not published any articles about how to fit models that involve those features. If you can write down a likelihood function, you can use standard MLE.

Re: Multivariate Gaussian Mixture Model in SAS/STAT or SAS/IML ???

ODENWALD — Tue, 16 Jun 2020 16:07:06 GMT

I'm looking for SAS functionality just as in SAS/STAT Proc MI (which is limited to the univariate case) but for multivariate distributions, i.e. fitting the mixture probabilities and the components' parameters at best via EM or MCMC (because missing values are possible). The (new) Proc GMM in SAS Visual Data Mining and Machine Learning 8.3: Procedures - as is being written in a summary - does clustering based on Multivariate Gaussian Mixture Models, hence I would need and want just the first part, the model fitting. If possible in SAS/STAT or /IML. Proc FMM seems to work only for univariate cases, too.

Re: Multivariate Gaussian Mixture Model in SAS/STAT or SAS/IML ???

Rick_SAS — Tue, 16 Jun 2020 16:58:30 GMT

Please post some data and give an example that shows what you are trying to accomplish. PROC MI, PROC MCMC, and PROC FMM all handle multivariate data, so I do not understand what you mean when you say they are "limited to the univariate case." I think an example would be helpful.

Re: Multivariate Gaussian Mixture Model in SAS/STAT or SAS/IML ???

ODENWALD — Thu, 18 Jun 2020 09:04:51 GMT

I have several problems at hand, from 2-variate (== bivariate) to 10-variate. The easiest is the following :

data MIX_of_Four_NORMALS ;
input X Y ;
datalines ;
5.8236317e-01 3.2144090e+00
-2.3075091e-01 5.7222812e-02
2.1169226e+00 3.2754264e+00
-4.3229182e-02 -3.7732497e-01
5.2826550e-01 1.3233708e+00
3.1512625e-01 6.0427425e-01

...................................................

;

where I want to fit a mixture with 4 components of bivariate normals, if possible internally using some kind of EM algo.

I might have overlooked something, but terms as <multivariate normals> for starters (then maybe using non-normals) I have only detected with Proc GMM and not with FMM . The latter seems to be using univariate distributions and co-variates, but no truly multivariate distributions. It should be clarified.

I think I can techn. use all SAS/Stat Procedures but I do not know how to access Proc GMM.

Any ideas ?

Odenwald

Re: Multivariate Gaussian Mixture Model in SAS/STAT or SAS/IML ???

Rick_SAS — Thu, 18 Jun 2020 10:00:29 GMT

Thanks for the clarification. It sounds like you want to perform an analysis that will output the following

four mean vectors. Each is the center of a bivariate cluster
four covariance matrices. Each is the covariance of a bivariate cluster.

Do you need anything else? For example, the mixing probabilities that define the probability of an observation belonging to each cluster?

IMHO, this sounds like model-based clustering, as implemented in the MBC procedure in SAS Viya. I had linked to this paper in an earlier response.

Re: Multivariate Gaussian Mixture Model in SAS/STAT or SAS/IML ???

ODENWALD — Thu, 18 Jun 2020 10:31:22 GMT

Yes, the mixture probabilities are needed, too.

There seems to be at least some overlap between Procs MBC and GMM , both being Viya only.

The basic question was : Is such an approach available via SAS/STAT or /IML, too ?

The new question would be : Where can I find a step-by-step recipe on how to solve it via Viya ?

Where the point is how to access Viya faster than R: mvnormalmixEM - for an old sas-ler.

Re: Multivariate Gaussian Mixture Model in SAS/STAT or SAS/IML ???

Rick_SAS — Thu, 18 Jun 2020 11:03:04 GMT

> The basic question was : Is such an approach available via SAS/STAT or /IML, too ?

I do not think it is available in SAS/STAT. It is certainly possible to implement EM in SAS/IML (and others have done so), but I have not done it myself.

> The new question would be : Where can I find a step-by-step recipe on how to solve it via Viya ?

If you have a license for Viya, I would start with the Kessler (2019) paper. The documentation for PROC MBC in the SAS Visual Statistics Procedures has other examples. The doc also has sections such as "Quick Start for SAS Viya" and "An Introduction to SASViya Programming" for learning how to interact with Viya procedures.

Re: Multivariate Gaussian Mixture Model in SAS/STAT or SAS/IML ???

ODENWALD — Wed, 24 Jun 2020 08:18:44 GMT

Thanks for your tips.

Problem solved.

It turned out to be the most straightforward way to use R's mvnormalmixEM package. That did the jobs.