topic Re: IML sparse matrices in SAS/IML Software and Matrix Computations

IML sparse matrices

deleted_user — Tue, 20 Oct 2009 08:59:21 GMT

Hi to all,

I've search through the internet but I haven't found a way to manage sparse matrices in SAS/IML, like in R.

Is there a way? I'm working with a very sparse matrix with 250000 rows and 17500 columns but few nonzero data.

Thanks to all

Massimo

Re: IML sparse matrices

Hutch_sas — Wed, 28 Oct 2009 13:32:08 GMT

What do you want to do with sparse matrices? IML does support several methods for solving large sparse linear systems. See

http://support.sas.com/documentation/cdl/en/imlug/59656/HTML/default/sma_sect1.htm

Re: IML sparse matrices

deleted_user — Wed, 28 Oct 2009 14:26:22 GMT

I'm writing a custom implementation of PLSA (probabilsitic latent semantical analysis) in SAS/IML with an EM algorithm, but I need to manage a large dataset with circa 17000 words and 250000 documents (a 10% cut dataset for the training job is also too much for IML), but this matrix is very sparse.
So when I calculate the product between word-prob matrix and occurrence matrix, with a non sparse matrix "object", IML returns a "not enough memory" error.