Statistical programming, matrix languages, and more

IML sparse matrices

Reply
N/A
Posts: 0

IML sparse matrices

Hi to all,

I've search through the internet but I haven't found a way to manage sparse matrices in SAS/IML, like in R.

Is there a way? I'm working with a very sparse matrix with 250000 rows and 17500 columns but few nonzero data.

Thanks to all

Massimo
SAS Employee
Posts: 94

Re: IML sparse matrices

What do you want to do with sparse matrices? IML does support several methods for solving large sparse linear systems. See

http://support.sas.com/documentation/cdl/en/imlug/59656/HTML/default/sma_sect1.htm
N/A
Posts: 0

Re: IML sparse matrices

I'm writing a custom implementation of PLSA (probabilsitic latent semantical analysis) in SAS/IML with an EM algorithm, but I need to manage a large dataset with circa 17000 words and 250000 documents (a 10% cut dataset for the training job is also too much for IML), but this matrix is very sparse.
So when I calculate the product between word-prob matrix and occurrence matrix, with a non sparse matrix "object", IML returns a "not enough memory" error.
Ask a Question
Discussion stats
  • 2 replies
  • 348 views
  • 0 likes
  • 2 in conversation