Statistical programming, matrix languages, and more

Solving least squares problem with sparse matrix

Occasional Contributor
Posts: 16

Solving least squares problem with sparse matrix

This question is related to a question I posted in the SAS/STAT forum ().

I have a large and sparse design matrix of ones and zeros (about 450k rows, 2500 columns). I'm trying to use this to run a regression, but SAS/STAT procedures are taking too long. (I ran proc hpmixed and it had not produced any results after 24 hours.)

If I read the data into Matlab as a sparse matrix, I can run a QR decomposition and invert the resulting matrix in less than 10 seconds. Is it possible to take this approach in IML? I feel like I must be missing something because it doesn't seem like this is a hard problem to solve, so I thought SAS would handle everything on the fly, but it seems that I'm going wrong somewhere.

Thanks for any advice.

Posts: 4,239

Re: Solving least squares problem with sparse matrix

Posted in reply to stoffprof

In your MATLAB example, if X = QR, then R and Q are dense 2500x2500 matrices, So I guess you are asking whether IML has a sparse QR decomposition?  No, it does not.

Ask a Question
Discussion stats
  • 1 reply
  • 2 in conversation