Programming the statistical procedures from SAS

A huge dataset with 100+ variables and 2 million observations

Reply
Frequent Contributor
Posts: 131

A huge dataset with 100+ variables and 2 million observations

My company has a very dataset which requires to do a regession analysis. The dataset has over 2million observations for the customers, and 100 variables (many are categorical).

I want to use SAS to run a regression model. But just worry about the massive size of the dataset. Will this cause computer crash if running for days or weeks?

An suggestions?
Trusted Advisor
Posts: 2,114

Re: A huge dataset with 100+ variables and 2 million observations

Regression of that size. No problem. The linear regression procedures (REG, GLM, etc), use memory based on the number of variables and generally make one pass through the data for each execution of the procedure.

Other regression type procedures (LOGISTIC, PHREG) use search algorithms that do run through the data multiple times and potentially take days to run.

The reference manual has some information on the execution algorithms in the "details" section of each chapter.

Doc Muhlbaier
Duke
Frequent Contributor
Posts: 131

Re: A huge dataset with 100+ variables and 2 million observations

Many thanks indeed, Duke.
Ask a Question
Discussion stats
  • 2 replies
  • 126 views
  • 0 likes
  • 2 in conversation