02-16-2012 10:56 AM
I understand that, in data step, SAS process data record by record; so memory should not be a problem when dealing with large data set.
Now suppose I have a dataset "test" which is 100G, and available memory is 16G; if I want to do some statistics about the data, such as a logistic regression:
PROC LOGISTIC DATA=test;
How does SAS deal with such Procs? still record by record? will memory shortage be a problem in such case?
Thank you in advance for educating me.
02-16-2012 11:18 AM
Memory should only be a problem if the procedure or datastep is using a method that requires such memory, like the hash method.
In other cases, it should only affect performance. However, at least with some of the cluster algorithms, that degredation of performance is "almost" equivalent to non-functionality (e.g., a 2 second task still running after 10 hours).
However, with 16GB, I doubt if you will often confront such issues.
02-16-2012 11:31 AM
So, you suggest that memory should not be a problem, even if the memory is lower than 16G (say 2 G); only the calculation speed might be lower. Right?
02-16-2012 11:54 AM
I think that the correct answer depends upon the options selected. Take a look at: