BookmarkSubscribeRSS Feed
littlestone
Fluorite | Level 6

Hello, All

I understand that, in data step, SAS process data record by record; so memory should not be a problem when dealing with large data set.

Now suppose I have a dataset "test" which is 100G, and available memory is 16G; if I want to do some statistics about the data, such as a logistic regression:

PROC LOGISTIC DATA=test;

How does SAS deal with such Procs? still record by record? will memory shortage be a problem in such case?

Thank you in advance for educating me.

6 REPLIES 6
art297
Opal | Level 21

Memory should only be a problem if the procedure or datastep is using a method that requires such memory, like the hash method.

In other cases, it should only affect performance.  However, at least with some of the cluster algorithms, that degredation of performance is "almost" equivalent to non-functionality (e.g., a 2 second task still running after 10 hours).

However, with 16GB, I doubt if you will often confront such issues.

littlestone
Fluorite | Level 6

Thank you.

So, you suggest that memory should not be a problem, even if the memory is lower than 16G (say 2 G); only the calculation speed might be lower. Right?

art297
Opal | Level 21

Yes, with those few exceptions, at least from my own experience.

littlestone
Fluorite | Level 6

I guess in " PROC LOGISTIC DATA=test;" SAS does NOT process data record by record, or does it?

littlestone
Fluorite | Level 6

Thank you very much. I will look into the article you recommended.

sas-innovate-2024.png

Join us for SAS Innovate April 16-19 at the Aria in Las Vegas. Bring the team and save big with our group pricing for a limited time only.

Pre-conference courses and tutorials are filling up fast and are always a sellout. Register today to reserve your seat.

 

Register now!

What is Bayesian Analysis?

Learn the difference between classical and Bayesian statistical approaches and see a few PROC examples to perform Bayesian analysis in this video.

Find more tutorials on the SAS Users YouTube channel.

Click image to register for webinarClick image to register for webinar

Classroom Training Available!

Select SAS Training centers are offering in-person courses. View upcoming courses for:

View all other training opportunities.

Discussion stats
  • 6 replies
  • 951 views
  • 3 likes
  • 2 in conversation