I have used SAS in an elementary way for many years, but I am now working on a project that is bigger than what I'm used to. I am extracting about five million observations from a government data source that has about 15 columns of data.
What kind of hardware is required for a data set of that size? Can I used my usual laptop, or do I need to get access to a mainframe? Also, my college, CUNY, has a VMware-accessed version of SAS in the cloud. Will that work?
Thanks for any advice, and please forgive my naivete.
--Mitchell
@Reeza wrote:
Mainframes aren't really used anymore
Interesting statement. 🙂 In a lot of large corporations, mainframes are still used for OLTP (think ATM networks, airline reservations, etc.). A large university might have one even if only to run legacy systems on.
In analytical environments, I think mainframe users are few and far between unless perhaps the mainframe is there for some other reason.
Jim
@jimbarbour wrote:
@Reeza wrote:
Mainframes aren't really used anymoreInteresting statement. 🙂 In a lot of large corporations, mainframes are still used for OLTP (think ATM networks, airline reservations, etc.). A large university might have one even if only to run legacy systems on.
In analytical environments, I think mainframe users are few and far between unless perhaps the mainframe is there for some other reason.
Jim
Definitely speaking from an analytical standpoint! I've worked with some really old data sources in my 15 years but have managed to avoid a mainframe system to date.
I'll check. I'm working with someone who knows more computerese than I.
Probably your laptop will work. I would certainly try it before going to another platform. 5 million isn't that big, but of course size is dependent on how many columns and the size of the columns.
Make sure you have OPTIONS COMPRESS=YES in effect if your columns are mainly character or COMPRESS=BINARY if your columns are mostly numeric -- at least that's the standard advice. In actual practice I've found that sometimes binary compression is more space efficient than character compression even if there are a lot of character columns present. To really know, one has to experiment, but for now, just making sure that you are using compression should go a long way towards efficient space allocation.
Jim
Are you ready for the spotlight? We're accepting content ideas for SAS Innovate 2025 to be held May 6-9 in Orlando, FL. The call is open until September 25. Read more here about why you should contribute and what is in it for you!
Follow along as SAS’ Robert Blanchard explains three aspects of autotuning in a deep learning context: globalized search, localized search and an in parallel method using SAS.
Find more tutorials on the SAS Users YouTube channel.