BookmarkSubscribeRSS Feed
hamcap
Fluorite | Level 6

Has anyone done any speed comparisons between the two?


I have only had access to the trial environment, so the testing I've done have been on very small datasets.

These results have shown no whatsoever faster speed using SWAT, compared to Pandas. When manipulating data, they were both around the same speed, but Pandas has even been a couple of thousand times faster when doing grouping reports.

 

Anyone managed to do testing with large datasets and/or any other sort of testing?

4 REPLIES 4
joeFurbee
Community Manager

Hi hamcap

I spoke with a developer and he indicated that:

When doing small data sets, pandas is always going to win.  That's comparing an in-memory local operation (pandas) against a network service over possibly multiple servers being combined into a single result (CAS / SWAT).  However, try running analytics on a few gigabytes of data in pandas and CAS.  Pandas will probably just run out of memory and quit; CAS will be done in a few seconds.


Join us for SAS Community Trivia
SAS Bowl XXXVIII, SAS Programming: Getting Started
Wednesday, February 14, 2024, at 10 a.m. ET | #SASBowl

hamcap
Fluorite | Level 6

Hi Joe,

Thanks for your reply!

This was my thoughts too, that's why I wanted to know if anyone has ever tried comparing operations on large files (preferably >10GB).

I've never tried SAS for this, but pandas handles around 10GB with no problem. Though it takes a few minutes. I'm sure SAS might be faster here, but it would be nice to see comparisons on a few different operations (and a list of specs on the hardware), in the exactly same datasets.

joeFurbee
Community Manager

I'll dig deeper into this to see if I can find any metrics that have been published or any general guidelines on performance. I'll post anything that I find here.


Join us for SAS Community Trivia
SAS Bowl XXXVIII, SAS Programming: Getting Started
Wednesday, February 14, 2024, at 10 a.m. ET | #SASBowl

hamcap
Fluorite | Level 6

Hi Joe,

I was wondering whether you've had the chance to find any metrics yet?

KR

sas-innovate-2024.png

📢

ANNOUNCEMENT

The early bird rate has been extended! Register by March 18 for just $695 - $100 off the standard rate.

 

Check out the agenda and get ready for a jam-packed event featuring workshops, super demos, breakout sessions, roundtables, inspiring keynotes and incredible networking events. 

 

Register now!

How to Concatenate Values

Learn how use the CAT functions in SAS to join values from multiple variables into a single value.

Find more tutorials on the SAS Users YouTube channel.

Click image to register for webinarClick image to register for webinar

Classroom Training Available!

Select SAS Training centers are offering in-person courses. View upcoming courses for:

View all other training opportunities.

Discussion stats
  • 4 replies
  • 1941 views
  • 3 likes
  • 2 in conversation