We've posted a few times about referencing files directly from CAS actions in CASL code (1,2,3). It's a great way to utilize CAS action functionality in programs without explicitly loading tables to CAS. The referenced file is transiently loaded to CAS and dropped immediately after the requested CAS action is run. For context, when referencing files directly, the CASL code looks like this:
Select any image to see a larger version.
Mobile users: To view the images, select the "Full" version at the bottom of the page.
As you can see, the simple.Summary action references the SASHDAT file directly instead of an explicitly loaded CAS table. SAS7BDAT, CSV, XLSX, and parquet files can also be referenced in this way. Referencing files directly is great for traditional programming where we don't need tables loaded for extended ad hoc analysis and/or reporting. However, there is a more efficient way for most scenarios, setting singlepass=true. With singlepass=true, the files aren't loaded in whole. Their data are read by block as the CAS action requires and are released immediately. This generally makes for faster processing.
For our 500,000 row, 24 column SASHDAT table, setting singlepass=true reduces our total execution time from 2.56 to 1.55 seconds. Examining the log, we see that while setting singlepass=true increased CPU on the summary action, real time for both the load and the summary decreased.
The same pattern exists for the other file types, SAS7BDAT, CSV, XLSX, and parquet. CAS CPU on the summary action increases but overall performance for the action improves.
As the name implies, "singlepass=true" can only be used on CAS actions that perform their processing with a single pass of the input data. Actions that require multiple passes will error as shown below.
Find more articles from SAS Global Enablement and Learning here.
Registration is open! SAS is returning to Vegas for an AI and analytics experience like no other! Whether you're an executive, manager, end user or SAS partner, SAS Innovate is designed for everyone on your team. Register for just $495 by 12/31/2023.
If you are interested in speaking, there is still time to submit a session idea. More details are posted on the website.
Data Literacy is for all, even absolute beginners. Jump on board with this free e-learning and boost your career prospects.