We are using explicit SQL pass through statements to query a large table in Redshift. The Proc SQL completes without a SAS error but has only about half the number of records we were expecting. Our DBA reviewed the Redshift database log and found the query had an out of memory error and stopped fetching additional rows from the table.
Since these are production Jobs we need to have the SAS Proc SQL Step fail with an error as we do not want to continue the process with an incomplete data set. Additionally, since this is a "silent" error, our users would never know there was problem.
Does anyone know how to pass this Redshift error down to our SAS job?
I first contacted SAS support and the response was to work with our DBA to optimize our queries to get them to run with the size of our memory. This won't work as the database is shared resource and the total amount of available memory at any given time will be unknown. I'll continue to experiment to see if I can find a solution.
Registration is open! SAS is returning to Vegas for an AI and analytics experience like no other! Whether you're an executive, manager, end user or SAS partner, SAS Innovate is designed for everyone on your team. Register for just $495 by 12/31/2023.
If you are interested in speaking, there is still time to submit a session idea. More details are posted on the website.
Learn the difference between classical and Bayesian statistical approaches and see a few PROC examples to perform Bayesian analysis in this video.
Find more tutorials on the SAS Users YouTube channel.