PROC PYTHON on SAS Viya (Stable 2025.06): need to load entire ~80GB SAS table into pandas

[Zaid_Sanchez]

Environment

- SAS Viya: Stable 2025.06

- Using PROC PYTHON in SAS Studio (Compute)

- Goal: materialize a large SAS table (~80GB) as a pandas DataFrame

What I tried (works on small samples, fails before completion on ~80GB):

1) PROC PYTHON callback: SAS.sd2df('LIB.TABLE')  (SAS↔pandas bridge)

2) pandas.read_sas('.../file.sas7bdat') from a filesystem path

3) SWAT from CAS (connects fine; pulling the full table to pandas fails)

- What’s the proven approach to fully load ~80GB into pandas within Viya?

- Which admin/infra settings should be adjusted in Compute/CAS to support this (memory limits, timeouts, I/O, pod/context config)?

- For PROC PYTHON callbacks: any known limits that could explain early failures on big tables?

[ahmedalattar]

Hi @Zaid_Sanchez 

While this does not directly answers your questions, but I hope it gives you another perspective to consider.

Check this article Parquet Support in SAS Compute Server

If you convert your 80 GB *.sas7bdat file to parquet first, then you would be able to use various Python DataFrame packages beside Pandas to read that Parquet file, even if it was larger than the available RAM in the SAS Compute machine.

Hope this helps