topic Re: Python Program in Sas Viya in SAS Data Science

Python Program in Sas Viya

HuseyinKaya — Mon, 25 Dec 2023 13:35:21 GMT

I have a dataframe contains 3399 rows and 16 columns running in a Python program. I want to create a table from it on SAS library.

I used

SAS.df2sd(df,"LIBRARY.Table_Name")

After i create the table there no rows in the table.

Then i used the following code for understanding the problem

SAS.df2sd(df.iloc[:1000],"LIBRARY.Table_Name")

Now i can see all the 1000 rows and 16 columns but i want to create all the rows.

There is no space problem in the server, I cannot identify the problem, can somebody help me?

Re: Python Program in Sas Viya

Kerim1 — Mon, 25 Dec 2023 14:41:40 GMT

I have same problem, please help me.

Re: Python Program in Sas Viya

IbrahimTurkoglu — Tue, 26 Dec 2023 10:45:43 GMT

I would like to ask for your help on this issue. It does not convert the existing dataframe into a SAS table, regardless of the number of columns and records, for no reason. Sometimes I experience this situation, sometimes I don't.

Re: Python Program in Sas Viya

Patrick — Wed, 27 Dec 2023 09:52:45 GMT

@HuseyinKaya , @IbrahimTurkoglu , @Kerim1

What really would help us help you is if you could provide some as simple as possible sample code that allows to replicate the issue (i.e. load sashelp.class into the data frame).

Also please state what version of saspy and Viya you are using.

Re: Python Program in Sas Viya

HuseyinKaya — Thu, 28 Dec 2023 13:52:53 GMT

Actually, the code is so basic, I prepare a python df with 3k+ columns and try to create a sas table and insert it with df2sd function and get no error, only it says "Socket connection failed during data transfer", column's name lenth less than 22, you may try with dummy data, thanks for reply.

Re: Python Program in Sas Viya

HuseyinKaya — Thu, 28 Dec 2023 14:02:38 GMT

By the way i don't use saspy and viya = 2023.3

Re: Python Program in Sas Viya

Patrick — Thu, 28 Dec 2023 14:29:05 GMT

@HuseyinKaya wrote:
By the way i don't use saspy and viya = 2023.3

Method df2sd comes with the saspy package so not sure what you're talking about.

https://sassoftware.github.io/saspy/api.html

Without seeing your code and ideally log there isn't much more I can say.

Re: Python Program in Sas Viya

sastpw — Tue, 02 Jan 2024 19:53:42 GMT

I have to believe the user is referring to using Proc Python in Viya. The proc also has similarly named methods for moving data between pandas and SAS, though the method signatures are different.

Have you looked at the log after the failure to see what errors or other information might be available?

SAS.df2sd(df, "LIBRARY.Table_Name")

SAS.printLOG()

Do you know what's in your dataframe? Realize that dataframes can contain things SAS datasets cannot. Each cell of a dataframe can contain anything. They aren't bound by 'relation table' constraints like SAS data sets or database tables. Relational tables require that the data type in every row of a table be the same for a given column. data frames don't require that. There are all kinds of ways a dataframe can't be converted into a SAS data set. But that's speculation at this point as I have no idea what's in your dataframe of what errors might be in log stating what went wrong.

So, let's start by looking at the log for the method you ran that didn't seem to work right. BTW, you can change the setting so the log is simply displayed to begin with:

SAS.hideLOG(False)

SAS.df2sd(df, "LIBRARY.Table_Name")

It's also possible that there could be an enhancement or bug fix that's happened since the version of viya you're using. it's almost a year old. So, again, seeing what actually happened will help with that. Looking at the LOG is the first thing to do.

Thanks,

Tom

Re: Python Program in Sas Viya

HuseyinKaya — Mon, 08 Jan 2024 10:55:07 GMT

Hi, thanks for the help.

I used "SAS.hideLOG(False)" code and now i can identify what is the problem.

My data takes up too much space.(One row exceeds 32768 Bytes)

How can i resolve this issue, is there any options for increasing 32768 Bytes?

Re: Python Program in Sas Viya

Kurt_Bremser — Mon, 08 Jan 2024 11:44:14 GMT

Character variables in SAS datasets are technically limited to 32767 bytes. I guess this limit also kicks in when processing rows from the dataframe.

How is that dataframe defined?

Please post the complete log of the step(s) creating the dataframe, and the complete log of the failing step by copy/pasting the log text into a window opened with this button:

Re: Python Program in Sas Viya

Patrick — Mon, 08 Jan 2024 11:57:43 GMT

With .sas7bdat files character variables are limited to 32KB.

Depending on what you need the data in SAS for you could eventually investigate if loading the data directly into CAS would be an option using VARCHAR(*).

I have never done it but the SWAT package looks promising.

https://communities.sas.com/t5/SAS-Communities-Library/Better-Open-source-Integration-through-the-Python-Load-Objects/ta-p/902222

https://sassoftware.github.io/python-swat/

Re: Python Program in Sas Viya

Namanfg — Thu, 13 Mar 2025 11:19:50 GMT

I am also facing the same problem. socket connection fails during data transfer