Python Program in Sas Viya

[HuseyinKaya]

I have a dataframe contains 3399 rows and 16 columns running in a Python program. I want to create a table from it on SAS library. 

I used 

SAS.df2sd(df,"LIBRARY.Table_Name")

After i create the table there no rows in the table. 

Then i used the following code for understanding the problem 

SAS.df2sd(df.iloc[:1000],"LIBRARY.Table_Name")

Now i can see all the 1000 rows and 16 columns but i want to create all the rows.

There is no space problem in the server, I cannot identify the problem, can somebody help me?

[Kerim1]

I have same problem, please help me.

[IbrahimTurkoglu]

I would like to ask for your help on this issue. It does not convert the existing dataframe into a SAS table, regardless of the number of columns and records, for no reason. Sometimes I experience this situation, sometimes I don't.

[Patrick]

@HuseyinKaya , @IbrahimTurkoglu , @Kerim1 

What really would help us help you is if you could provide some as simple as possible sample code that allows to replicate the issue (i.e. load sashelp.class into the data frame). 

Also please state what version of saspy and Viya you are using.

[HuseyinKaya]

Actually, the code is so basic, I prepare a python df with 3k+ columns and try to create a sas table and insert it with df2sd function and get no error, only it says "Socket connection failed during data transfer", column's name lenth less than 22, you may try with dummy data, thanks for reply.

[HuseyinKaya]

By the way i don't use saspy and viya = 2023.3

[Patrick]

---

@HuseyinKaya wrote:
By the way i don't use saspy and viya = 2023.3

---

Method df2sd comes with the saspy package so not sure what you're talking about. 

https://sassoftware.github.io/saspy/api.html 

Without seeing your code and ideally log there isn't much more I can say.

[sastpw]

I have to believe the user is referring to using Proc Python in Viya. The proc also has similarly named methods for moving data between pandas and SAS, though the method signatures are different. 

Have you looked at the log after the failure to see what errors or other information might be available?

SAS.df2sd(df, "LIBRARY.Table_Name")

SAS.printLOG()

Do you know what's in your dataframe? Realize that dataframes can contain things SAS datasets cannot. Each cell of a dataframe can contain anything. They aren't bound by 'relation table' constraints like SAS data sets or database tables. Relational tables require that the data type in every row of a table be the same for a given column. data frames don't require that. There are all kinds of ways a dataframe can't be converted into a SAS data set. But that's speculation at this point as I have no idea what's in your dataframe of what errors might be in log stating what went wrong.

So, let's start by looking at the log for the method you ran that didn't seem to work right. BTW, you can change the setting so the log is simply displayed to begin with:

SAS.hideLOG(False)

SAS.df2sd(df, "LIBRARY.Table_Name")

It's also possible that there could be an enhancement or bug fix that's happened since the version of viya you're using. it's almost a year old. So, again, seeing what actually happened will help with that. Looking at the LOG is the first thing to do.

Thanks,

Tom

[HuseyinKaya]

Hi, thanks for the help. 

I used "SAS.hideLOG(False)" code and now i can identify what is the problem. 

My data takes up too much space.(One row exceeds  32768 Bytes) 

How can i resolve this issue, is there any options for increasing 32768 Bytes?

[Kurt_Bremser · Accepted solution]

Character variables in SAS datasets are technically limited to 32767 bytes. I guess this limit also kicks in when processing rows from the dataframe.

How is that dataframe defined?

Please post the complete log of the step(s) creating the dataframe, and the complete log of the failing step by copy/pasting the log text into a window opened with this button:

[Patrick]

With .sas7bdat files character variables are limited to 32KB.

Depending on what you need the data in SAS for you could eventually investigate if loading the data directly into CAS would be an option using VARCHAR(*). 

I have never done it but the SWAT package looks promising. 

https://communities.sas.com/t5/SAS-Communities-Library/Better-Open-source-Integration-through-the-Py... 

https://sassoftware.github.io/python-swat/