topic Re: Passing a Datagrid from Python Code file in Intelligent Decisioning in Developers

Passing a Datagrid from Python Code file in Intelligent Decisioning

klay_martens — Mon, 05 Apr 2021 10:12:11 GMT

Hi all.

I was wondering if anyone had a more elegant way of doing this: the solution I found was to convert the python object to a json string and then pass this back from the code file to the decision flow, then I use DS2 to convert the json to a datagrid. Works fine, but it seems pretty clumsy.

Just wondering if there was a better way to do this...

I created some utility methods to assist with creating the appropriate objects in Python...

BTW, does anyone else find the lack of proper datatypes for the datagrids in ID a little lame? The fact that we cant pass datetime fields or booleans is really quite silly!

Re: Passing a Datagrid from Python Code file in Intelligent Decisioning

alexal — Mon, 05 Apr 2021 10:54:19 GMT

For a Python code that is absolutely correct approach.

BTW, does anyone else find the lack of proper datatypes for the datagrids in ID a little lame? The fact that we cant pass datetime fields or booleans is really quite silly!

Data Grids in SAS Intelligent Decision currently supports Characters, Decimal, and Integer columns. There is no boolean data type in DS2, so you have to use integers 1 or 0. Also, you could store datetime in a decimal format.

Re: Passing a Datagrid from Python Code file in Intelligent Decisioning

klay_martens — Mon, 05 Apr 2021 13:09:36 GMT

Lame. You should be able to work with the types natively. RTDM had this facility...its dumb that the latest and greatest does not.

Re: Passing a Datagrid from Python Code file in Intelligent Decisioning

klay_martens — Mon, 05 Apr 2021 13:37:05 GMT

And what further irks me is that if the computed json strings are not returned in the outputs of the decision flow, they DONT get populated by the program either. I really dont like duplicated nonsense in my interfaces!

Re: Passing a Datagrid from Python Code file in Intelligent Decisioning

alexal — Thu, 08 Apr 2021 01:45:04 GMT

I've created two feature requests for you. May I ask you to describe your problem with computed JSON strings in details? It will be better if you provide an example.

Re: Passing a Datagrid from Python Code file in Intelligent Decisioning

klay_martens — Thu, 08 Apr 2021 06:05:48 GMT

Hi...

Thanks for that - the enhancement will be usefull to most, I think. With regards to the JSON issue I mentioned....here is a sample. Start with a Python program which returns a data grid JSON object

''' List all output parameters as comma-separated values in the "Output:" docString. Do not specify "None" if there is no output parameter. '''
def execute ():
'Output:PythonString,PythonNumeric'
PythonString='''
[
{
"metadata": [
{
"ProductAuthenticationQuestionID": "int"
},
{
"AnswerID": "int"
},
{
"Answer": "string"
},
{
"IsEnteredAnswerYN": "int"
}
]
},
{
"data": [
[
28038,
21388640,
"No",
1
],
[
28038,
21388639,
"Yes",
0
],
[
27003,
21388647,
"None of these",
1
],
[
27003,
21388646,
"011 DAINFERN",
0
]
]
} 
]
'''
PythonNumeric=100
return PythonString,PythonNumeric

And then create a DS2 program which converts the string to a datagrid

package "${PACKAGE_NAME}" /inline;
method execute(varchar(2048) PythonString, in_out package datagrid AGrid);
Ds2String = PythonString; 
DATAGRID_CREATE(AGrid ,PythonString );
end;
endpackage;

Now create a simple decision flow, adding first the Python code file, then the DS2 code file...

Leave the Python variables selected as outputs, publish and submit a test score. You get the expected results

requestBody='''{
    "inputs":[
        
    ]
}'''
moduleID = "testdecision1_0"
# Define the request URL.
masModuleUrl = "/microanalyticScore/modules/" + moduleID
requestUrl = baseUrl1 + masModuleUrl + "/steps/execute"

# Execute the decision.
masExecutionResponse = post(requestUrl, contentType, 
                       acceptType, accessToken1, requestBody) 
print(masExecutionResponse)
# Display the response.
print ("response=", masExecutionResponse, end='\n\n')
print ("response content=", "\n", json.dumps(json.loads(masExecutionResponse.content), indent=4), end='\n\n')

<Response [201]>
response= <Response [201]>

response content= 
 {
    "links": [],
    "version": 2,
    "moduleId": "testdecision1_0",
    "stepId": "execute",
    "executionState": "completed",
    "outputs": [
        {
            "name": "AGrid",
            "value": [
                {
                    "metadata": [
                        {
                            "PRODUCTAUTHENTICATIONQUESTIONID": "int"
                        },
                        {
                            "ANSWERID": "int"
                        },
                        {
                            "ANSWER": "string"
                        },
                        {
                            "ISENTEREDANSWERYN": "int"
                        }
                    ]
                },
                {
                    "data": [
                        [
                            28038,
                            21388640,
                            "No",
                            1
                        ],
                        [
                            28038,
                            21388639,
                            "Yes",
                            0
                        ],
                        [
                            27003,
                            21388647,
                            "None of these",
                            1
                        ],
                        [
                            27003,
                            21388646,
                            "011 DAINFERN",
                            0
                        ]
                    ]
                }
            ]
        },
        {
            "name": "PythonNumeric",
            "value": 100.0
        },
        {
            "name": "PythonString",
            "value": "\n  [\n{\n\"metadata\": [\n{\n\"ProductAuthenticationQuestionID\": \"int\"\n},\n{\n\"AnswerID\": \"int\"\n},\n{\n\"Answer\": \"string\"\n},\n{\n\"IsEnteredAnswerYN\": \"int\"\n}\n]\n},\n{\n\"data\": [\n[\n28038,\n21388640,\n\"No\",\n1\n],\n[\n28038,\n21388639,\n\"Yes\",\n0\n],\n[\n27003,\n21388647,\n\"None of these\",\n1\n],\n[\n27003,\n21388646,\n\"011 DAINFERN\",\n0\n]\n]\n}\n]\n   "
        }
    ]
}

Now. Switch off the PythonString variable from the reply, and republish:

Run the same test again...

<Response [201]>
response= <Response [201]>

response content= 
 {
    "links": [],
    "version": 2,
    "moduleId": "testdecision1_0",
    "stepId": "execute",
    "executionState": "completed",
    "outputs": [
        {
            "name": "AGrid",
            "value": [
                {
                    "metadata": [
                        {
                            "PRODUCTAUTHENTICATIONQUESTIONID": "int"
                        },
                        {
                            "ANSWERID": "int"
                        }
                    ]
                }
            ]
        },
        {
            "name": "PythonNumeric",
            "value": 100.0
        }
    ]

As you can see, most of the structure and all of the data for the grid is now not populated. I suppose the Python string variable might be getting truncated somewhere down the line or something. But this is definitely a problem...

Re: Passing a Datagrid from Python Code file in Intelligent Decisioning

Prasenjit_Sen — Thu, 23 Dec 2021 07:09:28 GMT

This is a response to the feature request for supporting more data types in datagrid.

I am evaluating the feature request.

Could you please include customer use cases for each of the types that you requested (date-time, boolean)?

Please include the type of processing you plan to do with the data of those types in a decision flow.

Thanks