Exploring the use of Python Code with SAS Intelligent Decisioning
Have you ever wanted to create a decision flow using SAS Intelligent Decisioning capable of invoking an externalised public RESTful API using Python code but were not sure where or how to start? I had this challenge, solved it and thought I'd share my results. I'm not a Python programmer, but I can find my way around programming code which helped a fair bit.
I wrote this article to help others who are curious about this topic like I am, but who may find it challenging to figure out where and how to start. The assumption here is that the reader has some knowledge of SAS Intelligent Decisioning, some programming experience, Postman, PyCharm, and a general understanding of RESTful APIs and how to consume them. Let's go on a journey - shall we?
To keep this organised I will follow these steps to explain the development process:
1st Challenge - Finding a free public API to test my Python code with
Fortunately with the advent of APIs and open systems, today there are literally thousands of hosted services exposed through public REST APIs which can be consumed by anyone wanting to use those services. These services are provided by organisations which, more than often, provide free as well as paid-for subscription services to their APIs.
To keep things interesting I decided to use the "Tomorrow.io" weather API. Tomorrow.io provides numerous weather API recipes and I decided on this particular API - https://docs.tomorrow.io/recipes/build-your-own-weather-app-with-one-call. Note that you can sign-up for free, a step you will have to complete in order to get your own access API key. These APIs are very well documented and another cool feature is the fact that you can run this directly through Postman. Postman will be your friend through this process. You will use Postman to test the API and its parameters and the format of the output the API delivers. You will need the output structure in order to construct your Python code correctly, issue the request and parse the results within SAS Intelligent Decisioning.
2nd Challenge - Testing the public API using Postman
The "Tomorrow.io" UI allows you to test the recipe using Postman. Selecting "Run in Postman" launches Postman - assuming Postman is already installed -
A "Tomorrow.io API - Build Your Own Weather App With One Call" collection will be created and a "Retrieve Timelines" definition tab will be created. The right pane below shows the input parameters required. Run the request by "Sending" it to the end-point.
If all goes well the request will be sent and a result will be returned within a few seconds. The result should mimic the structure below as per this example. The resulting response will be displayed below and this is handy as it will help understand the JSON response structure and help when writing code to parse the results using Python later.
3rd Challenge - Develop Python code using PyCharm (optional)
Three are two possible approaches to this next step. If you are familiar with SAS Intelligent Decisioning (SID) and using the code node, you could opt to code Python directly in the code node. Alternatively, you could code the raw Python using a Python editor like PyCharm - which you can download from here - and once you are comfortable the code works, you can use the code in SID. If you are new to this, I'd suggest you use PyCharm first as Python has its own programming quirks.
Let us break down the Python code we will need - note that later when the code is pasted into the SID Python Code Node, a few code line items will be added specifically to facilitate the use of Python in SID.
We will use two Python packages which will help us submit RESTful API requests - the requests package - as well as the JSON package to help with parsing JSON result sets.
import requests, json
Next, we will assign variables to the required API input parameters (the _apikey variable is obfuscated for obvious reasons - you can enter yours here):
url = 'https://data.climacell.co/v4/timelines'
_apikey = "XXXXXXXXXXXXXXXXXX"
_location = location_inp
_fields = "precipitationIntensity,precipitationType,windSpeed,windGust,windDirection,temperature,temperatureApparent,cloudCover,cloudBase,cloudCeiling,weatherCode"
_startTime = "2022-06-30T01:10:40.609Z"
_endTime = "2022-07-01T01:10:40.610Z"
_timesteps = "current,1h,1d"
_units = "imperial"
_timezone = "America/New_York"
As a coding standard, I use an underscore to indicate variables associated with input parameters for the API but you are welcome to use your own standards. Also, note the API variable will require your API key to work. At the moment we are hard coding the input parameters in the code but later you can substitute these with input variables passed through from SID as part of a decision flow. The _location variable is an example here of an input variable being passed to the Python code from SID.
Now we will construct the API request using the "requests.get" python method (you can read more about the Python Request module here😞
response = requests.get(url, params={'apikey': _apikey, 'location': _location, 'fields': _fields, 'startTime': _startTime,'endTime': _endTime, 'timesteps': _timesteps, 'units': _units, 'timezone': _timezone})
... and then load up the results into result variables:
responseText = response.text
jResp = json.loads(responseText)
For the purpose of this example, only certain return or result variables are required and this is one of the tricky parts. What I'm looking for are the following output variables: timelines, temperature and windDirection. The complexity of retrieving the values of the return variables lies with the structure in which the results are returned. We define this structure as well-formed JSON. Meaning the JSON is returned in a nested and hierarchical structure. Let's revisit the Postman request output. You will notice a nested structure with repeating sections separated for example by "intervals". Each section has repeating variables for example "cloudBase" or "windDirection". The question is how to programmatically pick the correct variables given this structure. Now please remember we are learning here and to a proficient Python programmer this is most likely a no-brainer.
The next code snippet shows examples of how the required variables are selected using Pythons' coding structure:
timelines = jResp["data"]["timelines"][0]["timestep"]
temperature = str(jResp["data"]["timelines"][0]["intervals"][0]["values"]["temperature"])
windDirection = str(jResp["data"]["timelines"][0]["intervals"][1]["values"]["windDirection"])
Compare the code above with the POstman screenshot above that. Notice the nested structure of the JSON -> data -> timelines -> intervals -> values -> temperature.
Notice the use of [0] and [1] which one can compare to an array where [0] depicts the first row and [1] the second row in the array. In this example, this code format is used to pick variables from the repeating result sections in the "intervals" section of the JSON body. An easy way to reference this in JSON output is the nested sections wrapped inside square brackets [].
Finally, the entirety of the code is as follows:
import requests, json
url = 'https://data.climacell.co/v4/timelines'
_apikey = "XXXXXXXXXXXXXXXXXXXX"
_location = "40.758,-73.9855"
_fields = "precipitationIntensity,precipitationType,windSpeed,windGust,windDirection,temperature,temperatureApparent,cloudCover,cloudBase,cloudCeiling,weatherCode"
_startTime = "2022-07-01T01:06:19.232Z"
_endTime = "2022-07-02T01:07:54.244Z"
_timesteps = "current,1h,1d"
_units = "imperial"
_timezone = "America/New_York"
response = requests.get(url,params={'apikey': _apikey, 'location': _location, 'fields': _fields, 'startTime': _startTime,'endTime': _endTime, 'timesteps': _timesteps, 'units': _units, 'timezone': _timezone})
responseText = response.text
jResp = json.loads(responseText)
timelines = jResp["data"]["timelines"][0]["timestep"]
temperature = str(jResp["data"]["timelines"][0]["intervals"][0]["values"]["temperature"])
windDirection = str(jResp["data"]["timelines"][0]["intervals"][1]["values"]["windDirection"])
bigResult = json.dumps(jResp)
print (temperature)
print (windDirection)
print (bigResult)
Submitting this code snippet will return the following "trimmed" result set:
76.44
180.94
{"data": {"timelines": [{"timestep": "1h", "endTime": "2022-07-01T21:00:00-04:00", "startTime": "2022-06-30T21:00:00-04:00", "intervals": [{"startTime": "2022-06-30T21:00:00-04:00", "values": {"cloudBase": null, "cloudCeiling": null, "cloudCover": 0, "precipitationIntensity": 0, "precipitationType": 0, "temperature": 76.44, "temperatureApparent": 76.44, "weatherCode": 1000, "windDirection": 2.31, "windGust": 12.02, "windSpeed": 4.89}}, {"startTime": "2022-06-30T22:00:00-04:00", "values": {"cloudBase": null, "cloudCeiling": ......
Process finished with exit code 0
4th Challenge - Create the Python Code Node in SAS Intelligent Decisioning
Now that the Python code is returning the results without error we can move to the next step of creating the code node in SAS Intelligent Decisioning (SID). Open SID, pick the "Code File" tab on the left and click the "New Code File" button on the top right. A dialogue box will appear and you can complete this by naming your code file and selecting "Python Code File" as the type.
A code editor window will appear. We are presented with the following default code snippet:
def execute ():
'Output:'
return NoneThis default code snippet forms the most basic requirements for the code node to function. However, we will extend the code to address the requirements for the weather API. A quick breakdown of the default code snippet will provide some useful information.
- def execute () - this is the default execution method for the code node. You can extend this by providing input variable names, which will allow SID to pass input variable data to the code node from a decision flow. This can be done by adding input variable names between the brackets. For example: "def execute (location_inp)" will expect a variable called "location" to pass input data to the python code. This input variable is a decision flow variable passing data onto the Python code node. Variables can be separated by a comma.
- 'output:' - this is used to pass back variable data to SID populated during the Python code execution. For example 'Output:timelines,temperature,windDirection,bigResult' - will return four variables to SID after the Python code has been executed namely: timelines, temperature, windDirection and bigResult.
- return None - this code instructs Python to return variable data to an outside application, in this case, SID. If you omit this step none of the response data collected during the Python code execution will make its way back to SID. For example "return timelines,temperature,windDirection,bigResult" will instruct Python to return the variable data externally.
These three code line items are crucial in the Python Code node functioning correctly in SID. Here is the final code snippet for the Python Code Node:
import requests, json
def execute (location_inp):
'Output:timelines,temperature,windDirection,bigResult'
url = 'https://data.climacell.co/v4/timelines'
_apikey = "XXXXXXXXXXXXXXXXXXXXXXXXXXX"
_location = location_inp
_fields = "precipitationIntensity,precipitationType,windSpeed,windGust,windDirection,temperature,temperatureApparent,cloudCover,cloudBase,cloudCeiling,weatherCode"
_startTime = "2022-06-30T01:10:40.609Z"
_endTime = "2022-07-01T01:10:40.610Z"
_timesteps = "current,1h,1d"
_units = "imperial"
_timezone = "America/New_York"
response = requests.get(url, params={'apikey': _apikey,'location': _location,'fields': _fields,'startTime': _startTime,'endTime': _endTime,'timesteps': _timesteps,'units': _units,'timezone': _timezone})
responseText = response.text
jResp = json.loads(responseText)
timelines = jResp["data"]["timelines"][0]["timestep"]
temperature = str(jResp["data"]["timelines"][0]["intervals"][0]["values"]["temperature"])
windDirection = str(jResp["data"]["timelines"][0]["intervals"][1]["values"]["windDirection"])
bigResult = json.dumps(jResp)
return timelines,temperature,windDirection,bigResult
Here is the code ready in SID:
The next step is to click on the "Sync Variables" button. This will parse the code and populate the SID variables table with the input variables found in the "def execute (location_inp)" statement and output variables found in the 'Output:timelines,temperature,windDirection,bigResult' statement. The variables tab in SID will show the input and output variables for this Python code node.
5th Challenge - Testing the Decision Flow
This walkthrough assumes the reader is familiar with SAS Intelligent Decisioning and will not go into any depth on creating decision flows. This example continues with the "Python Weather API" Python Code Node being added to a basic decision flow diagram and the user starting the process of testing the decision flow using the "Scenarios" Scoring type.
Here is an example of the populated "scenario". Notice the input and output variables reflect the variables as defined in the Python code.
If the code executes correctly the following output should be presented:
Pro-tip is to use the "log" tab in SID when testing (scoring) a code node or a decision flow. Any errors will be presented in RED at the end of the code-submission section of the code log, especially Python-specific errors will appear here.
Finally
Hopefully, by the end of this, you managed to go through the steps and had a successful outcome. For me, it was a case of trying, researching a lot and re-trying. It is most likely that on your own you will have numerous goes at this. Python for example is full of programming quirks which may take time to iron out. Simple things in Python like indentations can trip you up. A good reference for Python code can be found here at w3schools.
