Streaming Live Weather Data into a SAS ESP Model

zephstemle — Tue, 02 Jun 2020 14:26:01 GMT

Weather is part of our every day lives. However, weather plays a critical role in business and industry. There are obvious industries that use weather data such as agriculture and outdoor entertainment. Weather data is also used by non-obvious industries such as energy, insurance, and police work.

For example, insurance companies use weather data to alert customers to avoid or minimize claims, manage reserves for high claim situations, and even flag suspicious claims. Weather data can even be used as evidence by police departments.

Live weather data is available from a variety of sources, including free and subscription accounts. There are several types of weather data available, as well. These include live weather, forecasts, historical weather, agricultural and environmental data, and even satellite imagery.

SAS Event Stream Processing provides a URL connector that can publish data to a model through the use of API calls to a weather server. I am using OpenWeather (openweathermap.org) for my examples. OpenWeather provides a set of free API calls. You must obtain an APPID from OpenWeather to use them.

APIs

The Application Programming Interface (API) allows you to send server requests to a data source server. This is done by way of the http protocol by adding endpoints to a URL. This is also known as an API call.

API Call Structure

The following is the structure of an API call to OpenWeather.

uniform-resource-locator?key1=value1&key2=value2&key3=value3…

The portion of the API call before the ? is the URL of the server to which we are making requests. The portion following the ? contains key-value pairs to pass parameters to the server. The following shows an example of an API call:

http://api.openweathermap.org/data/2.5/weather?zip=46142&appid=@APPID@&units=imperial

Single Location API Calls

Live weather data is available for a single location or multiple locations with a single request. The following are examples of single location API calls:

You can provide the Zip Code of a U.S. city as a parameter:

http://api.openweathermap.org/data/2.5/weather?zip=46142&appid=@APPID@&units=imperial

You can also use geographic coordinates (i.e., latitude, longitude) to specify a location:

http://api.openweathermap.org/data/2.5/weather?lat=-86&lon=40&appid=@APPID@&units=imperial

Finally, you can use City IDs in an API Call. OpenWeather provides a city.list.json file that lists the City IDs for locations globally.

http://api.openweathermap.org/data/2.5/weather?id=4258313&appid=@APPID@&units=imperial

Multiple Locations API Calls

You can also request multiple locations in a single API call. When you do so the JSON message returned includes an array of cities. One example of requesting multiple locations is by using several City IDs as a group:

http://api.openweathermap.org/data/2.5/group?id=4861716,4180439,5368361&appid=@APPID@&units=imperial

You can enter the latitude and longitude of a center point, followed by the number of locations you want, to form a circle of cities:

http://api.openweathermap.org/data/2.5/find?lat=-86&lon=40&cnt=20&appid=@APPID@&units=imperial

Here is a map of locations returned from this example.

You can also specify the coordinates of the lower left and upper right corners of a bounding box, and the zoom factor to request locations in a rectangle:

http://api.openweathermap.org/data/2.5/box/city?bbox=-86,40,-87,39,10&appid=@APPID@&units=imperial

Here is a map of the cities returned in the above example:

Publishing Data from a URL

ESP includes a URL connector to publish data from a URL into a model.

A source window is used in the model to provide a schema for the data and an input data connector (class is URL) that points to an external configuration file. The configuration file contains the actual URL connector. It requests the data using an API call, transforms the data, and publishes events to the model. If the JSON message contains an array (multiple locations), the configuration file uses event loop technology to transform the array into multiple events.

This GitHub page includes examples that you can execute.

Re: Streaming Live Weather Data into a SAS ESP Model

JJMajor — Wed, 03 Jun 2020 01:09:54 GMT

This is a great example - thanks so much for sharing!