Hi IoT Streaming Analytics Community,   Dedicated to highlighting the innovative ways SAS Event Stream Processing (ESP) empowers customers, this new web series, IoT Innovators, will explore the ideas, designs, and implementation for cutting-edge IoT solutions to help you accelerate your time to value on IoT projects.      Episode 1: From Pixel to Actionable Insights   Available on-demand, the first episode host Brandon Kooman (@bkooman) discusses computer vision and streaming analytics with experts, Juthika Khargharia (@Juthika) and Tom Tuning (@TomTuning) – sharing real-world examples, field challenges and advances they foresee the industry making in the next few years.   From practical use cases to groundbreaking advancements, this new series promises insights you won’t want to miss. Tune in to gain valuable insights that will help shape the way you think about SAS and IoT.  2.sas.com/61105eh4KX     ... View more

SAS Analytics for IoT is a complete AI-embedded IoT analytics solution that covers the entire IoT analytics life cycle. It was created for all types of users --business users, engineers, data scientist and IT professionals -- who want to accelerate time to value (new insight, fast & accurate decision making, desirable outcomes) from their use cases. The solution enables them to organize their diverse IoT data, create data selections, visually explore massive volumes of high frequency data, and launch data sets that can be leveraged in SAS, third party and open source applications. With this capability, organizations can extend the use of IoT analytics and collaboration across the enterprise, while optimizing their ecosystem of IoT investments –SAS, third party and open source.   Check out this GitHub repo to learn more about SAS Analytics for IoT and its different components. ... View more

Check out this GitHub repo to learn how the Japan IoT Team used an accelerometer, a simple rotating fan, and an SVDD model to demonstrate anomaly detection capabilities.  This demo was developed to be an way to demonstrate anomaly detection using a real-time vibration data and SAS ESP.  Input data is flowing in via MQTT at a rate of 12.8 kHz with RMSA calculated every second.  After that, the SVDD model for is used for scoring. (Note: The SVDD model was trained using SAS Studio)   As shown in the video below, we made a device with 2 rotating fans (one with normal fan and another with a blade broken) and used SVDD to detect when the condition of the machine (fan) changes. The output is SVDD distance (top graph) and SVDD score (bottom graph). When the normal fan is rotating, the score is -1 (normal state) but when the device is switched to the broken fan, the SVDD score changes to 1 (faulty state).       Below is an instructional video on how to built your own fan simulator device that you can switch between normal and failure modes. The team attached an accelerometer to gather the vibration data and send to an ADLink USB-2405 DAQ.  (Note: It is also possible to use ADLink MCM-100 DAQ (edge device) which is basically a DAQ device and also a PC)   ... View more

SAS ESP can be a bit overwhelming at first. The functionality is so vast that knowing where to begin is difficult. One of the best learning techniques is learning by example. The SAS Event Stream Processing Studio Examples GitHub repository provides several examples of ESP code that can be used in various scenarios. You can even edit the provided code and try things yourself.    There are 42 examples in the repository divided into four categories:  Getting Started  Basic  Advance  Analytics    The Getting Started and Basic examples are a great place to start. Each example covers a specific ESP technique and uses  a single ESP window. The Advanced examples are more complex and involve the use of multiple ESP windows. These examples are closer to “real-world” examples.   ESP supports numerous algorithms for performing analytics on streaming data. The Analytics section of the repository includes an example for many of these algorithms. The SAS Help Center contains step-by-step instructions for many of the Analytics examples.    Be sure to check out this GitHub repository as these examples will surely improve your ESP skills!  ... View more

One of the best features of SAS ESP is that it can be integrated with many third-party environments. One of these environments is Microsoft Power Automate.    Microsoft Power Automate (formerly Microsoft Flow) allows you to create automated workflows that use a trigger and then perform an action. It has 500 flow connectors to the entire Microsoft family, several third-party products, and any publicly available API. You can use ESP as the trigger to execute a flow or use ESP to receive events as the action portion of the flow.      Azure Event Hubs is a data streaming platform and event ingestion service. It sits between event publishers and event consumers and manages the event stream between the two. The above diagram illustrates Event Hubs providing bi-directional flow between ESP and Power Automate.    The Integration of SAS Event Stream Processing with Microsoft Power Automate using Microsoft Azure Event Hubs GitHub site provides the information you need to connect ESP with Power Automate through Azure Event Hubs including:  Prerequisites  Installation  Finding your Event Hubs connection strings  Connecting ESP to Azure  ... View more

Munehisa Homma was a rice merchant from Sakata, Japan who traded in the Dojima Rice market in Osaka during the Tokugawa Shogunate (18th century). He is sometimes considered to be the father of the candlestick chart, a form of technical analysis used in stock markets. He earned the nickname “the god of the markets” while accumulating a rumored net worth of $100 billion in today’s US currency.  Homma discovered that the price of rice was not only dictated by supply and demand, but also by the emotions of traders. Greed, fear, anxiety, and excitement can all contribute to market psychology. Market psychology is used to explain market movement that may not be explained by other metrics.    As previously stated, Homma is considered the father of the candlestick chart. A candlestick chart, or Japanese candlestick chart displays price movements over time. A candlestick contains the important information for one trading period (open, high, low, and close values). The trading period can be one day, 15 minutes, one hour, or any trading period as long as each candlestick represents the same time period. The candlestick chart identifies repeating patterns and trends.    SAS Event Stream Processing (ESP) is very powerful when it comes to detecting patterns of interest in real time. However, before any patterns can be detected the ESP model must create groups of values for each trading period (e.g., 60 minutes) and then find the open, high, low, and close values for each group (candlestick). This creates a continuously updating virtual candlestick chart.    ESP then analyses the virtual candlestick chart looking for specific patterns and alerts the user when a known pattern is detected. This allows the user to respond quickly. The patterns used in this project include Engulfing Bearish, Bullish Engulfing, Three Inside Up, and Three Black Crows. Each pattern includes its own forecast for short term price movement of the market.    This project is a great way to learn about creating groups, aggregating values by group, and detecting patterns using ESP. You can download the project from GitHub and execute the project yourself. Be sure to check it out.  ... View more

Global resources suffer a cost of around $1.6 trillion per year due to money laundering efforts across the world, according to the UN United States’ banks spend between 0.4% and 2.4% of their total operating expenses trying to prevent fraud and money laundering. Additionally, millions are being spent on compliance with the Treasury Department’s Customer Due Diligence (CDD) Rule.    This example illustrates how SAS Event Stream Processing (ESP) can detect possible fraud and money laundering transactions without the use of advanced analytics. The detection is performed using data comparison and analysis with rules, predominantly in sliding, temporal windows. This allows ESP to look for patterns in specific time windows.    There are three fraud and anti-money laundering rules associated with this example:  Four consecutive “cash out” transactions by the same customer in a 10-minute span. If this occurs ESP issues a decline message.  Many to one fund transfers. If three or more distinct customers are sending transactions to the same beneficiary in a one-day period, ESP issues a decline message  Transactions originating or terminating from a high-risk location. If a beneficiary name is flagged after being compared to a restricted list, ESP issues an on-hold message indicating further investigation is needed.    The Streaming Fraud and Anti-Money Laundering Simple Example GitHub page has all you need to execute this example yourself. Be sure to check it out.  ... View more

Potholes are the result of weakened pavement from the expansion and contraction of water over time. They can form in all sorts of different environments due to natural conditions like seasonal shifts between winter and spring, which means all roadways are subject to damages caused by potholes.   The financial impact of potholes and other effects of under-maintained roadways has a real impact on the wallets of motorists across the U.S. Each year potholes are causing an estimated $3 billion in blown tires, busted axles, and other collateral damages. A 2016 study from AAA that analyzed data from the previous five years, estimated that around 16 million drivers across the U.S. suffered damage from a pothole. As costly repairs to cars and trucks increase from the reduced budgets of state governments to repair roads, and as more drivers use highways, public roadways are in more need than ever for significant improvements.   According to the same AAA survey, lower-income individuals are most affected by potholes damages because oftentimes a costly bill can have a substantial impact on their financial stability. Once they suffer damage, they are less likely to repair their vehicles, adding to the potential for unsafe driving conditions for themselves as well as for other motorists. Damage to vehicles causes other problems too as damaged cars often pull over to the side of the road where they can sit for long periods of time before removal. This increases the probability of vehicular accidents, and increases traffic congestion and gas consumption.   Municipalities have developed some ways to prevent potholes from forming in city streets like crack sealing, road patching, and overlays, and these preventative efforts can help reduce damage to vehicles and costly roadway repairs. The main problem is that the same municipalities don't have ways to timely identify the areas that need intervention.   This demo shows how visual detection technology can dramatically help cities keep road maintenance costs down and improve overall public safety by identifying and ranking potholes based on their condition so that repairs can be pre-emptively executed. The solution is cost-effective because it uses cameras mounted on garbage trucks, which already drive through cities to pick up waste. As they follow their routes, each pothole they find is scored and identified via GPS coordinates. This information is used to feed an ESP project and a visualization component that displays the potholes on a map.   Click here to watch a video overview of the demo to learn more.     ... View more

Remote patient monitoring (RPM) uses digital technologies to collect medical and other forms of health data from individuals in one location and electronically transmit that information securely to health care providers in a different location for assessment and recommendations. Remote Patient Monitoring has benefits for both the patient as well as the healthcare systems relative to the following: Speed and Accuracy Faster Diagnosis and Treatment 24/7 monitored health Efficient and preventative diagnostics and treatments Reduce patient visits, organized health checkups and well manages healthcare Automatic alerts to the concerned family members/doctors/GP etc. Parkinson's disease is a progressive nervous system disorder that affects movement. Symptoms start gradually, sometimes starting with a barely noticeable tremor in just one hand. Tremors are common, but the disorder also commonly causes stiffness or slowing of movement. Parkinson’s disease symptoms are most often charted using the MDS-UPDRS. Limitations of this approach include the subjective character of the assessments and a discrepant performance in the clinic compared to the home situation. Continuous monitoring using wearable devices is believed to eventually replace this golden standard, but measurements often lack a parallel ground truth or are only tested in lab settings. Through this demo we seek to add a quantitative aspect to monitoring Parkinson’s progression. If a patient is diagnosed with Parkinson's, this demo will help monitor the progression over time as well as what/if any effect medication is having on the patient.   Additional details about the demo can be found here or on GitLab.  ... View more

The Global Positioning System (GPS) is a satellite-based navigation system made up of at least 24 satellites. These GPS satellites orbit the earth in a precisely defined and monitored orbit. Each GPS receiver measures the time it takes for each satellite’s signal to arrive, and the exact location of the receiver can then be calculated through multilateration.   GPS spoofing occurs when a radio transmitter (e.g., software defined radio, as shown in this demo) transmits a counterfeit GPS satellite signal that is stronger than the signal from the corresponding actual satellites. The GPS receiver will preferentially use the counterfeit signal, because it is stronger than the signal from the actual satellite. This counterfeit signal includes a false time base, which fools the GPS receiver into calculating that it is in a different location.   Why is spoofing an issue? Many applications, such as a ship’s navigation system, require precise location and/or precise time. For example, with just a laptop, a small antenna, and a GPS “spoofing” device, a group of college students fed a stronger signal to a yacht’s steering system than the genuine one from actual GPS satellites (article). By doing this, they essentially tricked the ship’s computer into believing it was somewhere it was not, causing it to adjust its heading to stay on course. A nefarious actor could do the same to cause a ship to collide with another ship or a known hazard, with disastrous consequences.   The cost of spoofing is hard to calculate. However, downtime of critical infrastructure that relies on an accurate location or time is a bigger issue than just a financial one. If spoofing goes undetected, applications that rely on location and time will produce inaccurate results or crash. Therefore, the detection of spoofing as soon as it occurs is paramount.   Click here to learn how SAS addressed this challenge using SAS ESP.  ... View more

Food loss occurring between the farm and the market results in over 100 Billion pounds of wasted food every year. This equates to a financial loss of over $100 Billion. One cause of food spoilage is equipment malfunction during transportation, which can result in a lack of adequate climate control. Obviously, temperature and humidity can be good indicators of a potential problem with a food shipment, but with more types of sensors, such as those sensitive to different gasses, it may be possible to discover a problem before a significant amount of food is compromised. Using a combination of inexpensive sensors and controllers, Azure IoT services, and SAS Analytics, we have developed a demonstration of how these pieces can come together to yield timely alerts for food beginning to spoil. IoT applications almost always have both edge and cloud components: individual assets are monitored at the edge, while fleets are monitored and analytics are developed and trained in the cloud.   Key take-aways from the demonstration: Learn how to design a streaming model for real-time food spoilage detection Learn how to create an ESP model on the edge and in the cloud Best practices for integrating ESP on edge in in the cloud with Azure IOT Services     Click here to listen to a video overview of the demo!  ... View more

In this GitHub project, we will share different reference architectures of ESP and how one can evaluate which one out of these is the best fit for your customer/partner use-case. Figure 1 below provides the steps that one must follow to determine the right architecture for their business problem. NOTE: In this project we do not provide any guidelines on the ESP project modeling.         Figure 1: Steps to Determine the Reference Architecture   2. Determine the Use Case Requirements To determine the right architecture, it is important to understand the customers' long-term business objectives. The below sections explains The different types of design requirements to consider and their importance, Questionnaire helps to ask the right questions to determine the right requirements, and Following the flowchart/Decision-Tree to reach the architecture that fulfills all the business goals.   Understanding the ESP Architecture Design Requirements Figure 2 presents the non-functional, functional, infrastructure, and streaming input/output requirements for a use-case to determine the right ESP architecture.   Non-Functional requirements define the quality attributes of the applications, i.e., ESP. These requirements allow the architect to impose constraints or restrictions on the design of the overall architecture.   Functional requirements define a system and its components that are required by the application to perform the desired behavior. They primarily are derived from the use-case objectives, what will the input data sources and outputs, what should be the format of the data in consideration, how the application must process it, and finally, verifying the correctness of the obtained output. It is usually defined in terms of calculation, transformations, data manipulation, business processes, user interaction, and various tasks to be performed by the ESP.   By Infrastructure requirement we mean the platform to be used for deploying the ESP. Using Cloud adds several benefits as the cloud-native services can be leveraged and many 3rd party tools/applications in need can be managed services provided by the Cloud platforms. This comes at a cost.   We will also look into Streaming Input/Output which means how from where and what format of the input data/events are streamed to the ESP and where the output results must be streamed to. We also consider the volume, velocity, and variety of the streaming events as well.   In this GitHub project, we will focus on the non-functional and streaming input/output design requirements to determine the right infrastructure and select the right architecture.   Figure 2: ESP Architecture Design Requirements   Definitions of Non-Functional Requirements The ESP reference architectures use the standard definitions from Cloud Computing (Distributed Systems).   Real-time Streaming Architecture Design Guide Questionnaire During designing or architecting a solution, we must ask questions! A lot of questions! The Streaming Architecture Design Guide Questionaire provides a comprehensive list of questions one must ask to architect a complete solution based on the provided business requirements.   3. Decision-Tree to Determine the Reference Architecture Based on the Requirements Figure 3 presents the Design-Tree/Flowchart which is built using the possible business requirements a customer/partner would have. The flowchart helps in determining the right ESP reference architecture that fulfils the business needs. It is also possible to have mix of many architectures, if many paths in the flowchart are realized. It is extremely necessary to have the complete list of requirements to obtain the most suitable architecture.   Figure 3: Design Tree to Determine the Reference Architecture   4. Reference Architectures 01. Single-Node, Single ESP Server Deployment 02. ESP with traditional High Availability with and without Kubernetes 03. Multi-Node, ESP Operator Kubernetes Framework 04. Multi-Node, ESP with Kubernetes using Kafka 05. Multi-Node, ESP with Kubernetes using Load Balancer   ... View more

  If you are a user of Microsoft Power Automate or Microsoft Dynamics 365, and you want to integrate with SAS ESP check out this GitHub repository on how to do it!  The repository presents the concepts of integrating the Microsoft products with ESP using the Azure Event Hubs as well as provides the configuration steps needed to make the connection. The above diagram depicts the architecture for connecting ESP to Power Automate by way of an Event Hub. Microsoft Power Automate is an automated workflow tool that you can use to automate paperless tasks. Event Hubs is Azure’s real-time data ingestion service that serves as a pipeline from ESP to Power Automate.   There are two options for connecting ESP to the Event Hub, the SAS ESP Event Hub Connector, or the SAS ESP Kafka Connector. The SAS ESP Event Hub connector requires two things: The path to the Event Hub The connection string   The GitHub repository includes the steps to configure and use of the SAS ESP Event Hub connector to connect ESP to the Event Hub. There is another option, the SAS ESP Kafka connector. SAS ESP can integrate with Microsoft Azure Event Hub by leveraging the Kafka broker. Kafka performs publishing and subscription tasks.   Be sure to check out the GitHub site. It is chocked full of information and steps to get you up and running. ... View more

Deploying real-time analytics projects at scale requires two main components: having an approach so simple that a subject matter expert can configure and deploy projects, and extensive automation that transforms simple configuration files into complex deployment-ready projects, complete with state-of-the-art machine learning models. With these principles in mind, the Streaming Analytics Framework was developed as an overlay to VIYA and Event Stream Processing. Users create a visual map of their devices and sensors and then configure real-time updating analytic base tables in a few simple steps. Machine learning models are automatically trained and managed based on the specification of just the target and predictor variable list. Streaming Analytics Framework then automatically builds the Event Stream Processing project which can be deployed in minutes.   The approach has been used successfully for projects focused on energy monitoring, prediction of device failure, optimization of manufacturing plant settings, and is also being positioned for analysis of electronic health records and disaster aversion.   This video introduces the framework and walks you through a real-world example to help you understand the approach. If you are interested in accelerating the deployment of real-time analytics projects for your organization, please see the attached services brief and reach out to our today! ... View more