ESP Pattern Windows

[#1]

One of the traditional differentiators of SAS DATA Step is its ability to compare field values across records using BY Group Processing (P.S. EEL do this too). In general, SQL does not do this easily or efficiently, requiring cursors and/or repeated self-joins.

The major disadvantage of all traditional cross-record analysis methods is that the data must be stored and analyzed all at once.  This makes real time processing difficult if not impossible.

SAS Event Stream Processing avoids this limitation by processing the data live as its created. The ESP Pattern window compares field values across events (rows) without storing any data on disk.

When using the Pattern window, you first define "Events of Interest" and then combine those events using logical operators into a pattern. To understand, think about the example of Broker Front Running. When front running, a stock broker buys some shares in a stock for him/herself. Then he/she buys a large quantity of the same stock for a customer. The size of the customer order "moves the market" raising the price of the stock. With the price now higher, the broker illegally sells his/her shares for a profit.

To detect front running, we need to look for three events of interest:

1. The broker buys some stock X for himself
2. The broker buys some stock X for a customer
3. The broker sells his shares in stock X for a profit

Once identified, we combine these events of interest with logical operators into a pattern that defines front running. In addition to the usual, AND, OR, and NOT, ESP has a special logical operator named FBY which translates to "Followed by." So we combine the above events of interest using the FBY operator. Thus the front running pattern is defined by event 1 followed by event 2 followed by event 3.

SAS ESP models are built using ESP Studio but looking at the underlying XML gives useful context here.  In XML, our front running model might look like this:

As the graphic shows, the XML defines the events of interest, the logic used to combine them into a notable pattern as well as the output event definition. Binding variables are used to compare variables across events.

Again, the pattern window (and all of SAS ESP) operates on live event feeds with no need store any data on disk.

The pattern window has many other capabilities not shown here. For more, please see the User's Guide.