Sorting data in SAS ESP

upadhi — Fri, 27 Sep 2019 15:10:21 GMT

Hello,

We want to group data according to an id being processed in the streaming data. Is there any way we can group data in SAS ESP? and Is there a way where we can pause data processing until all the parallel nodes are done processing one record, so that new record is taken only after all the parallel nodes have processed one record.

Thanks in advance.

Re: Sorting data in SAS ESP

ScottK — Fri, 27 Sep 2019 15:34:45 GMT

The second part of the question may be easier, so let's discuss that first. If you have a single source window where a event enters the ESP model, and splits out to parallel compute paths and is rejoined later on, you have what is called a fork-join. SAS ESP supports tagged token data flow which will do exactly what you want in this case, allow all forks to complete and a single result produced by the model where all the paths rejoin. In xml this is enabled by specifying the use-tagged-token='true' attribute on the project element. So you get what you want without pausing the processing at all.

The sorting question is more difficult in general for any streaming engine. Events that are produced by a streaming engine are normally supplied to a consumer as soon as the streaming engine finishes its computation. This means the streaming engine does not hold a bunch of events, and then a consumer issues a query against the window and gets a result set back. That is more of a database paradigm. If you want to talk through the use case you have that requires sorting data, I would be happy to work with you and see how it might be architected.

Re: Sorting data in SAS ESP

upadhi — Mon, 30 Sep 2019 09:50:58 GMT

Hey Scott,

Thank you so much for the help. It worked exactly the way we wanted.!!

topic Re: Sorting data in SAS ESP in Streaming Analytics

Sorting data in SAS ESP

Re: Sorting data in SAS ESP

Re: Sorting data in SAS ESP