BookmarkSubscribeRSS Feed
🔒 This topic is solved and locked. Need further help from the community? Please sign in and ask a new question.
upadhi
Quartz | Level 8

Hello,

 

We want to group data according to an id being processed in the streaming data. Is there any way we can group data in SAS ESP? and Is there a way where we can pause data processing until all the parallel nodes are done processing one record, so that new record is taken only after all the parallel nodes have processed one record.

 

Thanks in advance.

1 ACCEPTED SOLUTION

Accepted Solutions
ScottK
SAS Employee

The second part of the question may be easier, so let's discuss that first. If you have a single source window where a event enters the ESP model, and splits out to parallel compute paths and is rejoined later on, you have what is called a fork-join. SAS ESP supports tagged token data flow which will do exactly what you want in this case, allow all forks to complete and a single result produced by the model where all the paths rejoin. In xml this is enabled by specifying the use-tagged-token='true' attribute on the project element. So you get what you want  without pausing the processing at all.

 

The sorting question is more difficult in general for any streaming engine. Events that are produced by a streaming engine are normally supplied to a consumer as soon as the streaming engine finishes its computation. This means the streaming engine does not hold a bunch of events, and then a consumer issues a query against the window and gets a result set back. That is more of a database paradigm. If you want to talk through the use case you have that requires sorting data, I would be happy to work with you and see how it might be architected.  

View solution in original post

2 REPLIES 2
ScottK
SAS Employee

The second part of the question may be easier, so let's discuss that first. If you have a single source window where a event enters the ESP model, and splits out to parallel compute paths and is rejoined later on, you have what is called a fork-join. SAS ESP supports tagged token data flow which will do exactly what you want in this case, allow all forks to complete and a single result produced by the model where all the paths rejoin. In xml this is enabled by specifying the use-tagged-token='true' attribute on the project element. So you get what you want  without pausing the processing at all.

 

The sorting question is more difficult in general for any streaming engine. Events that are produced by a streaming engine are normally supplied to a consumer as soon as the streaming engine finishes its computation. This means the streaming engine does not hold a bunch of events, and then a consumer issues a query against the window and gets a result set back. That is more of a database paradigm. If you want to talk through the use case you have that requires sorting data, I would be happy to work with you and see how it might be architected.  

upadhi
Quartz | Level 8

Hey  Scott,

 

Thank you so much for the help. It worked exactly the way we wanted.!!

Whether you're already using SAS Event Stream Processing or thinking about it, this is where you can connect with your peers, ask questions and find resources.

 

Multiple Linear Regression in SAS

Learn how to run multiple linear regression models with and without interactions, presented by SAS user Alex Chaplin.

Find more tutorials on the SAS Users YouTube channel.

Discussion stats
  • 2 replies
  • 1211 views
  • 2 likes
  • 2 in conversation