SAS Data Integration Studio, DataFlux Data Management Studio, SAS/ACCESS, SAS Data Loader for Hadoop and others

Event stream processing - Best practice

Reply
Senior User
Posts: 1

Event stream processing - Best practice

Hi,
I have a problem with ESP flow implementation, which costs us a lot of RAM usage. As an input, I have a lot of raw records that are accumulating on daily, 7-days and 30-days level in realtime, and as a result, we have a huge amount of data. The doubt is whether to use stateful windows for these accumulated datasets or some persistent database (some in-memory DB). I would appreciate some best practices based on your experience with ESP, which methodology would suit the use case best? I am also considering that using persistent database between processing steps can help in failover, in case that the whole ESP is down.

 

Thanks,
Darko

Ask a Question
Discussion stats
  • 0 replies
  • 148 views
  • 1 like
  • 1 in conversation