BookmarkSubscribeRSS Feed
🔒 This topic is solved and locked. Need further help from the community? Please sign in and ask a new question.
upadhi
Quartz | Level 8

Hi all,

 

I am using ESP for the first time. below is the command  I am using to fetch data from CSV file:

 

$DFESP_HOME/bin/dfesp_fs_adapter -k pub -h "dfESP://sasserver.demo.sas.com:8701/SGF_Test_V2/sgf_cq/sgf_source" -f "/home/sas/data2_root.csv" -t csv -b 1 -r 1

 

I am using ESP 5.2

 

The issue is that the streaming of data is happening very slowly. What are the parameter /configurations that could be changed to get a faster stream of data.

 

Thanks and regards,

Upadhi

 

1 ACCEPTED SOLUTION

Accepted Solutions
ScottK
SAS Employee

If you remove the -r 1, but leave the -b 1, they will indeed be streamed in 1 at a time. If you only have a few rows, it will appear that they are all coming in at once, since ESP an handle multiple thousands of csv events per second. if you have only a small data set, experiment with -r N for various values of N. N is the target rate the adapter tries to maintain. For slow rates (< 1000) it is more approximate. If you have 1000 rows, and set -r 100, it should stream data for approximates 10 seconds. 

 

Hope that helps, 

Scott.

 

View solution in original post

4 REPLIES 4
FredCombaneyre
SAS Employee

Hi, 

just remove the -r 1 option. it sets the input rate to 1 event per second. 

upadhi
Quartz | Level 8

hi Fred, If i remove -r, all the records come in once and i can not see stream of data incoming. Thats why i kept -r to 1

 

FredCombaneyre
SAS Employee

They cannot be coming in once, even more that you force the block size to 1 (-b 1). This is just the refresh delay of the tool you are using for displaying the events that makes it look as such. 

ScottK
SAS Employee

If you remove the -r 1, but leave the -b 1, they will indeed be streamed in 1 at a time. If you only have a few rows, it will appear that they are all coming in at once, since ESP an handle multiple thousands of csv events per second. if you have only a small data set, experiment with -r N for various values of N. N is the target rate the adapter tries to maintain. For slow rates (< 1000) it is more approximate. If you have 1000 rows, and set -r 100, it should stream data for approximates 10 seconds. 

 

Hope that helps, 

Scott.

 

Whether you're already using SAS Event Stream Processing or thinking about it, this is where you can connect with your peers, ask questions and find resources.

 

Multiple Linear Regression in SAS

Learn how to run multiple linear regression models with and without interactions, presented by SAS user Alex Chaplin.

Find more tutorials on the SAS Users YouTube channel.

Discussion stats
  • 4 replies
  • 1017 views
  • 0 likes
  • 3 in conversation