AndyT_SAS Tracker

Re: Authentication when using the URL_Connector to query data from a REST API

AndyT_SAS — Wed, 31 Jan 2024 16:58:36 GMT

I don't think the url connector is expecting user/password information in the url. I will submit a request to update ESP to support that. What release of ESP are you using? That will help us find a work around.

Re: SAS ESP 5.2 Oracle pub/sub connector fails to start

AndyT_SAS — Wed, 27 Mar 2019 11:40:38 GMT

ESP Studio is reflecting the message from the ESP XML server about the connector not starting. Can you try running without the -logconfig option to see if there are any additional messages about the connector not starting?

Re: SAS ESP 5.2 Oracle pub/sub connector fails to start

AndyT_SAS — Tue, 26 Mar 2019 18:29:03 GMT

There are normally error messages that go with the "connector failed to start". The DB connector/adapter will try to connect to the database as part of the start up process.

This could be because of how the logging is set up. What do you have configured in the esp.logger file?

Re: How can I stream Twitter data in SAS ESP 5.2

AndyT_SAS — Thu, 14 Mar 2019 14:00:38 GMT

You can use the Twitter Publisher Adapter to publish tweets into an ESP model. Here is a blog post that shows more details on using the twitter adapter and the ESP model that can be designed to process the tweets.

Re: removing Opcode and eventflag chars in CSV output

AndyT_SAS — Mon, 21 Jan 2019 14:33:38 GMT

Hi Ebele,

The file/socket connector does not allow an option to remove opcode or the flags from the format. What is the intended target of the events? Who will process the data? Is the model only going to output insert events?

Thanks,

Andy

Re: removing Opcode and eventflag chars in CSV output

AndyT_SAS — Tue, 15 Jan 2019 17:53:09 GMT

Hi Ebele,

There is not a way to remove the opcode or event flag from the CSV output. The format was designed so that it could be read back in by another ESP source window.

Thanks,

Andy

Re: Discarding duplicate events produced by QKB matchcode function

AndyT_SAS — Thu, 03 Jan 2019 16:39:47 GMT

Hi Rain,

This is a little tricky. How many match codes could you potentially get? An aggregate window could group events by the matchcode. But you would need to worry about storage growth in the aggregate window because nothing will remove the old events.

You could write a Python or DS2 routine with a hash table that uses the matchcode has the hash key. Then, you could keep a record of the matchcode values that have been processed. An event would only be output if it was a new matchcode. That wouldn't have any memory concerns, as you could make the window pi_EMPTY.

Thanks,

Andy

Re: What would be the best approach to handle offline situations for ESP on edge devices?

AndyT_SAS — Fri, 09 Nov 2018 12:02:33 GMT

Hi Ebele,

I see you opened a track for this. We will work with you on the track, then update this with the solution.

Thanks,

Andy

Re: Convert Procedural DS2 code into MAS module and run it in Calculate window

AndyT_SAS — Mon, 15 Oct 2018 19:29:49 GMT

In ESP, a date or a timestamp is stored as a int64. A date is the number of seconds since Unix epoch. A timestamp is the number of milliseconds since Unix epoch. So, it should be able to be passed as a bigint to DS2, which will correctly hold the value. I'm not sure why the model won't start when LAST_TO_BOUGHT_DTTM is set to bigint. Can you add the model so I can take a look? Or we can create a track for this, if you would like?

Re: Convert Procedural DS2 code into MAS module and run it in Calculate window

AndyT_SAS — Fri, 12 Oct 2018 17:03:27 GMT

The timestamp type isn't supported as a DS2 data type by MAS. I think there is a debug message about this if you turn up the debug levels enough. Here is a table for the conversion between ESP data types and MAS datatypes for DS2 and Python

https://go.documentation.sas.com/?docsetId=masag&docsetTarget=n0nsznpd1r9wd8n1sd6qsm8efyjg.htm&docsetVersion=5.1&locale=en

You want the timestamp to be a bigint.

Re: Hex values in ESP

AndyT_SAS — Wed, 10 Oct 2018 16:16:45 GMT

There isn't a built in function for this in the functional window or compute window. You could use a python window or write a C++ function to do this.

Re: SAS ESP reading data from Apache NiFi

AndyT_SAS — Tue, 09 Oct 2018 17:00:28 GMT

I would open a new question. That way the discussions don't get confusing for someone else who finds this thread in the future.

Re: SAS ESP reading data from Apache NiFi

AndyT_SAS — Tue, 09 Oct 2018 14:31:14 GMT

Hi Sinisa,

I'm not sure what is happening here. I'm going to create a track for you and we can investigate it from there. You will get an email from the track.

Thanks,

Andy

Re: SAS ESP reading data from Apache NiFi

AndyT_SAS — Fri, 05 Oct 2018 14:28:22 GMT

You can get the AVRO format of an ESP window's schema using this REST request:

http://hostname:http_port/SASESP/window?name=xxxxx&schema=true&avro=true

Substitute the window name you are trying to publish to for xxxxx.

But if the Record Schema isn't specified, ESP should connect to the running project and obtain the schema for you. Is the ESP project up when you try to start the processor, it should connect ok. What error message do you get?

Converting an Event Stream Processing DS2 table server mode function to run as a MAS module

AndyT_SAS — Wed, 12 Sep 2018 20:55:57 GMT

In Event Stream Processing(ESP) 5.2, support for the Procedural window input handler that used DS2 code run in table server mode is deprecated. This means that to migrate from previous releases of Event Stream Processing to release 5.2, the DS2 table server mode code must be changed to run as a MAS module. The MAS module then needs to be moved to the calculate window. The following code snippet changes the opcode of incoming events. Here’s how to convert it from using DS2 to MAS.

Re: Generate a message when there is no message

AndyT_SAS — Wed, 05 Sep 2018 18:24:57 GMT

Hi Ebele,

Sorry, I had the timer in the wrong spot. This should work for you:

fby(e1, not{10 seconds}(e2))

Every time an e1 is found, the 10 second timer will start. If another event arrives(e2), the pattern will collapse with no output. If the timer pops after 10 seconds, then the pattern will complete and output an event.

The timer connector is documented here. Hopefully the pattern window by itself will solve this issue.

Thanks,

Andy

Re: Generate a message when there is no message

AndyT_SAS — Tue, 04 Sep 2018 17:08:19 GMT

Hi Ebele,

I think a pattern window could help with this. The pattern would use clock time and be specific to each sensor by setting the pattern index to the sensor id. This way your pattern has an instance for each specific sensor. The pattern would be "fby{10 seconds}(a,not(a))" where event "a" is just an event from the input window. That pattern would complete after 10 seconds of no activity, which would output an event you could use to send a message in a notification window. If another event from the sensor came in before 10 seconds, the pattern should collapse and produce no output event.

Pattern windows can be tricky to get the patterns just right, so this would take some testing to validate this pattern is correct.

There is also a timer connector/adapter, which can publish an event at a specified interval. That could be helpful, but there would need to be some thing else to maintain a field for the last heard from time for a sensor and then look for sensors that haven't responded within 10 seconds when a timer event comes in. I think that would need to be DS2 or Python code.

Thanks,

Andy

Re: Combining Output of various scenario - SAS ESP

AndyT_SAS — Wed, 15 Aug 2018 16:50:47 GMT

Hi Sandeep,

ESP can combine multiple outputs in a few ways. You could use multiple join windows to join everything back together after running each scenario in parallel on the data. You could also use a pattern window with a pattern that detects each of the scenarios produced an answer, then output an event.

I'm not sure I understand the second part of your requirement, "Also for multiple outputs of different scenario we want to prioritize single output."

What do you mean by prioritize single output? Would the same input event create multiple output events for the same scenario?

Thanks,

Andy

Re: I want to connect OPC UA Server Using SAS ESP. How to establish connection ?

AndyT_SAS — Thu, 09 Aug 2018 18:17:12 GMT

Tech Support(TS) investigated this. The commas in the nodeid were causing issues parsing the nodeid. A patch is available from TS to support a nodeid with commas in it. Contact SAS Tech Support if you need the patch.

How to parse a string field containing a timestamp or date with the functional window

AndyT_SAS — Tue, 03 Jul 2018 19:53:47 GMT

In many SAS Event Stream Processing(ESP) models, the connector or adapter can convert a date or timestamp while reading data from the source. However, if the model reads in an opaque string of XML or JSON or a base64 encoded string, the date or timestamp might be in a string field, like 2018-05-28T09:33:55.670Z. To convert the string field into a date or timestamp field, use a functional window. Here is a simple example of how to implement the functional window. The complete model and input data are attached.