topic Re: SAS CIS communication export to kafka topic in Administration and Deployment

SAS CIS communication export to kafka topic

John_Wick — Tue, 13 Aug 2024 16:34:28 GMT

Hello, experts!

Challenge: A customer has a request with the request to send customer segments from CIS directly to Kafka topic.

Question: has anyone had experience in integrating SAS and kafka via a communication node? Perhaps someone can share ready-made solutions or suggest how it can be done in the best possible way with minimum number of steps.

So far we see a couple of options:
- export to json and console producer
- csv connector
- java profile with Kafka Producer

Re: SAS CIS communication export to kafka topic

Sajid01 — Fri, 16 Aug 2024 18:06:18 GMT

Hello @John_Wick
SAS event processing has an Apache Kafka Connector. SAS Help Center: Using the Kafka Connector and Adapter.
This is also another option Stream SAS Data Sets Data into Apache Kafka Topics (cdata.com)

I would appreciate if you can share your success story in connecting to Kafka from SAS.

Re: SAS CIS communication export to kafka topic

John_Wick — Wed, 06 Nov 2024 19:10:56 GMT

Hello @Sajid01

Now we have stopped at the integration of Java+sas in order to connect the communication node with kafka in SAS CIStudio. Ideologically it looks as follows:
1. At the SAS level in the additional characteristics selects a bunch of json template of a certain format + Kafka topic for each communication channel (email, sms, push) from the table-dictionary.
2. In the communication node (maexp script) the selected segment is written to the integration table
3. The segment is fetched from the integration table with a certain interval by the java profile.
4. The profile pulls up the json template required by the topic and enriches it with data for each customer
5. Then using Kafka template the generated json is sent to the required topic