Hello,
Has anyone implemented real time data monitoring using SAS DataFlux Data Management Studio?
I have build some Data monitoring jobs that runs in batch, looking for some guidance on how real time monitoring can be set up using MD studio.
Best regards,
Chetan
Hi Chetan -
Assuming that by "real-time monitoring" you mean capturing data metrics or violations on a per-record basis through a web service, you have already done most of the work. By creating a variation of your batch process that uses an External Data Provider node instead of a data source, you will have created a job that can be deployed as a web service. Upload that new job to Data Management Server and then you can call that job as a web service (SOAP or REST based, your choice).
If you want to differentiate the results that end up in your data monitoring repository as real-time versus batch sources, you can add new data elements as properties to the business rules (and selectable in the Data Monitoring node) to call out the systems, owners, priorities, or statuses associated with the system that made the web service call. These can be parameterized too such that you can pass the parameters into the web service from the web-service invoking system when the call is made.
Ron
Hello Ron,
First thanks for your time and response to my post. By real time, I mean if any new record is added to database that meets business rule defined, I would like to send an email or write that record to a separate table.So business users are notified quickly and they will take action asap.
Your business rules in combination with the Data Monitoring node can take care of this. Review this page in the Data Management Studio User's Guide. When a business rule is triggered, you can use events to write to tables and send emails.
Thanks Ron.
I will try to implement this and get back to you if I have any questions.
Best regards,
Chetan
Hello Ron,
Here is what I am trying to accomplish:
I have a table with few million records which gets updated in real time. I have to monitor value of one column and if I find a record with that value, I need to report it or write it to a table with create date and derive a real time dashboard.
What I have Done so far:
I set up fields, tasks and business rule using SAS Data Management Studio.
I have set up data job which read from table and uses monito node which in result triggers a event which essentially write that record to a table.
My SQL currently checks for any new records that were added in the last five minutes, so reporting is always on new records added in last five minutes.
Question:
For real time monitoring, do I need to run this job every five minute or is there a way to inject this at database level?
Regarding dashboard that I talked about, do you recommend on using DM Studio for this or I should look for some other tool?
Any inputs/help here is greatly appreciated.
Best regards,
Chetan
Join us for SAS Innovate 2025, our biggest and most exciting global event of the year, in Orlando, FL, from May 6-9. Sign up by March 14 for just $795.
Need to connect to databases in SAS Viya? SAS’ David Ghan shows you two methods – via SAS/ACCESS LIBNAME and SAS Data Connector SASLIBS – in this video.
Find more tutorials on the SAS Users YouTube channel.