topic Re: Performance Issue in Data-Flux in SAS Data Management

Performance Issue in Data-Flux

Shakti_Sourav — Mon, 11 Apr 2022 09:58:41 GMT

Dear Team,

I am executing the Data-flux job having 3 crores record and It's taking long time to read.

I have attached one screenshot, Please suggest me how to handle performance size.

Thank You

Shakti

Re: Performance Issue in Data-Flux

audrey — Tue, 12 Apr 2022 12:27:32 GMT

Hi,

From what I see in the screenshot, you're reading from one dataset over 60M records, appending it with another data source. So the load is not so small for a start.

Then you're running Matchcode, Clustering and Surviving operations which also requires lots of ressources.

This is going to take some time! Depends what you mean by "long".

This would probably need to run on a Data Management Server, with appropriate ressources, rather than on a Studio.

Hope this helps.

Audrey

Re: Performance Issue in Data-Flux

RacheLGomez123 — Mon, 26 Dec 2022 07:09:51 GMT

Start by checking your wait stats
Ensure your TempDB database is configured optimally.
Make sure you're running index maintenance frequently.
Implement indexes that provide a benefit to your queries.
Check your most expensive queries and stored procedures.
Monitor your performance counters.
Regards,
Rachel Gomez