Performance Issue in Data-Flux

[Shakti_Sourav]

Dear Team,

I am executing the Data-flux job having 3 crores record and  It's taking long time to read.

I have attached one screenshot, Please suggest me how to handle performance size.

Thank You 

Shakti

[audrey]

Hi,

From what I see in the screenshot, you're reading from one dataset over 60M records, appending it with another data source. So the load is not so small for a start.

Then you're running Matchcode, Clustering and Surviving operations which also requires lots of ressources.

This is going to take some time! Depends what you mean by "long".

This would probably need to run on a Data Management Server, with appropriate ressources, rather than on a Studio.

Hope this helps.

Audrey

[RacheLGomez123]

Start by checking your wait stats
Ensure your TempDB database is configured optimally.
Make sure you're running index maintenance frequently.
Implement indexes that provide a benefit to your queries.
Check your most expensive queries and stored procedures.
Monitor your performance counters.
Regards,
Rachel Gomez