Data Validating in Data management Studio

Shakti_Sourav — Tue, 13 Sep 2022 10:04:19 GMT

Dear Team,

I have created a data job which is under process job to identifying duplicate records.

I want to validate the data rows flag wise means which batch rows matched with Golden Flag.

I have attached screenshots for your references.

My data is like this,

Note: I have tried by Cluster Number, I want it like if cluster number count is repeated or having count >1.

Ex: Want data:

ID Name Marks Flag Matchcode Cluster Number Cond_1

want only Batch flag which is False, and if batch flag is true then will come similar duplicate data from Golden Flag.

Please let me know if required any further details

Thank You

Shakti

Re: Data Validating in Data management Studio

audrey — Wed, 14 Sep 2022 09:37:11 GMT

Hi,

Have you tried to use the Surviving Record Identification node?
It's meant to keep only one record per cluster, based on some conditions.

Hope this helps.