Hi All
I am brand new to Data Remediation and need some help with getting started. Our design is as follows (steps 1-3 have already been developed, I need help with the following steps):
1. Data Quality assessment is run to identify any records that violate business rules
2. These records are written to a SQL table and surfaced in a view (v_AllRuleViolations)
3. SAS code is run to derive suggested remediation values for these rule violations, and the relevant data is written to a SQL table (Remediation)
4. Violation records, together with the suggested remediation value, are extracted from the Remediation table and sent to SAS Data Remediation
5. Workflow is instantiated:
- Data Steward accepts or rejects suggested remediation
- Accept:
- Remediation table updated (FinalRemediation=SuggestedRemediation and ProcessedDttm populated with current date and time)
- SQL script is invoked to apply final remediation to source data
- Reject:
- (Optional) Data Steward enters alternative remediation: populates Remediation table (AlternativeRemediation)
- Enters reason for rejection: populates Rejection table (Rejector = username, RejectionReason = reason entered, ValueRejected=SUGGESTED and RejectedDttm=current date and time)
- Workflow advances to next task:
- Alerts data owner
- Data owner may accept or reject either the suggested or alternative remediation, suggest another remediation or reject all remediation suggestions
- Accept suggested remediation
- Populates Remediation table (FinalRemediation = SuggestedRemediation, ProcessedDttm = current date and time)
- Invokes SQL script to apply Remediation.FinalRemediation to source data
- Accept alternative remediation
- Populates Remediation table (FinalRemediation = AlternativeRemediation, ProcessedDttm = current date and time)
- Invokes SQL script to apply Remediation.FinalRemediation to source data
- Enter alternative remediation
- Populates Remediation table (FinalRemediation = newly supplied value)
- Invokes SQL script to apply FinalRemediation to source data
- Reject
- Populates RemediationRejection table (Rejector = username, RejectionReason = reason entered, ValueRejected=ALL and RejectedDttm=current date and time)
- Manual intervention will be required – process tbc
If anyone is able and willing to assist me, please could you let me know? Unfortunately there is currently no Data Remediation training available, and the documentation does not provide enough guidance w.r.t. the integration between the assessment and remediation.
Thanks!
