DataFlux - How to archive profiling reports

[smicha]

Hello Community,

I have read in the link below, about archiving profiling reports. How can this be achieved:

• http://support.sas.com/documentation/onlinedoc/dfdmstudio/2.7/dmpdmsug/Content/dfU_T_Profile_perform...

I use Microsoft SQL Server as the data storage for my repository.

Thank you in advance,

smicha

[Rama_V]

I hope this blog should give you some ideas

https://blogs.sas.com/content/sgf/2016/02/11/share-dataflux-data-quality-profiling-metrics-in-sas-vi...

Once you extracted the results stored elsewhere, you can clean the repository tables, especially for profiling related tables.

[afroditiz]

@Rama_V 

In continuance of the thread, is there an exhaustive list of profiling-related tables stored in the repository, so that someone proceeds to cleaning them? Also, by cleaning you mean deleting or truncating those tables?

Because I have a case where a FRQDSTxxx table never got created for some reason (FRQDSTxxx-1 & FRQDSTxxx+1 are created), and i would like to examine the possibility "cleaning" the whole repository for profiling only.

Plus, does deleting the profile history inside DMS also delete the FRQDSTxxx tables or not?

Thank you very much

[Rama_V]

You need to look at the Repository schema, accessing it shown in the blog link my previous reply.  Rule executions, pattern distributions etc stores row logs which takes lot of repository space . 

I am not sure this is recommended approach for cleaning. I never truncated the DMS tables. If you do need truncate then I am not sure how the relationships in the repository tables will behave. So need some caution around that especially folders,profile jobs, business rules etc. 

If I am deleting and recreating the repository then I usually export the jobs, business rules, profile jobs and all the results are extracted and loaded into a Data mart. This way I have the results ,Jobs and business rules which will be used to recreate my repository. If using RDBMS as repository then its even better to use scripts to create the repository. The import the exported objects back into repository.

[VincentRejany]

Hi

Find in attachment the process job mentioned in the blog. It allows to extract data profiling metrics from a DataFlux repository (file or db).

Hope that helps

Vincent

[afroditiz]

Hi @VincentRejany thank you very much for your replies.

It seems that the problem I am facing was irrelevant with FRQDST dynamic construction logic, but it's good to know more information about cleaning.

I can't see the attachment, but this is the job I am using as well based on the link you have sent 🙂

Regarding the repository "migration" part as mentioned in your previous reply, I guess that, having the repository in RDBMS, in case of a new environment I would just have to export/import profiles, rules etc. and define the repository definition to 'look' at the RDBMS, so I imaginge no further action would be required for profiling tables.

[VincentRejany]

Strange that the file has been removed

[VincentRejany]

And yes you can import export data profiling jobs (not reports), rules from on repository to another (file or db)