BookmarkSubscribeRSS Feed
smicha
Obsidian | Level 7

Hello Community,

 

I have read in the link below, about archiving profiling reports. How can this be achieved:

I use Microsoft SQL Server as the data storage for my repository.

 

Thank you in advance,

smicha

7 REPLIES 7
Rama_V
Obsidian | Level 7

I hope this blog should give you some ideas

https://blogs.sas.com/content/sgf/2016/02/11/share-dataflux-data-quality-profiling-metrics-in-sas-vi...

 

Once you extracted the results stored elsewhere, you can clean the repository tables, especially for profiling related tables.

afroditiz
Calcite | Level 5

@Rama_V 

In continuance of the thread, is there an exhaustive list of profiling-related tables stored in the repository, so that someone proceeds to cleaning them? Also, by cleaning you mean deleting or truncating those tables?

Because I have a case where a FRQDSTxxx table never got created for some reason (FRQDSTxxx-1 & FRQDSTxxx+1 are created), and i would like to examine the possibility "cleaning" the whole repository for profiling only.

 

Plus, does deleting the profile history inside DMS also delete the FRQDSTxxx tables or not?

 

Thank you very much

Rama_V
Obsidian | Level 7

You need to look at the Repository schema, accessing it shown in the blog link my previous reply.  Rule executions, pattern distributions etc stores row logs which takes lot of repository space . 

 

I am not sure this is recommended approach for cleaning. I never truncated the DMS tables. If you do need truncate then I am not sure how the relationships in the repository tables will behave. So need some caution around that especially folders,profile jobs, business rules etc. 


If I am deleting and recreating the repository then I usually export the jobs, business rules, profile jobs and all the results are extracted and loaded into a Data mart. This way I have the results ,Jobs and business rules which will be used to recreate my repository. If using RDBMS as repository then its even better to use scripts to create the repository. The import the exported objects back into repository.

VincentRejany
SAS Employee

Hi

 

Find in attachment the process job mentioned in the blog. It allows to extract data profiling metrics from a DataFlux repository (file or db).

Hope that helps

 

Vincent

afroditiz
Calcite | Level 5

Hi @VincentRejany thank you very much for your replies.

 

It seems that the problem I am facing was irrelevant with FRQDST dynamic construction logic, but it's good to know more information about cleaning.

I can't see the attachment, but this is the job I am using as well based on the link you have sent 🙂

 

Regarding the repository "migration" part as mentioned in your previous reply, I guess that, having the repository in RDBMS, in case of a new environment I would just have to export/import profiles, rules etc. and define the repository definition to 'look' at the RDBMS, so I imaginge no further action would be required for profiling tables.

VincentRejany
SAS Employee

Strange that the file has been removed

VincentRejany
SAS Employee
And yes you can import export data profiling jobs (not reports), rules from on repository to another (file or db)

Ready to join fellow brilliant minds for the SAS Hackathon?

Build your skills. Make connections. Enjoy creative freedom. Maybe change the world. Registration is now open through August 30th. Visit the SAS Hackathon homepage.

Register today!
How to connect to databases in SAS Viya

Need to connect to databases in SAS Viya? SAS’ David Ghan shows you two methods – via SAS/ACCESS LIBNAME and SAS Data Connector SASLIBS – in this video.

Find more tutorials on the SAS Users YouTube channel.

Discussion stats
  • 7 replies
  • 1369 views
  • 0 likes
  • 4 in conversation