Suggestions about sharing data among different development teams

Reply
New Contributor
Posts: 4

Suggestions about sharing data among different development teams

Hi all,

we are working on a distributed development environment (I mean several development groups working on different business areas) on a single SAS platform. For the sake of security, each development team is working on its own application server.

These projects manage both private and confidencial data and data that needs to be shared to one, a few or all the development teams so that they can include this data in their own ETL processes. We are intending to avoid each of the teams accessing the transactional database n-times and use the data already in SAS tables instead. E.g.: suppose an organization that has several branches. We would like to have only one Branch datasource in the platform even though there are several BI projects (marketing, human resources, etc.) that need this datasource as an entry table to their processes.

How would you handle this? Where would you put the information that needs to be shared? How would you manage access to this information, especially, when a development team should access a part of the shared table to include in their own processes?

Thanks very much.

Respected Advisor
Posts: 4,173

Re: Suggestions about sharing data among different development teams

A reasonable database gives you much more granularity when it comes to securing data. This is especially true when it comes to row level and column level security. I therefore would re-consider your approach from moving away from the DB.

The best approach for securing SAS files is OS level security - so access permission settings on physical folder/file level. You could then consider to use concatenated libraries defining the paths according to the data needs of the groups.

When using concatenated libraries then be aware that when creating a table it will be written to the first folder in the concatenated path. So if someone needs to actually create a table in the common folder you would need an other library for this.

You could also consider using SAS Metadata bound libraries.

Both of the above approaches will allow you to secure SAS files but it's no solution for row level and folder level security. To implement this you need either information maps (with hidden filters) or you need to split up your data into multiple tables stored in multiple folders (where you can apply OS level security) and then define views combining the split up tables.

Same named views could get defined for different user group specific folders including the table bits a user group has access to - and you could combine this with the concatenated library approach.

Again: I believe using DB functionality for such requirements is better. Depending on what you need the data for you could consider building up your own reporting data mart  (instead of copying stuff into SAS files for the same purpose).

New Contributor
Posts: 4

Re: Suggestions about sharing data among different development teams

Thanks for your comments, Patrick.

Even if we go for the db approach we still don´t want more than one etl process to bring the transactional data into the "datamart zone". Do you think we should use views to restrict access to the shared data? How should we handle this sharing in Data Integration ? I mean, would you have one library in the application server where the etl "lives" and share the views to the other application servers teams so they can use the data as input for their jobs (for example, as a dimentional table)? (As far as I understand, you cannot use information maps within a job. Is this correct?) Or repeat the library in every application server?

Thanks again! Smiley Happy

Trusted Advisor
Posts: 3,212

Re: Suggestions about sharing data among different development teams

What I am missing is a holostic concept on working in way you described.

Call it DTAP call it waterfall approach ITSM or whatever. You need to have some architecture on:

- developing code (business logic) with all stages, able to de testing in all kinds, deployment etc.  

- Segrations in business data as hat part is normally containing sensitive information it should be well protected.

At D,T parallel development en parallel testing should be possible, The goal is getting to P(roduction) as I fear the only environment people commonly are seeing and understanding. It is first about version-management and then about release- management (different subjects/tools).

Wiht DI you can support versioning (SVM) check-in check-out for developers. Developers are commonly blocked form production.

At het OS level this architecture is more easy to implement as it should be known stuff for 30-years (SDM).

At the DBMS the business data part is normally well isolated as required. It also easy to define shared autorizations.

I could give som generic prictures of that.

With SAS you will also get into the SAS metadata. Data describing code (business logic) and businsess data.

That is an other container of information to set up in a same way as described.   

The case "Orion-star" used at trainings by SAS is having a structure (limited) in that way.

With 9.3 you can share libraries and tabels between app-servers.

Are you still on 9.1.3? That is an outdated version now..

---->-- ja karman --<-----
Ask a Question
Discussion stats
  • 3 replies
  • 214 views
  • 1 like
  • 3 in conversation