SAS Data Management Products: HTML5 (Flash Free) Interfaces

[#1]

The post will review the SAS Data Management products on SAS 9.4 M6 and what they can bring to a business. Find out when and why you should use these products, you might learn a few new things about SAS Business Data Network, Lineage, Data Remediation and Federation Server. There is also a new product: SAS Reference Data Manager. Highlighted are the changes and improvements in the new HTML5 interfaces.

Welcome to the Age of HTML5. Previously, applications in SAS 9.4 used the Adobe Flash Player to provide interactive user interfaces. Adobe announced that it intends to end support for Flash technology and will cease to update and distribute the Flash Player at the end of 2020.

SAS® Data Management products have a new shiny HTML5 interface since the SAS® 9.4 M6 release. The SAS® Data Management Advanced bundle released in June 2019:

• SAS® Business Data Network 3.3 centralizes knowledge about business terms, data and share the knowledge with fellow workers.
• SAS® Lineage is a real help to your internal and technical audits. Lineage is built on a detailed repository of SAS business and technical metadata and beyond.
• SAS® Data Remediation is a “data issue organizer”. It enables you to dispatch and follow-up quality issues.
• SAS® Reference Data Manager centralizes and manages reference data.

SAS® Federation Server Manager was released in September 2019 and is a stand-alone product, not part of the above bundle.

Most of the changes affect the interface and navigation. From a visual perspective, the product interfaces are aligned with other interfaces in SAS Viya. From a technical perspective, the revamp aimed to make the search and the REST APIs faster.

SAS Business Data Network 3.3

Why should you use it?

• Centralize and share knowledge about business terms, responsibilities, physical data (tables, columns), relationships between terms and processes (jobs, services, business rules).
• Collaborate with other users, enrich the knowledge about data.
• The REST APIs allow to re-use the knowledge (business terms, responsible persons, physical data, etc.) in technical processes and reports.

SAS Business Data Network is an application that enables you to manage the business terms. It supports a collaborative approach to manage:

• Descriptions, including requirements and attributes.
• Related source data and reference data.
• Contacts (such as technical owners, business owners, and interested parties).

Short Demonstration

This SAS demo includes an overview of business terms, business metadata, associated items, technical metadata, search options, business term hierarchies, and term types, export or import, REST API and workflows.

 

Notable Features

• Links between business terms and metadata (tables, columns, jobs).
• Very flexible term templates.
• Import from CSV and XML. Export to XML.
• Extensive REST APIs.
• Well thought CRUD (create, update, delete) lifecycle.
• Pre-defined workflows to collaborate, define a new term, approve, update, delete.
• CLI (command line interface).

 

What has changed

• Navigation is different.
• Search has improved, has more filters available and seems to be faster than in the flash version. You can improve the search
• REST APIs are supposed to be faster (although I did not personally test).

Tips

• Search: searching by name only is the fastest option if you have many terms. Adding searching through notes takes a fraction longer. Searching through attributes takes longer depending on the number of terms and attributes.
• Import: Improve the import time by not publishing the terms to relationships. Meaning, the terms will not show up in SAS Lineage at import time. There is a CLI that can be used to publish terms to relationships later.

Import terms

SAS Business Data Network supports business terms import from .csv and .xml files. The import is a simple and efficient way of managing business terms. In this demo, you will learn about three key elements in the import process: the term hierarchy, the term type and the import template.

More Info

• SAS Business Data Network documentation.
• SAS Business Data Network REST API documentation.

SAS Lineage 3.3

Why should you use it?

• Features a shared store for all relationship information, called the SAS relationship service. Most SAS 9 products and object types are integrated into the SAS relationship service. Integration with SAS Viya is on the product roadmap.
• Import content from third-party sources (uses a SAS Metadata Bridge for [technology], based on Meta Integration® Model Bridge, or MIMB). E.g.: you can import ETL metadata from IBM or Informatica, link it with SAS objects: tables, columns, jobs and  display those in SAS Lineage.
• Relationships web viewer. SAS Lineage is a web-based diagram for visualizing relationships between objects.
• REST APIs and command line interfaces to manage the objects.

Notable Features

• Network diagram that displays all relationships (same looks and feel like in SAS Viya).

• Governance diagram that displays data governance information.
• Can link business terms with metadata (tables, columns, jobs).
• REST APIs to extract relationships, stitch, update, delete.

What has changed

• Look and feel is like in SAS Viya Lineage.
• Search has improved and seems to be faster than in the flashed version.
• REST APIs were designed to be lighter and faster.

More Info

• SAS Lineage documentation.

SAS Data Remediation 2.4

Why should you use it?

You might find Data Remediation as your “data issue organizer”. You can assign issues to users. Those issues are sent automatically in the remediation database. In the interface, users can visualize the issues, assign them to other users, resolve or reject the issues, comment, etc.

SAS Data Remediation makes it easy to capture and review problems found in your data. SAS Data Remediation has a web-based interface for data administrators and REST APIs for system integration.

Data Remediation organizes the data and gives users access to remediation issues, items, and packages. Issues, items, and packages are associated with an application and subject area in remediation.

Notable Features

• Organize and track by number of days an issue is open, responsible person.
• Workflows (out of the box) for your most common remediation tasks.
• Did you know you could make corrections, to a database, directly from Data Remediation? REST APIs are working behind the scenes:
• URL to retrieve item attributes (GET): retrieve data from an external process. Design the custom REST web service so that the HTTP call using the GET retrieves one data row.
• URL to send item attributes (PUT): send the data back to an external process. Design the custom REST web service so that the HTTP call using the PUT to take data elements from one row in data remediation and send it to the external process.

What has changed

• User experience:
• Look and feel and Navigation has been re-thought.
• Display on page.
• Grouping of issues.
• Search criteria has improved.

More Info

• SAS Data Remediation documentation.

SAS Reference Data Manager 3.3

Why should you use it?

SAS Reference Data Manager supplies hierarchy, data management capabilities and a repository for your organization's reference data to be:

• centrally managed.
• versioned.
• linked to business terms defined in the Business Data Network.
• exported to external systems.

SAS Reference Data Manager consists of the following:

• data jobs deployed as batch jobs and real-time web services that must exist on a Data Management Server.
• a standard unified repository.
• web interface capabilities.

Some examples of common reference data are as follows:

• a list of product codes.
• a list of securities symbols.
• a material taxonomy such as the United Nations Standard Products and Services Code (UNSPSC).

How it works

• Reference data are organized by domains:
• list domains (populate a list).
• lookup domains (return a standardized name associated with a value).
• hierarchy domains (parent-child relationships).
• In a domain there are items. Each item has a key, description and label.
• Items can have attributes, defined per language.

Interesting Features

• Manage domains, lock, version, publish, etc.
• Import domain items, search, manage.
• Export to a .csv file or use REST API to access the captured reference data.
• Multi-language support for attributes. See in this example, how to manage a list with country codes and labels in several languages.

More Info

• SAS Reference Data Manager documentation.
• SAS Reference Data Manager 3.3: Attributes in Multiple Languages

SAS Federation Server Manager 4.3

Why should you use it?

SAS Federation Server Manager is the editor used to set up, manage, configure and secure access to SAS Federation Server. Administrators use SAS Federation Server Manager, to create data services and DSNs that connect to data sources supported by any SAS Federation Server.

Imagine Fed Server as a window*. You stand on the inside. The window lets light and heat in but filters it at the same time. You can:

• Look through the window outside:
• Through data services <like access engines, but not the same>.
• Views on data.
• Get protection from wind, rain and heat:
• Secures and control data access, views, cached views. Read further.
• Open the window and have the wind come in (administrator’s interface).

Notable Features

With Fed Server you can:

CONNECT

• Administrators can define connections to a data source.
• Users downstream will see a Federation Server connection. You could not care less if behind was Hive, Oracle, Postgres SQL or Netezza. You use the same FedSQL to query the sources, Federation Server does the translation for you.
• Sources you can connect to:
• Apache HIVE, DB2, Greenplum, MDS (Memory Data Store), Netezza, ODBC, Oracle, PostgreSQL, SAP, SAP HANA, SAS Federation Server, SASHDAT, Scalable Performance Data Server (SPD Server), SQL Server, Teradata.
• SAS.

CONTROL

• access data sources. For example:
• Finance Users should have access to the Salary data in a data source.
• Business Users should NOT have access.
• A Business User should have access to all other tables in the SCHEMA except for a SALARY table.

• access data rows and columns. For instance, a Finance user should have access to salaries of certain employees.

QUERY

Built-in query interface for administrators.

MASK

• Handle Encryption, hashing, transliteration, randomization, random string replacement.
• You could mask in a view what is sensitive, then let the users select data access according to their authorization.

TRACE

• Functionality: every query is logged in a database, if enabled. Absolute auditability, performance review and traceability. You can easily exploit even to put rules on the SQL log.

RUN

• Data Quality functions and cleanse post or at access time: Standardization, Matching, Pattern Analysis, Identification, Gender, Case, Parse, Extraction. The SAS Quality Knowledge Base at work.
• Score models, via DS2.

More info

• SAS Federation Server Manager documentation.

Acknowledgements

Mary Kathryn Queen, Nancy Rausch.

Conclusions

The SAS Data Management products: Business Data Network, Lineage, Reference Data Manager, Data Remediation, Federation Server have now a brand new shiny HTML5 interface. Most of the changes affect the interface and navigation. Visually, the product interfaces align now to other interfaces in SAS Viya. In some cases, the search has improved, the REST APIs are faster. Or the product was entirely re-thought, such as the SAS Reference Data Manager.