Picturing SAS and Microsoft Azure data analytics integration
A modern data analytics architecture usually involves the integration of a number of technologies with data passing from one to the next. This is typically illustrated in a left-to-right diagram of lines connecting product icons.
Major cloud vendors offer a range of these architecture diagrams for various solutions. Of course, this kind of diagram can only represent one view of an architecture but it’s a very useful view for communicating to stakeholders a target landscape and the relationships between technologies within it.
What is the diagram called?
Although these diagrams have become popular in recent years, there is no standard name or definition for them. They are a kind of data flow diagram but do not use the DFD notation and the entities they depict are products rather than data stores. You could call it a data pipeline diagram but the term usually refers to a lower level description of the extraction, transformation and loading (ELT) of data or similar operations in another order (e.g. ELT). Perhaps the most appropriate term is system pipeline diagram.
Characteristics
System pipeline diagrams have common characteristics:
- Each node is a product, tool or service for processing or storing data, often shown as an icon.
- The nodes are connected with arrows representing a flow of data between nodes.
- Arrows are mostly one-directional although there may be some bi-directional arrows.
- The flow is mainly left-to-right although there may be some up/down arrows and a possibly a feedback loop from right to left.
- Nodes or groups of nodes may be labelled with the processing stage they are used for.
A system pipeline for data analytics with SAS and Microsoft
As SAS has a strategic partnership with Microsoft is seems appropriate to create a system pipeline diagram showing how the two sets of products can work together. Microsoft provides example architecture diagrams for wide range of solutions or scenarios including the following:
Data Warehousing and Analytics
Here is a diagram in a similar style showing the integration of SAS and Microsoft products in an architecture for data analytics. It is just one example of how the products can be arranged.
- Azure Synapse Analytics pipelines read data from sources and load to Azure Data Lake Storage Gen 2 (ADLS2). Similar capabilities are available outside Synapse using Azure Data Factory.
- Azure Synapse Analytics SQL pools read data from ADLS2 and hold it in a relational data model.
- Power BI reads the data in the Azure Synapse Analytics SQL pools via Azure Analysis Services, which presents a semantic view.
- A data analyst using SAS Studio builds data pipelines to read the data from ADLS2 and prepare it for analysis. The CAS in-memory engine can access the data in CSV, Parquet or ORC format and perform data cleansing and transformation. Prepared data can then be stored back in ADLS2 in an open format, available for other applications to use.
- Data pipelines running in SAS can also write output to Azure Synapse Analytics SQL. This would enable data sharing with Power BI.
- A business analyst using SAS Visual Analytics can explore the prepared data from SAS Studio, ADLS2 or Azure Synapse Analytics SQL.
Other scenarios
Many variations on this are possible. If SAS is used to create and deploy machine learning models, then the diagram will need to show the model repository and deployment targets. Streaming analytics will involve data streams and the running of models into those streams using SAS Event Stream Processing. Real-time transactional analytics will use the SAS Micro Analytic Service or container runtimes configured as API endpoints.
Whichever way you intend to use SAS in your landscape, you can create a diagram that shows how SAS integrates with other technologies to deliver analytics insights.
