What is functional data? We work with data that is typically represented as discrete measurements at specific points in time or space. An example shown below informs us that for each product/batch ID, we have readings available for variables x1 and x2.  We can predict y using the variables x1 and x2 at the ID level using supervised learning techniques.   Table 1: Regular structured data   What if we encounter a scenario where instead of discrete values, we are given continuous curves to predict the same target y? In the table below, we have a scenario where x2 is replaced with continuous curves i.e. multiple measurements for x2 are available for each ID. These continuous curves could be a function of time or any other metric such as wavelength, voltage, etc.   These types of data are called functional data.   Table 2: Functional data   The key difference between the two scenarios is that while regular data only provides snapshots at specific points, functional data captures the changes of a phenomena across a continuum. In this article we will investigate functional data analysis for IoT in SAS Viya. Where do we encounter functional data? The simple answer is that functional data is encountered whenever measurements are recorded continuously over a domain rather than at isolated points. Consider an IoT use case in the semiconductor manufacturing industry where wafer analysis plays a critical role in ensuring high-quality chips and optimizing production processes. Functional Data Analysis (FDA) offers a powerful framework for examining continuous, high-dimensional data, making it particularly valuable in wafer inspection, metrology, and process control. During semiconductor fabrication, wafers undergo multiple processes, including lithography, deposition, doping, metrology, etc. In the deposition process, sensors continuously record key metrics such as gas flow, temperature, heater modes, power monitors, and others throughout the operation, capturing their variations as continuous functions over time. FDA can leverage these functional inputs from the deposition step to predict wafer thickness during the subsequent metrology stage, enabling more accurate process optimization and defect detection. Other areas of IoT where functional data occurs quite commonly (but not limited to) include: IoT-enabled wearables and medical devices collect real-time functional data on heart rate, blood pressure, glucose levels, oxygen level, etc. which enables early diagnosis of conditions and preventative care. In smart agriculture and precision farming, IoT sensors gather functional data on environmental elements such as soil, moisture, humidity, etc. which can be used to reduce resource consumption and improve yield. In supply chain and logistics optimization, IoT enabled trackers collect functional data on temperature, humidity, location, and handling conditions for cold chain monitoring in pharmaceutical and food supply chains. How do we analyze functional data? The easiest way to work with functional data is perform summarization of the curves such as mean, min, median, max, etc. Additionally, we can summarize the shape of the curve itself such as slope, inflection point, and others. This however comes at the expense of throwing away a lot of the information from the original curves thereby losing temporal trends and inherent structure. Available as part of SAS Viya 2024.11, functional principal component analysis (FPCA) is a technique appropriate for analyzing functional data. FPCA is an extension of Principal Component Analysis (PCA), a widely used dimensionality reduction technique. PCA identifies principal components—vectors that maximize variance along their direction—where each component captures a portion of the total variance in the data. FPCA, while conceptually similar, differs in that it operates on functions rather than vectors. Consequently, the principal components in FPCA are also functions or curves, reflecting the continuous nature of functional data. There are two CAS actions available for SAS Viya users fPca and fPcaScore. Let’s understand how these CAS actions work on functional data. Consider a simple example where we have ten curves (x1, x2..., x10) measured at equally spaced timestamps with their own intrinsic characteristic shape features represented in Fig 1 below.                                                                              Fig 1: Functional data represented by curves x1-x10   The CAS action fPca trains the data on the input curves such that every individual function/curve above can be reconstructed by a combination of the mean curve and the functional principal component scores along with the eigenfunctions that are generated by the code snippet below. result = s.fPca( table={"name": "fpca_train_data"}, output={"casout": {"name": "SCORE", "replace": True}, "npc": 2}, eigenVec={"name": "EIGENVEC", "replace": True}, eigenVal={"name": "EIGENVAL", "replace": True}, saveState={"name": "trainStore", "replace": True} ) In the above code ‘npc’ specifies the number of principal components which has been set to 2 in this case. The code produces four different output tables “output” -contains the principal component scores of the training data “eigenVal” – contains the eigenvalue matrix “eigenVec” - contains the eigenvector matrix  “saveState” – contains the state of eigenvector matrix for scoring Using the output tables, we can approximate the original curves from Fig 1 by using a combination of the mean function with the corresponding eigenfunctions and their respective principal component scores. Figure 2 below shows the reconstruction of the curves x1 and x10 using this technique.   Fig 2: Reconstruction of curves x1 and x10 using FPCA   The two eigenfunctions encapsulate the variability of all individual curves in two distinct directions, while the principal component scores quantify each curve's specific contribution to these eigenfunctions. Throughout this process, the mean function and eigenfunctions remain consistent across all curves, with differentiation arising from the principal component scores. This transformation reduces a function to a few key numerical values, simplifying its representation. For instance, the curve x1​ can now be characterized solely by the scores PC_1 Score1 and PC_1Score2. Following this process, one can reconstruct all ten signals from Figure 1 as shown below.    Fig 3: Reconstruction of curves x1 through x10 using FPCA   An additional CAS action available for functional data analysis is fPcaScore. This action scores new functional data based on eigenfunctions that are derived from a prior FPCA training analysis. The table saved from the “savestate” parameter in the fPca CAS action needs to be invoked in this step. Let’s go back to the original problem we posed at the beginning of the article that stems from working with data of the form below where instead of discrete values, we have continuous curves to predict the target y.   Table 2: Functional Data  We can now use fPca to represent the structural patterns of the curves within variable x2 by their functional principal components (2 components used)  and reformat the same dataset into the form below. These fpca scores combined with variable x1 can easily predict y using any of the supervised learning techniques from SAS.   Table 3: Representation of functional data using fpca scores only   Functional Data Analysis (FDA) is critical in IoT because it provides a structured way to handle complex, continuous data streams efficiently. By leveraging FDA techniques, IoT systems can enhance accuracy, reduce storage costs, improve decision-making, and enable real-time monitoring—critical for the success of smart and connected technologies.       ... View more

  The term IoT, or Internet of Things, refers to the collective network of connected devices and the technology that facilitates communication between devices and the cloud, as well as between the devices themselves. The measurements being sent by the sensors (wearables, connected devices, machinery, etc.) are always captured at a specific moment in time. This makes it vitally important to have the right tools to work with timestamp information be it for visualization of the timestamp data, creation of aggregated fields, or building analytical models.   In this article, we provide tips to makes it easier to work with timestamped IoT data using SAS’ Wrapper for Analytics Transfer (SWAT). SWAT is the Python module used to interface with SAS' Cloud Analytics Services (CAS).  It provides the best of both worlds by combining SAS Analytics in the cloud with Python’s large collection of open-source packages.   To help demonstrate this, let’s use a simple IoT csv file that has the following fields: Timestamp –Time at which the measurement was taken Measure – Measurement from a device taken every minute Device – Nominal variable indicating the device name   Below is a sample record from a csv file: Timestamp,Measure,Device 2/24/2015 0:00,35,A 2/24/2015 0:01,32,A 2/24/2015 0:02,30,A 2/24/2015 0:03,31,A 2/24/2015 0:04,44,A 2/24/2015 0:05,29,A 2/24/2015 0:06,45,A 2/24/2015 0:07,43,A   First, load the data into CAS using the loadtable CAS action. If you load the file 'as is' it will render the timestamp variable to be of type varchar. You will need to ensure the timestamp variable has the right datetime format so it is amenable for any further processing using CAS actions, SAS functions, etc.   Tip #1:  Read in the Timestamp variable with the correct informat and format while loading the data using the loadtable CAS action. It ensures that the data type and format for the Timestamp variable is captured in a desired way. s.loadtable("timestamp_test.csv", casout= dict(name='Timeseries', replace=True), importoptions= dict(filetype='csv', vars=[dict(name='Timestamp', informat='anydtdtm19.', format='DATETIME19.')]) )   As a data analyst/scientist explores the timestamp data, he/she might be interested in sub-setting the data on a time-based condition.   Tip #2: The 'where' option in the CASTable object enables sub setting operations on the CAS table quite easily. The convenience of using the CASTable object enables the user to run different sub-setting scenarios on the fly without making changes to the actual in-memory CAS table.  tbl = s.CASTable('timeseries', where = "Timestamp < '2015-02-24 00:06:00'dt ") Any other operation you want to perform on the sub-setted data can be applied directly to the  above CASTable object, tbl, by using a variety of CAS actions available to the end user. For example, a simple summary statistic on the Measure variable of the sub-setted data can be achieved by simply applying the summary CAS action directly on the CASTable object, tbl. tbl.summary(inputs='Measure') The SWAT package uses many features of the Pandas Python package and other dependencies of Pandas. This enables the user to directly apply Pandas methods on the CASTable object. For example, a frequency count on the Device column can be computed using the value_counts methods of Pandas API.  tbl['Device'].value_counts() Since the SWAT API tries to blend the world of CAS and Pandas into a single entity, you have to be aware of whether you are calling a CAS action or a method from the Pandas API and understand the differences in the output objects produced from those operations. This link is very useful to the end user trying to navigate through these concepts.    Often times when exploring the data, one is also interested in creating new computed columns based off certain conditions etc.   Tip #3: Let us assume that there is a 3-hour lag in the process of collecting the data. Essentially, we would like to create a new column that captures the time at which the measurements were truly registered by subtracting 3 hours from the current Timestamp variable. A quick way to do this is by use of the options computedvars and computedvars program.  We first declare the variables with the correct format in the computedvars option. The computedvarsprogram allows you to define those calculations. In the code snippet below the SAS function intnx is used to account for the 3-hour difference. We capture this in the new variable StartDate  with the desired datetime format. The second computed variable, MonthName, in the code snippet below,  shows an example of extracting the month value from the Timestamp variable which can be easily done by using the SAS datepart and PUT function with the choice of date format we want.  tbl1 = s.CASTable('TimeSeries', computedvars=[ dict(format='DATETIME19.', name='StartDate'), 'MonthName' ], computedvarsprogram = "StartDate = intnx('HOUR',TimeStamp,-3,'SAME'); MonthName=put(datepart(Timestamp),MONNAME3.); ") It is important to note that CASTable objects are client-side views of the data. Setting the parameters as shown in the tips will have no effect on the actual table in the server. However, it is a great tool for on-the-fly analysis during data exploration.   Tip #4: Once you have created a CASTable object after experimenting with all your filters and computed columns, you will want to materialize them to the server as an in-memory table. Let’s say we want to materialize the newly computed variables into an actual in-memory CAS table. Any CAS action that reads a CAS table and writes it to a new location (e.g. Partition, Index, Shuffle, etc.) can be used to do this. Below, we will use the partition CAS action whose output has a key named casTable (note that this is different from the CASTable object we referenced earlier) that contains the reference to the new in-memory CAS table with the computed columns.  tbl2 = tbl1.partition().casTable   We hope you found this helpful. For additional examples about working with IoT data, please visit developer.sas.com. ... View more

Computer Vision is a field of artificial intelligence that seeks to understand digital images from cameras and videos by leveraging state of the art deep learning models. Business use cases of computer vision covers the entire gamut from defect detection of products in a manufacturing plant to identification of potentially malignant tumor in the human body, to traffic monitoring for smart city initiatives, and much more. These applications will continue to improve with faster processors, more advanced deep learning algorithms, and deeper integration to edge devices. For computer vision to bring value to a business, it is important to focus not only on the development of deep learning models but also on the deployment of these models. In the Internet of Things (IoT) era, when large amounts of streaming data are generated continuously, it is extremely inefficient to store all these data in the cloud. For real-time image or video processing, moving the analytics to edge devices not only saves a device-to-cloud data round trip but also improves data privacy and governance.   SAS tackles the end-to-end lifecycle of a computer vision pipeline from model building to deployment in two phases: Model building using SAS® Visual Data Mining and Machine Learning and SAS DLPy Model Deployment on edge devices using SAS® Event Stream Processing Please take a look at this video  to learn the details behind solving computer vision use cases using SAS.   ... View more

Deep Convolution Neural Networks (CNN) are powerful tools for image classification tasks. Image classification involves training a model using a large dataset of examples of each class.  The fitted model is then used to make predictions about the instances of each class. For example: if we want to classify images of vehicles, we would ideally train the model with many images of each type of vehicle. Some networks can identify thousands of classes with very good performance because they are trained with large data sets. However, training on a large dataset is not always feasible.   What if we don’t have enough images of each class to build a robust model? What if the classes change frequently so that retraining and redeploying the model is not feasible? We can address these issues by using the concept of image embedding—a method used to represent an input image in a condensed form known as a low-dimensional feature vector. When we build an image embedding model, the low-dimensional vector representation we create of the image is then used to map similar images close together and dissimilar images far apart within the embedding space. In order to compare two images, we would simply compute the distance between their respective embeddings; if the distance is less than a threshold, the two images are the same. Thus, an image classification problem can be transformed into an image similarity problem.  An application of image embedding would be facial recognition systems. Instead of training a model with many images of each person’s face, we can use image embedding to find out how similar one picture of a face is to another. Additionally, this technique could be used for image searching, anomaly detection, feature generation, etc.   Figure1 represents the architecture of an image embedding model. Passing an image through a series of convolution, pooling, and fully connected layers would result in a feature vector— a set of numbers that represents the embedding of the input image. Typically, for a classification model, this feature vector feeds into an output softmax layer. But we will not use that output layer for building an image embedding model. Instead, the feature vectors for each image class is compared for similarity as shown in Figure 1. The output of the embedding layer can be further passed on to other machine learning techniques such as clustering, k nearest-neighbor analysis, etc. Figure 1: High-level architecture of an Image Embedding Model             Building an Image Embedding Model using SAS DLPy   SAS provides a variety of different layer types within its deep learning framework including an embedding loss layer. The embedding loss layer computes losses such as contrastive loss, triplet loss, and quartet loss, in order to compare the feature vectors. The source layers for these three loss functions must be fully connected dense layers that have the exact same number of neurons. In this context, the number of neurons define the embedding dimension. As the name suggests, contrastive loss (Siamese) networks, triplet loss networks, and quartet loss networks each ingests two, three and four input image data streams respectively.   Figure 2 shows each of the three architectures. What do the different input streams represent in each of the three architectures? In a Siamese network, the similarity/difference between the input streams is tracked by an additional ‘target’ variable. In triplet network, we require three input image data streams. The first input layer contains anchor(reference) examples, the second contains positive examples (images similar to anchors), and the third contains negative examples (images dissimilar to anchors). As such, there is no need to specify the target column. All the weights from three model branches are shared. The triplet loss minimizes the distance between anchors and positives while it maximizes the distance between anchors and negatives up to a user-specified threshold. In Quartet loss the first three input layers follow the same requirements as the triplet loss. The fourth input layer contains another set of negative examples. Figure 2: Siamese, Triplet, and Quartet networks   Building the Architecture in Three Easy Steps   An image embedding architecture can be defined in a few easy steps: First, define a base network. A  variety of architectures exist that can be used to build the base network. The code below defines a RESNET-18 base architecture in DLPy: Resnet18_model = ResNet18_Caffe(s, width=w, height=h, random_mutation=’random’) Next, define the embedding layer, which is a dense fully connected network consisting of n neurons. The length of the feature vector(embedding) is determined by the number of neurons in this layer. The following DLPy code defines the embedding layer. Embedding_layer = Dense(n=n, act=’identity’) Finally, invoke the built-in DLPy function called “build_embedding_model”, where you can specify the base architecture (such as the RESNET-18 model in step 1), the embedding_model_type as either “Siamese”, “triplet”, or “quartet”, and the embedding layer as the fully connected dense layer defined in step 2. Model_tr = EmbeddingModel.build_embedding_model(resnet18_model, model_table=’test_tr’, embedding_model_type=’siamese’, margin=m, embedding_layer=embedding_layer)   Figure 3 shows the architecture of a Siamese network (not all layers are shown) built in DLPy. There are two branches built from a base RESNET-18 network. DLPy handles replicating the RESNET-18 architecture across the two branches, removing the output layer from the base architecture, adding the fully connected layer to each branch, and finally bringing them all together in the embedding loss layer. The two branches share the same parameters and weights. For a triplet and quartet network, there would be three and four similar branches respectively. Note the option called “margin” in above code snippet. For a Siamese network, the margin parameter is the upper bound for the distance between two dissimilar data samples. It helps to control the distance from exploding.   Figure 3: Siamese network built in SAS DLPy   Training the model   The training data resides in a CASLIB-accessible location so we can employ a server-side load of the data. Images are separated into directories by class, with the directory name indicating the class label. Creation of the input CAS table and resizing of the images can be performed on-the-fly during the training process by simply specifying the path to the data. The model training is carried out in three steps:   Create a batch of n_samples images. Train on that batch for max_epochs specified in the DLPy optimizer object. Repeat steps 1 and 2 for max_iter iterations. res = model_tr.fit_embedding_model(optimizer=optimizer, gpu=True, seed=1234, path='path_to_image_files_on the_server_side', n_samples=n_samples, max_iter=max_iter, resize_width=w, resize_height=h) If the model performs as desired, it can be applied on the validation/test data to verify the predictions.   Deploying the model using an ASTORE   An analytic store, or ASTORE, is a binary file that you create to save the information about the state of an analytic object (such as a predictive model) after the training stage is completed. DLPy allows us to save the model ASTORE from an image embedding model in two modes: A single branch of the image embedding model can be saved by specifying model_type=’branch’. In this case the embedding loss layer is removed and replaced by a generic output layer. The full model can be saved by specifying the option model_type=’full’. branch_model = model_tr.deploy_embedding_model(output_format='astore', model_type='full', path='path_to_astore') In order to score a new data set, simply invoke the saved ASTORE and apply it on new data. The result of scoring new data with an ASTORE will produce an n-dimensional feature vector, where n is determined from the training model. Note that when scoring with an ASTORE saved as option 1 above, we can pass a single stream of images. However, an ASTORE saved under option 2 will require as many streams of images as determined by the type of network (Siamese/triplet/quartet). This example takes you through the steps for creating an embedding table in DLPy. The resulting table could be scored against a full ASTORE. Finally, the feature vector obtained as a result of scoring can be ingested by other analytical tasks such as k-means clustering, k nearest neighbors etc. as shown in this example to rank or cluster the most similar objects.   Several applications in the industry, ranging from facial recognition to document ranking can be tackled via placing contextually similar data points close by and dissimilar data points far away from each other in the projected embedding space. In this article, we looked at how image embedding can be performed using SAS DLPy. For more information on creating image embedding models please visit the SAS DLPy GitHub Page and let us know how you plan to use it in your business problems. ... View more