How to implement a LSTM Autoencoder model in CAS on Viya?

[Edoedoedo]

I need to implement a LSTM Autoencoder neural network in CAS. How can it be done?

https://towardsdatascience.com/step-by-step-understanding-lstm-autoencoder-layers-ffab055b6352

Thank you

[WendyCzika]

This might help (includes link to code in GitHub):

https://communities.sas.com/t5/SAS-Communities-Library/Autoencoder-analysis-using-PROC-NNET-and-neur... 

[Edoedoedo]

Thanks, however nnet seems to support only the standard Autoencoder and not the LSTM form.

I believe the model should be created "by hand" layer per layer with the deepLearn action set, but I don't know how to implement it.

[BrianGaines]

Hi @Edoedoedo, 

The documentation has an example of using the deepLearn action set to build a recurrent neural network.  It does not build the specific architecture that you are interested in, but it will give you a good idea of the code and mechanics behind it, and you can reference the syntax for the addLayer action to tweak the code to fit your needs.  For example, for recurrent layers, you can specify rnnType="LSTM" for those layers.  

There is also a nice blog that provides a high-level overview of the steps involved with training deep learning models with SAS that may also be helpful.  

Hopefully these resources are helpful!

-Brian

[Edoedoedo]

Thanks a lot!

Just as an exercise I'm trying to recreate the standard MLP Autoencoder. Here's what I did:

cas session sessopts=(caslib="CASUSER");
libname CASUSER cas caslib="CASUSER";

data CASUSER.TRAIN;
    do id=1 to 10000;
        x=id/10000; 
        /* Fake relation */
        y=x+2;
        z=x*2-sqrt(x);
        output;
    end;
run;

data CASUSER.SCORE;
    set CASUSER.TRAIN;
    if id = 1 then z=z*10; /* anomaly */
    if id > 10 then delete;
run;

proc cas;
    deepLearn.buildModel /
        modelTable={name="MODELTABLE" replace=true}
        type="DNN";
    run;
      
    deepLearn.addLayer /
        layer={type="INPUT" std="STD"}
        modelTable={name="MODELTABLE"}
        name="LInput";
    run;
    
    deepLearn.addLayer /
        layer={type="FC" n=1}
        modelTable={name="MODELTABLE"}
        name="LHidden"
        srcLayers="LInput";
    run;
    
    deepLearn.addLayer /
        layer={type="OUTPUT"}
        modelTable={name="MODELTABLE"}
        name="LOutput"
        srcLayers="LHidden";
    run; 
    
    deepLearn.modelInfo / 
        modelTable={name="MODELTABLE"};
    run;
    
    deepLearn.dlTrain /
        inputs={"x" "y" "z"}
        modelTable={name="MODELTABLE"}
        modelWeights={name="MODELWEIGHTS" replace=true}
        table={name="TRAIN"}
        optimizer={loglevel=1};
    run;
quit;

proc cas;
    deepLearn.dlScore /
        initWeights={name="MODELWEIGHTS"}
        modelTable={name="MODELTABLE"}
        table={name="SCORE"}
        casout={name="SCORED" replace=true}
        copyvars={"id"};
   run;
quit;

However I can't see the scored layers values to calculate the mse row by row. What am I doing wrong? Is this simple code correct?

Thanks!

[Edoedoedo]

I'm looking at "layers" and "layersOut" but it's not working, the output is weird with columns like "_layeract_0_img_0_" which contain characters. It seems something related to images but my model has nothing to do with images. What's wrong? Any help?

EDIT: Even with the example I couldn't find a way to see all the layers values row by row: https://documentation.sas.com/?docsetId=casdlpg&docsetTarget=p024gf3bj0rmwcn1tfkyy68dzs4g.htm&docset...

Thank you very much

Regards