topic Re: How to implement a LSTM Autoencoder model in CAS on Viya? in SAS Data Science

How to implement a LSTM Autoencoder model in CAS on Viya?

Edoedoedo — Wed, 18 Dec 2019 10:40:18 GMT

I need to implement a LSTM Autoencoder neural network in CAS. How can it be done?

https://towardsdatascience.com/step-by-step-understanding-lstm-autoencoder-layers-ffab055b6352

Thank you

Re: How to implement a LSTM Autoencoder model in CAS on Viya?

WendyCzika — Wed, 18 Dec 2019 16:27:23 GMT

This might help (includes link to code in GitHub):

https://communities.sas.com/t5/SAS-Communities-Library/Autoencoder-analysis-using-PROC-NNET-and-neuralNet-action-set/ta-p/447135

Re: How to implement a LSTM Autoencoder model in CAS on Viya?

Edoedoedo — Wed, 18 Dec 2019 18:25:58 GMT

Thanks, however nnet seems to support only the standard Autoencoder and not the LSTM form.

I believe the model should be created "by hand" layer per layer with the deepLearn action set, but I don't know how to implement it.

Re: How to implement a LSTM Autoencoder model in CAS on Viya?

BrianGaines — Wed, 18 Dec 2019 19:02:48 GMT

Hi @Edoedoedo,

The documentation has an example of using the deepLearn action set to build a recurrent neural network. It does not build the specific architecture that you are interested in, but it will give you a good idea of the code and mechanics behind it, and you can reference the syntax for the addLayer action to tweak the code to fit your needs. For example, for recurrent layers, you can specify rnnType="LSTM" for those layers.

There is also a nice blog that provides a high-level overview of the steps involved with training deep learning models with SAS that may also be helpful.

Hopefully these resources are helpful!

-Brian

Re: How to implement a LSTM Autoencoder model in CAS on Viya?

Edoedoedo — Tue, 24 Dec 2019 12:18:01 GMT

Thanks a lot!

Just as an exercise I'm trying to recreate the standard MLP Autoencoder. Here's what I did:

cas session sessopts=(caslib="CASUSER");
libname CASUSER cas caslib="CASUSER";

data CASUSER.TRAIN;
    do id=1 to 10000;
        x=id/10000; 
        /* Fake relation */
        y=x+2;
        z=x*2-sqrt(x);
        output;
    end;
run;

data CASUSER.SCORE;
    set CASUSER.TRAIN;
    if id = 1 then z=z*10; /* anomaly */
    if id > 10 then delete;
run;

proc cas;
    deepLearn.buildModel /
        modelTable={name="MODELTABLE" replace=true}
        type="DNN";
    run;
      
    deepLearn.addLayer /
        layer={type="INPUT" std="STD"}
        modelTable={name="MODELTABLE"}
        name="LInput";
    run;
    
    deepLearn.addLayer /
        layer={type="FC" n=1}
        modelTable={name="MODELTABLE"}
        name="LHidden"
        srcLayers="LInput";
    run;
    
    deepLearn.addLayer /
        layer={type="OUTPUT"}
        modelTable={name="MODELTABLE"}
        name="LOutput"
        srcLayers="LHidden";
    run; 
    
    deepLearn.modelInfo / 
        modelTable={name="MODELTABLE"};
    run;
    
    deepLearn.dlTrain /
        inputs={"x" "y" "z"}
        modelTable={name="MODELTABLE"}
        modelWeights={name="MODELWEIGHTS" replace=true}
        table={name="TRAIN"}
        optimizer={loglevel=1};
    run;
quit;

proc cas;
    deepLearn.dlScore /
        initWeights={name="MODELWEIGHTS"}
        modelTable={name="MODELTABLE"}
        table={name="SCORE"}
        casout={name="SCORED" replace=true}
        copyvars={"id"};
   run;
quit;

However I can't see the scored layers values to calculate the mse row by row. What am I doing wrong? Is this simple code correct?

Thanks!

Re: How to implement a LSTM Autoencoder model in CAS on Viya?

Edoedoedo — Fri, 03 Jan 2020 11:33:31 GMT

I'm looking at "layers" and "layersOut" but it's not working, the output is weird with columns like "_layeract_0_img_0_" which contain characters. It seems something related to images but my model has nothing to do with images. What's wrong? Any help?

EDIT: Even with the example I couldn't find a way to see all the layers values row by row: https://documentation.sas.com/?docsetId=casdlpg&docsetTarget=p024gf3bj0rmwcn1tfkyy68dzs4g.htm&docsetVersion=8.5&locale=en

Thank you very much

Regards