SAS Deep Learning - "Floating-point exception" Error with MultiHeadAttention Layer

[Gaozhe]

I'm encountering an error when using SAS's deep learning capabilities. I've created a sequential model with a MultiHeadAttention layer for a regression task with a dataset of 1024 rows and 5 columns (4 features and 1 target column).

My model architecture is:

model = Sequential(conn, model_table='MHA_Regression')

# Input layer
model.add(InputLayer(n_channels=4, name='input_layer'))

# Dense layer
model.add(Dense(n=32, act='relu', name='dense2'))

# Multi-head attention layer
model.add(MultiHeadAttention(n=32, n_attn_heads=2, name='mha_layer'))

# Output layers
model.add(Dense(n=16, act='relu', name='pre_output'))
model.add(OutputLayer(n=1, act='IDENTITY', name='output_layer'))

When I try to fit the model, I receive this error:
ERROR: A floating-point exception or access violation has occurred, halting the analysis. This condition is usually caused by improperly scaled inputs, a large learning rate, or exploding gradients. ERROR: The action stopped due to errors.

I have tried to normalize my data and add Reshape layer, but it doesn't work. How to solve the problem?

[Season]

I am not an expert on deep learning, so I thought I should not respond to your question before. But now that three weeks have passed since you raised your question and no response have been offered, I would like to express my concern so that you may find a better place to ask your question.

The concern is: are you sure you are dealing with SAS codes? I do not have access to SAS Viya, the software developed by SAS Institute capable of doing deep learning, but the formats of your code really makes me think that your code is highly unlikely to be a SAS code. For instance, SAS does not use hashes to mark the beginning of appendices; SAS does not have codes like "model.add" (with dots in the middle); SAS does not stuff multiple options in a space flanked by parentheses. So I really speculate that you are coming to the wrong forum for help. The users here are all SAS users. While being a SAS user never implies that he/she cannot use other softwares, the fact that your question has been here for three weeks and no one has ever responded urges me to issue my concern. Nobody in this forum might be able to answer your question because you might had raised it in the wrong place.

Therefore, if your code is not a SAS code, then confirm the software on which the code should run and post your question on a forum on that software to minimize your time to get an answer; if I am wrong and your code really is a SAS code, then my response brings you back to the first place of the Statistical Procedures board so that other people visiting this board can find it more easily.

Good luck!