Problem with DLPy fit

[Michal_S_00]

Hi,

I'm working at Sequnce Labaling model for NLP task (dlpy.applications.SequenceLabeling) and while I was trying to fit the data, I came across an error "ERROR: A floating-point overflow exception occurred, halting the analysis. This condition is usually caused by improperly scaled inputs, a large learning rate, or exploding gradients.".

This is my code:

model.fit(data='train_data',

                 inputs=col_names[:10],

                 texts=col_names[:10],

                 target_sequence=col_names[10:],

                 nominals=col_names[10:],

                 text_parms=TextParms(init_input_embeddings='word_embeddings_100'),

                 mini_batch_size=10,

                 max_epochs=100,

                 lr=0.000000000000001,

                 log_level=2

                 )

Input consists of ten columns, there is one word in each column in each row, also the labels(varchar type) are represented in the same way.

I tried to run it with diffrent learning rate, but every time I get the same error, any ideas how to fix that?

[joeFurbee]

Hi @Michal_S_00,

This question is being worked on by the developer through the issue created on the DLPy GitHub repository. I will update this thread once a solution is posted there.

[joeFurbee]

Here's a copy of the response from @lipcai on GitHub:

If your model generates floating-point exception errors, you can consider specifying values for gradient clipping parameters (clip_grad_min and clip_grad_max). https://github.com/sassoftware/python-dlpy/blob/af4874e00edc7a4b7c31646e76057a76d566481c/dlpy/tests/...