topic Problem with DLPy fit in Developers

Problem with DLPy fit

Michal_S_00 — Thu, 13 Oct 2022 12:19:20 GMT

Hi,

I'm working at Sequnce Labaling model for NLP task (dlpy.applications.SequenceLabeling) and while I was trying to fit the data, I came across an error "ERROR: A floating-point overflow exception occurred, halting the analysis. This condition is usually caused by improperly scaled inputs, a large learning rate, or exploding gradients.".

This is my code:

model.fit(data='train_data',

inputs=col_names[:10],

texts=col_names[:10],

target_sequence=col_names[10:],

nominals=col_names[10:],

text_parms=TextParms(init_input_embeddings='word_embeddings_100'),

mini_batch_size=10,

max_epochs=100,

lr=0.000000000000001,

log_level=2

)

Input consists of ten columns, there is one word in each column in each row, also the labels(varchar type) are represented in the same way.

I tried to run it with diffrent learning rate, but every time I get the same error, any ideas how to fix that?

Re: Problem with DLPy fit

joeFurbee — Fri, 14 Oct 2022 15:14:51 GMT

Hi @Michal_S_00,

This question is being worked on by the developer through the issue created on the DLPy GitHub repository. I will update this thread once a solution is posted there.

Re: Problem with DLPy fit

joeFurbee — Wed, 19 Oct 2022 14:59:11 GMT

Here's a copy of the response from @lipcai on GitHub:

If your model generates floating-point exception errors, you can consider specifying values for gradient clipping parameters (clip_grad_min and clip_grad_max). https://github.com/sassoftware/python-dlpy/blob/af4874e00edc7a4b7c31646e76057a76d566481c/dlpy/tests/test_embedding_model.py