topic Re: How to calculate parameters for Generalized Additive Neural Network? in Statistical Procedures

How to calculate parameters for Generalized Additive Neural Network?

xliu1 — Wed, 28 Feb 2024 17:14:00 GMT

This is a sample question from the certification preparation for advanced predictive modeling:

Consider a Generalized Additive Neural Network (GANN) with 3 continuous inputs and 2 hidden nodes
for each input.
How many parameters do you need to estimate when training the neural network?
A. 19
B. 21
C. 22
D. 25
correct_answer = "C"

I would like to know why the correct answer is C? Is there a formula that could be used to calculate the parameters estimated in GANN or can someone please draw a diagram to illustrate the correct answer? Thanks.

Re: How to calculate parameters for Generalized Additive Neural Network?

sbxkoenk — Fri, 01 Mar 2024 23:16:09 GMT

Hello,

I think (!) it's like this :

The basic architecture for a generalized additive neural network (GANN) has a separate MLP (multilayer perceptron) with a single hidden layer of h units for each input variable (h=2 in your example).
Each individual univariate function has 3*h parameters (where h could vary across inputs).
The individual output bias terms are absorbed into the overall bias
An enhanced architecture includes an additional parameter for a direct connection (skip layer)

So, 3 continuous inputs with each of them having two hidden nodes (in one hidden layer) and a skip layer ... that makes:
3 * (3 * 2 + 1) + 1 = 22

First 3 due to 3 continuous inputs
3 * 2 = 3 * h (see point 2 above)
+ 1 between the brackets is for the direct connection (skip layer)
the last + 1 (outside the brackets) is for the overall bias (see point 3 above)

Koen

Re: How to calculate parameters for Generalized Additive Neural Network?

xliu1 — Mon, 04 Mar 2024 04:12:32 GMT

Thanks for the explanation!