How to calculate parameters for Generalized Additive Neural Network?

[xliu1]

This is a sample question from the certification preparation for advanced predictive modeling:

Consider a Generalized Additive Neural Network (GANN) with 3 continuous inputs and 2 hidden nodes
for each input.
How many parameters do you need to estimate when training the neural network?
A. 19
B. 21
C. 22
D. 25
correct_answer = "C"

I would like to know why the correct answer is C? Is there a formula that could be used to calculate the parameters estimated in GANN or can someone please draw a diagram to illustrate the correct answer? Thanks.

[sbxkoenk]

Hello,

I think (!) it's like this :

1. The basic architecture for a generalized additive neural network (GANN) has a separate MLP (multilayer perceptron) with a single hidden layer of h units for each input variable (h=2 in your example).
2. Each individual univariate function has 3*h parameters (where h could vary across inputs).
3. The individual output bias terms are absorbed into the overall bias
4. An enhanced architecture includes an additional parameter for a direct connection (skip layer)

So, 3 continuous inputs with each of them having two hidden nodes (in one hidden layer) and a skip layer ... that makes:
3 * (3 * 2 + 1) + 1 = 22

• First 3 due to 3 continuous inputs
• 3 * 2 = 3 * h (see point 2 above)
• + 1 between the brackets is for the direct connection (skip layer)
• the last + 1 (outside the brackets) is for the overall bias (see point 3 above)

Koen

[xliu1]

Thanks for the explanation!