Understand neural network output in EMiner (XOR problem)

AndrewVOA — Tue, 11 Jul 2017 11:20:24 GMT

Split this topic from this solved question about neural network output.

I managed to see the predicted values for a given observation, following the procedures detailed in this previous post. However, that predicted value seems to be different from what I calculated manually.

I am trying to learn the XOR problem, so I have 2 binary input Variables A and B, and 1 binary target y=0 if A=B, otherwise y=1. I created a single layer neural network that have 2 hidden nodes and a bias node (see figure 1 for network settings, see figure 2 for structure). From the exported trianing data, I can see that all observations are predicted correctly. However, I then extracted the final weights of all the connections, and then proceed to try out all 4 combinations of A and B (see figure 2), but they all seems to predict y=1 which is obviously wrong. Did i do something wrong? Is this not how I am suppose to intepret the weights?

Re: Understand neural network output in EMiner (XOR problem)

WendyCzika — Mon, 17 Jul 2017 18:06:13 GMT

Are you using the activation functions in your calculations? That's the only thing I can think that could be missing. You can see the exact calculations that are being done in the score code if you open up the Results of the Neural Network node, then go to View>Scoring>SAS Code. You will see something like this for the hidden units:

*** *************************;
*** Writing the Node H1 ;
*** *************************;
IF _DM_BAD EQ 0 THEN DO;
H11 = -1.52129446502195 * BAD0 ;
H12 = 0.79120505194756 * BAD0 ;
H13 = 0.57109362454256 * BAD0 ;
H11 = 0.17517854710157 + H11 ;
H12 = -1.44433268465385 + H12 ;
H13 = 0.45098528984172 + H13 ;
H11 = TANH(H11 );
H12 = TANH(H12 );
H13 = TANH(H13 );
END;

Then for the posterior probabilities, code like this:

P_REASONHomeImp = 0.11560366817131 * H11 + 0.11890802193169 * H12
+ -0.12101594764922 * H13 ;
P_REASONHomeImp = -0.97997241580015 + P_REASONHomeImp ;
P_REASONDebtCon = 0;
_MAX_ = MAX (P_REASONHomeImp , P_REASONDebtCon );
_SUM_ = 0.;
P_REASONHomeImp = EXP(P_REASONHomeImp - _MAX_);
_SUM_ = _SUM_ + P_REASONHomeImp ;
P_REASONDebtCon = EXP(P_REASONDebtCon - _MAX_);
_SUM_ = _SUM_ + P_REASONDebtCon ;
P_REASONHomeImp = P_REASONHomeImp / _SUM_;
P_REASONDebtCon = P_REASONDebtCon / _SUM_;

topic Re: Understand neural network output in EMiner (XOR problem) in SAS Data Science

Understand neural network output in EMiner (XOR problem)

Re: Understand neural network output in EMiner (XOR problem)