[Solved]: Problem with derivative of sigmoid activation function

Problem Detail: 

I'm following Jeff heatons book 'Introduction to Neural Networks with Java'. To get node deltas, we need to calculate $f'(sum)$. In the very first row for Training Element #1, we need to compute $f'(1.13) \cdot 0.25$ which Heaton evaluates to $0.045$.

Using my calculator, I get $f'(1.13)=-0.13$ (derivative of sigmoid activation function), then I multiply by $0.25$ to get $-0.0325$. I've been trying to figure out for days how heaton does his calculation but no success yet. Kindly assist.

Asked By : Leroy Kayanda

Answered By : MattD

The derivative of a sigmoid is always positive. Here's my math: Sigmoid(1.13) = 0.7558

I computed this on Wolfram Alpha

The derivative of a sigmoid y=S(x) is y * (1-y), so 0.7558 * 0.2442 = 0.0451

Best Answer from StackOverflow

Question Source : http://cs.stackexchange.com/questions/24070

3.2K people like this

 Download Related Notes/Documents