CSE 5160 Machine Learning (Spring 2021) Assignment #4 (Due on April 23, 2021)

All assignments are to be submitted to Blackboard. Please note that the due time of each assignment is at 11:55 pm (Blackboard time) on the due date. Please make sure to “submit” after uploading your files. Please do not attach unrelated files. You will not be able to change your files after deadline.

Given a neural network, the structure is shown below. Each neuron in the neural network uses the logistic or sigmoid function () = !

!” $!” as activation function. %

[‘] is the output of the

linear part of )* neuron in layer ; % [‘] is the output of the activation part of )* neuron in layer .

1. [20 points] (Forward propagation) Given a training example (�⃗�, ), ∈ ℝ+, what is the

output of the neural network .?

2. [50 points] (Backpropagation) The loss function is defined by logistic loss function (.,) =

−[. + (1 − )(1 − .)] . Please derive the partial derivatives of loss function with

respect to parameters in the stochastic gradient descent update rules, that is, derive ,- ,.[$]

and

,- ,/[$]

, = 1,2,3.

! 0

+

….

…. ….

….

! [!] !

[!]

0 [!] 0

[!]

1 [!] 1

[!]

! [0] !

[0]

0 [0] 0

[0]

! [1] !

[1]

.