1. Neural Network. Consider a neural net for a binary classification which has one hidden layer
as shown in the figure below. We use a linear activation function a(z) = cz at hidden units and
a sigmoid activation function a(z) = 1/(1 + e z ) at the output unit to learn the function for
P (y = 1|x, w) where x = (x1 , x2 ) and w = (w1 , w2 , . . . , w9 ).
(a) What is the output P (y = 1|x, w) from the above neural net? Express it in terms of xi , c and
weights wi . What is the final classification boundary?
(b) Draw a neural net with no hidden layer which is equivalent to the given neural net, and write
weights w? of this new neural net in terms of c and wi .
(c) Is it true that any multi-layered neural net with linear activation functions at hidden layers
can be represented as a neural net without any hidden layer? Explain your answer.
2. Feedforward Neural Network. Consider a three-layer neural network to learn a function
f : X ! Y , where X = [X1 , X2 ] consists of two features. The weights w1 , . . . , w6 can be
arbitrary. There are two possible choices for the function implemented by each unit in this network:
1
: sigmoid function, S(z) = 1+exp(
,
z)
1
: linear function, L(z) = cz,
P
where in both cases z = i wi Xi . Assign proper activation functions (S or L) to each unit in the
1
following graph so that we can generate functions of the form f (X1 , X2 ) = 1+exp( 1 X
at
1 + 2 X2 )
the output of the neural network Y . Derive 1 and 2 as a function of w1 , . . . , w6 and c.
2
3

Purchase answer to see full
attachment