Fig. 2 shows the activations in the three hidden units of a shallow network. The slopes in the hidden units are 1.0, 1.0, and -1.0, respectively, and the joints in the hidden units are at positions 1/6, 2/6, and 4/6. Find values of ϕ0, ϕ1, ϕ2, and ϕ3 that will combine the hidden unit activations as ϕ0+ϕ1h1+ϕ2h2+ϕ3h3 to create a function with four linear regions that oscillate between output values of zero and one. The slope of the leftmost region should be positive, the next one negative, and so on. How many linear regions will we create if we compose this network with itself? How many will we create if we compose it with itself K times?

Question
0 5 months 0 Answers Reply 7 views

Fig. 2 shows the activations in the three hidden units of a shallow network. The slopes in the hidden units are 1.0, 1.0, and -1.0, respectively, and the joints in the hidden units are at positions 1/6, 2/6, and 4/6. Find values of ϕ0, ϕ1, ϕ2, and ϕ3 that will combine the hidden unit activations as ϕ0+ϕ1h1+ϕ2h2+ϕ3h3 to create a function with four linear regions that oscillate between output values of zero and one. The slope of the leftmost region should be positive, the next one negative, and so on. How many linear regions will we create if we compose this network with itself? How many will we create if we compose it with itself K times?

Question
0 5 months 0 Answers Reply 4 views

Show that: ReLU h β1+λ1 ·Ω1ReLU [β0+λ0 · Ω0x] i =λ0λ1 · ReLU 1 λ0λ1 β1+Ω1ReLU 1 λ0 β0+Ω0x , where λ0 and λ1 are non-negative scalars, Ωk is the weights matrix that are applied to the k-th layer (contribute to the (k + 1)-th layer) and βk is the the vector of biases that contribute to hidden layer k + 1. From this, we see that the weight matrices can be rescaled by any magnitude as long as the biases are also adjusted, and the scale factors can be re-applied at the end of the network. Hint: Use non-negative homogeneity property of the ReLU function.

Question
0 5 months 0 Answers Reply 1 views