Layer Activations
The most
straight-forward visualization technique is to show the activations of the
network during the forward pass. For ReLU networks, the activations usually
start out looking relatively blobby and dense, but as the training progresses
the activations usually become more sparse and localized. One dangerous pitfall
that can be easily noticed with this visualization is that some activation maps
may be all zero for many different inputs, which can indicate dead filters, and can be a symptom
of high learning rates.
Convolutional/FC Filters
The
second common strategy is to visualize the weights. These are usually most
interpretable on the first CONV layer which is looking directly at the raw
pixel data, but it is possible to also show the filter weights deeper in the
network. The weights are useful to visualize because well-trained networks
usually display nice and smooth filters without any noisy patterns. Noisy
patterns can be an indicator of a network that hasn't been trained for long
enough, or possibly a very low regularization strength that may have led to over
fitting.
Back propagation
Primary reason
we are interested in this problem is that in the specific case of Neural Networks, f will correspond to the
loss function ( L ) and the inputs x
will consist of the training data and the neural network weights. Training data
is given and fixed so it is a constant factor in the equation. So here we are
left with two variables which are weights and biases of each layer. In back
propagation Convolutional Neural Network compute the gradient at every layer according to the loss
function at output and these calculated new weights are then updated to
converse the network for the final solution.
No comments:
Post a Comment