caffe Batch normalization
Introduction
From the docs:
"Normalizes the input to have 0-mean and/or unit (1) variance across the batch.
This layer computes Batch Normalization as described in [1].
[...]
[1] S. Ioffe and C. Szegedy, "Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift." arXiv preprint arXiv:1502.03167 (2015)."
Parameters
| Parameter | Details |
|---|---|
| use_global_stats | From rohrbach's post from 2nd March 2016 - maybe he knows: |
| (use_global_stats) | "By default, during training time, the network is computing global mean/ variance statistics via a running average, which is then used at test time to allow deterministic outputs for each input. You can manually toggle whether the network is accumulating or using the statistics via the use_global_stats option. IMPORTANT: for this feature to work, you MUST set the learning rate to zero for all three parameter blobs, i.e., param {lr_mult: 0} three times in the layer definition. |
| (use_global_stats) | This means by default (as the following is set in batch_norm_layer.cpp), you don't have to set use_global_stats at all in the prototxt. use_global_stats_ = this->phase_ == TEST;" |
