Add ResNet training code

[yuyu2172]

Merge after #652
Merge after #432, #435, #321

Aims at reproducing scheme used in https://github.com/facebook/fb.resnet.torch.
Edit:
The training code reproduces the scheme introduced below.
This performs better than fb.resnet.torch and scales to the situation with many GPUs.
https://arxiv.org/pdf/1706.02677.pdf

• Linear Scaling Rule for lr (see the paper)
• Set default value of gamma for BNs in the end of bottleneck to 0
• use corrected momentum SGD
• Have correct scale for weight decay (ChainerMN accomplishes this by default)
• Learning rate scheduling happens at 30, 60, 80-th epoch.
• stop using color related data augmentation

Edit 11/16:
Although "ResNet in 1 hour" paper is new, the training scheme is adapted by some researchers (e.g. mixup).

[yuyu2172]

@Hakuyume
Please review this

[Hakuyume]

Both a weight and weights are used for "a set of parameters of a model". Which one is better?

[Hakuyume]

Don't we need 'cv2': True?

[yuyu2172]

I am checking how slow the training gets without options like autotune.

RESULT
After 7382 iterations with 32 GPUs

none:  1771s
`No `autotune`, no `chainer.cuda.set_max_workspace_size(1 * 1024 * 1024 * 1024)`:  1734s
`autotune`, `chainer.cuda.set_max_workspace_size(1 * 1024 * 1024 * 1024)`: 1776s

[yuyu2172]

Thanks for reviewing.
I reflected ours discussions to the code.

[Hakuyume]

Why did you use 5th level? (4th level #### is not enough?)

[Hakuyume]

H and W are not used?

[Hakuyume]

For the consistency with Trained model, it would be nice to add something like ~ of the models converted from Caffe model.

[Hakuyume]

he and fb should be distinguished from each other in score board.

[Hakuyume]

LGTM