Skip to content
This repository has been archived by the owner on Oct 10, 2020. It is now read-only.
/ xnor-net Public archive
forked from jiecaoyu/XNOR-Net-PyTorch

PyTorch Implementation of XNOR-Net

Notifications You must be signed in to change notification settings

lee-man/xnor-net

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

82 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

XNOR-NET

This repo originates from XNOR-NET-PyTorch and cv-tricks.com. Several modifications are made for my own research purpose. I will list the modifications in details then.

Modifications

XNOR-NET-PYTORCH

This a PyTorch implementation of the XNOR-Net. I implemented Binarized Neural Network (BNN) for:

Dataset Network                 Accuracy                   Accuracy of floating-point
MNIST LeNet-5 99.23% 99.34%
CIFAR-10 Network-in-Network (NIN) 86.28% 89.67%
ImageNet AlexNet Top-1: 44.87% Top-5: 69.70% Top-1: 57.1% Top-5: 80.2%

MNIST

I implemented the LeNet-5 structure for the MNIST dataset. I am using the dataset reader provided by torchvision. To run the training:

$ cd <Repository Root>/MNIST/
$ python main.py

Pretrained model can be downloaded here. To evaluate the pretrained model:

$ cp <Pretrained Model> <Repository Root>/MNIST/models/
$ python main.py --pretrained models/LeNet_5.best.pth.tar --evaluate

CIFAR-10

I implemented the NIN structure for the CIFAR-10 dataset. You can download the training and validation datasets here and uncompress the .zip file. To run the training:

$ cd <Repository Root>/CIFAR_10/
$ ln -s <Datasets Root> data
$ python main.py

Pretrained model can be downloaded here. To evaluate the pretrained model:

$ cp <Pretrained Model> <Repository Root>/CIFAR_10/models/
$ python main.py --pretrained models/nin.best.pth.tar --evaluate

ImageNet

I implemented the AlexNet for the ImageNet dataset.

Dataset

The training supports torchvision.

If you have installed Caffe, you can download the preprocessed dataset here and uncompress it. To set up the dataset:

$ cd <Repository Root>/ImageNet/networks/
$ ln -s <Datasets Root> data

AlexNet

To train the network:

$ cd <Repository Root>/ImageNet/networks/
$ python main.py # add "--caffe-data" if you are training with the Caffe dataset

The pretrained models can be downloaded here: pretrained with Caffe dataset; pretrained with torchvision. To evaluate the pretrained model:

$ cp <Pretrained Model> <Repository Root>/ImageNet/networks/
$ python main.py --resume alexnet.baseline.pth.tar --evaluate # add "--caffe-data" if you are training with the Caffe dataset

The training log can be found here: log - Caffe dataset; log - torchvision.

Todo

  • NIN for ImageNet.

Notes

Gradients of scaled sign function

In the paper, the gradient in backward after the scaled sign function is

equation

However, this equation is actually inaccurate. The correct backward gradient should be

equation

Details about this correction can be found in the notes (section 1).

About

PyTorch Implementation of XNOR-Net

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Python 93.5%
  • TeX 6.5%