Skip to content

A Residual Network Design with less than 5 million trainable parameters achieving an accuracy of 96.04% on CIFAR-10.

License

Notifications You must be signed in to change notification settings

Nikunj-Gupta/Efficient_ResNets

 
 

Repository files navigation

Efficient ResNets: Residual Network Design

Paper: Efficient ResNets: Residual Network Design

Authors: Nikunj Gupta, Harish Chauhan, Aditya Thakur

Summary: We constructed a Residual Network Design with less than 5 million trainable parameters. We achieved an accuracy of 96.04% on CIFAR-10 dataset by using best-suited hyperparameters and multiple training strategies like data normalization, data augmentation, optimizers, gradient clipping, etc.

Introduction

ResNets (or Residual Networks) are one of the most commonly used models for image classification 5 tasks. In this project, you will design and train your own ResNet model for CIFAR-10 image 6 classification. In particular, your goal will be to maximize accuracy on the CIFAR-10 benchmark 7 while keeping the size of your ResNet model under budget. Model size, typically measured as the 8 number or trainable parameters, is important when models need to be stored on devices with limited 9 storage capacity, mobile devices for example.

Prerequisites

  • Python 3.6+
  • PyTorch 1.0+

To install all the dependencies, execute: pip install -r requirements.txt

Description of files in the repository

  • models/resnet.py : PyTorch description of ResNet model architecture (flexible to change/modify using config.yaml)
  • main.py : code to train and test ResNet architectures
  • config.yaml : contains the hyperparamters used for constructing and training a ResNet architecture
  • project1_model.pt : Trained parameters/weights for our final model.
  • project1_model.py : ResNet architecture used.

Training

# Start training with: 
python3 main.py  --config <path_to_config> --resnet_architecture <architecture_id>

To modify and test with new ResNet architectures, you can create a new configuration experiment in config.yaml. Currently, it includes descriptions for our model and ResNet18.

Reproduce the results

Train our best modified ResNet Architecture with:

python3 main.py  --config resnet_configs/config.yaml --resnet_architecture best_model

We have set the above as our default inputs in main.py and hence the following will reproduce our results too:

python3 main.py 

Train ResNet18 Architecture with:

python3 main.py  --config resnet_configs/config.yaml --resnet_architecture resnet18

Accuracy

Model Acc.
Our ResNet 96.04%
ResNet18 88.56%

Hyperparameters in our final model's architecture

Parameter Our Model
number of residual layers 3
number of residual blocks [4, 4, 3]
convolutional kernel sizes [3, 3, 3]
shortcut kernel sizes [1, 1, 1]
number of channels 64
average pool kernel size 8
batch normalization True
dropout 0
squeeze and excitation True
gradient clip 0.1
data augmentation True
data normalization True
lookahead True
optimizer SGD
learning rate (lr) 0.1
lr scheduler CosineAnnealingLR
weight decay 0.0005
batch size 128
number of workers 16
Total number of Parameters 4,697,742

Cite

@article{thakur2023efficient,
  title={Efficient ResNets: Residual Network Design},
  author={Thakur, Aditya and Chauhan, Harish and Gupta, Nikunj},
  journal={arXiv preprint arXiv:2306.12100},
  year={2023}
}

About

A Residual Network Design with less than 5 million trainable parameters achieving an accuracy of 96.04% on CIFAR-10.

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Python 90.3%
  • Shell 9.7%