Classifying Fashion-MNIST.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "3a1af8f1",
   "metadata": {},
   "source": [
    "# Classifying Fashion-MNIST\n",
    "\n",
    "Now it's your turn to build and train a neural network. You'll be using the [Fashion-MNIST dataset](https://github.com/zalandoresearch/fashion-mnist), a drop-in replacement for the MNIST dataset. MNIST is actually quite trivial with neural networks where you can easily achieve better than 97% accuracy. Fashion-MNIST is a set of 28x28 greyscale images of clothes. It's more complex than MNIST, so it's a better representation of the actual performance of your network, and a better representation of datasets you'll use in the real world.\n",
    "\n",
    "<img src='assets/fashion-mnist-sprite.png' width=500px>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a84623d2",
   "metadata": {},
   "source": [
    "## Import Resources"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "73725573",
   "metadata": {},
   "outputs": [],
   "source": [
    "import warnings\n",
    "warnings.filterwarnings('ignore')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0c131606",
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "%config InlineBackend.figure_format = 'retina'\n",
    "\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "import tensorflow as tf\n",
    "import tensorflow_datasets as tfds\n",
    "tfds.disable_progress_bar()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7ee20b03",
   "metadata": {},
   "outputs": [],
   "source": [
    "import logging\n",
    "logger = tf.get_logger()\n",
    "logger.setLevel(logging.ERROR)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e6a139a5",
   "metadata": {},
   "outputs": [],
   "source": [
    "print('Using:')\n",
    "print('\\t\\u2022 TensorFlow version:', tf.__version__)\n",
    "print('\\t\\u2022 tf.keras version:', tf.keras.__version__)\n",
    "print('\\t\\u2022 Running on GPU' if tf.test.is_gpu_available() else '\\t\\u2022 GPU device not found. Running on CPU')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "25900f07",
   "metadata": {},
   "source": [
    "## Load the Dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "97d40f69",
   "metadata": {},
   "outputs": [],
   "source": [
    "train_split = 60\n",
    "test_val_split = 20\n",
    "\n",
    "splits = tfds.Split.ALL.subsplit([train_split, test_val_split, test_val_split])\n",
    "\n",
    "dataset, dataset_info = tfds.load('fashion_mnist', split=splits, as_supervised=True, with_info=True)\n",
    "\n",
    "training_set, validation_set, test_set = dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6e581ee6",
   "metadata": {},
   "source": [
    "## Explore the Dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fbd40fcb",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Display dataset\n",
    "dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a6c8c0e0",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Display dataset_info\n",
    "dataset_info"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f7744a4e",
   "metadata": {},
   "outputs": [],
   "source": [
    "total_examples = dataset_info.splits['train'].num_examples + dataset_info.splits['test'].num_examples\n",
    "\n",
    "num_training_examples = (total_examples * train_split) // 100\n",
    "num_validation_examples = (total_examples * test_val_split) // 100\n",
    "num_test_examples = num_validation_examples\n",
    "\n",
    "print('There are {:,} images in the training set'.format(num_training_examples))\n",
    "print('There are {:,} images in the validation set'.format(num_validation_examples))\n",
    "print('There are {:,} images in the test set'.format(num_test_examples))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "508a629c",
   "metadata": {},
   "source": [
    "The images in this dataset are 28 $\\times$ 28 arrays, with pixel values in the range `[0, 255]`. The *labels* are an array of integers, in the range `[0, 9]`. These correspond to the *class* of clothing the image represents:\n",
    "\n",
    "<table>\n",
    "  <tr>\n",
    "    <th>Label</th>\n",
    "    <th>Class</th> \n",
    "  </tr>\n",
    "  <tr>\n",
    "    <td>0</td>\n",
    "    <td>T-shirt/top</td> \n",
    "  </tr>\n",
    "  <tr>\n",
    "    <td>1</td>\n",
    "    <td>Trouser</td> \n",
    "  </tr>\n",
    "    <tr>\n",
    "    <td>2</td>\n",
    "    <td>Pullover</td> \n",
    "  </tr>\n",
    "    <tr>\n",
    "    <td>3</td>\n",
    "    <td>Dress</td> \n",
    "  </tr>\n",
    "    <tr>\n",
    "    <td>4</td>\n",
    "    <td>Coat</td> \n",
    "  </tr>\n",
    "    <tr>\n",
    "    <td>5</td>\n",
    "    <td>Sandal</td> \n",
    "  </tr>\n",
    "    <tr>\n",
    "    <td>6</td>\n",
    "    <td>Shirt</td> \n",
    "  </tr>\n",
    "    <tr>\n",
    "    <td>7</td>\n",
    "    <td>Sneaker</td> \n",
    "  </tr>\n",
    "    <tr>\n",
    "    <td>8</td>\n",
    "    <td>Bag</td> \n",
    "  </tr>\n",
    "    <tr>\n",
    "    <td>9</td>\n",
    "    <td>Ankle boot</td> \n",
    "  </tr>\n",
    "</table>\n",
    "\n",
    "Each image is mapped to a single label. Since the *class names* are not included with the dataset, we create them here to use later when plotting the images:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e5329a22",
   "metadata": {},
   "outputs": [],
   "source": [
    "class_names = ['T-shirt/top', 'Trouser', 'Pullover', 'Dress', 'Coat', \n",
    "               'Sandal',      'Shirt',   'Sneaker',  'Bag',   'Ankle boot']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d3899373",
   "metadata": {},
   "outputs": [],
   "source": [
    "for image, label in training_set.take(1):\n",
    "    image = image.numpy().squeeze()\n",
    "    label = label.numpy()\n",
    "\n",
    "plt.imshow(image, cmap= plt.cm.binary)\n",
    "plt.colorbar()\n",
    "plt.show()\n",
    "\n",
    "print('The label of this image is:', label)\n",
    "print('The class name of this image is:', class_names[label])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c904f456",
   "metadata": {},
   "source": [
    "## Create Pipeline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "440064ac",
   "metadata": {},
   "outputs": [],
   "source": [
    "def normalize(image, label):\n",
    "    image = tf.cast(image, tf.float32)\n",
    "    image /= 255\n",
    "    return image, label\n",
    "\n",
    "batch_size = 64\n",
    "\n",
    "training_batches = training_set.cache().shuffle(num_training_examples//4).batch(batch_size).map(normalize).prefetch(1)\n",
    "validation_batches = validation_set.cache().batch(batch_size).map(normalize).prefetch(1)\n",
    "testing_batches = test_set.cache().batch(batch_size).map(normalize).prefetch(1)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "98dff923",
   "metadata": {},
   "source": [
    "## Build the Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "363c21aa",
   "metadata": {},
   "outputs": [],
   "source": [
    "model = tf.keras.Sequential([\n",
    "        tf.keras.layers.Flatten(input_shape=(28,28,1)),\n",
    "        tf.keras.layers.Dense(256, activation = 'relu'),\n",
    "        tf.keras.layers.Dense(128, activation = 'relu'),\n",
    "        tf.keras.layers.Dense(64, activation = 'relu'),\n",
    "        tf.keras.layers.Dense(10, activation = 'softmax')\n",
    "])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0c07e53e",
   "metadata": {},
   "source": [
    "## Train the Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "27c6605f",
   "metadata": {},
   "outputs": [],
   "source": [
    "model.compile(optimizer='adam',\n",
    "              loss='sparse_categorical_crossentropy',\n",
    "              metrics=['accuracy'])\n",
    "\n",
    "EPOCHS = 30\n",
    "\n",
    "history = model.fit(training_batches,\n",
    "                    epochs = EPOCHS,\n",
    "                    validation_data=validation_batches)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ec1bcefd",
   "metadata": {},
   "source": [
    "## Evaluate Loss and Accuracy on the Test Set"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2bb42138",
   "metadata": {},
   "outputs": [],
   "source": [
    "loss, accuracy = model.evaluate(testing_batches)\n",
    "\n",
    "print('\\nLoss on the TEST Set: {:,.3f}'.format(loss))\n",
    "print('Accuracy on the TEST Set: {:.3%}'.format(accuracy))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "47ec6786",
   "metadata": {},
   "source": [
    "## Loss and Validation Plots"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0a37743b",
   "metadata": {},
   "outputs": [],
   "source": [
    "training_accuracy = history.history['accuracy']\n",
    "validation_accuracy = history.history['val_accuracy']\n",
    "\n",
    "training_loss = history.history['loss']\n",
    "validation_loss = history.history['val_loss']\n",
    "\n",
    "epochs_range=range(EPOCHS)\n",
    "\n",
    "plt.figure(figsize=(8, 8))\n",
    "plt.subplot(1, 2, 1)\n",
    "plt.plot(epochs_range, training_accuracy, label='Training Accuracy')\n",
    "plt.plot(epochs_range, validation_accuracy, label='Validation Accuracy')\n",
    "plt.legend(loc='lower right')\n",
    "plt.title('Training and Validation Accuracy')\n",
    "\n",
    "plt.subplot(1, 2, 2)\n",
    "plt.plot(epochs_range, training_loss, label='Training Loss')\n",
    "plt.plot(epochs_range, validation_loss, label='Validation Loss')\n",
    "plt.legend(loc='upper right')\n",
    "plt.title('Training and Validation Loss')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "94727d82",
   "metadata": {},
   "source": [
    "## Early Stopping"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9160798d",
   "metadata": {},
   "outputs": [],
   "source": [
    "model = tf.keras.Sequential([\n",
    "        tf.keras.layers.Flatten(input_shape=(28,28,1)),\n",
    "        tf.keras.layers.Dense(256, activation = 'relu'),\n",
    "        tf.keras.layers.Dense(128, activation = 'relu'),\n",
    "        tf.keras.layers.Dense(64, activation = 'relu'),\n",
    "        tf.keras.layers.Dense(10, activation = 'softmax')\n",
    "])\n",
    "\n",
    "model.compile(optimizer='adam',\n",
    "              loss='sparse_categorical_crossentropy',\n",
    "              metrics=['accuracy'])\n",
    "\n",
    "# Stop training when there is no improvement in the validation loss for 5 consecutive epochs\n",
    "early_stopping = tf.keras.callbacks.EarlyStopping(monitor='val_loss', patience=5)\n",
    "\n",
    "history = model.fit(training_batches,\n",
    "                    epochs = 100,\n",
    "                    validation_data=validation_batches,\n",
    "                    callbacks=[early_stopping])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "62fa4490",
   "metadata": {},
   "outputs": [],
   "source": [
    "training_accuracy = history.history['accuracy']\n",
    "validation_accuracy = history.history['val_accuracy']\n",
    "\n",
    "training_loss = history.history['loss']\n",
    "validation_loss = history.history['val_loss']\n",
    "\n",
    "epochs_range=range(len(training_accuracy))\n",
    "\n",
    "plt.figure(figsize=(8, 8))\n",
    "plt.subplot(1, 2, 1)\n",
    "plt.plot(epochs_range, training_accuracy, label='Training Accuracy')\n",
    "plt.plot(epochs_range, validation_accuracy, label='Validation Accuracy')\n",
    "plt.legend(loc='lower right')\n",
    "plt.title('Training and Validation Accuracy')\n",
    "\n",
    "plt.subplot(1, 2, 2)\n",
    "plt.plot(epochs_range, training_loss, label='Training Loss')\n",
    "plt.plot(epochs_range, validation_loss, label='Validation Loss')\n",
    "plt.legend(loc='upper right')\n",
    "plt.title('Training and Validation Loss')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4553f282",
   "metadata": {},
   "source": [
    "## Dropout"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ba750b0f",
   "metadata": {},
   "outputs": [],
   "source": [
    "model = tf.keras.Sequential([\n",
    "        tf.keras.layers.Flatten(input_shape=(28,28,1)),\n",
    "        tf.keras.layers.Dense(256, activation = 'relu'),\n",
    "        tf.keras.layers.Dropout(0.2),\n",
    "        tf.keras.layers.Dense(128, activation = 'relu'),\n",
    "        tf.keras.layers.Dropout(0.2),\n",
    "        tf.keras.layers.Dense(64, activation = 'relu'),\n",
    "        tf.keras.layers.Dropout(0.2),\n",
    "        tf.keras.layers.Dense(10, activation = 'softmax')\n",
    "])\n",
    "\n",
    "model.compile(optimizer='adam',\n",
    "              loss='sparse_categorical_crossentropy',\n",
    "              metrics=['accuracy'])\n",
    "\n",
    "\n",
    "history = model.fit(training_batches,\n",
    "                    epochs = 30,\n",
    "                    validation_data=validation_batches)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a5879a41",
   "metadata": {},
   "outputs": [],
   "source": [
    "training_accuracy = history.history['accuracy']\n",
    "validation_accuracy = history.history['val_accuracy']\n",
    "\n",
    "training_loss = history.history['loss']\n",
    "validation_loss = history.history['val_loss']\n",
    "\n",
    "epochs_range=range(len(training_accuracy))\n",
    "\n",
    "plt.figure(figsize=(8, 8))\n",
    "plt.subplot(1, 2, 1)\n",
    "plt.plot(epochs_range, training_accuracy, label='Training Accuracy')\n",
    "plt.plot(epochs_range, validation_accuracy, label='Validation Accuracy')\n",
    "plt.legend(loc='lower right')\n",
    "plt.title('Training and Validation Accuracy')\n",
    "\n",
    "plt.subplot(1, 2, 2)\n",
    "plt.plot(epochs_range, training_loss, label='Training Loss')\n",
    "plt.plot(epochs_range, validation_loss, label='Validation Loss')\n",
    "plt.legend(loc='upper right')\n",
    "plt.title('Training and Validation Loss')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "36669b3c",
   "metadata": {},
   "source": [
    "## Inference"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fd96772e",
   "metadata": {},
   "outputs": [],
   "source": [
    "for image_batch, label_batch in testing_batches.take(1):\n",
    "    ps = model.predict(image_batch)\n",
    "    images = image_batch.numpy().squeeze()\n",
    "    labels = label_batch.numpy()\n",
    "\n",
    "\n",
    "plt.figure(figsize=(10,15))\n",
    "\n",
    "for n in range(30):\n",
    "    plt.subplot(6,5,n+1)\n",
    "    plt.imshow(images[n], cmap = plt.cm.binary)\n",
    "    color = 'green' if np.argmax(ps[n]) == labels[n] else 'red'\n",
    "    plt.title(class_names[np.argmax(ps[n])], color=color)\n",
    "    plt.axis('off')"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}