notebooks/supervised/classification/ensemble/classification_adaboost.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "clear-illustration",
   "metadata": {},
   "source": [
    "## Classification with AdaBoost"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "latter-morning",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "\n",
    "from sklearn import ensemble, datasets, metrics, model_selection, preprocessing, pipeline"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "hidden-aerospace",
   "metadata": {},
   "source": [
    "Load the data set"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "distinct-scene",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      ".. _wine_dataset:\n",
      "\n",
      "Wine recognition dataset\n",
      "------------------------\n",
      "\n",
      "**Data Set Characteristics:**\n",
      "\n",
      "    :Number of Instances: 178 (50 in each of three classes)\n",
      "    :Number of Attributes: 13 numeric, predictive attributes and the class\n",
      "    :Attribute Information:\n",
      " \t\t- Alcohol\n",
      " \t\t- Malic acid\n",
      " \t\t- Ash\n",
      "\t\t- Alcalinity of ash  \n",
      " \t\t- Magnesium\n",
      "\t\t- Total phenols\n",
      " \t\t- Flavanoids\n",
      " \t\t- Nonflavanoid phenols\n",
      " \t\t- Proanthocyanins\n",
      "\t\t- Color intensity\n",
      " \t\t- Hue\n",
      " \t\t- OD280/OD315 of diluted wines\n",
      " \t\t- Proline\n",
      "\n",
      "    - class:\n",
      "            - class_0\n",
      "            - class_1\n",
      "            - class_2\n",
      "\t\t\n",
      "    :Summary Statistics:\n",
      "    \n",
      "    ============================= ==== ===== ======= =====\n",
      "                                   Min   Max   Mean     SD\n",
      "    ============================= ==== ===== ======= =====\n",
      "    Alcohol:                      11.0  14.8    13.0   0.8\n",
      "    Malic Acid:                   0.74  5.80    2.34  1.12\n",
      "    Ash:                          1.36  3.23    2.36  0.27\n",
      "    Alcalinity of Ash:            10.6  30.0    19.5   3.3\n",
      "    Magnesium:                    70.0 162.0    99.7  14.3\n",
      "    Total Phenols:                0.98  3.88    2.29  0.63\n",
      "    Flavanoids:                   0.34  5.08    2.03  1.00\n",
      "    Nonflavanoid Phenols:         0.13  0.66    0.36  0.12\n",
      "    Proanthocyanins:              0.41  3.58    1.59  0.57\n",
      "    Colour Intensity:              1.3  13.0     5.1   2.3\n",
      "    Hue:                          0.48  1.71    0.96  0.23\n",
      "    OD280/OD315 of diluted wines: 1.27  4.00    2.61  0.71\n",
      "    Proline:                       278  1680     746   315\n",
      "    ============================= ==== ===== ======= =====\n",
      "\n",
      "    :Missing Attribute Values: None\n",
      "    :Class Distribution: class_0 (59), class_1 (71), class_2 (48)\n",
      "    :Creator: R.A. Fisher\n",
      "    :Donor: Michael Marshall (MARSHALL%PLU@io.arc.nasa.gov)\n",
      "    :Date: July, 1988\n",
      "\n",
      "This is a copy of UCI ML Wine recognition datasets.\n",
      "https://archive.ics.uci.edu/ml/machine-learning-databases/wine/wine.data\n",
      "\n",
      "The data is the results of a chemical analysis of wines grown in the same\n",
      "region in Italy by three different cultivators. There are thirteen different\n",
      "measurements taken for different constituents found in the three types of\n",
      "wine.\n",
      "\n",
      "Original Owners: \n",
      "\n",
      "Forina, M. et al, PARVUS - \n",
      "An Extendible Package for Data Exploration, Classification and Correlation. \n",
      "Institute of Pharmaceutical and Food Analysis and Technologies,\n",
      "Via Brigata Salerno, 16147 Genoa, Italy.\n",
      "\n",
      "Citation:\n",
      "\n",
      "Lichman, M. (2013). UCI Machine Learning Repository\n",
      "[https://archive.ics.uci.edu/ml]. Irvine, CA: University of California,\n",
      "School of Information and Computer Science. \n",
      "\n",
      ".. topic:: References\n",
      "\n",
      "  (1) S. Aeberhard, D. Coomans and O. de Vel, \n",
      "  Comparison of Classifiers in High Dimensional Settings, \n",
      "  Tech. Rep. no. 92-02, (1992), Dept. of Computer Science and Dept. of  \n",
      "  Mathematics and Statistics, James Cook University of North Queensland. \n",
      "  (Also submitted to Technometrics). \n",
      "\n",
      "  The data was used with many others for comparing various \n",
      "  classifiers. The classes are separable, though only RDA \n",
      "  has achieved 100% correct classification. \n",
      "  (RDA : 100%, QDA 99.4%, LDA 98.9%, 1NN 96.1% (z-transformed data)) \n",
      "  (All results using the leave-one-out technique) \n",
      "\n",
      "  (2) S. Aeberhard, D. Coomans and O. de Vel, \n",
      "  \"THE CLASSIFICATION PERFORMANCE OF RDA\" \n",
      "  Tech. Rep. no. 92-01, (1992), Dept. of Computer Science and Dept. of \n",
      "  Mathematics and Statistics, James Cook University of North Queensland. \n",
      "  (Also submitted to Journal of Chemometrics).\n",
      "\n"
     ]
    }
   ],
   "source": [
    "wine = datasets.load_wine()\n",
    "print(wine.DESCR)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "empty-scratch",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "X = pd.DataFrame(wine.data, columns=wine.feature_names)\n",
    "y = wine.target"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "straight-copyright",
   "metadata": {},
   "source": [
    "Stratify the data by the target label"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "interpreted-cheat",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "train samples: 89\n",
      "test samples 89\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAEGCAYAAABiq/5QAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAVdklEQVR4nO3dfZBV9Z3n8fdX6LEhMkCgJShBGMZypNS0sWWSMTXxIUQ0M0El6pr4QNbddtaHMtkJpZlKnJjKbNwyMepuYoopEV0lRiVKJtEd1CVrfIrTMIyiENGMjq1GOrAoqDhCvvvHvWCLPNzGPvfQnPer6laf8zvn3PO91frh9O/87u9EZiJJqo69yi5AktRcBr8kVYzBL0kVY/BLUsUY/JJUMYPLLqARo0ePzgkTJpRdhiQNKIsXL/5dZrZt3T4ggn/ChAl0dXWVXYYkDSgR8fy22u3qkaSKMfglqWIMfkmqmAHRxy9JffX222/T3d3Nhg0byi6lcK2trYwbN46WlpaG9jf4Je2Ruru7GTZsGBMmTCAiyi6nMJnJ6tWr6e7uZuLEiQ0dY1ePpD3Shg0bGDVq1B4d+gARwahRo/r0l43BL2mPtaeH/mZ9/ZwGvyRVjMEvSTuxdu1afvCDH5RdRr/x5q70PvzbNw8tu4Q+G3/ZE2WXMOBsDv7zzz+/7FL6hVf8krQTl156Kc8++yzt7e2ceuqp3HXXXVu2feELX2DBggXMnTuX6dOnc/TRR3PggQdy+eWXb9nn5ptvZsqUKbS3t3PeeeexadOmEj7FOwoL/ohojYjHIuJfIuLJiLi83j43Iv41IpbWX+1F1SBJ/eGKK65g0qRJLF26lAsvvJC5c+cC8Oqrr/Lwww/zmc98BoDHHnuM+fPn8/jjj3P77bfT1dXF8uXL+fGPf8xDDz3E0qVLGTRoELfcckuJn6bYrp63gGMzc31EtAAPRsQ99W2zMvOOAs8tSYX45Cc/yfnnn09PTw/z589nxowZDB5ci9KpU6cyatQoAE455RQefPBBBg8ezOLFiznyyCMBePPNN9l3331Lqx8KDP6sPcV9fX21pf7yye6SBryzzz6bm2++mVtvvZUbbrhhS/vWwyojgszknHPO4dvf/nazy9yuQm/uRsQgYDHwx8D3M/NXEfFfgL+LiMuA+4FLM/OtbRzbCXQCjB8/vsgytRs5YtZNZZfQJ3cOK7sCNcOwYcNYt27dlvWZM2cyZcoUPvShDzF58uQt7ffeey9r1qxhyJAh3HXXXcyZM4ehQ4cyffp0vvzlL7PvvvuyZs0a1q1bxwEHHFDGRwEKvrmbmZsysx0YB0yJiEOArwJ/AhwJfBC4ZDvHzs7MjszsaGt7z3MEJKlpRo0axVFHHcUhhxzCrFmzGDNmDAcffDBf/OIX37XflClTmDFjBocddhgzZsygo6ODyZMn861vfYtPf/rTHHbYYUydOpWXX365pE9S05ThnJm5NiIWAdMy8zv15rci4gbgK82oQZLej3nz5m1ZfuONN1i5ciVnnHHGu/YZN27cu0b8bHb66adz+umnF11iw4oc1dMWESPqy0OAqcCKiBhbbwvgJGBZUTVIUn+77777OPjgg7nooosYPnx42eXskiKv+McCN9b7+fcCbsvMn0XE/4mINiCApcBfFViDJPWrT33qUzz//HufaDhz5kxmzpzZ/IJ2QZGjeh4HDt9G+7FFnVOStHN+c1eSKsbgl6SKMfglqWKcnVNSJfT3lwMXX3n2TvdZu3Yt8+bN6/OsnieeeCLz5s1jxIgRu1jdjnnFL0kF2d48/hs3btzhcXfffXdhoQ9e8UtSYXpP59zS0kJraysjR45kxYoVPP3005x00km88MILbNiwgYsvvpjOzk4AJkyYQFdXF+vXr+eEE07gE5/4BA8//DD7778/CxYsYMiQIe+rLq/4JakgvadzvvLKK1myZAnXXHMNTz/9NABz5sxh8eLFdHV1ce2117J69er3vMfKlSu54IILePLJJxkxYgTz589/33V5xS9JTTJlyhQmTpy4Zf3aa6/lzjvvBOCFF15g5cqVW6Z13mzixIm0t7cDcMQRR/Dcc8+97zoMfklqkg984ANbln/xi19w33338cgjjzB06FCOPvpoNmzY8J5j9t577y3LgwYN4s0333zfddjVI0kF2Xo6595effVVRo4cydChQ1mxYgWPPvpo0+ryil9SJTQy/LK/9Z7OeciQIYwZM2bLtmnTpvHDH/6Qgw8+mIMOOoiPfexjTavL4JekAvWezrm3vffem3vuuWeb2zb3448ePZply96ZwPgrX+mfWezt6pGkijH4JaliDH5JqhiDX5IqxuCXpIox+CWpYhzOKakS/u2bh/br+42/7Imd7rOr0zIDXH311XR2djJ06NBdKW+HvOKXpIJsb1rmRlx99dW88cYb/VxRTWFX/BHRCjwA7F0/zx2Z+bcRMRG4FRgFLAbOysx/L6oOSSpL72mZp06dyr777sttt93GW2+9xcknn8zll1/O66+/zmmnnUZ3dzebNm3i61//Oq+88govvfQSxxxzDKNHj2bRokX9WleRXT1vAcdm5vqIaAEejIh7gP8KfC8zb42IHwLnAtcVWIckleKKK65g2bJlLF26lIULF3LHHXfw2GOPkZl89rOf5YEHHqCnp4f99tuPn//850BtDp/hw4dz1VVXsWjRIkaPHt3vdRXW1ZM16+urLfVXAscCd9TbbwROKqoGSdpdLFy4kIULF3L44Yfz0Y9+lBUrVrBy5UoOPfRQ7r33Xi655BJ++ctfMnz48MJrKfTmbkQMotad88fA94FngbWZufm5Y93A/ts5thPoBBg/fnzD5+zv52o2QxmTR0lqrszkq1/9Kuedd957ti1ZsoS7776br33taxx33HFcdtllhdZS6M3dzNyUme3AOGAK8Cd9OHZ2ZnZkZkdbW1tRJUpSYXpPy3z88cczZ84c1q+vdYS8+OKLrFq1ipdeeomhQ4dy5plnMmvWLJYsWfKeY/tbU4ZzZubaiFgEfBwYERGD61f944AXm1GDpGprZPhlf+s9LfMJJ5zA5z//eT7+8Y8DsM8++3DzzTfzzDPPMGvWLPbaay9aWlq47rraLc/Ozk6mTZvGfvvtN3Bu7kZEG/B2PfSHAFOB/w4sAj5HbWTPOcCComqQpLJtPS3zxRdf/K71SZMmcfzxx7/nuIsuuoiLLrqokJqKvOIfC9xY7+ffC7gtM38WEU8Bt0bEt4B/Bq4vsAZJ0lYKC/7MfBw4fBvtv6HW3y9JKoFTNuwG+vur5EUro69U2hWZSUSUXUbhMrNP+ztlg6Q9UmtrK6tXr+5zKA40mcnq1atpbW1t+Biv+CXtkcaNG0d3dzc9PT1ll1K41tZWxo0b1/D+Br+kPVJLSwsTJ04su4zdkl09klQxBr8kVYzBL0kVY/BLUsUY/JJUMQa/JFWMwS9JFWPwS1LFGPySVDEGvyRVjMEvSRVj8EtSxRj8klQxBr8kVYzBL0kVY/BLUsUUFvwR8eGIWBQRT0XEkxFxcb39GxHxYkQsrb9OLKoGSdJ7FfkEro3AX2fmkogYBiyOiHvr276Xmd8p8NySpO0oLPgz82Xg5fryuohYDuxf1PkkSY1pSh9/REwADgd+VW+6MCIej4g5ETFyO8d0RkRXRHRV4WHJktQshQd/ROwDzAe+lJmvAdcBk4B2an8RfHdbx2Xm7MzsyMyOtra2osuUpMooNPgjooVa6N+SmT8ByMxXMnNTZv4e+HtgSpE1SJLerchRPQFcDyzPzKt6tY/ttdvJwLKiapAkvVeRo3qOAs4CnoiIpfW2vwHOiIh2IIHngPMKrEGStJUiR/U8CMQ2Nt1d1DklSTtX5BW/pAo5YtZNZZfQZ3cOu7LsEvps/GVPvO/3cMoGSaoYg1+SKsbgl6SKMfglqWIMfkmqGINfkirG4JekijH4JaliDH5JqhiDX5IqxuCXpIox+CWpYgx+SaoYg1+SKsbgl6SKMfglqWIMfkmqmIaCPyLub6RNkrT72+GjFyOiFRgKjI6IkbzzDN0/BPYvuDZJUgF29szd84AvAfsBi3kn+F8D/ueODoyIDwM3AWOABGZn5jUR8UHgx8AE4DngtMz8f7tWviSpr3bY1ZOZ12TmROArmflHmTmx/vpIZu4w+IGNwF9n5mTgY8AFETEZuBS4PzMPBO6vr0uSmmRnV/wAZOb/iIg/o3aVPrhX+007OOZl4OX68rqIWE6te2g6cHR9txuBXwCX9L10SdKuaCj4I+J/AZOApcCmenNS68pp5PgJwOHAr4Ax9X8UAH5LrStoW8d0Ap0A48ePb+Q0kqQGNBT8QAcwOTOzryeIiH2A+cCXMvO1iNiyLTMzIrb5npk5G5gN0NHR0efzSpK2rdFx/MuAD/X1zSOihVro35KZP6k3vxIRY+vbxwKr+vq+kqRd1+gV/2jgqYh4DHhrc2NmfnZ7B0Tt0v56YHlmXtVr00+Bc4Ar6j8X9LVoSdKuazT4v7EL730UcBbwREQsrbf9DbXAvy0izgWeB07bhfeWJO2iRkf1/N++vnFmPsg74/63dlxf30+S1D8aHdWzjtooHoA/AFqA1zPzD4sqTJJUjEav+IdtXq733U+n9qUsSdIA0+fZObPmLuD4/i9HklS0Rrt6Tum1uhe1cf0bCqlIklSoRkf1/GWv5Y3UJleb3u/VSJIK12gf/xeLLkSS1ByNPohlXETcGRGr6q/5ETGu6OIkSf2v0Zu7N1D7xu1+9dc/1NskSQNMo8Hflpk3ZObG+msu0FZgXZKkgjQa/Ksj4syIGFR/nQmsLrIwSVIxGg3+/0htTp3fUnu4yueAmQXVJEkqUKPDOb8JnLP52bj15+Z+h9o/CJKkAaTRK/7Dej8QPTPXUHuiliRpgGk0+PeKiJGbV+pX/I3+tSBJ2o00Gt7fBR6JiNvr66cCf1dMSZKkIjX6zd2bIqILOLbedEpmPlVcWZKkojTcXVMPesNekga4Pk/LLEka2Ax+SaqYwoI/IubUJ3Rb1qvtGxHxYkQsrb9OLOr8kqRtK/KKfy4wbRvt38vM9vrr7gLPL0nahsKCPzMfANYU9f6SpF1TRh//hRHxeL0raOTOd5ck9admB/91wCSgndpkb9/d3o4R0RkRXRHR1dPT06TyJGnP19Tgz8xXMnNTZv4e+Htgyg72nZ2ZHZnZ0dbm1P+S1F+aGvwRMbbX6snAsu3tK0kqRmETrUXEj4CjgdER0Q38LXB0RLQDCTwHnFfU+SVJ21ZY8GfmGdtovr6o80mSGuM3dyWpYgx+SaoYg1+SKsbgl6SKMfglqWIMfkmqGINfkirG4JekijH4JaliDH5JqhiDX5IqxuCXpIox+CWpYgx+SaoYg1+SKsbgl6SKMfglqWIMfkmqGINfkirG4Jekiiks+CNiTkSsiohlvdo+GBH3RsTK+s+RRZ1fkrRtRV7xzwWmbdV2KXB/Zh4I3F9flyQ1UWHBn5kPAGu2ap4O3FhfvhE4qajzS5K2rdl9/GMy8+X68m+BMdvbMSI6I6IrIrp6enqaU50kVUBpN3czM4HcwfbZmdmRmR1tbW1NrEyS9mzNDv5XImIsQP3nqiafX5Iqr9nB/1PgnPryOcCCJp9fkiqvyOGcPwIeAQ6KiO6IOBe4ApgaESuBT9XXJUlNNLioN87MM7az6biizilJ2jm/uStJFWPwS1LFGPySVDEGvyRVjMEvSRVj8EtSxRj8klQxBr8kVYzBL0kVY/BLUsUY/JJUMQa/JFWMwS9JFWPwS1LFGPySVDEGvyRVjMEvSRVj8EtSxRj8klQxBr8kVUxhD1vfkYh4DlgHbAI2ZmZHGXVIUhWVEvx1x2Tm70o8vyRVkl09klQxZQV/AgsjYnFEdG5rh4jojIiuiOjq6elpcnmStOcqK/g/kZkfBU4ALoiIP996h8ycnZkdmdnR1tbW/AolaQ9VSvBn5ov1n6uAO4EpZdQhSVXU9OCPiA9ExLDNy8CngWXNrkOSqqqMUT1jgDsjYvP552Xm/y6hDkmqpKYHf2b+BvhIs88rSapxOKckVYzBL0kVY/BLUsUY/JJUMQa/JFWMwS9JFWPwS1LFGPySVDEGvyRVjMEvSRVj8EtSxRj8klQxBr8kVYzBL0kVY/BLUsUY/JJUMQa/JFWMwS9JFWPwS1LFGPySVDGlBH9ETIuIX0fEMxFxaRk1SFJVNT34I2IQ8H3gBGAycEZETG52HZJUVWVc8U8BnsnM32TmvwO3AtNLqEOSKikys7knjPgcMC0z/1N9/SzgTzPzwq326wQ666sHAb9uaqHNNRr4XdlFaJf4uxvY9vTf3wGZ2bZ14+AyKmlEZs4GZpddRzNERFdmdpRdh/rO393AVtXfXxldPS8CH+61Pq7eJklqgjKC/5+AAyNiYkT8AfAfgJ+WUIckVVLTu3oyc2NEXAj8IzAImJOZTza7jt1MJbq09lD+7ga2Sv7+mn5zV5JULr+5K0kVY/BLUsUY/CVy6oqBKyLmRMSqiFhWdi3qu4j4cEQsioinIuLJiLi47JqayT7+ktSnrngamAp0UxvtdEZmPlVqYWpIRPw5sB64KTMPKbse9U1EjAXGZuaSiBgGLAZOqsr/f17xl8epKwawzHwAWFN2Hdo1mflyZi6pL68DlgP7l1tV8xj85dkfeKHXejcV+g9P2l1ExATgcOBXJZfSNAa/pMqKiH2A+cCXMvO1sutpFoO/PE5dIZUoIlqohf4tmfmTsutpJoO/PE5dIZUkIgK4HliemVeVXU+zGfwlycyNwOapK5YDtzl1xcARET8CHgEOiojuiDi37JrUJ0cBZwHHRsTS+uvEsotqFodzSlLFeMUvSRVj8EtSxRj8klQxBr8kVYzBL0kVY/Cr8iJiRESc34TznBQRk4s+j7QzBr8EI4CGgz9qduX/nZMAg1+lcxy/Ki8iNs+M+mtgEXAYMBJoAb6WmQvqE3n9I7WJvI4ATgTOBs4EeqhNuLc4M78TEZOA7wNtwBvAfwY+CPwMeLX+mpGZzzbrM0q9Nf1h69Ju6FLgkMxsj4jBwNDMfC0iRgOPRsTmqTQOBM7JzEcj4khgBvARav9ALKE2pzvUHuD9V5m5MiL+FPhBZh5bf5+fZeYdzfxw0tYMfundAvhv9Qet/J7aVNlj6tuez8xH68tHAQsycwOwISL+AbbM9vhnwO216WAA2LtZxUuNMPild/sCtS6aIzLz7Yh4Dmitb3u9geP3AtZmZnsx5Unvnzd3JVgHDKsvDwdW1UP/GOCA7RzzEPCXEdFav8r/C4D6nO7/GhGnwpYbwR/Zxnmk0hj8qrzMXA08VH9wejvQERFPULt5u2I7x/wTtWm0HwfuAZ6gdtMWan81nBsR/wI8yTuP1LwVmBUR/1y/ASyVwlE90i6KiH0yc31EDAUeADo3P8dV2p3Zxy/tutn1L2S1Ajca+hoovOKXpIqxj1+SKsbgl6SKMfglqWIMfkmqGINfkirm/wPjXfPkVn2NFAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "X_train, X_test, y_train, y_test = model_selection.train_test_split(X, y, train_size=0.5, stratify=y)\n",
    "\n",
    "df_train = pd.DataFrame(y_train, columns=['target'])\n",
    "df_train['type'] = 'train'\n",
    "\n",
    "df_test = pd.DataFrame(y_test, columns=['target'])\n",
    "df_test['type'] = 'test'\n",
    "\n",
    "df_set = df_train.append(df_test)\n",
    "\n",
    "_ = sns.countplot(x='target', hue='type', data=df_set)     \n",
    "\n",
    "print('train samples:', len(X_train))\n",
    "print('test samples', len(X_test))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "vocational-fortune",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "AdaBoostClassifier()"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model = ensemble.AdaBoostClassifier(n_estimators=50, algorithm='SAMME.R')\n",
    "model.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "registered-differential",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>28</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>34</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>23</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    0   1   2\n",
       "0  28   1   0\n",
       "1   2  34   0\n",
       "2   0   1  23"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "predicted = model.predict(X_test)\n",
    "\n",
    "confusion_matrix = pd.DataFrame(metrics.confusion_matrix(y_test, predicted))\n",
    "confusion_matrix"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "settled-quality",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAVoAAAD4CAYAAACt8i4nAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAVyElEQVR4nO3df5xVdZ3H8ffnzh1EQBJCLpNOiIIYYqFiapgSRhHWAuLPNpd6oFM+0mRzLZXCH6XittiuldaopO36Y11/rKhEEUGoqYBIMIgKuiQgDPmDEMSQmc/+MVecdGbOvTP33HP4zuvJ4/vg3nPufO9n7mN4853vOd9zzN0FAIhPJukCACB0BC0AxIygBYCYEbQAEDOCFgBilo37DT486S5Oa4jZ2l+ckXQJwavMMiYph65ZWUf72PuI8wvOnB3P/LTD71cIfnoAIGaxj2gBoKwsfeNHghZAWDIVSVfwAQQtgLBYWaZdi0LQAggLUwcAEDNGtAAQM0a0ABAzRrQAEDPOOgCAmDF1AAAxY+oAAGLGiBYAYkbQAkDMKjgYBgDxYo4WAGLG1AEAxIwRLQDEjBEtAMSMES0AxIwluAAQM6YOACBmKZw6SF/0A0BHWKbw1lY3Zl3NbJGZ/cnMVprZlfntA8zsKTNbY2b/bWZdokoiaAGEpURBK+lvkka5+yckDZM0xsyOlXSdpB+7+0BJb0iaHNURQQsgLJmKwlsbvMm2/NPKfHNJoyTdm99+u6TxkSW1+5sBgDQyK7iZWY2ZLWnWav6+K6sws2WSNkuaK+lFSVvcfVf+Jesl7R9VEgfDAISliLMO3L1WUm0b+xskDTOzfSU9IOnQ9pRE0AIISwxnHbj7FjObL+k4SfuaWTY/qj1A0oaor2fqAEBQrGlKoKAW0c9++ZGszGxvSaMlrZI0X9Kp+ZdNkvRgVE2MaAEEJSpAi1Al6XYzq1DToPQed3/YzJ6VdLeZ/VDSM5JujeqIoAUQFMuUJmjdfbmkI1rY/pKkTxbTV6cO2o/07qYba45V355d5ZJun79GtXNf0NCP7qsZk47WXpUVamhs1MW/WqKlL72edLlBuHLaVD22cIF69e6te+5/KOlygvX4owt13fSr1djQqAkTT9Pkc2uivygQJRzRlkynnqNtaGjUtLue0acum63PX/VbTf7sIA3+SE9dccYw/euDdRo5bY6uvX+FLj99WNKlBuNL48brJze1epAXJdDQ0KBrrr5KN/78Fj0w6xHNmf2wXlyzJumyyqZUc7Sl1KmDtv6vb2v5n9+QJG17e5dWv7JVVb26yV3ap2ulJKlnty7atGVHkmUG5cijjlbPnvsmXUbQ6lYsV3V1fx1QXa3KLl00ZuzJWjB/XtJllU0agzZy6sDMDpU0Tu+dlLtB0ix3XxVnYeVW3ae7Du/fS0+/+Kqm3rFU/3PxSF115jBlMqYxP5ibdHlAwTbX16tfVb/dz/vmclqxfHmCFZVZ+mYO2h7Rmtl3Jd2tptIX5ZtJusvMLom/vPLovldWt11wvKbesVRvvr1LXxs1UN+7c6k+/u1ZmnrnUt0w+ZikSwRQoDSOaKOmDiZLOtrdp7v7f+XbdDUdcWv1QgrNl7W9/UK6f2XJVphuu+B43fvHtXr46fWSpDOPH6CHljQ9fnDROh150IeTLBEoSt9cTps2btr9fHN9vXK5XIIVlVcmkym4la2miP2Nkj7Swvaq/L4WuXutuw939+FdDzmpI/XF7obJx+iFV7bqpt88v3vbpi07NOLQvpKkE4bk9GL9m0mVBxTtsKGH6+WX12r9+nV6Z+dOzZn9iE78zKikyyqbNI5oo+Zop0iaZ2arJa3Lb/uopIGSzo+xrrI4ZlAfnTFigFau26IFV42RJP3w3j9pysxFuuYrRymbMf3tnQZ9+5eLEq40HJd99yI9vWSRtmzZorGjR6rmvPM1/pRTo78QBctms7p06jSdV3OOGhsbNH7CRA0cOCjpssonhXO05u5tv8Aso6apguYHwxbnL7YQ6cOT7mr7DdBha39xRtIlBK8y26lP0CmbrtmOx2Sfr95dcOa8etuZZYnlyLMO3L1R0pNlqAUAOiyNCxY69cowAOEp1RLcUiJoAQSFES0AxIygBYCYEbQAEDOCFgDilr6cJWgBhKWcS2sLRdACCApTBwAQt/TlLEELICyMaAEgZgQtAMSMoAWAmKXxWgfpOw8CADqgVBf+NrNqM5tvZs+a2UozuzC//Qoz22Bmy/JtbFRNjGgBBKWEUwe7JF3k7kvNbB9JT5vZu3dq/bG7/1uhHRG0AIJSqpx1942SNuYfv2lmq/TeDRCKwtQBgKAUM3XQ/Eay+VbTSp8HSjpC0lP5Teeb2XIzm2lmvaJqImgBBCWTsYJb8xvJ5lvt+/szsx6S7pM0xd23SrpJ0sGShqlpxDsjqiamDgAEpZRnd5lZpZpC9g53v1+S3L2+2f6bJT0c1Q9BCyAomRKd3mVNR9VulbTK3a9vtr0qP38rSRMk1UX1RdACCEoJR7QjJJ0taYWZLctvu0zSWWY2TJJLWivp61EdEbQAglKq07vc/TG1fIma2cX2RdACCEoKV+AStADCwoW/ASBmjGgBIGZcvQsAYpbCnCVoAYSFES0AxCyFOUvQAghLqVaGlVLsQfv8z06L+y06vb7HfSvpEoL3xuKfJl0CCsTUAQDELIU5S9ACCAsjWgCIWQpzlqAFEJZOeTAMAMqJqQMAiBlBCwAxS2HOErQAwsKIFgBilsKcJWgBhIWzDgAgZpkUDmkJWgBBSWHOErQAwsLBMACIWQqnaJW+20UCQAdkMlZwa4uZVZvZfDN71sxWmtmF+e29zWyuma3O/90rsqYSfW8AkApWxJ8IuyRd5O5DJB0r6ZtmNkTSJZLmufsgSfPyz9tE0AIISsYKb21x943uvjT/+E1JqyTtL2mcpNvzL7td0vjImjrw/QBA6phZMa3GzJY0azWt9HmgpCMkPSUp5+4b87s2ScpF1cTBMABBKeakA3evlVTbdn/WQ9J9kqa4+9bmZzW4u5uZR70PQQsgKKVcsGBmlWoK2Tvc/f785nozq3L3jWZWJWlzZE0lqwgAUqCEZx2YpFslrXL365vtmiVpUv7xJEkPRtXEiBZAUEo4oB0h6WxJK8xsWX7bZZKmS7rHzCZL+rOk06M6ImgBBKVUUwfu/pjU6jlgJxXTF0ELICgpXBhG0AIIC9c6AICYpfFaBwQtgKBw4W8AiBlTBwAQsxQOaAlaAGFhRAsAMUtfzBK0AAJTkcK5A4I2r37TRv1g2qV64/XXJDONm3CaTv/y2UmXFYS9umT1u1unqEuXrLIVFXrgd8/ohz+fvXv/jO+cqn8ad5z2G3FRglWG5fFHF+q66VersaFREyaepsnntnj1vyAxdZBiFRVZXfDP39Hgjw3R9u3bNfkrp+noY4/TgIMGJl3aHu9vO3dpTM0N2r5jp7LZjH4/89v67ePPatGKtTpyyEe17z7dki4xKA0NDbrm6qv0i5t/qVwupy+fcapGfmaUDh7YOX6WU5izXL3rXX3220+DPzZEktS9e3f1H3CQ/rI58upnKND2HTslSZXZCmWzFXJ3ZTKma6aM19T/+N9kiwtM3Yrlqq7urwOqq1XZpYvGjD1ZC+bPS7qsssmYFdzKVlN7v9DMvlbKQtJk4ysbtPq5VTps6MeTLiUYmYzpybsv0cvzpuv3Tz6nxXV/1nlnnKhH/rBCm17dmnR5QdlcX69+Vf12P++by6m+vj7BisrLrPBWLh0Z0V7Z2o7mt4f41cybO/AW5ffWW9s19eIp+ta/XKLuPXokXU4wGhtdx545XQM//z0NH9pfI448WKeMPkI33v2HpEtDYIq5lU25tDlHa2bLW9ulNu6T0/z2EK9u2xV5m4e02PXOO5p68RR97gsna+So0UmXE6S/btuhPyx5QScOP0QHVe+nlbMulyR161qpugcv19Bxrf7/jQL1zeW0aeOm3c8319crl4u8rVUwKlI4SRt1MCwn6fOS3njfdpP0x1gqSoi769ofTFP/AQfpzK98NelygtKnVw+9806D/rpth7ruVamTjjlUM277nQaMvmz3a/7y+AxCtkQOG3q4Xn55rdavX6dc35zmzH5E1/5oRtJllU0Kz+6KDNqHJfVw92Xv32FmC+IoKCnLly3VnEdm6eCBh2jSWadIkr7+zSn61PEnJFzZnq9fn566+aqzVZHJKJMx3Td3qX79aF3SZQUrm83q0qnTdF7NOWpsbND4CRM1cOCgpMsqmzQGrbnH+5v9njR1sKeq/vSUpEsI3huLf5p0CZ1C12zHF3Zd9NDzBWfOjC8NLksscx4tgKCkcURL0AIISgqPhRG0AMKSTWHSErQAgpLCnCVoAYSlnEtrC8W1DgAEpZRLcM1sppltNrO6ZtuuMLMNZrYs38ZG9UPQAghKxgpvBbhN0pgWtv/Y3Yfl2+wW9v8dpg4ABKWUF/5294VmdmBH+2FECyAoxYxom18AK98KvUL6+Wa2PD+10Cuypg5+TwCQKlbEH3evdffhzVptAW9xk6SDJQ2TtFFS5IUkmDoAEJS4V4a5++6L+5rZzWq6JkybCFoAQYk7aM2syt035p9OkBR5hSSCFkBQSnlBbzO7S9JISX3MbL2kyyWNNLNhklzSWklfj+qHoAUQlIoSHnly97Na2Hxrsf0QtACCksaVYQQtgKBwmUQAiFkKB7QELYCwZDp+k4aSI2gBBIURLQDELJvCSVqCFkBQGNECQMw4vQsAYpbCnCVoAYQljZckJGgBBIWpAwCIGUELADFLX8wStAACk8IBLUELICylvB5tqRC0AILCWQcAELNOeTCsR1eyPG6bn7gh6RKC9+nrFiRdQqeweOrIDvfB1AEAxIypAwCIGSNaAIhZ+mKWoAUQmApGtAAQrxTmbCrnjQGg3ayIP5F9mc00s81mVtdsW28zm2tmq/N/94rqh6AFEBSzwlsBbpM05n3bLpE0z90HSZqXf94mghZAUDKyglsUd18o6fX3bR4n6fb849sljY+uCQACUsyI1sxqzGxJs1ZTwFvk3H1j/vEmSbmoL+BgGICgFLME191rJdW2973c3c3Mo15H0AIIShnuNl5vZlXuvtHMqiRtjqwp9pIAoIxKedZBK2ZJmpR/PEnSg1FfQNACCEopzzows7skPSFpsJmtN7PJkqZLGm1mqyV9Nv+8TUwdAAhKB0aqH+DuZ7Wy66Ri+iFoAQSlDHO0RSNoAQSlU174GwDKKX0xS9ACCAwjWgCIWfpilqAFEJoUJi1BCyAoTB0AQMzSF7MELYDQpDBpCVoAQSnlyrBSIWgBBCWFU7QELYCwpDBnCVoAYbEUDmkJWgBBSWHOErQAwpLCnCVoAQQmhUlL0AIICqd3pdzjjy7UddOvVmNDoyZMPE2Tzy3kzsMoxpXTpuqxhQvUq3dv3XP/Q0mXE4zcPnvpin84VL27d5EkPfDMK7p78QZ948QDdcKgPnJJr2/fqSsfek6vbtuZbLExS+McLfcMy2toaNA1V1+lG39+ix6Y9YjmzH5YL65Zk3RZwfnSuPH6yU3tvrszWrHLXf8+70WdUbtYX7ttqU49an8N6NNN//nEOn35liX6x1uW6LHVr+mcTx+YdKmxK+U9w0qFoM2rW7Fc1dX9dUB1tSq7dNGYsSdrwfx5SZcVnCOPOlo9e+6bdBnBeW3bTj2/aZsk6a2dDVr72lvab5+9tH1nw+7X7N2lQu6eVIllU4a74BYtcurAzA6VtL+kp9x9W7PtY9x9TpzFldPm+nr1q+q3+3nfXE4rli9PsCKgfao+1FWDcz20csNWSdJ5Iwfo5MNz2vZ2g75xx7JkiyuDPW7qwMy+paZ7ll8gqc7MxjXbfU2chQEo3t6VFbpu4mG6fu6a3aPZmxb8n774kyc1Z2W9Th++f8IVxs+KaOUSNXVwrqSj3H28pJGSvm9mF+b3tVqnmdWY2RIzW3LrzXvGfFzfXE6bNm7a/Xxzfb1yuVyCFQHFqciYrpt4mObU1Wv+869+YP+v6+o1avB+CVRWZilM2qipg8y70wXuvtbMRkq618z6q40y3b1WUq0kvb1Le8Sk0GFDD9fLL6/V+vXrlOub05zZj+jaH81IuiygYN8/ebDWvvaW7ly0fve26l57a90bOyRJJx7SR2tfeyup8sqmlBf+NrO1kt6U1CBpl7sPb08/UUFbb2bD3H2ZJLn7NjP7oqSZkg5vzxumVTab1aVTp+m8mnPU2Nig8RMmauDAQUmXFZzLvnuRnl6ySFu2bNHY0SNVc975Gn/KqUmXtcf7xAEf0skf76fV9dt0xzlNWfCz+S9p3LAq9e/dTY3u2rT1bV376xcSrjR+MQxUP+PuH/wVoQjW1lFIMztATSm+qYV9I9z98ag32FNGtHuyd3Y1Jl1C8EbNWJh0CZ3C4qkjO5yTL9S/VXDmHJLr1ub75Ue0wzsatG3O0br7+pZCNr8vMmQBoNyKOb2r+fGkfHv/KiWX9Fsze7qFfQVjZRiAoBQzRdv8eFIrjnf3DWbWV9JcM3vO3Yv+9YYFCwCCUsqTDtx9Q/7vzZIekPTJ9tRE0AIIipkV3CL66W5m+7z7WNLnJNW1pyamDgAEpYRnd+UkPZAP5KykO9u7GpagBRCUUuWsu78k6ROl6IugBRCWFF7rgKAFEBQu/A0AMUvj1bsIWgBByRC0ABC39CUtQQsgKEwdAEDMUpizBC2AsDCiBYCYRS2tTQJBCyAo6YtZghZAYFI4oCVoAYSFlWEAELf05SxBCyAsKcxZghZAWEp5u/FSIWgBBCWFOcutbAAgboxoAQQljSNaghZAUDi9CwBixogWAGJG0AJAzJg6AICYpXFEy+ldAIJiRbTIvszGmNnzZrbGzC5pb00ELYCwlChpzaxC0s8kfUHSEElnmdmQ9pTE1AGAoJRwCe4nJa1x95ckyczuljRO0rPFdhR70HbNpnBmOoKZ1bh7bdJ1FKprds/7xWRP+4wXTx2ZdAlF29M+41IpJnPMrEZSTbNNtc0+s/0lrWu2b72kY9pT0573L7Q8aqJfgg7iM44fn3EEd6919+HNWiz/MRG0ANCyDZKqmz0/IL+taAQtALRssaRBZjbAzLpIOlPSrPZ0xMGwlnW6ea0E8BnHj8+4A9x9l5mdL+k3kiokzXT3le3py9y9pMUBAP4eUwcAEDOCFgBiRtA2U6rldmidmc00s81mVpd0LaEys2ozm29mz5rZSjO7MOmaOjvmaPPyy+1ekDRaTScmL5Z0lrsXvQoErTOzEyRtk/Qrdx+adD0hMrMqSVXuvtTM9pH0tKTx/CwnhxHte3Yvt3P3nZLeXW6HEnL3hZJeT7qOkLn7Rndfmn/8pqRValrlhIQQtO9pabkdP5zYo5nZgZKOkPRUwqV0agQtECgz6yHpPklT3H1r0vV0ZgTte0q23A5ImplVqilk73D3+5Oup7MjaN9TsuV2QJLMzCTdKmmVu1+fdD0gaHdz912S3l1ut0rSPe1dbofWmdldkp6QNNjM1pvZ5KRrCtAISWdLGmVmy/JtbNJFdWac3gUAMWNECwAxI2gBIGYELQDEjKAFgJgRtAAQM4IWAGJG0AJAzP4fjV5fFA1LGrwAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "_ = sns.heatmap(confusion_matrix, annot=True, cmap=\"Blues\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "placed-thickness",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "accuracy: 0.955\n",
      "precision: 0.956\n",
      "recall: 0.955\n",
      "f1 score: 0.955\n"
     ]
    }
   ],
   "source": [
    "print(\"accuracy: {:.3f}\".format(metrics.accuracy_score(y_test, predicted)))\n",
    "print(\"precision: {:.3f}\".format(metrics.precision_score(y_test, predicted, average='weighted')))\n",
    "print(\"recall: {:.3f}\".format(metrics.recall_score(y_test, predicted, average='weighted')))\n",
    "print(\"f1 score: {:.3f}\".format(metrics.f1_score(y_test, predicted, average='weighted')))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}