Movie_Recommendation_System.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/alvitay/Movie-Recommendation-System/blob/main/Movie_Recommendation_System.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "c06bc9a4",
      "metadata": {
        "id": "c06bc9a4"
      },
      "source": [
        "# **Movie Recommendation System**\n",
        "\n",
        "\n",
        "-----------------\n",
        "## **Objective**\n",
        "-----------------\n",
        "\n",
        "Using the above dataset, we will build two different types of recommendation systems that are listed below.\n",
        "- **Clustering-based recommendation system**.\n",
        "- **Content-based collaborative filtering**.\n",
        "\n",
        "-----------------\n",
        "## **Dataset**\n",
        "-----------------\n",
        "\n",
        "We will use the following three datasets for this case study:\n",
        "- **ratings** dataset - This dataset contains the following attributes:\n",
        "    - userId\n",
        "    - movieId\n",
        "    - rating\n",
        "    - timestamp\n",
        "\n",
        "- **movies** dataset - This dataset contains the following attributes:\n",
        "    - movieId\n",
        "    - title\n",
        "    - genres\n",
        "\n",
        "- **tags** dataset- This dataset contains the following attributes:\n",
        "    - userId\n",
        "    - movieId\n",
        "    - tag\n",
        "    - timestamp"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "z9hGQQwAaq3C",
      "metadata": {
        "id": "z9hGQQwAaq3C"
      },
      "source": [
        "Sometimes, the installation of the surprise library, which is used to build recommendation systems, faces issues in Jupyter. To avoid any issues, it is advised to use **Google Colab** for this case study.\n",
        "\n",
        "Let's start by mounting the Google drive on Colab."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "id": "Qs0mS3ydnnQa",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Qs0mS3ydnnQa",
        "outputId": "93a51cec-390f-4935-a6ce-51edf27e45d2"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Mounted at /content/drive\n"
          ]
        }
      ],
      "source": [
        "# Mounting the drive\n",
        "from google.colab import drive\n",
        "drive.mount('/content/drive')"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "83046c58",
      "metadata": {
        "id": "83046c58"
      },
      "source": [
        "## **Importing the necessary libraries and overview of the dataset**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "id": "6366469b",
      "metadata": {
        "id": "6366469b"
      },
      "outputs": [],
      "source": [
        "import warnings                                 # Used to ignore the warning given as output of the code\n",
        "warnings.filterwarnings('ignore')\n",
        "\n",
        "import numpy as np                              # Basic libraries of python for numeric and dataframe computations\n",
        "import pandas as pd\n",
        "\n",
        "import matplotlib.pyplot as plt                 # Basic library for data visualization\n",
        "import seaborn as sns                           # Slightly advanced library for data visualization\n",
        "\n",
        "from collections import defaultdict             # A dictionary that does not raise a key error\n",
        "\n",
        "from sklearn.metrics.pairwise import cosine_similarity # To find the similarity between two vectors\n",
        "\n",
        "from sklearn.metrics import mean_squared_error  # A performance metric in sklearn"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "280d594a",
      "metadata": {
        "id": "280d594a"
      },
      "source": [
        "### **Loading the data**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "id": "689f2b57",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 204
        },
        "id": "689f2b57",
        "outputId": "fa14ae42-51a8-4ab2-97f9-e6d820b30705"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "   movieId                               title  \\\n",
              "0        1                    Toy Story (1995)   \n",
              "1        2                      Jumanji (1995)   \n",
              "2        3             Grumpier Old Men (1995)   \n",
              "3        4            Waiting to Exhale (1995)   \n",
              "4        5  Father of the Bride Part II (1995)   \n",
              "\n",
              "                                        genres  \n",
              "0  Adventure|Animation|Children|Comedy|Fantasy  \n",
              "1                   Adventure|Children|Fantasy  \n",
              "2                               Comedy|Romance  \n",
              "3                         Comedy|Drama|Romance  \n",
              "4                                       Comedy  "
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-deb9e2ca-e902-4806-8d36-d57d9ddcb3fb\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>movieId</th>\n",
              "      <th>title</th>\n",
              "      <th>genres</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>1</td>\n",
              "      <td>Toy Story (1995)</td>\n",
              "      <td>Adventure|Animation|Children|Comedy|Fantasy</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>2</td>\n",
              "      <td>Jumanji (1995)</td>\n",
              "      <td>Adventure|Children|Fantasy</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>3</td>\n",
              "      <td>Grumpier Old Men (1995)</td>\n",
              "      <td>Comedy|Romance</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>4</td>\n",
              "      <td>Waiting to Exhale (1995)</td>\n",
              "      <td>Comedy|Drama|Romance</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>5</td>\n",
              "      <td>Father of the Bride Part II (1995)</td>\n",
              "      <td>Comedy</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-deb9e2ca-e902-4806-8d36-d57d9ddcb3fb')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-deb9e2ca-e902-4806-8d36-d57d9ddcb3fb button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-deb9e2ca-e902-4806-8d36-d57d9ddcb3fb');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-a0c84173-a98b-42ef-bc0f-ebe72788eda1\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-a0c84173-a98b-42ef-bc0f-ebe72788eda1')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-a0c84173-a98b-42ef-bc0f-ebe72788eda1 button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "dataframe",
              "variable_name": "movies",
              "summary": "{\n  \"name\": \"movies\",\n  \"rows\": 9742,\n  \"fields\": [\n    {\n      \"column\": \"movieId\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 52160,\n        \"min\": 1,\n        \"max\": 193609,\n        \"num_unique_values\": 9742,\n        \"samples\": [\n          45635,\n          1373,\n          7325\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"title\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 9737,\n        \"samples\": [\n          \"Teenage Mutant Ninja Turtles (2014)\",\n          \"America's Sweethearts (2001)\",\n          \"Cast Away (2000)\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"genres\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 951,\n        \"samples\": [\n          \"Crime|Mystery|Romance|Thriller\",\n          \"Action|Adventure|Comedy|Western\",\n          \"Crime|Drama|Musical\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"
            }
          },
          "metadata": {},
          "execution_count": 4
        }
      ],
      "source": [
        "# Loading the movies dataset\n",
        "movies = pd.read_csv('/content/drive/MyDrive/Movie/movies.csv')\n",
        "\n",
        "# Let us see the first five records of the dataset\n",
        "movies.head()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "id": "2cc76d37",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "2cc76d37",
        "outputId": "b4afb676-b57b-48ef-87bd-8cb1f08eed6c"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(9742, 3)"
            ]
          },
          "metadata": {},
          "execution_count": 5
        }
      ],
      "source": [
        "# Shape of the DataFrame\n",
        "movies.shape"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 6,
      "id": "afcb1ebf",
      "metadata": {
        "id": "afcb1ebf"
      },
      "outputs": [],
      "source": [
        "# Loading the ratings dataset\n",
        "ratings = pd.read_csv('/content/drive/MyDrive/Movie/ratings.csv')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 7,
      "id": "8c76ffe4",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "8c76ffe4",
        "outputId": "6e8740b2-b7f8-4f6d-8786-dd34a3e28fd4"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(100836, 4)"
            ]
          },
          "metadata": {},
          "execution_count": 7
        }
      ],
      "source": [
        "# Shape of the ratings dataset\n",
        "ratings.shape"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "64af1755",
      "metadata": {
        "id": "64af1755"
      },
      "source": [
        "Let's **merge both the datasets** to get the title and rating of each movie in a single DataFrame."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 8,
      "id": "ffa6437d",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 204
        },
        "id": "ffa6437d",
        "outputId": "921f16be-047a-4125-e4b0-ffafbf184e2b"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "   userId  movieId  rating   timestamp             title\n",
              "0       1        1     4.0   964982703  Toy Story (1995)\n",
              "1       5        1     4.0   847434962  Toy Story (1995)\n",
              "2       7        1     4.5  1106635946  Toy Story (1995)\n",
              "3      15        1     2.5  1510577970  Toy Story (1995)\n",
              "4      17        1     4.5  1305696483  Toy Story (1995)"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-d6a71acb-d767-4025-878e-c5eb575ca94b\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>userId</th>\n",
              "      <th>movieId</th>\n",
              "      <th>rating</th>\n",
              "      <th>timestamp</th>\n",
              "      <th>title</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>4.0</td>\n",
              "      <td>964982703</td>\n",
              "      <td>Toy Story (1995)</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>5</td>\n",
              "      <td>1</td>\n",
              "      <td>4.0</td>\n",
              "      <td>847434962</td>\n",
              "      <td>Toy Story (1995)</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>7</td>\n",
              "      <td>1</td>\n",
              "      <td>4.5</td>\n",
              "      <td>1106635946</td>\n",
              "      <td>Toy Story (1995)</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>15</td>\n",
              "      <td>1</td>\n",
              "      <td>2.5</td>\n",
              "      <td>1510577970</td>\n",
              "      <td>Toy Story (1995)</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>17</td>\n",
              "      <td>1</td>\n",
              "      <td>4.5</td>\n",
              "      <td>1305696483</td>\n",
              "      <td>Toy Story (1995)</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-d6a71acb-d767-4025-878e-c5eb575ca94b')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-d6a71acb-d767-4025-878e-c5eb575ca94b button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-d6a71acb-d767-4025-878e-c5eb575ca94b');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-a8f4dfdd-e092-48a2-bbfc-fe4f4a5f435e\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-a8f4dfdd-e092-48a2-bbfc-fe4f4a5f435e')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-a8f4dfdd-e092-48a2-bbfc-fe4f4a5f435e button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "dataframe",
              "variable_name": "ratings_with_title"
            }
          },
          "metadata": {},
          "execution_count": 8
        }
      ],
      "source": [
        "# Merging datasets on movieId\n",
        "ratings_with_title = pd.merge(ratings, movies[['movieId', 'title']], on = 'movieId', how = 'inner')\n",
        "\n",
        "# See the first five records of the dataset\n",
        "ratings_with_title.head()"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "6e20ed5c",
      "metadata": {
        "id": "6e20ed5c"
      },
      "source": [
        "Let's check the **info** of the data."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 9,
      "id": "37e9b1dd",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "37e9b1dd",
        "outputId": "3465aae0-67fb-4f46-dcce-6e4d58a22cef"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "<class 'pandas.core.frame.DataFrame'>\n",
            "RangeIndex: 100836 entries, 0 to 100835\n",
            "Data columns (total 5 columns):\n",
            " #   Column     Non-Null Count   Dtype  \n",
            "---  ------     --------------   -----  \n",
            " 0   userId     100836 non-null  int64  \n",
            " 1   movieId    100836 non-null  int64  \n",
            " 2   rating     100836 non-null  float64\n",
            " 3   timestamp  100836 non-null  int64  \n",
            " 4   title      100836 non-null  object \n",
            "dtypes: float64(1), int64(3), object(1)\n",
            "memory usage: 3.8+ MB\n"
          ]
        }
      ],
      "source": [
        "# Checking info of the merged dataset\n",
        "ratings_with_title.info()"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "6717b540",
      "metadata": {
        "id": "6717b540"
      },
      "source": [
        "- There are **100,836 observations** and **5 columns** in the data.\n",
        "- All the columns are of **numeric data type** except the **title column**. The title column is of **object data type**.\n",
        "- The data type of the timestamp column is int64 which is not correct. We can convert this to DateTime format but **we don't need a timestamp for our analysis**. Hence, **we can drop this column**."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 10,
      "id": "0b153fe9",
      "metadata": {
        "id": "0b153fe9"
      },
      "outputs": [],
      "source": [
        "# Dropping the timestamp column\n",
        "rating = ratings_with_title.drop(['timestamp'], axis = 1)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 12,
      "id": "335edf2b",
      "metadata": {
        "id": "335edf2b"
      },
      "outputs": [],
      "source": [
        "# Calculating average ratings\n",
        "average_rating = rating.groupby('movieId')['rating'].mean()\n",
        "\n",
        "# Calculating the count of ratings\n",
        "count_rating = rating.groupby('movieId').count()['rating']\n",
        "\n",
        "# Making a dataframe with the count and average of ratings\n",
        "final_rating = pd.DataFrame({'avg_rating': average_rating, 'rating_count': count_rating})"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 13,
      "id": "945f9a1c",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 235
        },
        "id": "945f9a1c",
        "outputId": "45ffe97a-1b54-46b8-c877-e5e421e6461c"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "         avg_rating  rating_count\n",
              "movieId                          \n",
              "1          3.920930           215\n",
              "2          3.431818           110\n",
              "3          3.259615            52\n",
              "4          2.357143             7\n",
              "5          3.071429            49"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-0b2fa0ac-203b-45b3-a963-b25449a5b8c8\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>avg_rating</th>\n",
              "      <th>rating_count</th>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>movieId</th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>3.920930</td>\n",
              "      <td>215</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>3.431818</td>\n",
              "      <td>110</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>3.259615</td>\n",
              "      <td>52</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>2.357143</td>\n",
              "      <td>7</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5</th>\n",
              "      <td>3.071429</td>\n",
              "      <td>49</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-0b2fa0ac-203b-45b3-a963-b25449a5b8c8')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-0b2fa0ac-203b-45b3-a963-b25449a5b8c8 button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-0b2fa0ac-203b-45b3-a963-b25449a5b8c8');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-d5c6f8e3-4901-4e33-bf3f-db07c3462616\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-d5c6f8e3-4901-4e33-bf3f-db07c3462616')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-d5c6f8e3-4901-4e33-bf3f-db07c3462616 button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "dataframe",
              "variable_name": "final_rating",
              "summary": "{\n  \"name\": \"final_rating\",\n  \"rows\": 9724,\n  \"fields\": [\n    {\n      \"column\": \"movieId\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 52191,\n        \"min\": 1,\n        \"max\": 193609,\n        \"num_unique_values\": 9724,\n        \"samples\": [\n          1500,\n          114662,\n          5490\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"avg_rating\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0.869873634487782,\n        \"min\": 0.5,\n        \"max\": 5.0,\n        \"num_unique_values\": 1286,\n        \"samples\": [\n          3.9934210526315788,\n          3.657142857142857,\n          3.4423076923076925\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"rating_count\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 22,\n        \"min\": 1,\n        \"max\": 329,\n        \"num_unique_values\": 177,\n        \"samples\": [\n          89,\n          17,\n          39\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"
            }
          },
          "metadata": {},
          "execution_count": 13
        }
      ],
      "source": [
        "# See the first five records of the final_rating dataset\n",
        "final_rating.head()"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "6520f1ec",
      "metadata": {
        "id": "6520f1ec"
      },
      "source": [
        "## **Exploring the dataset**\n",
        "\n",
        "Let's explore the dataset and answer some basic data-related questions."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 14,
      "id": "56b5b632",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "56b5b632",
        "outputId": "9bf55755-d452-41c9-b299-1e60fa3a99d5"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "610"
            ]
          },
          "metadata": {},
          "execution_count": 14
        }
      ],
      "source": [
        "# Find the number of unique users\n",
        "ratings_with_title['userId'].nunique()"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "5fad323e",
      "metadata": {
        "id": "5fad323e"
      },
      "source": [
        "- There are **610 unique users** in the dataset."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 15,
      "id": "1a5f609f",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "1a5f609f",
        "outputId": "9c86f282-c1d7-499c-aad9-d3f71988a1ae"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "9719"
            ]
          },
          "metadata": {},
          "execution_count": 15
        }
      ],
      "source": [
        "# Find the number of unique movies\n",
        "ratings_with_title['title'].nunique()"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "238c98f3",
      "metadata": {
        "id": "238c98f3"
      },
      "source": [
        "- There are **9719 unique movies** in the dataset."
      ]
    },
    {
      "cell_type": "markdown",
      "id": "69c47aaa",
      "metadata": {
        "id": "69c47aaa"
      },
      "source": [
        "To demonstrate the clustering-based recommendation system, we are going to use the **surprise library** in this case study.\n",
        "\n",
        "\n",
        "**!pip install surprise**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 16,
      "id": "oiZdEaArobw1",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "oiZdEaArobw1",
        "outputId": "72bdada7-a8c9-4cae-84fd-0ffdaa32235f"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Collecting surprise\n",
            "  Downloading surprise-0.1-py2.py3-none-any.whl.metadata (327 bytes)\n",
            "Collecting scikit-surprise (from surprise)\n",
            "  Downloading scikit_surprise-1.1.4.tar.gz (154 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m154.4/154.4 kB\u001b[0m \u001b[31m2.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25h  Installing build dependencies ... \u001b[?25l\u001b[?25hdone\n",
            "  Getting requirements to build wheel ... \u001b[?25l\u001b[?25hdone\n",
            "  Preparing metadata (pyproject.toml) ... \u001b[?25l\u001b[?25hdone\n",
            "Requirement already satisfied: joblib>=1.2.0 in /usr/local/lib/python3.10/dist-packages (from scikit-surprise->surprise) (1.4.2)\n",
            "Requirement already satisfied: numpy>=1.19.5 in /usr/local/lib/python3.10/dist-packages (from scikit-surprise->surprise) (1.26.4)\n",
            "Requirement already satisfied: scipy>=1.6.0 in /usr/local/lib/python3.10/dist-packages (from scikit-surprise->surprise) (1.13.1)\n",
            "Downloading surprise-0.1-py2.py3-none-any.whl (1.8 kB)\n",
            "Building wheels for collected packages: scikit-surprise\n",
            "  Building wheel for scikit-surprise (pyproject.toml) ... \u001b[?25l\u001b[?25hdone\n",
            "  Created wheel for scikit-surprise: filename=scikit_surprise-1.1.4-cp310-cp310-linux_x86_64.whl size=2357279 sha256=9739ce20f1a1834de451e281e20d248a8b6a0c4e5c4c0aed56bed16c98272978\n",
            "  Stored in directory: /root/.cache/pip/wheels/4b/3f/df/6acbf0a40397d9bf3ff97f582cc22fb9ce66adde75bc71fd54\n",
            "Successfully built scikit-surprise\n",
            "Installing collected packages: scikit-surprise, surprise\n",
            "Successfully installed scikit-surprise-1.1.4 surprise-0.1\n"
          ]
        }
      ],
      "source": [
        "# Installing the surprise package\n",
        "!pip install surprise"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 17,
      "id": "8165b52f",
      "metadata": {
        "id": "8165b52f"
      },
      "outputs": [],
      "source": [
        "# To compute the accuracy of models\n",
        "from surprise import accuracy\n",
        "\n",
        "# Class to parse a file containing ratings, data should be in structure - user; item; rating\n",
        "from surprise.reader import Reader\n",
        "\n",
        "# Class for loading datasets\n",
        "from surprise.dataset import Dataset\n",
        "\n",
        "# For tuning model hyperparameters\n",
        "from surprise.model_selection import GridSearchCV\n",
        "\n",
        "# For splitting the rating data in train and test dataset\n",
        "from surprise.model_selection import train_test_split\n",
        "\n",
        "# For implementing a clustering-based recommendation system\n",
        "from surprise import CoClustering"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "196036fd",
      "metadata": {
        "id": "196036fd"
      },
      "source": [
        "**Before building the recommendation systems, let's  go over some basic terminologies we are going to use:**"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "923e64f6",
      "metadata": {
        "id": "923e64f6"
      },
      "source": [
        "**Relevant item:** An item (product in this case) that is actually **rated higher than the threshold rating (here 3.5)** is relevant, and an item that is **actually rated lower than the threshold rating is a non-relevant item**.  \n",
        "\n",
        "**Recommended item:** An item whose **predicted rating is higher than the threshold (here 3.5) is a recommended item**, and an item whose **predicted rating is lower the threshold rating is a non-recommended item**, i.e., it will not be recommended to the user.  "
      ]
    },
    {
      "cell_type": "markdown",
      "id": "5fe84d1f",
      "metadata": {
        "id": "5fe84d1f"
      },
      "source": [
        "**False Negative (FN):** It is the **frequency of relevant items that are not recommended to the user**. If the relevant items are not recommended to the user, then the user might not buy the product/item. This would result in the **loss of opportunity for the service provider**, which they would like to minimize.\n",
        "\n",
        "**False Positive (FP):** It is the **frequency of recommended items that are actually not relevant**. In this case, the recommendation system is not doing a good job of finding and recommending the relevant items to the user. This would result in **loss of resources for the service provider**, which they would also like to minimize."
      ]
    },
    {
      "cell_type": "markdown",
      "id": "fb0a1266",
      "metadata": {
        "id": "fb0a1266"
      },
      "source": [
        "**Recall:** It is the **fraction of actually relevant items that are recommended to the user**, i.e., if out of 10 relevant products, 6 are recommended to the user, then recall is 0.60. Higher the value of recall, better is the model. It is one of the metrics to do the performance assessment of classification models.\n",
        "\n",
        "**Precision:** It is the **fraction of recommended items that are relevant actually**, i.e., if out of 10 recommended items, 6 are found relevant by the user, then precision is 0.60. The higher the value of precision, better is the model. It is one of the metrics to do the performance assessment of classification models."
      ]
    },
    {
      "cell_type": "markdown",
      "id": "0d888c5b",
      "metadata": {
        "id": "0d888c5b"
      },
      "source": [
        "**While making a recommendation system, it becomes customary to look at the performance of the model. In terms of how many recommendations are relevant and vice-versa, below are some most used performance metrics used in the assessment of recommendation systems.**"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "1ef08ba2",
      "metadata": {
        "id": "1ef08ba2"
      },
      "source": [
        "### **Precision@k, Recall@ k, and F1-score@k**"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "06ca7a4a",
      "metadata": {
        "id": "06ca7a4a"
      },
      "source": [
        "**Precision@k** - It is the **fraction of recommended items that are relevant in `top k` predictions**. The value of k is the number of recommendations to be provided to the user. One can choose a variable number of recommendations to be given to a unique user.  \n",
        "\n",
        "**Recall@k** - It is the **fraction of relevant items that are recommended to the user in `top k` predictions**.\n",
        "\n",
        "**F1-score@k** - It is the **harmonic mean of Precision@k and Recall@k**. When **precision@k and recall@k both seem to be important**, it is useful to use this metric because it is representative of both of them."
      ]
    },
    {
      "cell_type": "markdown",
      "id": "a589174a",
      "metadata": {
        "id": "a589174a"
      },
      "source": [
        "### **Some useful functions**"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "427f71c9",
      "metadata": {
        "id": "427f71c9"
      },
      "source": [
        "- Below function takes the **recommendation model** as input and gives the **precision@k, recall@k, and F1-score@k** for that model.  \n",
        "- To compute **precision and recall**, **top k** predictions are taken under consideration for each user.\n",
        "- We will use the precision and recall to compute the F1-score."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 18,
      "id": "82f78eb0",
      "metadata": {
        "id": "82f78eb0"
      },
      "outputs": [],
      "source": [
        "def precision_recall_at_k(model, k = 10, threshold = 3.5):\n",
        "    \"\"\"Return precision@k and recall@k metrics for each user\"\"\"\n",
        "\n",
        "    # First map the predictions to each user\n",
        "    user_est_true = defaultdict(list)\n",
        "\n",
        "    # Making predictions on the test data\n",
        "    predictions=model.test(testset)\n",
        "\n",
        "    for uid, _, true_r, est, _ in predictions:\n",
        "        user_est_true[uid].append((est, true_r))\n",
        "\n",
        "    precisions = dict()\n",
        "    recalls = dict()\n",
        "    for uid, user_ratings in user_est_true.items():\n",
        "\n",
        "        # Sort user ratings by estimated value\n",
        "        user_ratings.sort(key = lambda x: x[0], reverse = True)\n",
        "\n",
        "        # Number of relevant items\n",
        "        n_rel = sum((true_r >= threshold) for (_, true_r) in user_ratings)\n",
        "\n",
        "        # Number of recommended items in top k\n",
        "        n_rec_k = sum((est >= threshold) for (est, _) in user_ratings[ : k])\n",
        "\n",
        "        # Number of relevant and recommended items in top k\n",
        "        n_rel_and_rec_k = sum(((true_r >= threshold) and (est >= threshold))\n",
        "                              for (est, true_r) in user_ratings[ : k])\n",
        "\n",
        "        # Precision@K: Proportion of recommended items that are relevant\n",
        "        # When n_rec_k is 0, Precision is undefined. We here set Precision to 0 when n_rec_k is 0\n",
        "\n",
        "        precisions[uid] = n_rel_and_rec_k / n_rec_k if n_rec_k != 0 else 0\n",
        "\n",
        "        # Recall@K: Proportion of relevant items that are recommended\n",
        "        # When n_rel is 0, Recall is undefined. We here set Recall to 0 when n_rel is 0\n",
        "\n",
        "        recalls[uid] = n_rel_and_rec_k / n_rel if n_rel != 0 else 0\n",
        "\n",
        "    # Mean of all the predicted precisions are calculated\n",
        "    precision = round((sum(prec for prec in precisions.values()) / len(precisions)), 3)\n",
        "\n",
        "    # Mean of all the predicted recalls are calculated\n",
        "    recall = round((sum(rec for rec in recalls.values()) / len(recalls)), 3)\n",
        "\n",
        "    accuracy.rmse(predictions)\n",
        "\n",
        "    # Command to print the overall precision\n",
        "    print('Precision: ', precision)\n",
        "\n",
        "    # Command to print the overall recall\n",
        "    print('Recall: ', recall)\n",
        "\n",
        "    # Formula to compute the F-1 score\n",
        "    print('F_1 score: ', round((2 * precision * recall) / (precision + recall), 3))"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "e9a649e1",
      "metadata": {
        "id": "e9a649e1"
      },
      "source": [
        "Below, we are loading the **`rating` dataset**, which is a **pandas DataFrame**, into a **different format called `surprise.dataset.DatasetAutoFolds`**. This is required by this library. To do this we will be **using the classes `Reader` and `Dataset`**."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 19,
      "id": "052e4b4d",
      "metadata": {
        "id": "052e4b4d"
      },
      "outputs": [],
      "source": [
        "# Instantiating Reader scale with expected rating scale\n",
        "reader = Reader(rating_scale = (0, 5))\n",
        "\n",
        "# Loading the rating dataset\n",
        "data = Dataset.load_from_df(rating[['userId', 'movieId', 'rating']], reader)\n",
        "\n",
        "# Splitting the data into train and test dataset\n",
        "trainset, testset = train_test_split(data, test_size = 0.2, random_state = 42)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "57b31de5",
      "metadata": {
        "id": "57b31de5"
      },
      "source": [
        "## **Model 1: Cluster-Based Recommendation System**"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "06fe547e",
      "metadata": {
        "id": "06fe547e"
      },
      "source": [
        "In **clustering-based recommendation systems**, we explore the **similarities and differences** in people's tastes in movies based on how they rate different movies. We cluster similar users together and recommend movies to a user based on ratings from other users in the same cluster."
      ]
    },
    {
      "cell_type": "markdown",
      "id": "6b568c64",
      "metadata": {
        "id": "6b568c64"
      },
      "source": [
        "- **Co-clustering** is a set of techniques in **Cluster Analysis**. Given some **matrix A, we want to cluster rows of A and columns of A simultaneously**, this is a common task for user-item matrices.\n",
        "\n",
        "- As it clusters both the rows and the columns simultaneously, it is also called **bi-clustering**. To understand the working of the algorithm, let A be m x n matrix, the goal is to generate co-clusters: a subset of rows that exhibit similar behavior across a subset of columns, or vice versa.\n",
        "\n",
        "- Co-clustering is defined as two map functions:\n",
        "rows -> row cluster indexes\n",
        "columns -> column cluster indexes.\n",
        "\n",
        "These map functions are learned simultaneously. It is **different from other clustering techniques** where we cluster **first the rows and then the columns**."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 20,
      "id": "0c4b20e4",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "0c4b20e4",
        "outputId": "f899ee8f-f64e-40c0-9ad6-5fac3ada6701"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "RMSE: 0.9490\n",
            "Precision:  0.717\n",
            "Recall:  0.502\n",
            "F_1 score:  0.591\n"
          ]
        }
      ],
      "source": [
        "# Using CoClustering algorithm\n",
        "clust_baseline = CoClustering(random_state = 1)\n",
        "\n",
        "# Training the algorithm on the train set\n",
        "clust_baseline.fit(trainset)\n",
        "\n",
        "# Let us compute precision@k, recall@k, and F_1 score with k = 10\n",
        "precision_recall_at_k(clust_baseline)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "20f67577",
      "metadata": {
        "id": "20f67577"
      },
      "source": [
        "- We have calculated **RMSE** to check **how far the overall predicted ratings** are from the **actual ratings**.\n",
        "- Here **F_1 score** of the **baseline model is ~ 0.60**. It indicates that **mostly recommended movies were relevant, and relevant movies were recommended**. We will try to improve this later by using **GridSearchCV by tuning different hyperparameters** of this algorithm."
      ]
    },
    {
      "cell_type": "markdown",
      "id": "09549c5a",
      "metadata": {
        "id": "09549c5a"
      },
      "source": [
        "Now, let's **predict the rating for the user with `userId = 4` and the movie with `movieId = 10`** as shown below. Here, the user has already interacted or watched the movie with movieId 10."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 21,
      "id": "11dbdc0f",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "11dbdc0f",
        "outputId": "de459d90-3456-4634-ee2a-581a2319524e"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "user: 4          item: 10         r_ui = 4.00   est = 3.68   {'was_impossible': False}\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Prediction(uid=4, iid=10, r_ui=4, est=3.6757402992691386, details={'was_impossible': False})"
            ]
          },
          "metadata": {},
          "execution_count": 21
        }
      ],
      "source": [
        "# Making prediction for userId 4 and movieId 10\n",
        "clust_baseline.predict(4, 10, r_ui = 4, verbose = True)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "90276146",
      "metadata": {
        "id": "90276146"
      },
      "source": [
        "- The actual rating for this user-item pair is 4 and the **predicted rating by this Co-clustering is 3.68 which is close to the predicted rating**. The model has slightly under-estimated the rating. We will try to fix this later by tuning the hyperparameters of the model using GridSearchCV."
      ]
    },
    {
      "cell_type": "markdown",
      "id": "70e86f9c",
      "metadata": {
        "id": "70e86f9c"
      },
      "source": [
        "Below, we are predicting the rating for the same `userId = 4` but for a movie with which this user has not interacted before, i.e., `movieId = 3`, as shown below."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 22,
      "id": "dab1aaed",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "dab1aaed",
        "outputId": "b893cc25-09f2-4734-b1e5-ebd2c914748c"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "user: 4          item: 3          r_ui = None   est = 3.26   {'was_impossible': False}\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Prediction(uid=4, iid=3, r_ui=None, est=3.258169827544438, details={'was_impossible': False})"
            ]
          },
          "metadata": {},
          "execution_count": 22
        }
      ],
      "source": [
        "# Making prediction for userId 4 and movieId 3\n",
        "clust_baseline.predict(4, 3, verbose = True)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "c2fd66f5",
      "metadata": {
        "id": "c2fd66f5"
      },
      "source": [
        "### **Improving clustering-based recommendation system by tuning its hyperparameters**"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "5bc7c680",
      "metadata": {
        "id": "5bc7c680"
      },
      "source": [
        "Below, we will be tuning hyperparameters for the `CoClustering` algorithm. Let's try to understand the different hyperparameters of this algorithm."
      ]
    },
    {
      "cell_type": "markdown",
      "id": "31d97b16",
      "metadata": {
        "id": "31d97b16"
      },
      "source": [
        "- **n_cltr_u** (int) – Number of **user clusters**. The default value is 3.\n",
        "- **n_cltr_i** (int) – Number of **item clusters**. The default value is 3.\n",
        "- **n_epochs** (int) – Number of **iteration of the optimization loop**. The default value is 3.\n",
        "- **random_state** (int, RandomState instance from NumPy, or None) – Determines the RNG that will be used for initialization. If int, random_state will be used as a seed for a new RNG. This is useful to get the same initialization over multiple calls to fit(). If RandomState instance, this same instance is used as RNG. If None, the current RNG from NumPy is used. The default value is None.\n",
        "- **verbose** (bool) – If True, the current epoch will be printed. The default value is False."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 23,
      "id": "efe7d8e6",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "efe7d8e6",
        "outputId": "ab66261f-da82-4bcf-bd29-b5d3b92d27ef"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "0.9538203463635423\n",
            "{'n_cltr_u': 6, 'n_cltr_i': 3, 'n_epochs': 50}\n"
          ]
        }
      ],
      "source": [
        "# Set the parameter space to tune\n",
        "param_grid = {'n_cltr_u': [3, 4, 5, 6], 'n_cltr_i': [3, 4, 5, 6], 'n_epochs': [30, 40, 50]}\n",
        "\n",
        "# Performing 3-Fold gridsearch cross-validation\n",
        "gs = GridSearchCV(CoClustering, param_grid, measures = ['rmse'], cv = 3, n_jobs = -1)\n",
        "\n",
        "# Fitting data\n",
        "gs.fit(data)\n",
        "\n",
        "# Printing the best RMSE score\n",
        "print(gs.best_score['rmse'])\n",
        "\n",
        "# Printing the combination of parameters that gives the best RMSE score\n",
        "print(gs.best_params['rmse'])"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "568aeef9",
      "metadata": {
        "id": "568aeef9"
      },
      "source": [
        "Once the grid search is **complete**, we can get the **optimal values** for each of those hyperparameters as shown above."
      ]
    },
    {
      "cell_type": "markdown",
      "id": "06d378d1",
      "metadata": {
        "id": "06d378d1"
      },
      "source": [
        "We will build **final model** by using tuned values of the hyperparameters received after using the Grid search cross-validation above."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 24,
      "id": "5a7a8a30",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "5a7a8a30",
        "outputId": "62373f87-ec42-4a96-e121-ed16a25e5603"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "RMSE: 0.9499\n",
            "Precision:  0.715\n",
            "Recall:  0.5\n",
            "F_1 score:  0.588\n"
          ]
        }
      ],
      "source": [
        "# Using tuned Coclustering algorithm\n",
        "clust_tuned = CoClustering(n_cltr_u = 3, n_cltr_i = 3, n_epochs = 30, random_state = 1)\n",
        "\n",
        "# Training the algorithm on the train set\n",
        "clust_tuned.fit(trainset)\n",
        "\n",
        "# Let us compute precision@k, recall@k, and F_1 score with k = 10\n",
        "precision_recall_at_k(clust_tuned)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "9ec38e2a",
      "metadata": {
        "id": "9ec38e2a"
      },
      "source": [
        "- We can see that the F_1 score for tuned co-clustering model is slightly lower than the F_1 score for the baseline Co-clustering model."
      ]
    },
    {
      "cell_type": "markdown",
      "id": "e04c0665",
      "metadata": {
        "id": "e04c0665"
      },
      "source": [
        "Let's now **predict the rating** for the user with `userId = 4` and for the movie with `movieId = 10` as shown below. Here, the user has already rated the movie."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 25,
      "id": "6ba5b26b",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "6ba5b26b",
        "outputId": "df9ce76b-3d4a-423d-ce2a-dde9ede253ab"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "user: 4          item: 10         r_ui = 4.00   est = 3.65   {'was_impossible': False}\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Prediction(uid=4, iid=10, r_ui=4, est=3.6549811878747214, details={'was_impossible': False})"
            ]
          },
          "metadata": {},
          "execution_count": 25
        }
      ],
      "source": [
        "# Using co-clustering_optimized model to recommend for userId 4 and movieId 10\n",
        "clust_tuned.predict(4, 10, r_ui = 4, verbose = True)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "xviM9-Ptd7oD",
      "metadata": {
        "id": "xviM9-Ptd7oD"
      },
      "source": [
        "Below, we are predicting the rating for the same `userId = 4` but for a movie with which this user has not interacted before, i.e., `movieId = 3`, as shown below."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 26,
      "id": "ec582940",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ec582940",
        "outputId": "1680a8e5-4177-48ab-9554-df5320ca7204"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "user: 4          item: 3          r_ui = None   est = 3.23   {'was_impossible': False}\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Prediction(uid=4, iid=3, r_ui=None, est=3.2318175022354345, details={'was_impossible': False})"
            ]
          },
          "metadata": {},
          "execution_count": 26
        }
      ],
      "source": [
        "# Using Co-clustering based optimized model\n",
        "clust_tuned.predict(4, 3, verbose = True)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "df9e28ba",
      "metadata": {
        "id": "df9e28ba"
      },
      "source": [
        "### **Implementing the recommendation algorithm based on the optimized KNNBasic model**"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "4bbbdf9d",
      "metadata": {
        "id": "4bbbdf9d"
      },
      "source": [
        "Below we will be implementing a function where the input parameters are:\n",
        "\n",
        "- data: A **rating** dataset\n",
        "- user_id: A user id **against which we want the recommendations**\n",
        "- top_n: The **number of movies we want to recommend**\n",
        "- algo: The algorithm we want to use **for predicting the ratings**\n",
        "- The output of the function is a **set of top_n items** recommended for the given user id based on the given algorithm"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 27,
      "id": "4b8753bc",
      "metadata": {
        "id": "4b8753bc"
      },
      "outputs": [],
      "source": [
        "def get_recommendations(data, user_id, top_n, algo):\n",
        "\n",
        "    # Creating an empty list to store the recommended movie IDs\n",
        "    recommendations = []\n",
        "\n",
        "    # Creating an user-item interactions matrix\n",
        "    user_item_interactions_matrix = data.pivot(index = 'userId', columns = 'movieId', values = 'rating')\n",
        "\n",
        "    # Extracting those movie IDs which the userId has not interacted yet\n",
        "    non_interacted_movies = user_item_interactions_matrix.loc[user_id][user_item_interactions_matrix.loc[user_id].isnull()].index.tolist()\n",
        "\n",
        "    # Looping through each of the movie IDs which userId has not interacted yet\n",
        "    for item_id in non_interacted_movies:\n",
        "\n",
        "        # Predicting the ratings for those non interacted movie IDs by this user\n",
        "        est = algo.predict(user_id, item_id).est\n",
        "\n",
        "        # Appending the predicted ratings\n",
        "        recommendations.append((item_id, est))\n",
        "\n",
        "    # Sorting the predicted ratings in descending order\n",
        "    recommendations.sort(key = lambda x: x[1], reverse = True)\n",
        "\n",
        "    # Returing top n highest predicted rating movies for this user\n",
        "    return recommendations[:top_n]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 28,
      "id": "e0f36e15",
      "metadata": {
        "id": "e0f36e15"
      },
      "outputs": [],
      "source": [
        "# Getting top 5 recommendations for userId 4 using Co-clustering based optimized algorithm\n",
        "clustering_recommendations = get_recommendations(rating, 4, 5, clust_tuned)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "f1696941",
      "metadata": {
        "id": "f1696941"
      },
      "source": [
        "### **Correcting the Ratings and Ranking the above movies**"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "1de9ddd8",
      "metadata": {
        "id": "1de9ddd8"
      },
      "source": [
        "While comparing the ratings of two movies, it is not only the **ratings** that describe the **likelihood of the user to that movie**. Along with the rating, the **number of users who have watched that movie** also becomes important to consider. Due to this, we have calculated the **\"corrected_ratings\"** for each movie. Commonly higher the **\"rating_count\" of a movie more it is liked by users**. To interpret the above concept, a **movie rated 4 with rating_count 3 is less liked in comparison to a movie rated 3 with a rating count of 50**. It has been **empirically found that the likelihood of the movie is directly proportional to the inverse of the square root of the rating_count of the movie**."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 29,
      "id": "2a2f430d",
      "metadata": {
        "id": "2a2f430d"
      },
      "outputs": [],
      "source": [
        "def ranking_movies(recommendations, final_rating):\n",
        "\n",
        "    # Sort the movies based on ratings count\n",
        "    ranked_movies = final_rating.loc[[items[0] for items in recommendations]].sort_values('rating_count', ascending = False)[['rating_count']].reset_index()\n",
        "\n",
        "    # Merge with the recommended movies to get predicted ratings\n",
        "    ranked_movies = ranked_movies.merge(pd.DataFrame(recommendations, columns = ['movieId', 'predicted_ratings']), on = 'movieId', how = 'inner')\n",
        "\n",
        "    # Rank the movies based on corrected ratings\n",
        "    ranked_movies['corrected_ratings'] = ranked_movies['predicted_ratings'] - 1 / np.sqrt(ranked_movies['rating_count'])\n",
        "\n",
        "    # Sort the movies based on corrected ratings\n",
        "    ranked_movies = ranked_movies.sort_values('corrected_ratings', ascending = False)\n",
        "\n",
        "    return ranked_movies"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "411fa3ff",
      "metadata": {
        "id": "411fa3ff"
      },
      "source": [
        "**Note:** In the **above-corrected rating formula**, we can add the **quantity `1 / np.sqrt(n)` instead of subtracting it to get more optimistic predictions**. But here we are **subtracting this quantity**, as there are some movies with ratings of 5 and **we can't have a rating more than 5 for a movie**."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 30,
      "id": "c186f13b",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 204
        },
        "id": "c186f13b",
        "outputId": "dc861f45-3aa2-4fbd-d1bb-1ec6bc06dd70",
        "scrolled": true
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "   movieId  rating_count  predicted_ratings  corrected_ratings\n",
              "0      304             3                  5           4.422650\n",
              "1       53             2                  5           4.292893\n",
              "2       99             2                  5           4.292893\n",
              "3      238             2                  5           4.292893\n",
              "4      148             1                  5           4.000000"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-ba29dd77-9fcc-4e0b-8e9d-5f71f0efb738\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>movieId</th>\n",
              "      <th>rating_count</th>\n",
              "      <th>predicted_ratings</th>\n",
              "      <th>corrected_ratings</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>304</td>\n",
              "      <td>3</td>\n",
              "      <td>5</td>\n",
              "      <td>4.422650</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>53</td>\n",
              "      <td>2</td>\n",
              "      <td>5</td>\n",
              "      <td>4.292893</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>99</td>\n",
              "      <td>2</td>\n",
              "      <td>5</td>\n",
              "      <td>4.292893</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>238</td>\n",
              "      <td>2</td>\n",
              "      <td>5</td>\n",
              "      <td>4.292893</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>148</td>\n",
              "      <td>1</td>\n",
              "      <td>5</td>\n",
              "      <td>4.000000</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-ba29dd77-9fcc-4e0b-8e9d-5f71f0efb738')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-ba29dd77-9fcc-4e0b-8e9d-5f71f0efb738 button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-ba29dd77-9fcc-4e0b-8e9d-5f71f0efb738');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-41467511-9a64-45f2-b94f-e7fe30e1f3f6\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-41467511-9a64-45f2-b94f-e7fe30e1f3f6')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-41467511-9a64-45f2-b94f-e7fe30e1f3f6 button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "dataframe",
              "summary": "{\n  \"name\": \"ranking_movies(clustering_recommendations, final_rating)\",\n  \"rows\": 5,\n  \"fields\": [\n    {\n      \"column\": \"movieId\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 102,\n        \"min\": 53,\n        \"max\": 304,\n        \"num_unique_values\": 5,\n        \"samples\": [\n          53,\n          148,\n          99\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"rating_count\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0,\n        \"min\": 1,\n        \"max\": 3,\n        \"num_unique_values\": 3,\n        \"samples\": [\n          3,\n          2,\n          1\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"predicted_ratings\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0,\n        \"min\": 5,\n        \"max\": 5,\n        \"num_unique_values\": 1,\n        \"samples\": [\n          5\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"corrected_ratings\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0.15596511870473115,\n        \"min\": 4.0,\n        \"max\": 4.422649730810374,\n        \"num_unique_values\": 3,\n        \"samples\": [\n          4.422649730810374\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"
            }
          },
          "metadata": {},
          "execution_count": 30
        }
      ],
      "source": [
        "# Ranking movies based on the above recommendations\n",
        "ranking_movies(clustering_recommendations, final_rating)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "bd6f4e5e",
      "metadata": {
        "id": "bd6f4e5e"
      },
      "source": [
        "**Let us now move to the final recommendation algorithm which is named the Content-based recommendation system.**"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "2a917624",
      "metadata": {
        "id": "2a917624"
      },
      "source": [
        "## **Model 2: Content-Based Recommendation System**"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "2e21e2a2",
      "metadata": {
        "id": "2e21e2a2"
      },
      "source": [
        "In a **content-based recommendation system**, we will use the feature - **text**, i.e., reviews to find similar movies."
      ]
    },
    {
      "cell_type": "markdown",
      "id": "cc0ff761",
      "metadata": {
        "id": "cc0ff761"
      },
      "source": [
        "Text data generally contains pronunciation, stopwords, and non-ASCII characters, which makes it **very noisy**. So, we will first need to **pre-process the text** and then we will **generate features from the text to compute similarities** between the texts/reviews.\n",
        "\n",
        "Let's load the **tags dataset.**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 31,
      "id": "b9c8a0bd",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 204
        },
        "id": "b9c8a0bd",
        "outputId": "51b6c132-3223-4287-991c-aa4c473ec29a",
        "scrolled": true
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "   userId  movieId              tag   timestamp\n",
              "0       2    60756            funny  1445714994\n",
              "1       2    60756  Highly quotable  1445714996\n",
              "2       2    60756     will ferrell  1445714992\n",
              "3       2    89774     Boxing story  1445715207\n",
              "4       2    89774              MMA  1445715200"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-09e27119-6827-4722-9073-e40d8c6485a0\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>userId</th>\n",
              "      <th>movieId</th>\n",
              "      <th>tag</th>\n",
              "      <th>timestamp</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>2</td>\n",
              "      <td>60756</td>\n",
              "      <td>funny</td>\n",
              "      <td>1445714994</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>2</td>\n",
              "      <td>60756</td>\n",
              "      <td>Highly quotable</td>\n",
              "      <td>1445714996</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>2</td>\n",
              "      <td>60756</td>\n",
              "      <td>will ferrell</td>\n",
              "      <td>1445714992</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>2</td>\n",
              "      <td>89774</td>\n",
              "      <td>Boxing story</td>\n",
              "      <td>1445715207</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>2</td>\n",
              "      <td>89774</td>\n",
              "      <td>MMA</td>\n",
              "      <td>1445715200</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-09e27119-6827-4722-9073-e40d8c6485a0')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-09e27119-6827-4722-9073-e40d8c6485a0 button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-09e27119-6827-4722-9073-e40d8c6485a0');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-b47495cb-23fe-44f8-ac3e-183cb2a2a0c2\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-b47495cb-23fe-44f8-ac3e-183cb2a2a0c2')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-b47495cb-23fe-44f8-ac3e-183cb2a2a0c2 button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "dataframe",
              "variable_name": "tags",
              "summary": "{\n  \"name\": \"tags\",\n  \"rows\": 3683,\n  \"fields\": [\n    {\n      \"column\": \"userId\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 158,\n        \"min\": 2,\n        \"max\": 610,\n        \"num_unique_values\": 58,\n        \"samples\": [\n          2,\n          62,\n          341\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"movieId\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 43490,\n        \"min\": 1,\n        \"max\": 193565,\n        \"num_unique_values\": 1572,\n        \"samples\": [\n          6296,\n          2939,\n          6708\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"tag\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 1589,\n        \"samples\": [\n          \"Grace\",\n          \"documentary\",\n          \"homosexuality\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"timestamp\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 172102450,\n        \"min\": 1137179352,\n        \"max\": 1537098603,\n        \"num_unique_values\": 3411,\n        \"samples\": [\n          1498456527,\n          1137192017,\n          1525554215\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"
            }
          },
          "metadata": {},
          "execution_count": 31
        }
      ],
      "source": [
        "# Importing the tags data\n",
        "tags = pd.read_csv('/content/drive/MyDrive/Movie/tags.csv')\n",
        "tags.head()"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "00015af4",
      "metadata": {
        "id": "00015af4"
      },
      "source": [
        "In this dataset, we don't have any movie review or plot of the movie, so we will combine the columns - **title**, **genres** from the other two datasets, and **tag** from the tags dataset to create a text-based feature and apply the **TF-IDF** feature extraction technique to extract features, which we later use to compute similar movies based on these texts."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 32,
      "id": "0494a4cd",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 419
        },
        "id": "0494a4cd",
        "outputId": "8224df38-ca0f-48c6-90f3-f1614650d044"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "        userId  movieId  rating   timestamp  \\\n",
              "0            1        1     4.0   964982703   \n",
              "1            1        1     4.0   964982703   \n",
              "2            1        1     4.0   964982703   \n",
              "3            5        1     4.0   847434962   \n",
              "4            5        1     4.0   847434962   \n",
              "...        ...      ...     ...         ...   \n",
              "233208     599   176419     3.5  1516604655   \n",
              "233209     599   176419     3.5  1516604655   \n",
              "233210     594     7023     4.5  1108972356   \n",
              "233211     606     6107     4.0  1171324428   \n",
              "233212     606     6516     3.5  1171755910   \n",
              "\n",
              "                                                    title  \\\n",
              "0                                        Toy Story (1995)   \n",
              "1                                        Toy Story (1995)   \n",
              "2                                        Toy Story (1995)   \n",
              "3                                        Toy Story (1995)   \n",
              "4                                        Toy Story (1995)   \n",
              "...                                                   ...   \n",
              "233208                                     Mother! (2017)   \n",
              "233209                                     Mother! (2017)   \n",
              "233210               Wedding Banquet, The (Xi yan) (1993)   \n",
              "233211  Night of the Shooting Stars (Notte di San Lore...   \n",
              "233212                                   Anastasia (1956)   \n",
              "\n",
              "                                             genres               tag  \n",
              "0       Adventure|Animation|Children|Comedy|Fantasy             pixar  \n",
              "1       Adventure|Animation|Children|Comedy|Fantasy             pixar  \n",
              "2       Adventure|Animation|Children|Comedy|Fantasy               fun  \n",
              "3       Adventure|Animation|Children|Comedy|Fantasy             pixar  \n",
              "4       Adventure|Animation|Children|Comedy|Fantasy             pixar  \n",
              "...                                             ...               ...  \n",
              "233208                Drama|Horror|Mystery|Thriller     uncomfortable  \n",
              "233209                Drama|Horror|Mystery|Thriller        unsettling  \n",
              "233210                         Comedy|Drama|Romance  In Netflix queue  \n",
              "233211                                    Drama|War      World War II  \n",
              "233212                                        Drama  In Netflix queue  \n",
              "\n",
              "[233213 rows x 7 columns]"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-14296e93-0dff-4476-a520-b87574123c20\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>userId</th>\n",
              "      <th>movieId</th>\n",
              "      <th>rating</th>\n",
              "      <th>timestamp</th>\n",
              "      <th>title</th>\n",
              "      <th>genres</th>\n",
              "      <th>tag</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>4.0</td>\n",
              "      <td>964982703</td>\n",
              "      <td>Toy Story (1995)</td>\n",
              "      <td>Adventure|Animation|Children|Comedy|Fantasy</td>\n",
              "      <td>pixar</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>4.0</td>\n",
              "      <td>964982703</td>\n",
              "      <td>Toy Story (1995)</td>\n",
              "      <td>Adventure|Animation|Children|Comedy|Fantasy</td>\n",
              "      <td>pixar</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>4.0</td>\n",
              "      <td>964982703</td>\n",
              "      <td>Toy Story (1995)</td>\n",
              "      <td>Adventure|Animation|Children|Comedy|Fantasy</td>\n",
              "      <td>fun</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>5</td>\n",
              "      <td>1</td>\n",
              "      <td>4.0</td>\n",
              "      <td>847434962</td>\n",
              "      <td>Toy Story (1995)</td>\n",
              "      <td>Adventure|Animation|Children|Comedy|Fantasy</td>\n",
              "      <td>pixar</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>5</td>\n",
              "      <td>1</td>\n",
              "      <td>4.0</td>\n",
              "      <td>847434962</td>\n",
              "      <td>Toy Story (1995)</td>\n",
              "      <td>Adventure|Animation|Children|Comedy|Fantasy</td>\n",
              "      <td>pixar</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>233208</th>\n",
              "      <td>599</td>\n",
              "      <td>176419</td>\n",
              "      <td>3.5</td>\n",
              "      <td>1516604655</td>\n",
              "      <td>Mother! (2017)</td>\n",
              "      <td>Drama|Horror|Mystery|Thriller</td>\n",
              "      <td>uncomfortable</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>233209</th>\n",
              "      <td>599</td>\n",
              "      <td>176419</td>\n",
              "      <td>3.5</td>\n",
              "      <td>1516604655</td>\n",
              "      <td>Mother! (2017)</td>\n",
              "      <td>Drama|Horror|Mystery|Thriller</td>\n",
              "      <td>unsettling</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>233210</th>\n",
              "      <td>594</td>\n",
              "      <td>7023</td>\n",
              "      <td>4.5</td>\n",
              "      <td>1108972356</td>\n",
              "      <td>Wedding Banquet, The (Xi yan) (1993)</td>\n",
              "      <td>Comedy|Drama|Romance</td>\n",
              "      <td>In Netflix queue</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>233211</th>\n",
              "      <td>606</td>\n",
              "      <td>6107</td>\n",
              "      <td>4.0</td>\n",
              "      <td>1171324428</td>\n",
              "      <td>Night of the Shooting Stars (Notte di San Lore...</td>\n",
              "      <td>Drama|War</td>\n",
              "      <td>World War II</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>233212</th>\n",
              "      <td>606</td>\n",
              "      <td>6516</td>\n",
              "      <td>3.5</td>\n",
              "      <td>1171755910</td>\n",
              "      <td>Anastasia (1956)</td>\n",
              "      <td>Drama</td>\n",
              "      <td>In Netflix queue</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>233213 rows × 7 columns</p>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-14296e93-0dff-4476-a520-b87574123c20')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-14296e93-0dff-4476-a520-b87574123c20 button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-14296e93-0dff-4476-a520-b87574123c20');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-612863a9-e23d-4cdc-851e-1e74dbf5872b\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-612863a9-e23d-4cdc-851e-1e74dbf5872b')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-612863a9-e23d-4cdc-851e-1e74dbf5872b button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "\n",
              "  <div id=\"id_f5c0d78e-8ac8-4245-aac6-deb3ba008af7\">\n",
              "    <style>\n",
              "      .colab-df-generate {\n",
              "        background-color: #E8F0FE;\n",
              "        border: none;\n",
              "        border-radius: 50%;\n",
              "        cursor: pointer;\n",
              "        display: none;\n",
              "        fill: #1967D2;\n",
              "        height: 32px;\n",
              "        padding: 0 0 0 0;\n",
              "        width: 32px;\n",
              "      }\n",
              "\n",
              "      .colab-df-generate:hover {\n",
              "        background-color: #E2EBFA;\n",
              "        box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "        fill: #174EA6;\n",
              "      }\n",
              "\n",
              "      [theme=dark] .colab-df-generate {\n",
              "        background-color: #3B4455;\n",
              "        fill: #D2E3FC;\n",
              "      }\n",
              "\n",
              "      [theme=dark] .colab-df-generate:hover {\n",
              "        background-color: #434B5C;\n",
              "        box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "        filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "        fill: #FFFFFF;\n",
              "      }\n",
              "    </style>\n",
              "    <button class=\"colab-df-generate\" onclick=\"generateWithVariable('final_ratings')\"\n",
              "            title=\"Generate code using this dataframe.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M7,19H8.4L18.45,9,17,7.55,7,17.6ZM5,21V16.75L18.45,3.32a2,2,0,0,1,2.83,0l1.4,1.43a1.91,1.91,0,0,1,.58,1.4,1.91,1.91,0,0,1-.58,1.4L9.25,21ZM18.45,9,17,7.55Zm-12,3A5.31,5.31,0,0,0,4.9,8.1,5.31,5.31,0,0,0,1,6.5,5.31,5.31,0,0,0,4.9,4.9,5.31,5.31,0,0,0,6.5,1,5.31,5.31,0,0,0,8.1,4.9,5.31,5.31,0,0,0,12,6.5,5.46,5.46,0,0,0,6.5,12Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "    <script>\n",
              "      (() => {\n",
              "      const buttonEl =\n",
              "        document.querySelector('#id_f5c0d78e-8ac8-4245-aac6-deb3ba008af7 button.colab-df-generate');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      buttonEl.onclick = () => {\n",
              "        google.colab.notebook.generateWithVariable('final_ratings');\n",
              "      }\n",
              "      })();\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "dataframe",
              "variable_name": "final_ratings"
            }
          },
          "metadata": {},
          "execution_count": 32
        }
      ],
      "source": [
        "# Merging all the three datasets on movieId\n",
        "ratings_with_title = pd.merge(ratings, movies[['movieId', 'title', 'genres']], on = 'movieId' )\n",
        "\n",
        "final_ratings = pd.merge(ratings_with_title, tags[['movieId', 'tag']], on = 'movieId' )\n",
        "\n",
        "# Let us see the dataset\n",
        "final_ratings"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "bb2d2201",
      "metadata": {
        "id": "bb2d2201"
      },
      "source": [
        "- We can observe that the **multiple genres are separated by | which we need to remove**.\n",
        "- We will combine the three columns title, genres, and tag."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 33,
      "id": "90377400",
      "metadata": {
        "id": "90377400"
      },
      "outputs": [],
      "source": [
        "# Replacing | character with space in genres column\n",
        "final_ratings['genres'] = final_ratings['genres'].apply(lambda x: \" \".join(x.split('|')))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 34,
      "id": "9477d9e0",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 204
        },
        "id": "9477d9e0",
        "outputId": "ba22f065-8b0d-4c57-f558-4fe65e41a651"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "   userId  movieId  rating  timestamp             title  \\\n",
              "0       1        1     4.0  964982703  Toy Story (1995)   \n",
              "1       1        1     4.0  964982703  Toy Story (1995)   \n",
              "2       1        1     4.0  964982703  Toy Story (1995)   \n",
              "3       5        1     4.0  847434962  Toy Story (1995)   \n",
              "4       5        1     4.0  847434962  Toy Story (1995)   \n",
              "\n",
              "                                        genres    tag  \\\n",
              "0  Adventure Animation Children Comedy Fantasy  pixar   \n",
              "1  Adventure Animation Children Comedy Fantasy  pixar   \n",
              "2  Adventure Animation Children Comedy Fantasy    fun   \n",
              "3  Adventure Animation Children Comedy Fantasy  pixar   \n",
              "4  Adventure Animation Children Comedy Fantasy  pixar   \n",
              "\n",
              "                                                text  \n",
              "0  Toy Story (1995) Adventure Animation Children ...  \n",
              "1  Toy Story (1995) Adventure Animation Children ...  \n",
              "2  Toy Story (1995) Adventure Animation Children ...  \n",
              "3  Toy Story (1995) Adventure Animation Children ...  \n",
              "4  Toy Story (1995) Adventure Animation Children ...  "
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-c07ecbc6-64b2-449b-a13f-f3fbde31c20b\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>userId</th>\n",
              "      <th>movieId</th>\n",
              "      <th>rating</th>\n",
              "      <th>timestamp</th>\n",
              "      <th>title</th>\n",
              "      <th>genres</th>\n",
              "      <th>tag</th>\n",
              "      <th>text</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>4.0</td>\n",
              "      <td>964982703</td>\n",
              "      <td>Toy Story (1995)</td>\n",
              "      <td>Adventure Animation Children Comedy Fantasy</td>\n",
              "      <td>pixar</td>\n",
              "      <td>Toy Story (1995) Adventure Animation Children ...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>4.0</td>\n",
              "      <td>964982703</td>\n",
              "      <td>Toy Story (1995)</td>\n",
              "      <td>Adventure Animation Children Comedy Fantasy</td>\n",
              "      <td>pixar</td>\n",
              "      <td>Toy Story (1995) Adventure Animation Children ...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>4.0</td>\n",
              "      <td>964982703</td>\n",
              "      <td>Toy Story (1995)</td>\n",
              "      <td>Adventure Animation Children Comedy Fantasy</td>\n",
              "      <td>fun</td>\n",
              "      <td>Toy Story (1995) Adventure Animation Children ...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>5</td>\n",
              "      <td>1</td>\n",
              "      <td>4.0</td>\n",
              "      <td>847434962</td>\n",
              "      <td>Toy Story (1995)</td>\n",
              "      <td>Adventure Animation Children Comedy Fantasy</td>\n",
              "      <td>pixar</td>\n",
              "      <td>Toy Story (1995) Adventure Animation Children ...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>5</td>\n",
              "      <td>1</td>\n",
              "      <td>4.0</td>\n",
              "      <td>847434962</td>\n",
              "      <td>Toy Story (1995)</td>\n",
              "      <td>Adventure Animation Children Comedy Fantasy</td>\n",
              "      <td>pixar</td>\n",
              "      <td>Toy Story (1995) Adventure Animation Children ...</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-c07ecbc6-64b2-449b-a13f-f3fbde31c20b')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-c07ecbc6-64b2-449b-a13f-f3fbde31c20b button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-c07ecbc6-64b2-449b-a13f-f3fbde31c20b');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-dd389e02-edef-4d11-8840-a52767757398\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-dd389e02-edef-4d11-8840-a52767757398')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-dd389e02-edef-4d11-8840-a52767757398 button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "dataframe",
              "variable_name": "final_ratings"
            }
          },
          "metadata": {},
          "execution_count": 34
        }
      ],
      "source": [
        "# Combining title, genres, and tag columns\n",
        "final_ratings['text'] = final_ratings['title'] + ' ' + final_ratings['genres'] + ' ' + final_ratings['tag']\n",
        "\n",
        "final_ratings.head()"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "ac3bdc9c",
      "metadata": {
        "id": "ac3bdc9c"
      },
      "source": [
        "Now, we will **keep only four columns** - userId, movieId, rating, and text. We will drop the duplicate titles from the data and make it the **title column as the index** of the dataframe."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 35,
      "id": "d8801412",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 235
        },
        "id": "d8801412",
        "outputId": "e68933e9-7529-410f-c29a-d5d87c1bdc6e"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "                             userId  movieId  rating  \\\n",
              "title                                                  \n",
              "Toy Story (1995)                  1        1     4.0   \n",
              "Grumpier Old Men (1995)           1        3     4.0   \n",
              "Seven (a.k.a. Se7en) (1995)       1       47     5.0   \n",
              "Usual Suspects, The (1995)        1       50     5.0   \n",
              "Bottle Rocket (1996)              1      101     5.0   \n",
              "\n",
              "                                                                          text  \n",
              "title                                                                           \n",
              "Toy Story (1995)             Toy Story (1995) Adventure Animation Children ...  \n",
              "Grumpier Old Men (1995)           Grumpier Old Men (1995) Comedy Romance moldy  \n",
              "Seven (a.k.a. Se7en) (1995)  Seven (a.k.a. Se7en) (1995) Mystery Thriller m...  \n",
              "Usual Suspects, The (1995)   Usual Suspects, The (1995) Crime Mystery Thril...  \n",
              "Bottle Rocket (1996)         Bottle Rocket (1996) Adventure Comedy Crime Ro...  "
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-4e8e0e6c-8401-4511-b10b-c0b64b88259b\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>userId</th>\n",
              "      <th>movieId</th>\n",
              "      <th>rating</th>\n",
              "      <th>text</th>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>title</th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>Toy Story (1995)</th>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>4.0</td>\n",
              "      <td>Toy Story (1995) Adventure Animation Children ...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Grumpier Old Men (1995)</th>\n",
              "      <td>1</td>\n",
              "      <td>3</td>\n",
              "      <td>4.0</td>\n",
              "      <td>Grumpier Old Men (1995) Comedy Romance moldy</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Seven (a.k.a. Se7en) (1995)</th>\n",
              "      <td>1</td>\n",
              "      <td>47</td>\n",
              "      <td>5.0</td>\n",
              "      <td>Seven (a.k.a. Se7en) (1995) Mystery Thriller m...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Usual Suspects, The (1995)</th>\n",
              "      <td>1</td>\n",
              "      <td>50</td>\n",
              "      <td>5.0</td>\n",
              "      <td>Usual Suspects, The (1995) Crime Mystery Thril...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Bottle Rocket (1996)</th>\n",
              "      <td>1</td>\n",
              "      <td>101</td>\n",
              "      <td>5.0</td>\n",
              "      <td>Bottle Rocket (1996) Adventure Comedy Crime Ro...</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-4e8e0e6c-8401-4511-b10b-c0b64b88259b')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-4e8e0e6c-8401-4511-b10b-c0b64b88259b button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-4e8e0e6c-8401-4511-b10b-c0b64b88259b');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-21b920fc-97c7-451b-b75a-1d44883cf026\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-21b920fc-97c7-451b-b75a-1d44883cf026')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-21b920fc-97c7-451b-b75a-1d44883cf026 button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "dataframe",
              "variable_name": "final_ratings",
              "summary": "{\n  \"name\": \"final_ratings\",\n  \"rows\": 1554,\n  \"fields\": [\n    {\n      \"column\": \"title\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 1554,\n        \"samples\": [\n          \"Letter to Three Wives, A (1949)\",\n          \"Congo (1995)\",\n          \"Saw (2004)\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"userId\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 162,\n        \"min\": 1,\n        \"max\": 606,\n        \"num_unique_values\": 153,\n        \"samples\": [\n          168,\n          177,\n          210\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"movieId\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 35448,\n        \"min\": 1,\n        \"max\": 193565,\n        \"num_unique_values\": 1554,\n        \"samples\": [\n          25952,\n          160,\n          8957\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"rating\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 1.0177865475066292,\n        \"min\": 0.5,\n        \"max\": 5.0,\n        \"num_unique_values\": 10,\n        \"samples\": [\n          1.5,\n          5.0,\n          4.5\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"text\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 1554,\n        \"samples\": [\n          \"Letter to Three Wives, A (1949) Comedy Drama In Netflix queue\",\n          \"Congo (1995) Action Adventure Mystery Sci-Fi Michael Crichton\",\n          \"Saw (2004) Horror Mystery Thriller brutal\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"
            }
          },
          "metadata": {},
          "execution_count": 35
        }
      ],
      "source": [
        "# Create the final_ratings dataset with specified columns\n",
        "final_ratings = final_ratings[['userId', 'movieId', 'rating', 'title', 'text']]\n",
        "\n",
        "# Let us drop the duplicate records\n",
        "final_ratings = final_ratings.drop_duplicates(subset = ['title'])\n",
        "\n",
        "# Set the index\n",
        "final_ratings = final_ratings.set_index('title')\n",
        "\n",
        "# See the first five records of the dataset\n",
        "final_ratings.head()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 36,
      "id": "6854a556",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "6854a556",
        "outputId": "58990a9b-acb4-4295-f855-244620e91f62"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(1554, 4)"
            ]
          },
          "metadata": {},
          "execution_count": 36
        }
      ],
      "source": [
        "# Let us see the shape of final_ratings data\n",
        "final_ratings.shape"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "1b97645b",
      "metadata": {
        "id": "1b97645b"
      },
      "source": [
        "**Now, let's process the text data and create features to find the similarity between movies.**"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "b620769e",
      "metadata": {
        "id": "b620769e"
      },
      "source": [
        "### **Loading libraries to handle the text data**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 37,
      "id": "3e1cfa66",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "3e1cfa66",
        "outputId": "e653e494-b26c-47f4-c350-bc545a86ced3"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "[nltk_data] Downloading package omw-1.4 to /root/nltk_data...\n",
            "[nltk_data] Downloading package punkt to /root/nltk_data...\n",
            "[nltk_data]   Unzipping tokenizers/punkt.zip.\n",
            "[nltk_data] Downloading package stopwords to /root/nltk_data...\n",
            "[nltk_data]   Unzipping corpora/stopwords.zip.\n",
            "[nltk_data] Downloading package wordnet to /root/nltk_data...\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "True"
            ]
          },
          "metadata": {},
          "execution_count": 37
        }
      ],
      "source": [
        "# Importing nltk (natural language toolkit library)\n",
        "import nltk\n",
        "nltk.download('omw-1.4')\n",
        "\n",
        "# Downloading punctuations\n",
        "nltk.download('punkt')\n",
        "\n",
        "# Downloading stopwords\n",
        "nltk.download('stopwords')\n",
        "\n",
        "# Downloading wordnet\n",
        "nltk.download('wordnet')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 38,
      "id": "212f0a82",
      "metadata": {
        "id": "212f0a82"
      },
      "outputs": [],
      "source": [
        "# This is importing regular expression\n",
        "import re\n",
        "\n",
        "# Word_tokenize is used to do tokenization\n",
        "from nltk import word_tokenize\n",
        "\n",
        "# Importing the Lematizer\n",
        "from nltk.stem import WordNetLemmatizer\n",
        "\n",
        "# Importing the stopwords\n",
        "from nltk.corpus import stopwords\n",
        "\n",
        "# Tfidf vectorizer used to create the computational vectors\n",
        "from sklearn.feature_extraction.text import TfidfVectorizer"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "f70a9db9",
      "metadata": {
        "id": "f70a9db9"
      },
      "source": [
        "We will create a **function to pre-process the text data**. Before that, let's see some **terminology**.\n",
        "- **stopwords:** A stop word is a commonly used word (such as “the”, “a”, “an”, or “in”) that does not contain information in the text and can be ignored.\n",
        "- **Lemmatization:** Lemmatization in linguistics is the process of grouping together the inflected forms of a word so they can be analyzed as a single item. For example, runs, running, and ran are all forms of the word run, therefore run is the lemma of all these words."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 39,
      "id": "202b9caf",
      "metadata": {
        "id": "202b9caf"
      },
      "outputs": [],
      "source": [
        "# Create the tokenize function\n",
        "def tokenize(text):\n",
        "\n",
        "    # Making each letter as lowercase and removing non-alphabetical text\n",
        "    text = re.sub(r\"[^a-zA-Z]\",\" \", text.lower())\n",
        "\n",
        "    # Extracting each word in the text\n",
        "    tokens = word_tokenize(text)\n",
        "\n",
        "    # Removing stopwords\n",
        "    words = [word for word in tokens if word not in stopwords.words(\"english\")]\n",
        "\n",
        "    # Lemmatize the words\n",
        "    text_lems = [WordNetLemmatizer().lemmatize(lem).strip() for lem in words]\n",
        "\n",
        "    return text_lems"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "a04c830d",
      "metadata": {
        "id": "a04c830d"
      },
      "source": [
        "### **Feature Extraction**"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "5827d7ae",
      "metadata": {
        "id": "5827d7ae"
      },
      "source": [
        "Below are some of the ways to extract features from texts:\n",
        "- Bag of words\n",
        "- TF-IDF\n",
        "- One hot encoding\n",
        "- Word vectors"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "b55b0405",
      "metadata": {
        "id": "b55b0405"
      },
      "source": [
        "Here, we will be using **TF-IDF** as a feature extraction technique."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 40,
      "id": "be360fe3",
      "metadata": {
        "id": "be360fe3"
      },
      "outputs": [],
      "source": [
        "# Creating the TF-IDF object\n",
        "tfidf = TfidfVectorizer(tokenizer = tokenize)\n",
        "\n",
        "movie_tfidf = tfidf.fit_transform(final_ratings['text'].values).toarray()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 41,
      "id": "6d434849",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 419
        },
        "id": "6d434849",
        "outputId": "7f2a2fc0-8f95-4266-b2fe-1f2d0f5a6632"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "      0     1     2     3     4     5     6     7     8     9     ...  2773  \\\n",
              "0      0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0  ...   0.0   \n",
              "1      0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0  ...   0.0   \n",
              "2      0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0  ...   0.0   \n",
              "3      0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0  ...   0.0   \n",
              "4      0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0  ...   0.0   \n",
              "...    ...   ...   ...   ...   ...   ...   ...   ...   ...   ...  ...   ...   \n",
              "1549   0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0  ...   0.0   \n",
              "1550   0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0  ...   0.0   \n",
              "1551   0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0  ...   0.0   \n",
              "1552   0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0  ...   0.0   \n",
              "1553   0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0  ...   0.0   \n",
              "\n",
              "      2774  2775  2776  2777  2778  2779  2780  2781  2782  \n",
              "0      0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0  \n",
              "1      0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0  \n",
              "2      0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0  \n",
              "3      0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0  \n",
              "4      0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0  \n",
              "...    ...   ...   ...   ...   ...   ...   ...   ...   ...  \n",
              "1549   0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0  \n",
              "1550   0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0  \n",
              "1551   0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0  \n",
              "1552   0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0  \n",
              "1553   0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0  \n",
              "\n",
              "[1554 rows x 2783 columns]"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-e8ba08b9-4787-4b57-ba2a-cf1de660c5fb\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>0</th>\n",
              "      <th>1</th>\n",
              "      <th>2</th>\n",
              "      <th>3</th>\n",
              "      <th>4</th>\n",
              "      <th>5</th>\n",
              "      <th>6</th>\n",
              "      <th>7</th>\n",
              "      <th>8</th>\n",
              "      <th>9</th>\n",
              "      <th>...</th>\n",
              "      <th>2773</th>\n",
              "      <th>2774</th>\n",
              "      <th>2775</th>\n",
              "      <th>2776</th>\n",
              "      <th>2777</th>\n",
              "      <th>2778</th>\n",
              "      <th>2779</th>\n",
              "      <th>2780</th>\n",
              "      <th>2781</th>\n",
              "      <th>2782</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1549</th>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1550</th>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1551</th>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1552</th>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1553</th>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>1554 rows × 2783 columns</p>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-e8ba08b9-4787-4b57-ba2a-cf1de660c5fb')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-e8ba08b9-4787-4b57-ba2a-cf1de660c5fb button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-e8ba08b9-4787-4b57-ba2a-cf1de660c5fb');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-5282ebae-db85-49e4-9408-195126566529\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-5282ebae-db85-49e4-9408-195126566529')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-5282ebae-db85-49e4-9408-195126566529 button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "dataframe"
            }
          },
          "metadata": {},
          "execution_count": 41
        }
      ],
      "source": [
        "# Making the DataFrame of movie_tfidf data\n",
        "pd.DataFrame(movie_tfidf)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "690a0300",
      "metadata": {
        "id": "690a0300"
      },
      "source": [
        "We have **extracted features from the text data**. Now, we can find similarities between movies using these features. We will use cosine similarity to calculate the similarity."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 42,
      "id": "fc645217",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "fc645217",
        "outputId": "849437b0-c032-43c2-b18d-12b5173ad9f7"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[1.        , 0.02268393, 0.        , ..., 0.02022472, 0.        ,\n",
              "        0.        ],\n",
              "       [0.02268393, 1.        , 0.        , ..., 0.04779055, 0.        ,\n",
              "        0.        ],\n",
              "       [0.        , 0.        , 1.        , ..., 0.        , 0.        ,\n",
              "        0.        ],\n",
              "       ...,\n",
              "       [0.02022472, 0.04779055, 0.        , ..., 1.        , 0.00719396,\n",
              "        0.19617374],\n",
              "       [0.        , 0.        , 0.        , ..., 0.00719396, 1.        ,\n",
              "        0.01217017],\n",
              "       [0.        , 0.        , 0.        , ..., 0.19617374, 0.01217017,\n",
              "        1.        ]])"
            ]
          },
          "metadata": {},
          "execution_count": 42
        }
      ],
      "source": [
        "# Calculating the cosine similarity\n",
        "similar_movies = cosine_similarity(movie_tfidf, movie_tfidf)\n",
        "\n",
        "# Let us see the above array\n",
        "similar_movies"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "c46232f8",
      "metadata": {
        "id": "c46232f8"
      },
      "source": [
        "**Finally, let's create a function to find the most similar movies to recommend for a given movie.**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 43,
      "id": "1f6244ae",
      "metadata": {
        "id": "1f6244ae"
      },
      "outputs": [],
      "source": [
        "# Function that takes in movie title as input and returns the top 10 recommended movies\n",
        "def recommendations(title, similar_movies):\n",
        "\n",
        "    recommended_movies = []\n",
        "\n",
        "    indices = pd.Series(final_ratings.index)\n",
        "\n",
        "    # Getting the index of the movie that matches the title\n",
        "    idx = indices[indices == title].index[0]\n",
        "\n",
        "    # Creating a Series with the similarity scores in descending order\n",
        "    score_series = pd.Series(similar_movies[idx]).sort_values(ascending = False)\n",
        "\n",
        "    # Getting the indices of 10 most similar movies\n",
        "    top_10_indexes = list(score_series.iloc[1 : 11].index)\n",
        "    print(top_10_indexes)\n",
        "\n",
        "    # Populating the list with the titles of the best 10 matching movies\n",
        "    for i in top_10_indexes:\n",
        "        recommended_movies.append(list(final_ratings.index)[i])\n",
        "\n",
        "    return recommended_movies"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 44,
      "id": "6e0d68f9",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "6e0d68f9",
        "outputId": "fcb6081b-6aff-49d5-d4ce-612873608ec9"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[71, 1186, 124, 551, 569, 77, 719, 766, 123, 658]\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "['Game, The (1997)',\n",
              " 'Andalusian Dog, An (Chien andalou, Un) (1929)',\n",
              " 'Town, The (2010)',\n",
              " 'Now You See Me (2013)',\n",
              " 'Charade (1963)',\n",
              " 'Negotiator, The (1998)',\n",
              " 'Following (1998)',\n",
              " '21 Grams (2003)',\n",
              " 'Inception (2010)',\n",
              " 'Insomnia (2002)']"
            ]
          },
          "metadata": {},
          "execution_count": 44
        }
      ],
      "source": [
        "recommendations('Usual Suspects, The (1995)', similar_movies)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "f7c9ab2b",
      "metadata": {
        "id": "f7c9ab2b"
      },
      "source": [
        "- The movie belongs to **Crime, Mystery, and Thriller** genres, and the **majority of our recommendations** lie in one or more of these genres. It implies that the resulting recommendation system is working well."
      ]
    },
    {
      "cell_type": "markdown",
      "id": "477c2a47",
      "metadata": {
        "id": "477c2a47"
      },
      "source": [
        "## **Conclusion**"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "ccf917ef",
      "metadata": {
        "id": "ccf917ef"
      },
      "source": [
        "- In this case study, we built recommendation systems using five different algorithms. They are as follows:\n",
        "\n",
        "  - **clustering-based recommendation systems**\n",
        "  - **content-based recommendation systems**\n",
        "- To demonstrate clustering-based recommendation systems, **surprise** library has been used. Grid search cross-validation is applied to find the best working model, and with that the corresponding predictions are made.\n",
        "- To evaluate the performance of these models, **precision@k and recall@k** are used in this case study. Using these two metrics F_1 score is calculated for each working model.\n",
        "- We can try to improve the performance of these models using hyperparameter tuning."
      ]
    }
  ],
  "metadata": {
    "colab": {
      "provenance": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "display_name": "Python 3 (ipykernel)",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.11.7"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 5
}