COVID_Mutated.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# COVID mutation\n",
    "### Supporting research: Covid19 and beyond for vaccine candidates"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "jxPMeugQ0okg"
   },
   "source": [
    "## Step 1. Set up your environment.\n",
    "\n",
    "AI Platform Pipelines will prepare a development environment to build a pipeline, and a Kubeflow Pipeline cluster to run the newly built pipeline.\n",
    "\n",
    "**NOTE:** To select a particular TensorFlow version, or select a GPU instance, create a TensorFlow pre-installed instance in AI Platform Notebooks.\n",
    "\n",
    "**NOTE:** There might be some errors during package installation. For example: \n",
    "\n",
    ">\"ERROR: some-package 0.some_version.1 has requirement other-package!=2.0.,&lt;3,&gt;=1.15, but you'll have other-package 2.0.0 which is incompatible.\" Please ignore these errors at this moment.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "-am1yWXt0okh"
   },
   "source": [
    "Install `tfx`, `kfp`, and `skaffold`, and add installation path to the `PATH` environment variable."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "XNiqq_kN0okj"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[31mERROR: google-cloud-storage 1.26.0 has requirement google-resumable-media<0.6dev,>=0.5.0, but you'll have google-resumable-media 0.4.1 which is incompatible.\u001b[0m\n",
      "\u001b[31mERROR: google-cloud-bigquery 1.17.1 has requirement google-resumable-media<0.5.0dev,>=0.3.1, but you'll have google-resumable-media 0.5.0 which is incompatible.\u001b[0m\n",
      "  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n",
      "                                 Dload  Upload   Total   Spent    Left  Speed\n",
      "100 36.7M  100 36.7M    0     0  45.5M      0 --:--:-- --:--:-- --:--:-- 45.5M\n"
     ]
    }
   ],
   "source": [
    "# Install tfx and kfp Python packages.\n",
    "!pip install --user --upgrade -q tfx==0.21.2\n",
    "!pip install --user --upgrade -q kfp==0.2.5\n",
    "# Download skaffold and set it executable.\n",
    "!curl -Lo skaffold https://storage.googleapis.com/skaffold/releases/latest/skaffold-linux-amd64 && chmod +x skaffold && mv skaffold /home/jupyter/.local/bin/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!python3 install.py --sys-prefix"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "dependency-licenses  ijava-1.3.0.jar  kernel.json\n"
     ]
    }
   ],
   "source": [
    "!ls /opt/conda/share/jupyter/kernels/java "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/bin/sh: 1: java: not found\n"
     ]
    }
   ],
   "source": [
    "!java -version"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!jupyter console --kernel=java"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Set `PATH` to include user python binary directory and a directory containing `skaffold`.\n",
    "PATH=%env PATH\n",
    "%env PATH={PATH}:/home/jupyter/.local/bin"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "43ncix2Q0okm"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "env: PATH=/usr/local/cuda/bin:/opt/conda/bin:/opt/conda/condabin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games:/opt/conda/share/jupyter/kernels/java:/java/jre1.8.0_251/bin:/home/jupyter/java/jre1.8.0_251/bin\n"
     ]
    }
   ],
   "source": [
    "# Set `PATH` to include user python binary directory and a directory containing `skaffold`.\n",
    "PATH=%env PATH\n",
    "%env PATH={PATH}:/home/jupyter/java/jre1.8.0_251/bin"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "java -version"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "hX1rqpbQ0okp"
   },
   "source": [
    "Let's check the versions of TFX."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "XAIoKMNG0okq"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting biopython\n",
      "  Downloading biopython-1.76-cp37-cp37m-manylinux1_x86_64.whl (2.3 MB)\n",
      "\u001b[K     |████████████████████████████████| 2.3 MB 4.7 MB/s eta 0:00:01\n",
      "\u001b[?25hRequirement already satisfied: numpy in /opt/conda/lib/python3.7/site-packages (from biopython) (1.18.1)\n",
      "Installing collected packages: biopython\n",
      "Successfully installed biopython-1.76\n"
     ]
    }
   ],
   "source": [
    "!pip install biopython"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting package metadata (current_repodata.json): done\n",
      "Solving environment: failed with initial frozen solve. Retrying with flexible solve.\n",
      "Collecting package metadata (repodata.json): done\n",
      "Solving environment: failed with initial frozen solve. Retrying with flexible solve.\n",
      "\n",
      "PackagesNotFoundError: The following packages are not available from current channels:\n",
      "\n",
      "  - y\n",
      "\n",
      "Current channels:\n",
      "\n",
      "  - https://conda.anaconda.org/omnia/linux-64\n",
      "  - https://conda.anaconda.org/omnia/noarch\n",
      "  - https://conda.anaconda.org/conda-forge/linux-64\n",
      "  - https://conda.anaconda.org/conda-forge/noarch\n",
      "  - https://repo.anaconda.com/pkgs/main/linux-64\n",
      "  - https://repo.anaconda.com/pkgs/main/noarch\n",
      "  - https://repo.anaconda.com/pkgs/r/linux-64\n",
      "  - https://repo.anaconda.com/pkgs/r/noarch\n",
      "\n",
      "To search for alternate channels that may provide the conda package you're\n",
      "looking for, navigate to\n",
      "\n",
      "    https://anaconda.org\n",
      "\n",
      "and use the search bar at the top of the page.\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "!conda install -c omnia openmm y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Copying file://AIHub.tar.gz [Content-Type=application/x-tar]...\n",
      "| [1 files][134.4 MiB/134.4 MiB]                                                \n",
      "Operation completed over 1 objects/134.4 MiB.                                    \n"
     ]
    }
   ],
   "source": [
    "!gsutil cp AIHub.tar.gz gs://virus_mutations/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "COVID19/corona/download_sequences.py:9: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.\n",
      "  seqs = yaml.load(requests.get(\"https://www.ncbi.nlm.nih.gov/core/assets/genbank/files/ncov-sequences.yaml\").text)\n",
      "Traceback (most recent call last):\n",
      "  File \"COVID19/corona/download_sequences.py\", line 9, in <module>\n",
      "    seqs = yaml.load(requests.get(\"https://www.ncbi.nlm.nih.gov/core/assets/genbank/files/ncov-sequences.yaml\").text)\n",
      "  File \"/home/jupyter/.local/lib/python3.7/site-packages/yaml/__init__.py\", line 114, in load\n",
      "    return loader.get_single_data()\n",
      "  File \"/home/jupyter/.local/lib/python3.7/site-packages/yaml/constructor.py\", line 49, in get_single_data\n",
      "    node = self.get_single_node()\n",
      "  File \"/home/jupyter/.local/lib/python3.7/site-packages/yaml/composer.py\", line 36, in get_single_node\n",
      "    document = self.compose_document()\n",
      "  File \"/home/jupyter/.local/lib/python3.7/site-packages/yaml/composer.py\", line 58, in compose_document\n",
      "    self.get_event()\n",
      "  File \"/home/jupyter/.local/lib/python3.7/site-packages/yaml/parser.py\", line 118, in get_event\n",
      "    self.current_event = self.state()\n",
      "  File \"/home/jupyter/.local/lib/python3.7/site-packages/yaml/parser.py\", line 193, in parse_document_end\n",
      "    token = self.peek_token()\n",
      "  File \"/home/jupyter/.local/lib/python3.7/site-packages/yaml/scanner.py\", line 129, in peek_token\n",
      "    self.fetch_more_tokens()\n",
      "  File \"/home/jupyter/.local/lib/python3.7/site-packages/yaml/scanner.py\", line 223, in fetch_more_tokens\n",
      "    return self.fetch_value()\n",
      "  File \"/home/jupyter/.local/lib/python3.7/site-packages/yaml/scanner.py\", line 579, in fetch_value\n",
      "    self.get_mark())\n",
      "yaml.scanner.ScannerError: mapping values are not allowed here\n",
      "  in \"<unicode string>\", line 14, column 10:\n",
      "    min-width: 950px;\n",
      "             ^\n"
     ]
    }
   ],
   "source": [
    "!python3 COVID19/corona/download_sequences.py"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Spike Glycoprotein MFVFLVLLPLVSSQCVNLTTRTQLPPAYTNSFTRGVYYPDKVFRSSVLHSTQDLFLPFFSNVTWFHAIHVSGTNGTKRFDNPVLPFNDGVYFASTEKSNIIRGWIFGTTLDSKTQSLLIVNNATNVVIKVCEFQFCNDPFLGVYYHKNNKSWMESEFRVYSSANNCTFEYVSQPFLMDLEGKQGNFKNLREFVFKNIDGYFKIYSKHTPINLVRDLPQGFSALEPLVDLPIGINITRFQTLLALHRSYLTPGDSSSGWTAGAAAYYVGYLQPRTFLLKYNENGTITDAVDCALDPLSETKCTLKSFTVEKGIYQTSNFRVQPTESIVRFPNITNLCPFGEVFNATRFASVYAWNRKRISNCVADYSVLYNSASFSTFKCYGVSPTKLNDLCFTNVYADSFVIRGDEVRQIAPGQTGKIADYNYKLPDDFTGCVIAWNSNNLDSKVGGNYNYLYRLFRKSNLKPFERDISTEIYQAGSTPCNGVEGFNCYFPLQSYGFQPTNGVGYQPYRVVVLSFELLHAPATVCGPKKSTNLVKNKCVNFNFNGLTGTGVLTESNKKFLPFQQFGRDIADTTDAVRDPQTLEILDITPCSFGGVSVITPGTNTSNQVAVLYQDVNCTEVPVAIHADQLTPTWRVYSTGSNVFQTRAGCLIGAEHVNNSYECDIPIGAGICASYQTQTNSPRRARSVASQSIIAYTMSLGAENSVAYSNNSIAIPTNFTISVTTEILPVSMTKTSVDCTMYICGDSTECSNLLLQYGSFCTQLNRALTGIAVEQDKNTQEVFAQVKQIYKTPPIKDFGGFNFSQILPDPSKPSKRSFIEDLLFNKVTLADAGFIKQYGDCLGDIAARDLICAQKFNGLTVLPPLLTDEMIAQYTSALLAGTITSGWTFGAGAALQIPFAMQMAYRFNGIGVTQNVLYENQKLIANQFNSAIGKIQDSLSSTASALGKLQDVVNQNAQALNTLVKQLSSNFGAISSVLNDILSRLDKVEAEVQIDRLITGRLQSLQTYVTQQLIRAAEIRASANLAATKMSECVLGQSKRVDFCGKGYHLMSFPQSAPHGVVFLHVTYVPAQEKNFTTAPAICHDGKAHFPREGVFVSNGTHWFVTQRNFYEPQIITTDNTFVSGNCDVVIGIVNNTVYDPLQPELDSFKEELDKYFKNHTSPDVDLGDISGINASVVNIQKEIDRLNEVAKNLNESLIDLQELGKYEQYIKWPWYIWLGFIAGLIAIVMVTIMLCCMTSCCSCLKGCCSCGSCCKFDEDDSEPVLKGVKLHYT\n",
      "+++++++++++++++++++++++++++++++++++++\n",
      "                                     \n",
      "+++++++++++++++++++++++++++++++++++++\n",
      "Spike Glycoprotein peptides MFVFLVLL\n",
      "Spike Glycoprotein peptides FVFLVLLP\n",
      "Spike Glycoprotein peptides VFLVLLPL\n",
      "Spike Glycoprotein peptides FLVLLPLV\n",
      "Spike Glycoprotein peptides LVLLPLVS\n",
      "Spike Glycoprotein peptides VLLPLVSS\n",
      "........\n",
      "+++++++++++++++++++++++++++++++++++++\n",
      "                                     \n",
      "+++++++++++++++++++++++++++++++++++++\n",
      "Translated {'untranslated_region': 'ATTAAAGGTTTATACCTTCCCAGGTAACAAACCAACCAACTTTCGATCTCTTGTAGATCTGTTCTCTAAACGAACTTTAAAATCTGTGTGGCTGTCACTCGGCTGCATGCTTAGTGCACTCACGCAGTATAATTAATAACTAATTACTGTCGTTGACAGGACACGAGTAACTCGTCTATCTTCTGCAGGCTGCTTACGGTTTCGTCCGTGTTGCAGCCGATCATCAGCACATCTAGGTTTCGTCCGGGTGTGACCGAAAGGTAAG', 'orf1a': 'MESLVPGFNEKTHVQLSLPVLQVRDVLVRGFGDSVEEVLSEARQHLKDGTCGLVEVEKGVLPQLEQPYVFIKRSDARTAPHGHVMVELVAELEGIQYGRSGETLGVLVPHVGEIPVAYRKVLLRKNGNKGAGGHSYGADLKSFDLGDELGTDPYEDFQENWNTKHSSGVTRELMRELNGGAYTRYVDNNFCGPDGYPLECIKDLLARAGKASCTLSEQLDFIDTKRGVYCCREHEHEIAWYTERSEKSYELQTPFEIKLAKKFDTFNGECPNFVFPLNSIIKTIQPRVEKKKLDGFMGRIRSVYPVASPNECNQMCLSTLMKCDHCGETSWQTGDFVKATCEFCGTENLTKEGATTCGYLPQNAVVKIYCPACHNSEVGPEHSLAEYHNESGLKTILRKGGRTIAFGGCVFSYVGCHNKCAYWVPRASANIGCNHTGVVGEGSEGLNDNLLEILQKEKVNINIVGDFKLNEEIAIILASFSASTSAFVETVKGLDYKAFKQIVESCGNFKVTKGKAKKGAWNIGEQKSILSPLYAFASEAARVVRSIFSRTLETAQNSVRVLQKAAITILDGISQYSLRLIDAMMFTSDLATNNLVVMAYITGGVVQLTSQWLTNIFGTVYEKLKPVLDWLEEKFKEGVEFLRDGWEIVKFISTCACEIVGGQIVTCAKEIKESVQTFFKLVNKFLALCADSIIIGGAKLKALNLGETFVTHSKGLYRKCVKSREETGLLMPLKAPKEIIFLEGETLPTEVLTEEVVLKTGDLQPLEQPTSEAVEAPLVGTPVCINGLMLLEIKDTEKYCALAPNMMVTNNTFTLKGGAPTKVTFGDDTVIEVQGYKSVNITFELDERIDKVLNEKCSAYTVELGTEVNEFACVVADAVIKTLQPVSELLTPLGIDLDEWSMATYYLFDESGEFKLASHMYCSFYPPDEDEEEGDCEEEEFEPSTQYEYGTEDDYQGKPLEFGATSAALQPEEEQEEDWLDDDSQQTVGQQDGSEDNQTTTIQTIVEVQPQLEMELTPVVQTIEVNSFSGYLKLTDNVYIKNADIVEEAKKVKPTVVVNAANVYLKHGGGVAGALNKATNNAMQVESDDYIATNGPLKVGGSCVLSGHNLAKHCLHVVGPNVNKGEDIQLLKSAYENFNQHEVLLAPLLSAGIFGADPIHSLRVCVDTVRTNVYLAVFDKNLYDKLVSSFLEMKSEKQVEQKIAEIPKEEVKPFITESKPSVEQRKQDDKKIKACVEEVTTTLEETKFLTENLLLYIDINGNLHPDSATLVSDIDITFLKKDAPYIVGDVVQEGVLTAVVIPTKKAGGTTEMLAKALRKVPTDNYITTYPGQGLNGYTVEEAKTVLKKCKSAFYILPSIISNEKQEILGTVSWNLREMLAHAEETRKLMPVCVETKAIVSTIQRKYKGIKIQEGVVDYGARFYFYTSKTTVASLINTLNDLNETLVTMPLGYVTHGLNLEEAARYMRSLKVPATVSVSSPDAVTAYNGYLTSSSKTPEEHFIETISLAGSYKDWSYSGQSTQLGIEFLKRGDKSVYYTSNPTTFHLDGEVITFDNLKTLLSLREVRTIKVFTTVDNINLHTQVVDMSMTYGQQFGPTYLDGADVTKIKPHNSHEGKTFYVLPNDDTLRVEAFEYYHTTDPSFLGRYMSALNHTKKWKYPQVNGLTSIKWADNNCYLATALLTLQQIELKFNPPALQDAYYRARAGEAANFCALILAYCNKTVGELGDVRETMSYLFQHANLDSCKRVLNVVCKTCGQQQTTLKGVEAVMYMGTLSYEQFKKGVQIPCTCGKQATKYLVQQESPFVMMSAPPAQYELKHGTFTCASEYTGNYQCGHYKHITSKETLYCIDGALLTKSSEYKGPITDVFYKENSYTTTIKPVTYKLDGVVCTEIDPKLDNYYKKDNSYFTEQPIDLVPNQPYPNASFDNFKFVCDNIKFADDLNQLTGYKKPASRELKVTFFPDLNGDVVAIDYKHYTPSFKKGAKLLHKPIVWHVNNATNKATYKPNTWCIRCLWSTKPVETSNSFDVLKSEDAQGMDNLACEDLKPVSEEVVENPTIQKDVLECNVKTTEVVGDIILKPANNSLKITEEVGHTDLMAAYVDNSSLTIKKPNELSRVLGLKTLATHGLAAVNSVPWDTIANYAKPFLNKVVSTTTNIVTRCLNRVCTNYMPYFFTLLLQLCTFTRSTNSRIKASMPTTIAKNTVKSVGKFCLEASFNYLKSPNFSKLINIIIWFLLLSVCLGSLIYSTAALGVLMSNLGMPSYCTGYREGYLNSTNVTIATYCTGSIPCSVCLSGLDSLDTYPSLETIQITISSFKWDLTAFGLVAEWFLAYILFTRFFYVLGLAAIMQLFFSYFAVHFISNSWLMWLIINLVQMAPISAMVRMYIFFASFYYVWKSYVHVVDGCNSSTCMMCYKRNRATRVECTTIVNGVRRSFYVYANGGKGFCKLHNWNCVNCDTFCAGSTFISDEVARDLSLQFKRPINPTDQSSYIVDSVTVKNGSIHLYFDKAGQKTYERHSLSHFVNLDNLRANNTKGSLPINVIVFDGKSKCEESSAKSASVYYSQLMCQPILLLDQALVSDVGDSAEVAVKMFDAYVNTFSSTFNVPMEKLKTLVATAEAELAKNVSLDNVLSTFISAARQGFVDSDVETKDVVECLKLSHQSDIEVTGDSCNNYMLTYNKVENMTPRDLGACIDCSARHINAQVAKSHNIALIWNVKDFMSLSEQLRKQIRSAAKKNNLPFKLTCATTRQVVNVVTTKIALKGGKIVNNWLKQLIKVTLVFLFVAAIFYLITPVHVMSKHTDFSSEIIGYKAIDGGVTRDIASTDTCFANKHADFDTWFSQRGGSYTNDKACPLIAAVITREVGFVVPGLPGTILRTTNGDFLHFLPRVFSAVGNICYTPSKLIEYTDFATSACVLAAECTIFKDASGKPVPYCYDTNVLEGSVAYESLRPDTRYVLMDGSIIQFPNTYLEGSVRVVTTFDSEYCRHGTCERSEAGVCVSTSGRWVLNNDYYRSLPGVFCGVDAVNLLTNMFTPLIQPIGALDISASIVAGGIVAIVVTCLAYYFMRFRRAFGEYSHVVAFNTLLFLMSFTVLCLTPVYSFLPGVYSVIYLYLTFYLTNDVSFLAHIQWMVMFTPLVPFWITIAYIICISTKHFYWFFSNYLKRRVVFNGVSFSTFEEAALCTFLLNKEMYLKLRSDVLLPLTQYNRYLALYNKYKYFSGAMDTTSYREAACCHLAKALNDFSNSGSDVLYQPPQTSITSAVLQSGFRKMAFPSGKVEGCMVQVTCGTTTLNGLWLDDVVYCPRHVICTSEDMLNPNYEDLLIRKSNHNFLVQAGNVQLRVIGHSMQNCVLKLKVDTANPKTPKYKFVRIQPGQTFSVLACYNGSPSGVYQCAMRPNFTIKGSFLNGSCGSVGFNIDYDCVSFCYMHHMELPTGVHAGTDLEGNFYGPFVDRQTAQAAGTDTTITVNVLAWLYAAVINGDRWFLNRFTTTLNDFNLVAMKYNYEPLTQDHVDILGPLSAQTGIAVLDMCASLKELLQNGMNGRTILGSALLEDEFTPFDVVRQCSGVTFQSAVKRTIKGTHHWLLLTILTSLLVLVQSTQWSLFFFLYENAFLPFAMGIIAMSAFAMMFVKHKHAFLCLFLLPSLATVAYFNMVYMPASWVMRIMTWLDMVDTSLSGFKLKDCVMYASAVVLLILMTARTVYDDGARRVWTLMNVLTLVYKVYYGNALDQAISMWALIISVTSNYSGVVTTVMFLARGIVFMCVEYCPIFFITGNTLQCIMLVYCFLGYFCTCYFGLFCLLNRYFRLTLGVYDYLVSTQEFRYMNSQGLLPPKNSIDAFKLNIKLLGVGGKPCIKVATVQSKMSDVKCTSVVLLSVLQQLRVESSSKLWAQCVQLHNDILLAKDTTEAFEKMVSLLSVLLSMQGAVDINKLCEEMLDNRATLQAIASEFSSLPSYAAFATAQEAYEQAVANGDSEVVLKKLKKSLNVAKSEFDRDAAMQRKLEKMADQAMTQMYKQARSEDKRAKVTSAMQTMLFTMLRKLDNDALNNIINNARDGCVPLNIIPLTTAAKLMVVIPDYNTYKNTCDGTTFTYASALWEIQQVVDADSKIVQLSEISMDNSPNLAWPLIVTALRANSAVKLQNNELSPVALRQMSCAAGTTQTACTDDNALAYYNTTKGGRFVLALLSDLQDLKWARFPKSDGTGTIYTELEPPCRFVTDTPKGPKVKYLYFIKGLNNLNRGMVLGSLAATVRLQAGNATEVPANSTVLSFCAFAVDAAKAYKDYLASGGQPITNCVKMLCTHTGTGQAITVTPEANMDQESFGGASCCLYCRCHIDHPNPKGFCDLKGKYVQIPTTCANDPVGFTLKNTVCTVCGMWKGYGCSCDQLREPMLQSADAQSFLNGFAV', 'orf1b': 'RVCGVSAARLTPCGTGTSTDVVYRAFDIYNDKVAGFAKFLKTNCCRFQEKDEDDNLIDSYFVVKRHTFSNYQHEETIYNLLKDCPAVAKHDFFKFRIDGDMVPHISRQRLTKYTMADLVYALRHFDEGNCDTLKEILVTYNCCDDDYFNKKDWYDFVENPDILRVYANLGERVRQALLKTVQFCDAMRNAGIVGVLTLDNQDLNGNWYDFGDFIQTTPGSGVPVVDSYYSLLMPILTLTRALTAESHVDTDLTKPYIKWDLLKYDFTEERLKLFDRYFKYWDQTYHPNCVNCLDDRCILHCANFNVLFSTVFPPTSFGPLVRKIFVDGVPFVVSTGYHFRELGVVHNQDVNLHSSRLSFKELLVYAADPAMHAASGNLLLDKRTTCFSVAALTNNVAFQTVKPGNFNKDFYDFAVSKGFFKEGSSVELKHFFFAQDGNAAISDYDYYRYNLPTMCDIRQLLFVVEVVDKYFDCYDGGCINANQVIVNNLDKSAGFPFNKWGKARLYYDSMSYEDQDALFAYTKRNVIPTITQMNLKYAISAKNRARTVAGVSICSTMTNRQFHQKLLKSIAATRGATVVIGTSKFYGGWHNMLKTVYSDVENPHLMGWDYPKCDRAMPNMLRIMASLVLARKHTTCCSLSHRFYRLANECAQVLSEMVMCGGSLYVKPGGTSSGDATTAYANSVFNICQAVTANVNALLSTDGNKIADKYVRNLQHRLYECLYRNRDVDTDFVNEFYAYLRKHFSMMILSDDAVVCFNSTYASQGLVASIKNFKSVLYYQNNVFMSEAKCWTETDLTKGPHEFCSQHTMLVKQGDDYVYLPYPDPSRILGAGCFVDDIVKTDGTLMIERFVSLAIDAYPLTKHPNQEYADVFHLYLQYIRKLHDELTGHMLDMYSVMLTNDNTSRYWEPEFYEAMYTPHTVLQAVGACVLCNSQTSLRCGACIRRPFLCCKCCYDHVISTSHKLVLSVNPYVCNAPGCDVTDVTQLYLGGMSYYCKSHKPPISFPLCANGQVFGLYKNTCVGSDNVTDFNAIATCDWTNAGDYILANTCTERLKLFAAETLKATEETFKLSYGIATVREVLSDRELHLSWEVGKPRPPLNRNYVFTGYRVTKNSKVQIGEYTFEKGDYGDAVVYRGTTTYKLNVGDYFVLTSHTVMPLSAPTLVPQEHYVRITGLYPTLNISDEFSSNVANYQKVGMQKYSTLQGPPGTGKSHFAIGLALYYPSARIVYTACSHAAVDALCEKALKYLPIDKCSRIIPARARVECFDKFKVNSTLEQYVFCTVNALPETTADIVVFDEISMATNYDLSVVNARLRAKHYVYIGDPAQLPAPRTLLTKGTLEPEYFNSVCRLMKTIGPDMFLGTCRRCPAEIVDTVSALVYDNKLKAHKDKSAQCFKMFYKGVITHDVSSAINRPQIGVVREFLTRNPAWRKAVFISPYNSQNAVASKILGLPTQTVDSSQGSEYDYVIFTQTTETAHSCNVNRFNVAITRAKVGILCIMSDRDLYDKLQFTSLEIPRRNVATLQAENVTGLFKDCSKVITGLHPTQAPTHLSVDTKFKTEGLCVDIPGIPKDMTYRRLISMMGFKMNYQVNGYPNMFITREEAIRHVRAWIGFDVEGCHATREAVGTNLPLQLGFSTGVNLVAVPTGYVDTPNNTDFSRVSAKPPPGDQFKHLIPLMYKGLPWNVVRIKIVQMLSDTLKNLSDRVVFVLWAHGFELTSMKYFVKIGPERTCCLCDRRATCFSTASDTYACWHHSIGFDYVYNPFMIDVQQWGFTGNLQSNHDLYCQVHGNAHVASCDAIMTRCLAVHECFVKRVDWTIEYPIIGDELKINAACRKVQHMVVKAALLADKFPVLHDIGNPKAIKCVPQADVEWKFYDAQPCSDKAYKIEELFYSYATHSDKFTDGVCLFWNCNVDRYPANSIVCRFDTRVLSNLNLPGCDGGSLYVNKHAFHTPAFDKSAFVNLKQLPFFYYSDSPCESHGKQVVSDIDYVPLKSATCITRCNLGGAVCRHHANEYRLYLDAYNMMISAGFSLWVYKQFDTYNLWNTFTRLQSLENVAFNVVNKGHFDGQQGEVPVSIINNTVYTKVDGVDVELFENKTTLPVNVAFELWAKRNIKPVPEVKILNNLGVDIAANTVIWDYKRDAPAHISTIGVCSMTDIAKKPTETICAPLTVFFDGRVDGQVDLFRNARNGVLITEGSVKGLQPSVGPKQASLNGVTLIGEAVKTQFNYYKKVDGVVQQLPETYFTQSRNLQEFKPRSQMEIDFLELAMDEFIERYKLEGYAFEHIVYGDFSHSQLGGLHLLIGLAKRFKESPFELEDFIPMDSTVKNYFITDAQTGSSKCVCSVIDLLLDDFVEIIKSQDLSVVSKVVKVTIDYTEISFMLWCKDGHVETFYPKLQSSQAWQPGVAMPNLYKMQRMLLEKCDLQNYGDSATLPKGIMMNVAKYTQLCQYLNTLTLAVPYNMRVIHFGAGSDKGVAPGTAVLRQWLPTGTLLVDSDLNDFVSDADSTLIGDCATVHTANKWDLIISDMYDPKTKNVTKENDSKEGFFTYICGFIQQKLALGGSVAIKITEHSWNADLYKLMGHFAWWTAFVTNVNASSSEAFLIGCNYLGKPREQIDGYVMHANYIFWRNTNPIQLSSYSLFDMSKFPLKLRGTAVMSLKEGQINDMILSLLSKGRLIIRENNRVVISSDVLVNN', 'spike_glycoprotein': 'MFVFLVLLPLVSSQCVNLTTRTQLPPAYTNSFTRGVYYPDKVFRSSVLHSTQDLFLPFFSNVTWFHAIHVSGTNGTKRFDNPVLPFNDGVYFASTEKSNIIRGWIFGTTLDSKTQSLLIVNNATNVVIKVCEFQFCNDPFLGVYYHKNNKSWMESEFRVYSSANNCTFEYVSQPFLMDLEGKQGNFKNLREFVFKNIDGYFKIYSKHTPINLVRDLPQGFSALEPLVDLPIGINITRFQTLLALHRSYLTPGDSSSGWTAGAAAYYVGYLQPRTFLLKYNENGTITDAVDCALDPLSETKCTLKSFTVEKGIYQTSNFRVQPTESIVRFPNITNLCPFGEVFNATRFASVYAWNRKRISNCVADYSVLYNSASFSTFKCYGVSPTKLNDLCFTNVYADSFVIRGDEVRQIAPGQTGKIADYNYKLPDDFTGCVIAWNSNNLDSKVGGNYNYLYRLFRKSNLKPFERDISTEIYQAGSTPCNGVEGFNCYFPLQSYGFQPTNGVGYQPYRVVVLSFELLHAPATVCGPKKSTNLVKNKCVNFNFNGLTGTGVLTESNKKFLPFQQFGRDIADTTDAVRDPQTLEILDITPCSFGGVSVITPGTNTSNQVAVLYQDVNCTEVPVAIHADQLTPTWRVYSTGSNVFQTRAGCLIGAEHVNNSYECDIPIGAGICASYQTQTNSPRRARSVASQSIIAYTMSLGAENSVAYSNNSIAIPTNFTISVTTEILPVSMTKTSVDCTMYICGDSTECSNLLLQYGSFCTQLNRALTGIAVEQDKNTQEVFAQVKQIYKTPPIKDFGGFNFSQILPDPSKPSKRSFIEDLLFNKVTLADAGFIKQYGDCLGDIAARDLICAQKFNGLTVLPPLLTDEMIAQYTSALLAGTITSGWTFGAGAALQIPFAMQMAYRFNGIGVTQNVLYENQKLIANQFNSAIGKIQDSLSSTASALGKLQDVVNQNAQALNTLVKQLSSNFGAISSVLNDILSRLDKVEAEVQIDRLITGRLQSLQTYVTQQLIRAAEIRASANLAATKMSECVLGQSKRVDFCGKGYHLMSFPQSAPHGVVFLHVTYVPAQEKNFTTAPAICHDGKAHFPREGVFVSNGTHWFVTQRNFYEPQIITTDNTFVSGNCDVVIGIVNNTVYDPLQPELDSFKEELDKYFKNHTSPDVDLGDISGINASVVNIQKEIDRLNEVAKNLNESLIDLQELGKYEQYIKWPWYIWLGFIAGLIAIVMVTIMLCCMTSCCSCLKGCCSCGSCCKFDEDDSEPVLKGVKLHYT', 'orf3a': 'MDLFMRIFTIGTVTLKQGEIKDATPSDFVRATATIPIQASLPFGWLIVGVALLAVFQSASKIITLKKRWQLALSKGVHFVCNLLLLFVTVYSHLLLVAAGLEAPFLYLYALVYFLQSINFVRIIMRLWLCWKCRSKNPLLYDANYFLCWHTNCYDYCIPYNSVTSSIVITSGDGTTSPISEHDYQIGGYTEKWESGVKDCVVLHSYFTSDYYQLYSTQLSTDTGVEHVTFFIYNKIVDEPEEHVQIHTIDGSSGVVNPVMEPIYDEPTTTTSVPL', 'envelope_protein': 'MYSFVSEETGTLIVNSVLLFLAFVVFLLVTLAILTALRLCAYCCNIVNVSLVKPSFYVYSRVKNLNSSRVPDLLV', 'membrane_glycoprotein': 'MADSNGTITVEELKKLLEQWNLVIGFLFLTWICLLQFAYANRNRFLYIIKLIFLWLLWPVTLACFVLAAVYRINWITGGIAIAMACLVGLMWLSYFIASFRLFARTRSMWSFNPETNILLNVPLHGTILTRPLLESELVIGAVILRGHLRIAGHHLGRCDIKDLPKEITVATSRTLSYYKLGASQRVAGDSGFAAYSRYRIGNYKLNTDHSSSSDNIALLVQ', 'orf6': 'MFHLVDFQVTIAEILLIIMRTFKVSIWNLDYIINLIIKNLSKSLTENKYSQLDEEQPMEID', 'orf7a': 'MKIILFLALITLATCELYHYQECVRGTTVLLKEPCSSGTYEGNSPFHPLADNKFALTCFSTQFAFACPDGVKHVYQLRARSVSPKLFIRQEEVQELYSPIFLIVAAIVFITLCFTLKRKTE', 'orf7b': 'MIELSLIDFYLCFLAFLLFLVLIMLIIFWFSLELQDHNETCHA', 'orf8': 'MKFLVFLGIITTVAAFHQECSLQSCTQHQPYVVDDPCPIHFYSKWYIRVGARKSAPLIELCVDEAGSKSPIQYIDIGNYTVSCLPFTINCQEPKLGSLVVRCSFYEDFLEYHDVRVVLDFI', 'nucleocapsid_phosphoprotein': 'MSDNGPQNQRNAPRITFGGPSDSTGSNQNGERSGARSKQRRPQGLPNNTASWFTALTQHGKEDLKFPRGQGVPINTNSSPDDQIGYYRRATRRIRGGDGKMKDLSPRWYFYYLGTGPEAGLPYGANKDGIIWVATEGALNTPKDHIGTRNPANNAAIVLQLPQGTTLPKGFYAEGSRGGSQASSRSSSRSRNSSRNSTPGSSRGTSPARMAGNGGDAALALLLLDRLNQLESKMSGKGQQQQGQTVTKKSAAEASKKPRQKRTATKAYNVTQAFGRRGPEQTQGNFGDQELIRQGTDYKHWPQIAQFAPSASAFFGMSRIGMEVTPSGTWLTYTGAIKLDDKDPNFKDQVILLNKHIDAYKTFPPTEPKKDKKKKADETQALPQRQKKQQTVTLLPAADLDDFSKQLQQSMSSADSTQA', 'orf10': 'MGYINVFAFPFTIYSLLLCRMNSRNYIAQVDVVNFNLT'}\n"
     ]
    }
   ],
   "source": [
    "!python3 COVID19/corona/corona.py"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/home/jupyter/AIHub\n"
     ]
    }
   ],
   "source": [
    "cd ../.."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "python3: can't open file 'COVID19/corona/corona.py': [Errno 2] No such file or directory\n"
     ]
    }
   ],
   "source": [
    "!python3 COVID19/corona/corona.py"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Traceback (most recent call last):\n",
      "  File \"COVID19/corona/fold.py\", line 2, in <module>\n",
      "    from simtk.openmm.app import *\n",
      "ModuleNotFoundError: No module named 'simtk'\n"
     ]
    }
   ],
   "source": [
    "!python3 COVID19/corona/fold.py"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "_7JLpaXT0okv"
   },
   "source": [
    "In AI Platform Pipelines, TFX is running in a hosted Kubernetes environment using [Kubeflow Pipelines](https://www.kubeflow.org/docs/pipelines/overview/pipelines-overview/).\n",
    "\n",
    "Let's set some environment variables to use Kubeflow Pipelines.\n",
    "\n",
    "First, get your GCP project ID."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'/home/jupyter/AIHub'"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pwd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Hw3nsooU0okv"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "GCP project ID:covid-19-271622\n"
     ]
    }
   ],
   "source": [
    "# Read GCP project id from env.\n",
    "shell_output=!gcloud config list --format 'value(core.project)' 2>/dev/null\n",
    "GCP_PROJECT_ID=shell_output[0]\n",
    "print(\"GCP project ID:\" + GCP_PROJECT_ID)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "A_6r4uzE0oky"
   },
   "source": [
    "We also need to access your KFP cluster. You can access it in your Google Cloud Console under \"AI Platform > Pipeline\" menu. The \"endpoint\" of the KFP cluster can be found from the URL of the Pipelines dashboard, or you can get it from the URL of the Getting Started page where you launched this notebook. Let's create an `ENDPOINT` environment variable and set it to the KFP cluster endpoint. **ENDPOINT should contain only the hostname part of the URL.** For example, if the URL of the KFP dashboard is `https://1e9deb537390ca22-dot-asia-east1.pipelines.googleusercontent.com/#/start`, ENDPOINT value becomes `1e9deb537390ca22-dot-asia-east1.pipelines.googleusercontent.com`.\n",
    "\n",
    ">**NOTE: You MUST set your ENDPOINT value below.**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Test Test"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "AzqEQORV0oky"
   },
   "outputs": [],
   "source": [
    "# This refers to the KFP cluster endpoint\n",
    "ENDPOINT='87808c6f40125b0-dot-us-central2.pipelines.googleusercontent.com' # Enter your ENDPOINT here.\n",
    "if not ENDPOINT:\n",
    "    from absl import logging\n",
    "    logging.error('Set your ENDPOINT in this cell.')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "K6T-KXeA0ok3"
   },
   "source": [
    "Set the image name as `tfx-pipeline` under the current GCP project."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "3ztxXOVD0ok4"
   },
   "outputs": [],
   "source": [
    "# Docker image name for the pipeline image \n",
    "CUSTOM_TFX_IMAGE='gcr.io/' + GCP_PROJECT_ID + '/tfx-pipeline'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "TOsQbkky0ok7"
   },
   "source": [
    "And, it's done. We are ready to create a pipeline."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Cxlbi1QM0ok8"
   },
   "source": [
    "## Step 2. Copy the predefined template to your project directory.\n",
    "\n",
    "In this step, we will create a working pipeline project directory and files by copying additional files from a predefined template.\n",
    "\n",
    "You may give your pipeline a different name by changing the `PIPELINE_NAME` below. This will also become the name of the project directory where your files will be put."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "cIPlt-700ok-"
   },
   "outputs": [],
   "source": [
    "PIPELINE_NAME=\"TCR_Specificity\"\n",
    "import os\n",
    "PROJECT_DIR=os.path.join(os.path.expanduser(\"~\"),\"AIHub\",PIPELINE_NAME)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Create Project directory"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "colab_type": "text",
    "id": "ozHIomcd0olB"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Copying gs://hla_peptide_dataset/Pipelines/mhc_pipeline/beam_dag_runner.py...\n",
      "Copying gs://hla_peptide_dataset/Pipelines/mhc_pipeline/configs.py...           \n",
      "Copying gs://hla_peptide_dataset/Pipelines/mhc_pipeline/data_validation.ipynb...\n",
      "- [3 files][ 12.0 KiB/ 12.0 KiB]                                                \n",
      "==> NOTE: You are performing a sequence of gsutil operations that may\n",
      "run significantly faster if you instead use gsutil -m cp ... Please\n",
      "see the -m section under \"gsutil help options\" for further information\n",
      "about when gsutil -m can be advantageous.\n",
      "\n",
      "Copying gs://hla_peptide_dataset/Pipelines/mhc_pipeline/features.py...\n",
      "Copying gs://hla_peptide_dataset/Pipelines/mhc_pipeline/features_test.py...     \n",
      "Copying gs://hla_peptide_dataset/Pipelines/mhc_pipeline/hparams.py...           \n",
      "Copying gs://hla_peptide_dataset/Pipelines/mhc_pipeline/kubeflow_dag_runner.py...\n",
      "Copying gs://hla_peptide_dataset/Pipelines/mhc_pipeline/model.py...             \n",
      "Copying gs://hla_peptide_dataset/Pipelines/mhc_pipeline/model_analysis.ipynb... \n",
      "Copying gs://hla_peptide_dataset/Pipelines/mhc_pipeline/model_test.py...        \n",
      "Copying gs://hla_peptide_dataset/Pipelines/mhc_pipeline/pipeline.py...          \n",
      "Copying gs://hla_peptide_dataset/Pipelines/mhc_pipeline/preprocessing.py...     \n",
      "Copying gs://hla_peptide_dataset/Pipelines/mhc_pipeline/preprocessing_test.py...\n",
      "Copying gs://hla_peptide_dataset/Pipelines/mhc_pipeline/data/data1.csv...       \n",
      "/ [14 files][  1.0 MiB/  1.0 MiB]                                               \n",
      "Operation completed over 14 objects/1.0 MiB.                                     \n"
     ]
    }
   ],
   "source": [
    "# Make sure to have folder Peptide_Prediction created under AIHub before copying the file.\n",
    "# Following command will copy template pipeline for peptide prediction example.\n",
    "# After executing once, you can comment this out so that you dont accidently overright changes with template files.\n",
    "\n",
    "!gsutil cp -r gs://hla_peptide_dataset/Pipelines/mhc_pipeline/* ./TCR_Specificity/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "!ls"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/home/jupyter/AIHub/TCR_Specificity\n"
     ]
    }
   ],
   "source": [
    "%cd TCR_Specificity"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "beam_dag_runner.py     features_test.py        model_test.py\n",
      "configs.py\t       hparams.py\t       pipeline.py\n",
      "data\t\t       kubeflow_dag_runner.py  preprocessing.py\n",
      "data_validation.ipynb  model_analysis.ipynb    preprocessing_test.py\n",
      "features.py\t       model.py\n"
     ]
    }
   ],
   "source": [
    "!ls"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Test !python3 features_test.py"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Running tests under Python 3.7.6: /opt/conda/bin/python3\n",
      "[ RUN      ] FeaturesTest.testNumberOfBucketFeatureBucketCount\n",
      "[       OK ] FeaturesTest.testNumberOfBucketFeatureBucketCount\n",
      "[ RUN      ] FeaturesTest.testTransformedNames\n",
      "[       OK ] FeaturesTest.testTransformedNames\n",
      "[ RUN      ] FeaturesTest.test_session\n",
      "[  SKIPPED ] FeaturesTest.test_session\n",
      "----------------------------------------------------------------------\n",
      "Ran 3 tests in 0.002s\n",
      "\n",
      "OK (skipped=1)\n"
     ]
    }
   ],
   "source": [
    "!python3 features_test.py"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "artifacts.covid-19-271622.appspot.com\n",
      "cancer_vaccine\n",
      "hla_peptide_dataset\n",
      "hostedkfp-default-y95ed9e0de\n",
      "hostedkfp-default-yfu8n9ppw9\n",
      "mhc_peptide_model\n",
      "mhc_pipelines\n"
     ]
    }
   ],
   "source": [
    "!gsutil ls | cut -d / -f 3"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "yxOT19QS0olH"
   },
   "source": [
    "Change the working directory context in this notebook to the project directory."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting package metadata (current_repodata.json): done\n",
      "Solving environment: done\n",
      "\n",
      "## Package Plan ##\n",
      "\n",
      "  environment location: /opt/conda\n",
      "\n",
      "  added / updated specs:\n",
      "    - fastai\n",
      "\n",
      "\n",
      "The following packages will be downloaded:\n",
      "\n",
      "    package                    |            build\n",
      "    ---------------------------|-----------------\n",
      "    ca-certificates-2020.4.5.1 |       hecc5488_0         146 KB  conda-forge\n",
      "    certifi-2020.4.5.1         |   py37hc8dfbb8_0         151 KB  conda-forge\n",
      "    conda-4.8.3                |   py37hc8dfbb8_1         3.0 MB  conda-forge\n",
      "    cudatoolkit-10.1.243       |       h6bb024c_0       347.4 MB\n",
      "    cymem-2.0.2                |   py37he1b5a44_0          40 KB  fastai\n",
      "    cython-blis-0.2.4          |   py37h516909a_1         3.1 MB  fastai\n",
      "    dataclasses-0.6            |             py_0          19 KB  fastai\n",
      "    fastai-1.0.60              |                1         176 KB  fastai\n",
      "    fastprogress-0.2.2         |             py_0          15 KB  fastai\n",
      "    murmurhash-1.0.0           |   py37h3340039_0          16 KB  conda-forge\n",
      "    ninja-1.10.0               |       hc9558a2_0         1.9 MB  conda-forge\n",
      "    nvidia-ml-py3-7.352.0      |             py_0          22 KB  fastai\n",
      "    openssl-1.1.1f             |       h516909a_0         2.1 MB  conda-forge\n",
      "    plac-0.9.6                 |           py37_0          38 KB\n",
      "    preshed-2.0.1              |   py37he1b5a44_0          85 KB  conda-forge\n",
      "    pytorch-1.4.0              |py3.7_cuda10.1.243_cudnn7.6.3_0       432.9 MB  pytorch\n",
      "    spacy-2.1.8                |   py37hc9558a2_0        62.1 MB  fastai\n",
      "    srsly-0.1.0                |   py37he1b5a44_0         208 KB  fastai\n",
      "    thinc-7.0.8                |   py37hc9558a2_0         1.6 MB  fastai\n",
      "    torchvision-0.5.0          |       py37_cu101         9.1 MB  pytorch\n",
      "    wasabi-0.2.2               |             py_0          18 KB  fastai\n",
      "    ------------------------------------------------------------\n",
      "                                           Total:       864.2 MB\n",
      "\n",
      "The following NEW packages will be INSTALLED:\n",
      "\n",
      "  cudatoolkit        pkgs/main/linux-64::cudatoolkit-10.1.243-h6bb024c_0\n",
      "  cymem              fastai/linux-64::cymem-2.0.2-py37he1b5a44_0\n",
      "  cython-blis        fastai/linux-64::cython-blis-0.2.4-py37h516909a_1\n",
      "  dataclasses        fastai/noarch::dataclasses-0.6-py_0\n",
      "  fastai             fastai/noarch::fastai-1.0.60-1\n",
      "  fastprogress       fastai/noarch::fastprogress-0.2.2-py_0\n",
      "  murmurhash         conda-forge/linux-64::murmurhash-1.0.0-py37h3340039_0\n",
      "  ninja              conda-forge/linux-64::ninja-1.10.0-hc9558a2_0\n",
      "  nvidia-ml-py3      fastai/noarch::nvidia-ml-py3-7.352.0-py_0\n",
      "  plac               pkgs/main/linux-64::plac-0.9.6-py37_0\n",
      "  preshed            conda-forge/linux-64::preshed-2.0.1-py37he1b5a44_0\n",
      "  pytorch            pytorch/linux-64::pytorch-1.4.0-py3.7_cuda10.1.243_cudnn7.6.3_0\n",
      "  spacy              fastai/linux-64::spacy-2.1.8-py37hc9558a2_0\n",
      "  srsly              fastai/linux-64::srsly-0.1.0-py37he1b5a44_0\n",
      "  thinc              fastai/linux-64::thinc-7.0.8-py37hc9558a2_0\n",
      "  torchvision        pytorch/linux-64::torchvision-0.5.0-py37_cu101\n",
      "  wasabi             fastai/noarch::wasabi-0.2.2-py_0\n",
      "\n",
      "The following packages will be UPDATED:\n",
      "\n",
      "  ca-certificates                     2019.11.28-hecc5488_0 --> 2020.4.5.1-hecc5488_0\n",
      "  certifi                         2019.11.28-py37hc8dfbb8_1 --> 2020.4.5.1-py37hc8dfbb8_0\n",
      "  conda                                        4.8.2-py37_0 --> 4.8.3-py37hc8dfbb8_1\n",
      "  openssl                                 1.1.1d-h516909a_0 --> 1.1.1f-h516909a_0\n",
      "\n",
      "\n",
      "\n",
      "Downloading and Extracting Packages\n",
      "ca-certificates-2020 | 146 KB    | ##################################### | 100% \n",
      "dataclasses-0.6      | 19 KB     | ##################################### | 100% \n",
      "pytorch-1.4.0        | 432.9 MB  | ##################################### | 100% \n",
      "certifi-2020.4.5.1   | 151 KB    | ##################################### | 100% \n",
      "cudatoolkit-10.1.243 | 347.4 MB  | ##################################### | 100% \n",
      "ninja-1.10.0         | 1.9 MB    | ##################################### | 100% \n",
      "fastprogress-0.2.2   | 15 KB     | ##################################### | 100% \n",
      "wasabi-0.2.2         | 18 KB     | ##################################### | 100% \n",
      "openssl-1.1.1f       | 2.1 MB    | ##################################### | 100% \n",
      "cython-blis-0.2.4    | 3.1 MB    | ##################################### | 100% \n",
      "torchvision-0.5.0    | 9.1 MB    | ##################################### | 100% \n",
      "thinc-7.0.8          | 1.6 MB    | ##################################### | 100% \n",
      "nvidia-ml-py3-7.352. | 22 KB     | ##################################### | 100% \n",
      "preshed-2.0.1        | 85 KB     | ##################################### | 100% \n",
      "conda-4.8.3          | 3.0 MB    | ##################################### | 100% \n",
      "plac-0.9.6           | 38 KB     | ##################################### | 100% \n",
      "fastai-1.0.60        | 176 KB    | ##################################### | 100% \n",
      "srsly-0.1.0          | 208 KB    | ##################################### | 100% \n",
      "spacy-2.1.8          | 62.1 MB   | ##################################### | 100% \n",
      "murmurhash-1.0.0     | 16 KB     | ##################################### | 100% \n",
      "cymem-2.0.2          | 40 KB     | ##################################### | 100% \n",
      "Preparing transaction: done\n",
      "Verifying transaction: done\n",
      "Executing transaction: done\n",
      "\n",
      "Note: you may need to restart the kernel to use updated packages.\n"
     ]
    }
   ],
   "source": [
    "conda install -c pytorch -c fastai fastai"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: fastai in /opt/conda/lib/python3.7/site-packages (1.0.60)\n",
      "Requirement already satisfied: bottleneck in /opt/conda/lib/python3.7/site-packages (from fastai) (1.3.2)\n",
      "Requirement already satisfied: fastprogress>=0.2.1 in /opt/conda/lib/python3.7/site-packages (from fastai) (0.2.2)\n",
      "Requirement already satisfied: beautifulsoup4 in /opt/conda/lib/python3.7/site-packages (from fastai) (4.8.2)\n",
      "Requirement already satisfied: matplotlib in /opt/conda/lib/python3.7/site-packages (from fastai) (3.2.0)\n",
      "Requirement already satisfied: numexpr in /opt/conda/lib/python3.7/site-packages (from fastai) (2.7.1)\n",
      "Requirement already satisfied: numpy>=1.15 in /opt/conda/lib/python3.7/site-packages (from fastai) (1.18.1)\n",
      "Collecting nvidia-ml-py3\n",
      "  Downloading nvidia-ml-py3-7.352.0.tar.gz (19 kB)\n",
      "Requirement already satisfied: pandas in /home/jupyter/.local/lib/python3.7/site-packages (from fastai) (0.25.3)\n",
      "Requirement already satisfied: packaging in /opt/conda/lib/python3.7/site-packages (from fastai) (20.1)\n",
      "Requirement already satisfied: Pillow in /opt/conda/lib/python3.7/site-packages (from fastai) (7.0.0)\n",
      "Requirement already satisfied: pyyaml in /home/jupyter/.local/lib/python3.7/site-packages (from fastai) (5.3.1)\n",
      "Requirement already satisfied: requests in /opt/conda/lib/python3.7/site-packages (from fastai) (2.23.0)\n",
      "Requirement already satisfied: scipy in /opt/conda/lib/python3.7/site-packages (from fastai) (1.4.1)\n",
      "Requirement already satisfied: torch>=1.0.0 in /opt/conda/lib/python3.7/site-packages (from fastai) (1.4.0)\n",
      "Requirement already satisfied: spacy>=2.0.18 in /opt/conda/lib/python3.7/site-packages (from fastai) (2.1.8)\n",
      "Requirement already satisfied: torchvision in /opt/conda/lib/python3.7/site-packages (from fastai) (0.5.0)\n",
      "Requirement already satisfied: soupsieve>=1.2 in /opt/conda/lib/python3.7/site-packages (from beautifulsoup4->fastai) (1.9.4)\n",
      "Requirement already satisfied: python-dateutil>=2.1 in /opt/conda/lib/python3.7/site-packages (from matplotlib->fastai) (2.8.1)\n",
      "Requirement already satisfied: cycler>=0.10 in /opt/conda/lib/python3.7/site-packages (from matplotlib->fastai) (0.10.0)\n",
      "Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /opt/conda/lib/python3.7/site-packages (from matplotlib->fastai) (2.4.6)\n",
      "Requirement already satisfied: kiwisolver>=1.0.1 in /opt/conda/lib/python3.7/site-packages (from matplotlib->fastai) (1.1.0)\n",
      "Requirement already satisfied: pytz>=2017.2 in /opt/conda/lib/python3.7/site-packages (from pandas->fastai) (2019.3)\n",
      "Requirement already satisfied: six in /opt/conda/lib/python3.7/site-packages (from packaging->fastai) (1.14.0)\n",
      "Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /home/jupyter/.local/lib/python3.7/site-packages (from requests->fastai) (1.24.3)\n",
      "Requirement already satisfied: chardet<4,>=3.0.2 in /opt/conda/lib/python3.7/site-packages (from requests->fastai) (3.0.4)\n",
      "Requirement already satisfied: idna<3,>=2.5 in /opt/conda/lib/python3.7/site-packages (from requests->fastai) (2.9)\n",
      "Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/lib/python3.7/site-packages (from requests->fastai) (2020.4.5.1)\n",
      "Requirement already satisfied: murmurhash<1.1.0,>=0.28.0 in /opt/conda/lib/python3.7/site-packages (from spacy>=2.0.18->fastai) (1.0.0)\n",
      "Requirement already satisfied: plac<1.0.0,>=0.9.6 in /opt/conda/lib/python3.7/site-packages (from spacy>=2.0.18->fastai) (0.9.6)\n",
      "Requirement already satisfied: srsly<1.1.0,>=0.0.6 in /opt/conda/lib/python3.7/site-packages (from spacy>=2.0.18->fastai) (0.1.0)\n",
      "Requirement already satisfied: thinc<7.1.0,>=7.0.8 in /opt/conda/lib/python3.7/site-packages (from spacy>=2.0.18->fastai) (7.0.8)\n",
      "Requirement already satisfied: cymem<2.1.0,>=2.0.2 in /opt/conda/lib/python3.7/site-packages (from spacy>=2.0.18->fastai) (2.0.2)\n",
      "Requirement already satisfied: preshed<2.1.0,>=2.0.1 in /opt/conda/lib/python3.7/site-packages (from spacy>=2.0.18->fastai) (2.0.1)\n",
      "Requirement already satisfied: wasabi<1.1.0,>=0.2.0 in /opt/conda/lib/python3.7/site-packages (from spacy>=2.0.18->fastai) (0.2.2)\n",
      "Requirement already satisfied: blis<0.3.0,>=0.2.2 in /opt/conda/lib/python3.7/site-packages (from spacy>=2.0.18->fastai) (0.2.4)\n",
      "Requirement already satisfied: setuptools in /opt/conda/lib/python3.7/site-packages (from kiwisolver>=1.0.1->matplotlib->fastai) (46.0.0.post20200311)\n",
      "Requirement already satisfied: tqdm<5.0.0,>=4.10.0 in /opt/conda/lib/python3.7/site-packages (from thinc<7.1.0,>=7.0.8->spacy>=2.0.18->fastai) (4.43.0)\n",
      "Building wheels for collected packages: nvidia-ml-py3\n",
      "  Building wheel for nvidia-ml-py3 (setup.py) ... \u001b[?25ldone\n",
      "\u001b[?25h  Created wheel for nvidia-ml-py3: filename=nvidia_ml_py3-7.352.0-py3-none-any.whl size=19189 sha256=119129e39b12bd4562aa9394bb0fcacdcad2188bf60b01efaf5e6a0e60631cf9\n",
      "  Stored in directory: /home/jupyter/.cache/pip/wheels/df/99/da/c34f202dc8fd1dffd35e0ecf1a7d7f8374ca05fbcbaf974b83\n",
      "Successfully built nvidia-ml-py3\n",
      "Installing collected packages: nvidia-ml-py3\n",
      "Successfully installed nvidia-ml-py3-7.352.0\n"
     ]
    }
   ],
   "source": [
    "!pip install fastai"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "%reload_ext autoreload\n",
    "%autoreload 2\n",
    "\n",
    "import string\n",
    "from fastai import *\n",
    "from fastai.vision import *\n",
    "from fastai.tabular import *\n",
    "from fastai.tabular.data import *\n",
    "from sklearn.model_selection import train_test_split"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'1.0.60'"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import fastai; fastai.__version__"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>gene</th>\n",
       "      <th>cdr3</th>\n",
       "      <th>species</th>\n",
       "      <th>antigen.epitope</th>\n",
       "      <th>antigen.gene</th>\n",
       "      <th>antigen.species</th>\n",
       "      <th>complex.id</th>\n",
       "      <th>v.segm</th>\n",
       "      <th>j.segm</th>\n",
       "      <th>v.end</th>\n",
       "      <th>j.start</th>\n",
       "      <th>mhc.a</th>\n",
       "      <th>mhc.b</th>\n",
       "      <th>mhc.class</th>\n",
       "      <th>reference.id</th>\n",
       "      <th>vdjdb.score</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>TRB</td>\n",
       "      <td>CASSFSGTQYF</td>\n",
       "      <td>HomoSapiens</td>\n",
       "      <td>NLVPMVATV</td>\n",
       "      <td>p65</td>\n",
       "      <td>CMV</td>\n",
       "      <td>0</td>\n",
       "      <td>TRBV28*01</td>\n",
       "      <td>TRBJ2-3*01</td>\n",
       "      <td>4</td>\n",
       "      <td>7</td>\n",
       "      <td>HLA-A*02</td>\n",
       "      <td>B2M</td>\n",
       "      <td>MHCI</td>\n",
       "      <td>PMID:28423320</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>TRB</td>\n",
       "      <td>CASSQVEEFHGELFF</td>\n",
       "      <td>HomoSapiens</td>\n",
       "      <td>GILGFVFTL</td>\n",
       "      <td>M1</td>\n",
       "      <td>InfluenzaA</td>\n",
       "      <td>0</td>\n",
       "      <td>TRBV7-9*01</td>\n",
       "      <td>TRBJ2-2*01</td>\n",
       "      <td>4</td>\n",
       "      <td>10</td>\n",
       "      <td>HLA-A*02</td>\n",
       "      <td>B2M</td>\n",
       "      <td>MHCI</td>\n",
       "      <td>PMID:28423320</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>TRB</td>\n",
       "      <td>CASSMSRSSNQPQYF</td>\n",
       "      <td>MacacaMulatta</td>\n",
       "      <td>TTPESANL</td>\n",
       "      <td>Tat</td>\n",
       "      <td>SIV</td>\n",
       "      <td>0</td>\n",
       "      <td>TRBV14*01</td>\n",
       "      <td>TRBJ1-5*01</td>\n",
       "      <td>4</td>\n",
       "      <td>8</td>\n",
       "      <td>Mamu-A*01</td>\n",
       "      <td>B2M</td>\n",
       "      <td>MHCI</td>\n",
       "      <td>PMID:15589168</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>TRB</td>\n",
       "      <td>CASSPRQGLEGANVLTF</td>\n",
       "      <td>HomoSapiens</td>\n",
       "      <td>LPPIVAKEI</td>\n",
       "      <td>Int</td>\n",
       "      <td>HIV-1</td>\n",
       "      <td>0</td>\n",
       "      <td>TRBV7-9*01</td>\n",
       "      <td>TRBJ2-6*01</td>\n",
       "      <td>4</td>\n",
       "      <td>10</td>\n",
       "      <td>HLA-B*42:01</td>\n",
       "      <td>B2M</td>\n",
       "      <td>MHCI</td>\n",
       "      <td>PMID:25911754</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>TRB</td>\n",
       "      <td>CASSLGSQNTLYF</td>\n",
       "      <td>MusMusculus</td>\n",
       "      <td>HGIRNASFI</td>\n",
       "      <td>M45</td>\n",
       "      <td>MCMV</td>\n",
       "      <td>1962,1907,1926,1914,1969</td>\n",
       "      <td>TRBV12-1*01</td>\n",
       "      <td>TRBJ2-4*01</td>\n",
       "      <td>5</td>\n",
       "      <td>6</td>\n",
       "      <td>H-2Db</td>\n",
       "      <td>B2M</td>\n",
       "      <td>MHCI</td>\n",
       "      <td>PMID:28636592</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  gene               cdr3        species antigen.epitope antigen.gene  \\\n",
       "0  TRB        CASSFSGTQYF    HomoSapiens       NLVPMVATV          p65   \n",
       "1  TRB    CASSQVEEFHGELFF    HomoSapiens       GILGFVFTL           M1   \n",
       "2  TRB    CASSMSRSSNQPQYF  MacacaMulatta        TTPESANL          Tat   \n",
       "3  TRB  CASSPRQGLEGANVLTF    HomoSapiens       LPPIVAKEI          Int   \n",
       "4  TRB      CASSLGSQNTLYF    MusMusculus       HGIRNASFI          M45   \n",
       "\n",
       "  antigen.species                complex.id       v.segm      j.segm  v.end  \\\n",
       "0             CMV                         0    TRBV28*01  TRBJ2-3*01      4   \n",
       "1      InfluenzaA                         0   TRBV7-9*01  TRBJ2-2*01      4   \n",
       "2             SIV                         0    TRBV14*01  TRBJ1-5*01      4   \n",
       "3           HIV-1                         0   TRBV7-9*01  TRBJ2-6*01      4   \n",
       "4            MCMV  1962,1907,1926,1914,1969  TRBV12-1*01  TRBJ2-4*01      5   \n",
       "\n",
       "   j.start        mhc.a mhc.b mhc.class   reference.id  vdjdb.score  \n",
       "0        7     HLA-A*02   B2M      MHCI  PMID:28423320            0  \n",
       "1       10     HLA-A*02   B2M      MHCI  PMID:28423320            0  \n",
       "2        8    Mamu-A*01   B2M      MHCI  PMID:15589168            0  \n",
       "3       10  HLA-B*42:01   B2M      MHCI  PMID:25911754            0  \n",
       "4        6        H-2Db   B2M      MHCI  PMID:28636592            1  "
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "PATH = Path(\"/home/jupyter/AIHub/TCR_Specificity/new_model\")\n",
    "tcr = pd.read_csv(PATH/\"vdjdb.slim.txt\", sep=\"\\t\")\n",
    "tcr.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "20941"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# human only\n",
    "tcr_human = tcr[tcr[\"species\"]==\"HomoSapiens\"]\n",
    "len(tcr_human)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "14941"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# TRB chain only\n",
    "tcr_human_b = tcr_human[tcr_human[\"gene\"]==\"TRB\"]\n",
    "len(tcr_human_b)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "NLVPMVATV               4459\n",
       "GILGFVFTL               2678\n",
       "ELAGIGILTV               976\n",
       "GLCTLVAML                863\n",
       "LLWNGPMAV                409\n",
       "FRDYVDRFYKTLRAEQASQE     390\n",
       "KRWIILGLNK               328\n",
       "PKYVKQNTLKLAT            295\n",
       "LLLGIGILV                233\n",
       "TPRVTGGGAM               206\n",
       "Name: antigen.epitope, dtype: int64"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "epitope_counts = tcr_human_b[\"antigen.epitope\"].value_counts()\n",
    "epitope_counts[:10]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "7137"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tcr_human_b_2 = tcr_human_b[tcr_human_b[\"antigen.epitope\"].isin(\n",
    "    epitope_counts.index.values[:2])]\n",
    "len(tcr_human_b_2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "colab": {
   "collapsed_sections": [],
   "name": "template.ipynb",
   "provenance": [],
   "toc_visible": true
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}