src/FFL.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Using device: cpu\n"
     ]
    }
   ],
   "source": [
    "import torch\n",
    "import utils\n",
    "\n",
    "from environments.foggy_frozen_lake.utils import argument_actions\n",
    "from environments.foggy_frozen_lake.FFL import FFL\n",
    "\n",
    "from agents.orla import ORLABaseline\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "from tqdm.notebook import tqdm\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')\n",
    "print('Using device:', device)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Using ORLA to solve Foggy Frozen Lake (FFL)\n",
    "This jupyter notebook showcases the use of [ORLA](https://github.com/omcandido/ORLA) to solve the FFL game, which is a variation of Frozen Lake (FL), an archetypal text-based game that comes with Gymnasium out-of-the-box.\n",
    "\n",
    "In its original formulation, FL consists of a squared grid-like world where some cells are frozen (F) and others are holes (H). The agent needs to go from the start (S) in the upper left corner to the goal (G) in the lower right corner. Here is a text-based example of ORLA solving this game:\n",
    "\n",
    "<center><img src=\"../imgs/FL.gif\" width=\"350\"></center>\n",
    "\n",
    "\n",
    "## Frozen Lake with a twist\n",
    "To make things more interesting, we introduce FFL, a variant of FL where the player can only observe its immediately neigbouring cells (as if some kind of dense fog shortened the player's sight) and the tile index it is at.\n",
    "Here is a graphics-rendered example where the elf is the player and the red square around it represents observations:\n",
    "<center><img src=\"../imgs/FFL.gif\" width=\"350\"></center>\n",
    "\n",
    "This new formulation of the game is done to create a hiden state of the game so that the player needs both factual knowledge (derived from the current state of the world) and memory knowledge (derived from past states) to achieve a high score.\n",
    "\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## An Argumentation Framework (AF) to play FFL\n",
    "\n",
    "An argumentation framework is a set of arguments and attacks, where each argument recommends an action when all its associated conditions (premises) are satisfied at some point in the game. Every two attacks that recommend different actions attack each other.\n",
    "\n",
    "With this in mind, we can come up with the following set of arguments:\n",
    "```\n",
    "-  U: go UP      if   cell-up     is   safe\n",
    "-  D: go DOWN    if   cell-down   is   safe\n",
    "-  L: go LEFT    if   cell-left   is   safe\n",
    "-  R: go RIGHT   if   cell-right  is   safe\n",
    "- nU: go UP      if   cell-up     is   safe   &   new\n",
    "- nD: go DOWN    if   cell-down   is   safe   &   new\n",
    "- nL: go LEFT    if   cell-left   is   safe   &   new\n",
    "- nR: go RIGHT   if   cell-right  is   safe   &   new\n",
    "```\n",
    "\n",
    "Where arguments X promote exploring safe tiles and arguments nX promote exploring safe and unexplored squares. The complete AF can be represented through a graph, where the directed arrows represent attacks between arguments (i.e., A->B means A attacks B) would look like this:\n",
    "\n",
    "<center><img src=\"../imgs/AF_FFL.png\" width=\"300\"></center>\n",
    "\n",
    "\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Learning a good ranking with ORLA\n",
    "\n",
    "The goal of ORLA is to learn a ranking of the arguments such that the derived value-based argumentation framework (VAF) can be used as an inference engine by the agent to solve FFL.\n",
    "\n",
    "ORLA is task-agnostic. This means that it just tries to maximise the reward of an MDP (see paper) regardless of what the specific task is. In this case, the task to solve is FFL. The rest of this notebook explains how to implement everything necessary to have ORLA solve FFL. If you want to apply ORLA to your own reinforcement learning task, follow along and create your own task-specific code.\n",
    "\n",
    "\n",
    "#### Create a dictionary for your arguments and actions\n",
    "ORLA needs a dictionary of the form ``{argument_name: action_name}``. This is what the arguments for ORLA look like in code:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'U': <FLActions.UP: 3>,\n",
       " 'L': <FLActions.LEFT: 0>,\n",
       " 'R': <FLActions.RIGHT: 2>,\n",
       " 'D': <FLActions.DOWN: 1>,\n",
       " 'nD': <FLActions.DOWN: 1>,\n",
       " 'nL': <FLActions.LEFT: 0>,\n",
       " 'nR': <FLActions.RIGHT: 2>,\n",
       " 'nU': <FLActions.UP: 3>}"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "argument_actions"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "#### Extend the ``Environment`` class\n",
    "The interaction between the task and ORLA takes place via the abstract ``Environment`` class. You will need to inherit this class and implement your own methods:\n",
    "- ``get_premises``: from the current observation and memory, which premises hold?\n",
    "- ``get_arguments``: from the current premises, what arguments are applicable?\n",
    "- ``update_memory``: how should memory be updated at each step?\n",
    "- ``reset_memory``: how should memory be initialised?\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Training\n",
    "Once the Arguments and the Environment are defined, we can instantiate an ORLA agent and train it for 10k of episodes. Note that we are sampling a new environment every episode, so ORLA will have to learn a policy for the distribution of FFL games with paramenters ``n`` and ``p``.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "54c9fe72dc2844ac8e95f4e70dde6a76",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/10000 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "arguments = list(argument_actions.keys())\n",
    "mode = utils.Mode.STRICT #whether the ranking should be strict or not\n",
    "n_episodes = 10000\n",
    "alpha_theta = 5e-4\n",
    "alpha_w = utils.estimate_alpha_w(len(arguments), n_episodes)\n",
    "orla = ORLABaseline(arguments, alpha_theta, alpha_w, device, mode)\n",
    "\n",
    "n=8 #map size\n",
    "p=0.8 #probability of tile being frozen\n",
    "\n",
    "batch_size = 8\n",
    "\n",
    "returns = []\n",
    "rankings_batch = []\n",
    "probs_batch = []\n",
    "returns_batch = []\n",
    "\n",
    "t_episodes = tqdm(range(1, n_episodes+1))\n",
    "for episode in t_episodes:\n",
    "    env = FFL(argument_actions, n, p)\n",
    "\n",
    "    orla.net.train()\n",
    "    ranking, prob = orla.decode_ranking()\n",
    "    score = env.play(ranking)\n",
    "    returns.append(score)\n",
    "\n",
    "    rankings_batch.append(ranking)\n",
    "    probs_batch.append(prob)\n",
    "    returns_batch.append(score)\n",
    "    \n",
    "    \n",
    "    if len(rankings_batch) == batch_size:\n",
    "        orla.learn(rankings_batch, probs_batch, returns_batch)\n",
    "        rankings_batch = []\n",
    "        probs_batch = []\n",
    "        returns_batch = []\n",
    "        past = 1000\n",
    "        t_episodes.set_postfix({'avg': np.average(returns[-past:]), 'last': np.sum(returns[-batch_size:])/batch_size})\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Learning curve\n",
    "We see that ORLA has learned a pretty good policy (1 is the maximum attainable reward)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjcAAAGwCAYAAABVdURTAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABj+ElEQVR4nO3dd3hUVfoH8O/0Sa+kJyT03iVSRUEQWayrrKICtrWwi2BlFVh1FdaCbVHUFduq2FdXEH8YKdI70qQGQkkCIaSXycyc3x/J3Nw7LTNhkkkm38/z8DC3n7mZ5L5zznvOUQkhBIiIiIgChNrfBSAiIiLyJQY3REREFFAY3BAREVFAYXBDREREAYXBDREREQUUBjdEREQUUBjcEBERUUDR+rsAzc1qteLMmTMICwuDSqXyd3GIiIjIA0IIlJaWIikpCWq1+7qZNhfcnDlzBqmpqf4uBhERETXCyZMnkZKS4nafNhfchIWFAai9OeHh4X4uDREREXmipKQEqamp0nPcnTYX3NiaosLDwxncEBERtTKepJQwoZiIiIgCCoMbIiIiCigMboiIiCigMLghIiKigMLghoiIiAIKgxsiIiIKKAxuiIiIKKAwuCEiIqKAwuCGiIiIAgqDGyIiIgoofg1u1q5di4kTJyIpKQkqlQr//e9/Gzxm9erVGDBgAAwGAzp16oQPPvigyctJRERErYdfg5vy8nL07dsXixYt8mj/7OxsTJgwAZdffjl27dqFhx56CHfffTd++umnJi4pERERtRZ+nThz/PjxGD9+vMf7L168GBkZGXj55ZcBAN27d8e6devwyiuvYNy4cU1VTCJqBCEEzpZWIy7M4NFEd/5gtQqUmcwIN+r8XRQHpVU1MJmtiAk1+LsoDipMZhSWmwAAUcF6hBia91FSabIgSK9xWC+EQF5JFdqFGqDVqBXrc4urEBGka/KyVposMOrUXn3mayxWVJutCJWVzWS24mxpFQAgOkSPYL3/57m2lSk21ACjzvH+A7W/U2eKK6HXqhEXZmzmEtbz/93ywsaNGzFmzBjFunHjxuGhhx5yeUx1dTWqq6ul5ZKSkqYqHhHJ/P37ffhw4wlM7JuEN27p7+/iOPXAJzuQ9Xs+vrxvKPqlRvq7OJLf80pwzRvrYbJYsfDmvrhhQIq/iyQ5X1aNUS+tRmmVGQAQpNNg5ayRSIkKbpbrf7frNGYs3YUFN/TGnwanKbbN+34fPtp4An1TI/Hdg8Ok9U98vQefbzuJIJ0GPz98GZIjg5qkbLnFlRj98hqM6toOb04e6NExFqvAVa+uxakLlVjx0EhkxIagxmLFmIVrkFNYAQAIM2ix6tFRiPVjoGuxClz12locO1eOxAgjVj0yymmAc89H25D1+1n0T4vEtw8Mc3Km5tGqEorz8vIQHx+vWBcfH4+SkhJUVlY6PWb+/PmIiIiQ/qWmpjZHUYnavA83ngAA/G/3GT+XxLUV+/JQYxH4evspfxdFYe/pEpgsVgDAzpwi/xbGzuGzZVJgo1IBlTUW/J5b2mzXn7F0FwDgiW/2OGzLOnAWALD7ZBEsViGt33aiEICtrE33Bfe/O8+gwmTB8j15Hh9TXFmDo+fKUW22Yu/pYgBAQVm1FNgAQGm1GYfzy3xeXm8UVZhw7Fw5ACC3uAq5xVVO99uecwEAcPqC82dyc2lVwU1jzJ49G8XFxdK/kydP+rtIRNTCWIRoeKdmVFVjcfq6JbCVp0diOC5Jj65dZ25ZZQTs76HV6WtfszbicyQvZ01dQGsrY5hBi17J4bXr/HyPq8zK++bqc2lb//X9Q5u8TO60qmaphIQE5OfnK9bl5+cjPDwcQUHOqxkNBgMMhpbXZk0UqIQQ2Hu64W/HlSYLNmefh9ki0D0pHOfLqlFVY8Wg9lFQq73P0TlTVImj58qQmREDvbbh721mS/0f6+y6b6QtRYXJLL0+dNa/39jtHakrj1GnlpoldpwoQpBOg4HtoxAZrG/wHIXlJuw5XYwOsSHILijH4Ixo6VyVJguOnitDz6Rwh7wVYRc8rDtcgMEZ0agyW7DteCEqZQ/c8mozQgxaCCFwuqi+FmFHzgUYtGp0TQhDarT3TWlCCOw7U4KM2BBF/k6NxYr1Rwqk5Z/35yPYUHtPdpwoQnm1GQkRRlTVWDAgrf4znltcX7ZdJ4sQbtRJ5TXoNDBqa+/L1uxCWCy17z8qRIcBaVFu83osVoGtxwtRWmWGbS+rEMgprMAfB6ZIPyeLVWBLdiHKq83QqFW4JCNakftjk1ukrImxD25qLFZsyS5EdV0Q5Conp7m0quBmyJAhWL58uWLdypUrMWTIED+ViIjs7TxZhBve3KBYV1pVgzC7pN2/f78Pn29zrEl9dVI/XNc/2atrmi1WjHtlLUqrzfjzZR0we3z3Bo9Zsj5ber3x2HmcL6tuMcm7B2TNPLtPFkkPan+rNFnwj2UHAAAhBi1C6pJ6l6zPxpL12RjUPgpfefCNfdLbG3FYFrTdfml7PHtdLwDArf/ehJ05RXjjlv6Y2DdJcdyKvcrmntve24y/ju6MnTkX8OvhAsW299ZlY/bV3fHL72cd1r+3Lhsheg22PjXG60TdNYfOYer7WzGuZzzevn2QtP6dtcew4eh5afnuj7YBANrHBOPE+QrFOebf0Bu31OUL3fvRdmn9RxtP4KO65lwACDFoEFz3c39z9VHFOd65fSDG9kxwWc7/bDqBed/vc7rtH8sO4PiCCQCAjzcex9//t1/aNr5XAt66zTFf6J6692Nz+GwZ+qdFSctvrzmKl/7vkLQc7CThuzn59belrKwMR44ckZazs7Oxa9cuREdHIy0tDbNnz8bp06fx0UcfAQDuu+8+/Otf/8Jjjz2GO++8E7/88gu++OILLFu2zF9vgYjs5Nj9IQeA82Umh+DmRKHz2hL7B4EnyqrNKK2ure04XuBZLYz9dXKLq1pMcBNlV/txvszUIoKbwgqT9PrOYRkw6jQ4X25CSWUNfs8rxYlCz352h+1qo3adLJJe23KMvtx+yiG4kZ9fo1bBYhXIOV8u5ad0iQ/FobrclOLKmtpjZD/nwenRMFms+O1UEcpNFpwvMyE42rv7+unmHADAT/uUrQj2n/uEcCPySqqcfp6X78mVgptqWXOPPKldpQJuGZyGuDADKk1m1NTV2uQUVqCw3KTIyXFmR13uCwBEBevQPiZEcZ9tbPc0WK9Bhcni8vev2q5Zyr7OyHZcUoQRNwxI8fvn1a9X37ZtGy6//HJpedasWQCAKVOm4IMPPkBubi5ycnKk7RkZGVi2bBlmzpyJ1157DSkpKfj3v//NbuBELYitunpM9zjsyClCYblJSo5V7uc896ExuQXyc5ktnuU92F+/ugXljdjfA3/nW9jYfrbhRi0u7xYHABjScQiOnivD6JfXNDo/SMDxZ6Zx0uJiO//kzDR0SwzHnP/uRVWNVVq/8OZ+2HTsPP6x7IC0znbvbhqYghdv6gsA6PfM/6GooqZRP3NXny7bdZ6a0B13j+iAt1YfxT9X/N7g+Wzl3DR7NBIinHedHtU1Tnr9xNe/YenWkw3ea42syerKHvF44Y99kf6EY0WA7fdgYPso/Hq4wOlnTQghXW9g+yhsP3HBMQenbvnuER1w5/AMt2VrDn4NbkaNGuXQhirnbPThUaNGYefOnU1YKqK24fe8Emw6eh43DExRjPOy9XghcourcI3dt2Z7ZosVX+84hbgwo/SgAyB9ozToNNDVPaFMdn8I/7f7jNNvkQAa7M1y5GwpVv1+DqnRwTicX9t8s+9M/TGHzjr23PlpX56UK2Kt60Vj65lis3jNMbw6KRz7c0vw6s+HML5XIiZdkgqd5uL7Xfy4Jxd7ThdDo1bhtkvbIz68/iF2MK8UPx/Ix+jucciIDcFX209ht9292ZlzAV3iwxzOW2Ey46vtpxAXZsCaQwU4eq4Ml3Vphz/0SUT7mBCnZdl3phgnCytwVa9Er99Hfb6NssnBtlxaZcanm3NwvqwaHeNCcXVv5TW2ZBdi07HzsLf3dAlOXajAD7/lSus0TvKuDtR9Now6DYx1eVUH80ulWhqjTgNDXVn2nSnBG1mHpaYieZlr81hq8P2uM4gK0eOPA1Ow+2QxdBoVMjvE4GRhBdYdKUByZBB2nyzCoPRoqFTA377Zg2OymsEjZ8vQKS4U50qr8d2uM4rrGHWuPze/Hi6Qfn/MdZ9Hd/vL2c7/2ZaTmH5FZwDA4fxSHMgrxcQ+iVIejjxvTeVQz1Jv49Ha5ryIoNq/AedKq/H0/2qbsx4d1xXBei0Ky02wdT6LrNvv2Dll7ZutV6S/c21s/F/PSUR+MWXJFuSXVCO3uAqzr67NURFC4KbFGwEAHWJD0Cs5wuXx644U4PGva7vj7pp7pZSguKcuaLBahRQY1Mhqbg7mleIvn7n+gnKsgWalez7ajmw3+1Tb1cicKarEnz/e7mLveiv352Pp1pN49ofa/IP1R84jIdyIMT3iGzjSvRPny3H/Jzuk5f1nSvDe1Euk5bs/2oqThZX4bEsOHr+qG578dq/DOf654iAmXZLmsP7bnacx9ztlXsWW7EJsPV6ID6YNdlqeCa+vqz32gaGKnAlPvPdrbZ6Sfa+gMGP9o+Rv39Z30V71yChkxNYGWRarwJ0fbEVZtRnOTHxjHS5U1EjLQU5yYWyJ6lYhpM+b/LMQHqSVHr6Hz5bh5ZX1OSC2hzdQ3zvu9V9q0yIKyqqxaFVtTsuR58bjoc93YfuJ+mYdnUYlNQvJ/e2bPfjiviFYtKo+vcJ2nchg9wNDvpZ1GG/UXV+nUTkdlNCZU3VdrE8XVaK4ogYRwTpc+cpaAECoQYMrutV+XuU1N5Eh9WUqqrvHh/JL0TkuFMfrmpMS6gLu0ioz3l9/HEBtrc78G3pLTXEApGT9HbL7cyi//guF/D77E4MbojYqv6R2cMudsloCebt6fkmV2+DmXGn94JgXKmqkh01Q3Te3tOhgHKz7oyd/MNhGtrV5+pqe+D2vFD/ty0NhuQkhDSR42gc2KVFB0h98AA5t/efLlNcz6tS4vi5hOSbEgKGdYnDru5sBOI7NUVBWjYtlf44suwTXk4W11zx1oVK6p+1jgjGhdyJCjVq8sOKgy+/dhXbvzdU1nfk9r9Tr4MbWfDTBrkYm3KjDsE4xWH9EWStTUFYtBTdVNRYpsPnjwBTsP1OCXsnh+GJb7RhD8sAGALrGhzpc35ak2i81EiO7xGLG6M7SKL69kyMRF2bEmO7x+MsVnRT3IESvxe1D2kvLOrtaoWOy3nJVZqsisAHgNLCxvT+g/nch3KjFmO61wcXYHgmYfnk5zpdXIypYj0HpUfhq+ylpDBx50vjLN/eDQetZcFMuCw6LKk2IkAVRu04WS8GNfHiDu4bVNhP98JfhGP7PVbVlL61G+5j63mLThmcgJSoIW44XSmX84bczmH9Db6lmLDU6CL1TIvDj3jxFMCb/WzC6e30trj8xuCFq4+SDncnb8dUNDB8vb3NXjClSt75LfBjWHDoHQFlzI78eUJsPMGVoOib2TcSt725W7OuJzIwYnLpQPwiffS6CfQ5BTIgB82/oo1h332UdsXjNURRVKoMFX4wx4824KrayXpIejceu6oYT58vxwoqDLsvh6l55ck37n4NH5as7rzwHxOa2zPYOwY2r8XpeuLGP1GxSbrJgmaw5KipYhwsVNU4DCtv9SY0OhkGrwcwruzjsE6TX4OGxXd2+j0syoqVmJEDZbOrJz/z76cNwzb/W1+f11P3/5ITu0kM/xKDFI+OU5biiWzzu+3g7VuzLQ3HdZ214p9gGm4Dl2oXVJ73b/5ytsp+p7bMx5w89EFdXK5MSFYw+KRH47VQxqswWxfHtQg2YOiwDo7rGOQxCaLvvN/RPQee4MIdr295/35QINksRUcsg/5Yq/4MlT/Lcd6YYE15fh8yMaKREBSPr93ypehsA1h8pQPfEcJjMVqytC2iMOo3ULPV61mEM6xQLwPGBbPtjqHfShGXP2YMnyq76P7e4ClU1Fum86+y6CJ9zUqthy3eQj1MCAP/7LRdT6771ni+rxssrD+HTzTnokxKBzIzaAMRdTk5RhQmPfrnbYf11i9ZjZJd2+G7XacX6/+6sXQ6S8jZq/y83WfDy/x3ErCu7KMY2MbmoUThytgxT39+C1QfPYfFtA3H8fDn2nCrGsj31QcQnm3Nw7Fw5LuvaDn2SI/DS/x2EVQj0To7EwpUHERdmxDt3DJSmVcgrrpKaHA1O8kOcPdSWbjmJEZ3bAYA0Bo1eq1bkgxjtaizCg2qDm9eyDuPUhUo8cHlHCCHw5uqjOFtX22h/jLfsj5fXpi3dkmO/u+Pxde+1oNyEBz/Zgd2nihXr3R9be+9+zytVLHtK/p3j2R/2IzqkvmfdJ5tPSAGV7XOvt8vMtr33f/1yRDpWrYKUHyevkSmtMte9v6K6smqk8u46WQQhBFQqlfQ3xNBCAhuAwQ0Rycir8k3m+genbcj7zdmF2Jxd6HDcD7/l4u4RHbBFti0hwoC4um+Zm7MLkVtcicSIIEXPKZWqdiwPoL4t31UTAAApcLKJCdEjJcpxAM9fDxfgyrpcGftxTuyTm4H6fANbU53Nobz6poP/7T4j5R78dqoYv50qxmVd4jC8c6zL8i7bk4szToap33WyyGlCta0bs63XTLhRB6NOjaoaK9745QjG90pEj6RwaX93geDqg7X36r7/OM83OpBbggO5JfhgQzaeubYXPql7b5+hduyhgjITXll5GC/fXNvD6Kvt9WMSyROi3a1bticXi+pe25ou7O9/QoSy+32Q7AH59Y5TCNZrYLZa8c2O2sBPrQJiwxoeKNCdeBe9kgAoxmpxZkBaJGJC9NCoVTCZrYqA0dk9cHVt2xQWnhwjN65nglTrtM4uGJc37bn6LbJdf4dsWo+EcKMUNNtPIit/fwkRBul3BajNj+vYLlQKeuWDT/obgxsikshzbuQPziMNjJJr6/VYUlX/x3VAWhSeu743suZnAQAqTBaH83734DAp18BWA+Ks27hNSVX9H89nru2JS9KjkREbgohgHaJDDHjgP9tRbrKgtEr+R77h5pfrByQjSK9BcWUNyqrN2JJdiNUHzymOLDc51hrJr+O0vJX15X3++t6KZFtneidHYMrQdIzvVTs4W5Beg6X3DsF1i9Y7vZ7tXsaE6HHToFQM6RiDKUu2uL2GPatQ/tzkjsh6ntnufdf4MHRs55gP0yMpHO9PvQSniiqREhmEaR9sBVDbq06rUUtBq96upuu+yzqifUwIqmosSI4MwtnSasyWzRtVWlXfRDWhdyJuH9L+omebvndkB6RGBaGs2owPNhxvcGylZ67tCZPZivJqC6YOS0dEkA6f3p0p5ZQBtUFKZkZ0g9d+8PJO6NguFFU1Fug1aoxzMxCfM1f1TMC8iT1wvsyECxUmKSh1ZUB7ZV7V3D/0wJAOMTBb63/PLu0QI70O0mvw+b2XYtI7m6R1V/dOwFW9EnFVzwTF6N+lVcpg5tq+3g2+2ZQY3BCRpNrJPDeesDVn2ZqNRnSOhUqlQkKEEbGhBhSUVUvns/0/onMs+qRESudw1rPK8Tq15x/bIx53DEmX1l/fv3bW7CEdY/HzgXyv5xIyaDW4tl/9H+br+1diyPxfnM77oyhPA+OkyMdluTUzDcv2nHHIS5HrnRKBPw5UzgDeLzUS3RPDcSC3xGFsEVuZpg1Ll7oFN4Yn90i69z1d9x6zDQlQaVLmYIVq1NLxHdopu6iHGXW4eVD9hMb2TXVVNVbFZ0b+IG6sUIMWN9VdMyJIh1lfODYd2vRPi1R81mwyO8QgsxFlCbd7v95Sq1WYVtdUuu5wgcvgxna/7adSaBdmwK2Zjj3v5DI7xOCqnglYsa8292Z4p3aKvKAO7UJw7Fy5Q85RspNaVH9hcEPUBrlKJpV36fzl97O4YUCK0/3sHcwvVQwQJu/5YWvzrzELVJstmPn57rr1ym/wtuWiihpMensjOsaF4vnreyv22VA3Joer3AYpd+ZoAW7NTIMQQqp1UqsAT3NobXkJZqvAzYs3IinSiP/KElBtZn6+G//deQZv3z4QH288gZX78/H6Lf1RVl2DR7/6TRptV1uXY9JQkrbL8tS9r90ni3BZl9oclv1nSvDZltqmoosdi+f1rMNO1+8+VYzJ/96ENycPxLd1+UCe5JUYZN/uhy34BSlRQdJYRGdLveuBZnvAenptb7WUBNjGcJWvY7UKqaa0se9Pfm7769h+Pz7edAKXdojB1uMX3JbHH1pOSYio2djXQtialSpl3+Dl375jQrzLcegUV99sodPampssigk1bV2EbaJD66+xObsQn27OQXGF8+aSCxXOu0DbkpxtIYS827lt7iL7mhFnwoxaKV9oy/FCp4GNzZpD57DrZBGeW34AW44X4pPNJ/DW6mNSYAMAy+vmRLpzmPORW209YMa46EabX5e3I+8GvGxPfZk6yJqJXF2jsdYfOY8vtp5EWF0NgM7Z0MF25AnDxZU1ikEW7YcCsGdfM6OVnSu2CabHkH9W66+jR/+0SACQmghborToYDibY1Y+VlRD4+24Ir8v9s2Qtu8IR/LLFE2laY2YiLSpsOaGqA0y21VhWKwCWo1KMVO2fJyM5KggnC834dbMNNw6OA01FiuSo4JgMlulcTNshneKxaOyLrBSLo1ZQKD+/LaBA22czURcbbEAqP/jbEtGtR/51mZcrwSsO1Ig7SefJXpyZnsM6RDj0UzQWo0aPz00EpfOz3KYUycuzIDxvRLwoWyCQ/l1iitrcL5cWTvRPbE2CfjybnFY9cgoBOs1OHG+AlHBOoQYtIgM1uFMUZXTBy0ATOyXhLfXHFOUpdJU+3pM93gpeRqo7Y5848BkdI0Pw23vbcamY7VJ3o+O64rOcaFIigxCWkwwfs8tRXm1Gc/+sN9h4MRuCWFSbx4AKCivlnKhRtbVHDXENreSt+LDjdjyt9EwWWo/W/LPaq/kcDdHNk6X+DCsffRyFFaYoFbVNoN1igtFkE4jzU7eUsWFG7H+iSuw8eh5zPpit1T7aWsmMurUHo+fY++BUZ0wsks7hBt1SLf7IjLnD91x67ubYbJYFZ99Z7lY/sLghqgBtu6OgcRsV3NTbrIg3KhS1OjUWKzSe7cFC+N7Jbgd2A8ABmdEK4bO18n+4Nq+0fdIDHc6vH6oQasYwbbGImC2WGEVtd/gbX9Ig1xUtQfXra+ssaDabJFqn2yjpnbw4o9vVIgeIzq3w88HlBMkpseEIMquJkueWGkVwmF+K3l3XFuNlX0vGVeBDQCE1g1saJsLySx7qPS2+3lo1Cr0TKpdd3nXOCm4iQszKGaRHlyX/LpkfbZDcDO8U6wiuDFbhPQePe2GHROqb1RwA9Q+tJ31vPHFVBjOpMUEIy3GMeht6LPeEiRGBEnDLJjqfmdtP6sEL3tiyanVKkVOnFyYofb3qcJkRnl1/e9kS/o7yeCGyI3c4kpMfGM9ru2XhDl/6OHv4viMrZuwTd+n/w9d48MU3Zpzi6tw6fwsXN07UXrQefJwsW93tz3Yp32wVWoykfe4sD9WPgzNP3/8Hd/vdmwSctW2b8sv+PVwAbo+taLB/Rvi7DijXuOQx/BX2XQS/9nkLMHz4v7oy+cTGt8rEfd+vE1KAnb33hTzKbkICMOdDJdv35Tx3rrsBs9jT94U1RjOgihXn5u2Tv57mTF7ufS6qfKJbJ+5/JJqXP7SasW6lqJllYaohflkUw4KyqoVf9wDgfxbuc3B/FIcl32DP3auHPkl1dI8M0BtN2B78hyWIJ0GA9sru8PaBnEDgLWHa4Mq+2RiG/smD2eBTWSwDr1dfKPsk+L8m7a8S7Y3hneKdchpmH55J1ySHuWy9siZ2Vd3a9T1bWw1Re3CDPjhtzOK3k3ucioGto9CiF6DcKMWfV3cs+mXd0KYQYsrusUhWK9BqEHrdARim9jQixtj5t93DPJoP7Va5VDzoHWWYEKIDNIhOdKxp9IIN2MwXYzU6GCk29V02U+f4W+suSFyo8LJ2CaBwNUQ8/bjVsiFGbQOzTEA8NJNffGP63pBrVJBQDi08T8yriuKKk34z6YcKflQp3X+kFp4cz88c20vDFvwizSfjVy/1Eh8ed8QlzVIqdHB6BIfKg2GZ/PQmMZ1k/7T4DRc2y8ZZqsVIXotrEJAW3ftXfOuRGG5CUPm/+L02LgwA7Ievgwheq0iwbYxbMmtVTUWhx5fA+3GMZHrlRyBnXPHQqVyXevWPTEcu+aNhUatQrXZAhVU0GvVeOmmvnjEbnTlVyb1ld6/p1KigpD18GWwWAWMWo1X9+LjuwZLk0LqNeoW1ezRkqjVKiyfMQJ9n/4/ad28iT2kLuO+ZtRpkPXwKDz7w358sOE4AOCSdO/mKWtqDG6IApAQAofPliE+3Oh0lt6j55wPyudqPQC3sxY3VP0dHVLby8UWsLhr3go1aBFm1DoNboJkUzq44qzpwtMZl52pPbb2eLWsecmg1SiGvrdn1GkQZmxcTxWHMtTd39IqM3IKlQPONXTvPWnKseU/yQNTZ72ighuY1NQZrVrV6KRW+XvzpJdWWxZs9xmPaYKeZXIatQrhstngW1qXejZLEbmxt25YccD9g7+l+XDDcYx9ZS1Gv7zaodv31uOF+NVuviWb82666bqr1WmIbdwTW3NKQ80Lrh7IG4+5HgBPOtZJ8OOqGexiuTtvarTvBjSTN4FtsZv+wpvmMW8YnAWJjbiWs9o+T8mDUh3zbdzSadSK36um+lzIGWU/n+a4njf4aSFyQz4Wy3c7T7vZs2U5WNcsU1BmQoldDYh8oL5+qZEen7PSg9mSXam2O7akgUDptsz2CDM2rmJ52rAMhzF0vJ//2jMqlQr3jKit+rfPeXj8qovLs5GLDNa57JLsrvboYgzrFIvBGdHQaVRoHxOMzIxot01g9pZMHYRB7aPwys39Gl2GmBA9bhiQjJSoIEwdmt7o87QVdw3PQEpUEC5Jj8Lg9IangrhYY3vEo2dSODrEhng0flRzYrMUkYcu5uHe3CyyeWPsy23rHn1N3yS8fkt/af3g5372evRYT3W06+Y8opP7RMc7h2cgKTLI5aSP7kzsm4SJfZPwx7c2YFvdbMXOJtf0lScn9MCTE5q2J51KpcJrf+qHMQvXOt3WFMKMOnzx5yGNPv6KbvG4opvrqRo8oVKpsPAigqO2ZvbV3R3Gj2pKneLCsOyvI5rtet5gzQ2RCwVl1fh8W/1MyB9uPAGrp+P3X4SSqhpct2g9vtlxCrM+3yU1h/12qggHcj3rXiufWbuqxoq1h87hZF2uhm0UX4ch1ZuwWtk+58KTa+ldJB17Sp6b09LyARqjqcZ4IQpErLkhcmHcK8pvySazFasOnsXo7hf3bbQhff5e2+Nh18kiAMA3O09j97yxuOZf66FWAQf/Mb7BB518Zu3Pt+bg3V+zkREbglWPjMLx87XdvVV2Y6/Im4FGdmmHtYfOIT0mGFU1VuSVVElTBDRGuF0TU6gHTU4RQY7NLX29aEYLMdQHNM5GP25tnP3MnSWLExGDGyKXnCXXrtyf3+TBjdOy1I1sZxVARbUFEcHug5sa2TD9thFqs+vGsLE9EGusykTjR8Z1xWebczCgfVRtk1XWYdwyOA3lJjM+2ZTT4EzC7lySEY2pQ9Nx6kIlokN0uKpnw/P1DEiLREyIHufLTVg5cySWrD/uVd7F/aM6QqdRo31MCLolOI7P09rIg5tHx3VFbnElJvROcnMEUdvF4IbICxc7XkljyWfxrjJbEAH339hNdtMoyNl6LHWxG5Dv8q5xuFw2eNuCG/tIr4d2vLjBwHQaNf5+TU+vjlGpVNg+50ppef4Nvd3s7Whg+2iHAQVbM3nPrBGdY10OjU9EzLkhcspVbo2mCQcRK62qwaN2g6bZPLf8gPS6rNqM7IJyvLn6iGKWaLk9p+q7sMtHI/56+yl8veMUAMDIrrWtirx7vJqD2RG5xZobIifWHDrndL0nuSKNtey3XHy5/ZTTbfK5oA7mleIvn+2ExSpwtqTaaY2Iqx40D8uCJ29HmiX/kgc3F5P/RNQWMLghcsKWdAsAj13VFYt+OYJykwXRwU0zpghQ20vKE0LUN1NtPV7odB9PJrG7tEOM54Ujv9OoVfjk7kyUVZsdZhQnIiUGN0QNGNm5HY4XlOOLbacUuSy+Ju++7X6/+jK4ap2wz7NxJjyIv/6tzbAGxgciolqslyZyYo9s2gX5pIMv/nQQG444n7rAU+uPFGDG0p24IOuNdfRcGV786aBHxz8vy79xlXvhSaDU0oZLJyLyFQY3RE5U19TXfCSEG2GSda2e/tnOizr35H9vxne7ziiClPmy1w2RjyLsKq1UXl5nQvSaRk2CSETUGvCvG5ETom42ohsHpCAm1ICJfZOkZN9CN5NLeuOEbHZneU2RXqvGkimXwGy1omO7UEx6eyPOFFc5PYf9HEo2DTWfrXhopDQTNBFRoGHNDZETtvmXMjNqx0lpil5S8rFr5F3M/zyyA4Z3jsWornFIjQ7Gdf2TXZ7DWY8nIYTbnJvIYB1So4MbWWoiopaPNTcUcIQQuPfj2gkX/3ljH9z67ia0CzPgg2mDPaqtEEJgVV3Xa0NdryOj1jf5KZuOnZdebz9xAd/sOIUbBqQoamb0Gs/nfNLK3s/ba45i/o+/N1gG29xSRESBijU3FHAuVNRg5f58rNyfj5/35+P3vFL8ergApy9UenR8hal+Fu2O7Wpns06L8U1Nx+3vbVYsL1p1BFV2s3aX2Q3MZ3YzWadBNvaJJ4ENEVFbwOCGAo5ZNmeSPFCoMluc7e6gUhZs9EgMB1A78eJ9l3WU1jd2dnD7XkyVJovUBGbTve6aNgPbR7k8n0W4L8evj13uZQmJiFo/BjcUcMyyAKK0Shbc1HgW3Nj2M2jVirmk5DMw2086KVdjseLI2TJUmMwNXlMAyLVLFrbv3W1wM01CpckKk9mq6FYulxQZ5Pb6RESBiDk3FHDkybRniuqbonKLq9AnpeHjL5TX5qTYBxXB+vrcF5PZCoOTPBwhBDo/+aO0HB2ix5pHRyHM6Hyiy9ziKlz9+q+KdfZj17jLE/p6R/1cUfb0WjV7RBFRm8SaGwo48qafClnNiaeP+eq65quSKmXuyx8H1kdGrgbJs9g1VxWWm5Aj6/LdEJ1Ghcu6tlOs650cgV7J4RjbIx4jOns+Qu3DV3YBANwwIBlRwTqE1AVnXe1mAyciCjSsuaGAI8+5qZDl3LhLzJWzjRHTOS5UsT7EoIVGrYLF6rqrtbPxZcweTqtwbb8kvPan/g7rjToNfvjLCGk5/YllHp3vz3U5Qgtv7ufR/kREgYI1NxRwXOXc2AckVqvAnlPFitF8hRD4bucZAM7nbdJpale6GgH4lJMeWbZgSzSQ/Our7uZERG0dgxsKOF9sOym9/u10kfTavgZlyfpsTPzXOsxYWj+dwqs/H8bndccfyi9zOLctH2ZLtuNs3EIIjH1lrcN6WxPWOdm0Cc5sPeF8hm8iIvIOgxsKOB9tPCG9Dpcl8trX3Lyz9hgA4Me9edK617IOuz23bQycCie9oKpd1ObYrnuuzH1wc+xcudvtNi/8sQ/6pES43eefN/b26FxERIGIwQ0FNEWzlF3Ojc7J1AUNua5fEgCg2klw46rbt63GyNaUlRIVhN1zxzrs52nPppsHpeL76cNdbt8x50pMuiTNo3MREQUiBjfUKm08eh6rD55tcD/5gHxmWc3N73klOC3rJp51IN+j69qmQvjHsgO4/b3NKKqoH1/mZKHzEZBtNTcH80oB1AYxaie/efY9rRpLq2H3byJq2xjcUKtjMltxy7ubMPX9rV7N0C3PuZn2/lbFtrs+3IY3Vx9p8BzysW9+PVyA+/+zo/6cH2x1dojUS+uJb/YAAE6cr3BaazS6W1yD1/eEzlnkRETUhvCvILU68mkULlQ4BjdDOsQ4PU4+qrD9qMAA8PnWk4rllTNHOuxz40DlKIAbZRNhFrjIqXHWbdyo0+D1W/qjV3I4hnWKQVSwDm/eNsDp8d5izQ0RtXUc54ZanRpZ4q6zphwB5807DY03U2OXENzZyWB3qVGeT6A5onMsfj1c4PK61/RNwjV9kzw+nz2VCnDWu1zLUYmJqI1jcEOtihACzy0/IC2XVNY47OMqmPjf7jM4dq4M64+ed7r9jJPaHHvOakVKqmoUvbJsrHWRx+dbTyLU2Dy/alq1CipnA/QQEbUhbJaiVmXfmRJ8s+O0tCxPCrax7xVlc/hsGf6760yD480AwB/6JDpd7yxX5qa3Njqse2BUR6w/UhtEbTleiD9/vF3a1jvZfTduT90y2LFHVLswg0/OTUTUmrHmhlqVsmpzg/uYXUyN4I1XJvVzut5Zk8/B/NpeUAPSIrEjpwjDOsVg+hWd8Obqo07PsWTqJRddPgCY+4ceGJgWBZPFinahBpwprsSlLvKNiIjaEgY31KrYz5jtbAJLT+dycsfVGDjuxqKxXXba0AwE613/avmqdsWo0zgkOBMREYMb8qMaixVDF/wiNRPdMjgN829wP7KufQLxtuOFitm6AWWvKF9zlc9y0+IN2H2yCACg07K1l4jIn/hXmPxm+Z5cRf7LZ1tyGjzGvlv1Urvu28DF19yM6R7v9TFbj1+QXpfXNZ3dOSzDYb8bB7CmhYioqTG4Ib9xNV2BO87GjLHnTc7Nl/cNUSzfMCAZ794x0KsyJUcGKZZjQvQAgNsudUz4femmPl6dm4iIvMfghvzGWROPtYEpCBoKboQQMHlRc3NJerRieWTndl53pS6pcuyODjjP22E3bSKipsfghvzmnz/+7rCuw9+W46vtp1wec/x8hcttucWVyJi93OVIwa7oZTkyrgYAdEc+OWftORzPS0REzYd/fclvzruYF+qRL3e7PCZYr1EsRwbXD573woqDTo+ZnOl+huwX/1jfVDSuZ4LbfQE4JDDbG5AWBaC+eYqIiJoXe0tRq2KymyJhWMdY6bV9k9XR56+GyWxFkF6DJyd0R4+5Pzk957X9kjGhdyLUKhXUHkxd8NJNfTH/ht4orqzBoH/8rNh2aYdoqcZGq1Hj8q7tsOrgOY/eGxER+QaDG2oVhBDIK6lCtV1wY7Y6n2dKo1ZBo1YhqK6mJ0inrPGxp3Uxro0rOo1aMUO4jV6rvA5zbIiImh+DG/KLhhKHLVahGDDvzdVH8eJPjs1O8m7frgbeA5omyHB2vVi7pqh1Rwqk1xFBjvNPERGR7zHnhvyiyuy+G3ilXTdxZ4ENAJhlQdLQju6nHrjvso4Y0z0Oqx4ZhUvSo/DuHYM8LK1zzqZieOLqbopleTPaa3/qd1HXIyIiz7DmhvxCPm3CUxO6IykyCA98ssPr88ibohqoDMIT4+sDjy/vG+r1tezZT8XQJyUCcWFGl/uP6hp30dckIqKGseaG/OJsSZX0WqVSOcwZZRWedcmWJxHnFLruJt4U7Ju6fDGnFRERXTwGN+QXV76yVnqdFGF0GK3Y4mGgIK+5Wbymfhbu9jHBF1lC73WOD232axIRkSMGN+R3V/aIl3o12dj3irL3t7rcFrOLtqj517ufgLMpPDK2q8O6dY9fDgB4dJzjNiIiahrMuSG/02rU0Ng18TQ071Tn+DAAjrOE22R2cJ9c7GuPXdUVqdGOtUUpUcE4vmBCs5aFiKitY3BDLVKFyYLCchNmLN3pkI8D1PdU8mQizeagAsezISJqKRjcUIuQZpcjk19ahf/9dga/Hi5w2LdjuxCpp5KrmpvmEhWsw4WKGvRKDvdrOYiIqJ7fc24WLVqE9PR0GI1GZGZmYsuWLW73f/XVV9G1a1cEBQUhNTUVM2fORFVVldtjqOXrEh+GtyYPkJZVAE666P30/tTB0KprP7r+Dm4+visTL/6xD4Z3im14ZyIiahZ+DW4+//xzzJo1C/PmzcOOHTvQt29fjBs3DmfPnnW6/6effoonnngC8+bNw4EDB/Dee+/h888/x9/+9rdmLjk1hfG9E9EvNRJA7Tg4zrqDv3xTX6TFBEOrqWuWsvq3WapXcgRuGpTKaRaIiFoQvwY3CxcuxD333INp06ahR48eWLx4MYKDg7FkyRKn+2/YsAHDhg3DrbfeivT0dIwdOxa33HKL29qe6upqlJSUKP6R71XVWPDwF7uxfE/uRZ3n1IVKAMBvp4qwfE+ew3ZbTY0t58bTLuNERNR2+C24MZlM2L59O8aMGVNfGLUaY8aMwcaNG50eM3ToUGzfvl0KZo4dO4bly5fj6quvdnmd+fPnIyIiQvqXmprq2zdCAICPN57A1ztOeT3K8NW9ExTLBWXVAIA3fjnidP+OcSEA6kcHdtYV/E+X8GdMRNSW+S24KSgogMViQXx8vGJ9fHw88vIcv7EDwK233opnnnkGw4cPh06nQ8eOHTFq1Ci3zVKzZ89GcXGx9O/kyZM+fR9UK7+kcXlPT03o4dX+A9KiAEDKuZEHN/q6WbrvHpHRqLIQEVFg8HtCsTdWr16N559/Hm+++SZ27NiBb775BsuWLcOzzz7r8hiDwYDw8HDFP/I9b/J65TOCG3UaN3sq9UgMl3JbbDk3ZllXcFtTlbvZwYmIKPD5rSt4bGwsNBoN8vPzFevz8/ORkJDg9Jg5c+bg9ttvx9133w0A6N27N8rLy3HvvffiySefhFrNh5q3ThdVYtiCXwAAW54c7XbiR3fczQU177u92JFThKX3XooQgxaPfLlb2mYLUjwRLBvF2BbIlFSZIYSASqVChclSd05+DoiI2jK/PQX0ej0GDhyIrKwsaZ3VakVWVhaGDBni9JiKigqHAEajqX3gCQ8nWiSl55cdkF5/sP54o8/j7v5/uPEE9pwuxv/tr21u/GbnaWmbzu7nGW50HW/fOby+uSk6RC+9Lq02K0Y0DtVz+CYiorbMr19xZ82ahXfffRcffvghDhw4gPvvvx/l5eWYNm0aAOCOO+7A7Nmzpf0nTpyIt956C0uXLkV2djZWrlyJOXPmYOLEiVKQQ96pMJml18WVNY0+jyehZYXJ4jAujX3NzcKb+7k8/opucdLrUEN9AFNjtsIka56yn6eKiIjaFr9+xZ00aRLOnTuHuXPnIi8vD/369cOKFSukJOOcnBxFTc1TTz0FlUqFp556CqdPn0a7du0wceJEPPfcc/56CwHlYoZqkVfcmC1Wp01DFqtwmC7B1rwkLbtppjJo68+pUqmgVatgtgrUWASA+vPqvGjqIiKiwOP3+vvp06dj+vTpTretXr1asazVajFv3jzMmzevGUrWNpRUmRveyQOHz5ZKrzs9+aM0WaQ8eTi7oNxhtm/7we/cJQPb72vrKXXkbJnURVynUXFAPSKiNo6Zl23c9hMXpNcXM/ljQrgyEdnW/CTvqh0ZpG9wokv7mpyUqKAGr/3eumOoMddehz2liIjI7zU31HKofVjhUWOxQqPWwCybHkHAsVnKnn1z1pw/9MC4ns57z9mYLPU5NwxuiIiIT4I27MT5csVyzUVMQmnfFGQLNuQ1N4fyS1FebYE79vkyeg+ClQO5pVLQpNfyI01E1NbxSdBGFZRV47IXVyvWfbo5x2fnr6nLrTHL5n5avicPN7/tfGoNG61d13BPamIKy01S85ova5+IiKh1YnDTRh3KL214p4tQY7Hl3CiboQrLTdLrh6/s4nCcfc2Npz2fbBNu5pdUe1VOIiIKPAxu2ihNE/cosjUTmd3M2v2X0Z0d1tnv7a6ZST7gX0lV7Rg9IzrHelFKIiIKRAxu2ihns2n7km0qBHfBjTP2+3uaIPzjnlwA3s1VRUREgYnBTRu1aNURn57PfvqF3aeKADg2SzUkPEjZgS/MzXQMw2W1NBcqamtuIoJ0Xl2PiIgCD4ObNmrD0fM+PZ99TZDKxXqbe0d2cLo+JSoYt2amAQAeHdcV7WNCXF7zyQk9HNY9MKqjB6UlIqJAxnFuyCfsm5NsQY2rZqmeSeEuz/X89b3x/PW9G7xmcmQQBqdHY8vxQgBAmEGLDu1CPS0yEREFKNbckIJ8dm0A2Hu6GOlPLMOKvbluj7OvoZn9zR6MWbgGV7/+q9P97bt8N5ZOW58YbWC+DRERgcEN2Xl++QHF8h/eWAcAuO8/O9we5yy35sjZMpf7u5sg0xuF5fUzmV+oMLnZk4iI2goGN6Sw+1Rxo47ztleUr2buHt4pRnod7ib5mIiI2g4GN6RQXeN+egRXvO0V5atmqXjZhJ1p0cE+OScREbVu/KoboL7afgqfb83BSzf1ddvjyJ5t5OLtJy7gxrc2NLj/zW9vxJbsQq/L56tmKfks4hrOvUBERGDNTcB65Mvd2Hr8Al7Pang8m2B9fSLuNX2TAMCjwMZiFY0KbADf1dzIZxG3n1GciIjaJj4NAtzpogqn6zu0q63NeWBUR+z9+zhc2y+pbr3nXaltUyzIheg967EUG6r3+DruyHN3fJXHQ0RErRuDmwBntQJWJwPp2RKAR3ePh1qtkkb2NTsJWFxxFtx42jTkq2kS5DVAGh/VBhERUevGp0GA23K8EB3+thwb7UYktgUxttoOW5BQ48WcU856SHnaNOSr4EYeTJVXm31yTiIiat0Y3LQRT367R7FsC2JsQY0tyPGq5sZJDyl3NTe2rtp9UiIQ6aM5oE4XVUqvt5+44JNzEhFR68beUm3E+XLlAHcONTd1/9e4GK/GqHOMg53W3LgJbnbNHSu9VvuoZxN7SBERkT3W3LQRxZU1yDlfn1xsC0xszUi2Gpytxwud5uhU1TjW0jhvlnIdbKjVKumfr7gLpoiIqG1icNOGjHxxFU4W1gY4tiYlW3CgVtX+v+9MCZ7+3z6nxy/dkqNY3nTMcWbxBNmges0h3Oib5i0iIgocDG7amJ/25QGor3XR1dXcVJnrRyb+cOMJp8fO+W6vYvm9ddkO+/x1dGf0To4AAMSE1Hf3/uaBoRdRatcm9ElskvMSEVHrxZybNqaqxgIhhDSLt60ZSedB8459Po6zhOKu8WH431+G+6Ckngkx8CNMRERKrLkJML+dKkLPuStcbv/l97M4U1wlLevqcm28Hd3XYhU4dq7cYT0TfImIyN8Y3ASYa/61HuUm15Nf7sgpwrAFv0jLhrpeUBFeds3+dLPzpit/TIFgG205JSqo2a9NREQtD4ObNs42mJ5t+gV7fVMjna5ftifX6Xp/TIHw4bTBmJyZho/vymz2axMRUcvD4CbA6bWuf8TdEsKk15HBejx7bU+Hfd6+baBX1/NHs1RqdDCeu743MmI9n/2ciIgCF4ObVmzZb7noPe8nnLpQ2717+qc7HPZxF2vYD+znLBDytiZGx/mdiIjIz/gkasUe/HQHSqvNGP7PVdh7uhg//ObYVBTvZtyZc6XViuVOcWEO+7iq+XFVQ+PLAfqIiIgag8FNgDgjm2NJLjHC80H1BraPclin06gx5w89AADJkfUJu/1c5OIQERH5G4ObAOFqMm9vc2CS7IIhnUaNYZ1iANSOkSNfT0RE1BJxBLQA8dupIqfrhYugxxWjXqNY1qhVCKrrUXW+3IRdJ4vw455cvL32WGOKSURE1OT49TtAvLn6qNP1Nw5I8eo8t1/a3mFdlGwahesWrWdgQ0RELRqDm1bK4qodys4NA5LxzQNDMbcub6YhtwxOk15PG5YOAAjjFAdERNSK8KnVStVYHOd1ckalUmFAWpTUtCTnbFRig6x3VGyoQToHERFRa8HgppUxma3o8tSPXh9ndZJ84ywpWB7IVLqZxoGIiKilYrNUK7M5+7zH+/ZPi5Red2wX6rD94bFd3B4v7x3lztPX1I5s/NSE7h6XjYiIqKmw5qaVMZk9a45a/8QVim7dRp0Ge58eh17zfpLWXdohxuvrhxm1KK0yS8vPX98bt2am4YYByQgzejf5JhERUVNgcNPKeJhHrBhwzybULjHY3bxTrq4lD2yA+rwdBjZERNRSsFmqlXGWO9NYugYG+PNkXimtH2YBJyIicofBTSsjPAhu/jq6s8ttE3onSq9djV782FVd0SkuFH++rKPb67SPCcaANMcpG4iIiPyJzVKtTEOxzcqZI9E53nECTJu/X9MTy/bUTrDpqov3A6M64YFRnRTr+qZEYPepYmk5LsyANY9e7mGpiYiImg9rbloZSwPRTUN5NI1lP9s355YiIqKWik+oVub33FK32w1ax8H65OR5NN7ky2TEhiiWmyqIIiIiulhslmplgg3ug5f4cIPb7ZHBeky/vBNUKiDcix5Odw3PwDc7TkvLniQbExER+QODm1amqsb9ODeeTJXwyLiuXl83WjZ5JsCaGyIiarn4hGplsg7k++W69j2rmHNDREQtFZ9Qrcy+MyV+ua5WrfyoeDpxJxERUXNjcNPKbZo9ulmuY19zkxThOAIyERFRS8DgppVLiDDi5kEpTX4drV1wM7A9B+8jIqKWicFNAHA10nBTXsOoc99ri4iIyF8Y3ASAkZ3bAWjaIMe+5iY+3OhiTyIiIv9iV/BW6onx3TC+VwIA4KpeCVgydRC6J4Y32fW0GjWW/XU41h0uQEpUMMZ0j2uyaxEREV0MBjetTEyIHufLTRjVtR3ax9SOGqxSqXBFt/gmv3bPpAj0TIpo8usQERFdDDZLtSJCCJwvNwFw7JpNREREtfiEbEVOnK+QXnP6AyIiIucY3LQi1eb6gfPYW4mIiMg5BjetiNnKUYGJiIgawuCmFTFbhL+LQERE1OIxuGlFgvX1TVHtQg1+LAkREVHLxeCmFbGI2pqb6BA91M0wKjEREVFrxOCmFbE1S7GnFBERkWsMbloRs7U2uOEYN0RERK75/Sm5aNEipKenw2g0IjMzE1u2bHG7f1FRER588EEkJibCYDCgS5cuWL58eTOV1r/MltreUlrW3BAREbnUqOkXysvLsWDBAmRlZeHs2bOw2nVRPnbsmEfn+fzzzzFr1iwsXrwYmZmZePXVVzFu3DgcPHgQcXGOcxeZTCZceeWViIuLw1dffYXk5GScOHECkZGRjXkbrU6NxVZzw+CGiIjIlUYFN3fffTfWrFmD22+/HYmJiVCpGvewXbhwIe655x5MmzYNALB48WIsW7YMS5YswRNPPOGw/5IlS1BYWIgNGzZAp9MBANLT0xt17dZmw9ECvLO2NmiUD+ZHRERESo0Kbn788UcsW7YMw4YNa/SFTSYTtm/fjtmzZ0vr1Go1xowZg40bNzo95vvvv8eQIUPw4IMP4rvvvkO7du1w66234vHHH4dG43zE3urqalRXV0vLJSUljS6zvxw5W4pb390sLZ+6UOnH0hAREbVsjcq5iYqKQnR09EVduKCgABaLBfHxytms4+PjkZeX5/SYY8eO4auvvoLFYsHy5csxZ84cvPzyy/jHP/7h8jrz589HRESE9C81NfWiyu0Pv50q9ncRiIiIWo1GBTfPPvss5s6di4qKioZ39iGr1Yq4uDi88847GDhwICZNmoQnn3wSixcvdnnM7NmzUVxcLP07efJkM5bYNzgyMRERkeca1Sz18ssv4+jRo4iPj0d6erqU/2KzY8eOBs8RGxsLjUaD/Px8xfr8/HwkJCQ4PSYxMRE6nU7RBNW9e3fk5eXBZDJBr9c7HGMwGGAwtO7RfAsrTP4uAhERUavRqODmuuuuu+gL6/V6DBw4EFlZWdL5rFYrsrKyMH36dKfHDBs2DJ9++imsVivUdWO9HDp0CImJiU4Dm0Bx4nzz1pARERG1Zl4HN2azGSqVCnfeeSdSUlIu6uKzZs3ClClTMGjQIAwePBivvvoqysvLpd5Td9xxB5KTkzF//nwAwP33349//etfmDFjBv7yl7/g8OHDeP755/HXv/71osrR0hl1ytbDju1C/FQSIiKils/r4Ear1eLFF1/EHXfccdEXnzRpEs6dO4e5c+ciLy8P/fr1w4oVK6Qk45ycHKmGBgBSU1Px008/YebMmejTpw+Sk5MxY8YMPP744xddlpZM2KXcZMQyuCEiInKlUc1SV1xxBdasWeOTMWamT5/ushlq9erVDuuGDBmCTZs2XfR1WxOLVRndlFSZ/VQSIiKilq9Rwc348ePxxBNPYM+ePRg4cCBCQpQ1Cddcc41PCke1NHYjEheUVbvYk4iIiBoV3DzwwAMAakcYtqdSqWCxWC6uVORWkM75gIVERETUyODGfi4palomi/J+WznsDRERkUt+nxWc3LNYBT7dnKNYJ+wzjImIiEjSqJqbZ555xu32uXPnNqow5Ci7oMxh3d+u7u6HkhAREbUOjQpuvv32W8VyTU0NsrOzodVq0bFjRwY3PiRvgnrm2p64vn8ywow61wcQERG1cY0Kbnbu3OmwrqSkBFOnTsX1119/0YWiejWyfJtQg5aBDRERUQN8lnMTHh6Op59+GnPmzPHVKQnA6oPnpNf2XcKJiIjIkU8Tim0zb5PvvPjTQen1uJ7OJxQlIiKieo1qlnr99dcVy0II5Obm4uOPP8b48eN9UjByZOT4NkRERA1qVHDzyiuvKJbVajXatWuHKVOmYPbs2T4pGBEREVFjNCq4yc7O9nU5iIiIiHyiUTk3d955J0pLSx3Wl5eX484777zoQlG9SYNSAQAd23EmcCIiIk80Krj58MMPUVlZ6bC+srISH3300UUXiurptbU/oj/0SfJzSYiIiFoHr5qlSkpKIISAEAKlpaUwGo3SNovFguXLlyMuLs7nhWzLNhwtAMBu4ERERJ7yKriJjIyESqWCSqVCly5dHLarVCo8/fTTPiscAUfPlQMA/rPpBP46urOfS0NERNTyeRXcrFq1CkIIXHHFFfj6668RHR0tbdPr9Wjfvj2Skth80hTOllb7uwhEREStglfBzWWXXQagtrdUWloaVCo2lRAREVHL0qiE4vbt22PdunW47bbbMHToUJw+fRoA8PHHH2PdunU+LSDV0jLnhoiIyCONCm6+/vprjBs3DkFBQdixYweqq2ubTIqLi/H888/7tIBUa3zvRH8XgYiIqFVoVHDzj3/8A4sXL8a7774Lna5+luphw4Zhx44dPiscAcH62ikXpg5t7+eSEBERtQ6NCm4OHjyIkSNHOqyPiIhAUVHRxZaJZCKCaoNHvYbzShEREXmiUcFNQkICjhw54rB+3bp16NChw0UXiurVWAQAQKthzg0REZEnGhXc3HPPPZgxYwY2b94MlUqFM2fO4JNPPsHDDz+M+++/39dlbNPMVisAQMfghoiIyCONmjjziSeegNVqxejRo1FRUYGRI0fCYDDg0Ucfxd133+3rMrZpZlvNjbpRcSgREVGb06gnpkqlwpNPPonCwkLs3bsXmzZtwrlz5xAREYGMjAxfl7HNEkLAZKmtuWGzFBERkWe8Cm6qq6sxe/ZsDBo0CMOGDcPy5cvRo0cP7Nu3D127dsVrr72GmTNnNlVZ25TiyhpkPp8Fk9nWLMWaGyIiIk941Sw1d+5cvP322xgzZgw2bNiAm266CdOmTcOmTZvw8ssv46abboKGvXp84qvtpxRTLoQYGtWCSERE1OZ49cT88ssv8dFHH+Gaa67B3r170adPH5jNZuzevZtTMTQxJhQTERF5xqu2jlOnTmHgwIEAgF69esFgMGDmzJkMbJqA/WwLGt5jIiIij3gV3FgsFuj1emlZq9UiNDTU54UiQG0XzGg4txQREZFHvGqWEkJg6tSpMBgMAICqqircd999CAkJUez3zTff+K6EbZTaLphh7RgREZFnvApupkyZoli+7bbbfFoYqseKGiIiosbxKrh5//33m6ocZMe+WYqIiIg8w8FTWiiOa0NERNQ4fIK2UOz6TURE1DgMblooi1X4uwhEREStEoObForBDRERUeMwuGmhGNwQERE1DoObFsrM4IaIiKhRGNy0UFbB4IaIiKgxGNy0UD/ty/N3EYiIiFolBjct1Poj56XX3RLC/FgSIiKi1oXBTSvw9f1D/V0EIiKiVoPBTSsQYvBqlgwiIqI2jcENERERBRQGNy3c2B7x/i4CERFRq8LgpoVKjQ4CANw3qqOfS0JERNS6MLhpoSqqLQAAnZo/IiIiIm/wydkCFVfU4Hy5CQCgUXN2cCIiIm8wuGmBsn7Pl17rNAxuiIiIvMHgpoVjzQ0REZF3GNy0QCpZPKPT8EdERETkDT45WyAV6qMbLZuliIiIvMLgpoVjsxQREZF3GNy0QFYhpNdadgUnIiLyCictakF+zyvBwbxSmC31wQ3rbYiIiLzD4KYFueO9LThbWo0bB6RI6/Ra1twQERF5g0/OFuRsaTUA4PDZUmkdZwQnIiLyDoObFshirW2WysyI9nNJiIiIWh8GNy3QvjMlANgNnIiIqDEY3LRg5+qaqYiIiMhzDG5asG4J4f4uAhERUavD4KYFS4gw+rsIRERErQ6DmxaitKrGYZ2WoxMTERF5rUUEN4sWLUJ6ejqMRiMyMzOxZcsWj45bunQpVCoVrrvuuqYtYDN4Z+0xh3VaTppJRETkNb8/PT///HPMmjUL8+bNw44dO9C3b1+MGzcOZ8+edXvc8ePH8cgjj2DEiBHNVNKmVVhuclinY80NERGR1/we3CxcuBD33HMPpk2bhh49emDx4sUIDg7GkiVLXB5jsVgwefJkPP300+jQoUMzlrbpWIXjOh1HJyYiIvKaX5+eJpMJ27dvx5gxY6R1arUaY8aMwcaNG10e98wzzyAuLg533XVXg9eorq5GSUmJ4l9LtO9MscM65twQERF5z6/BTUFBASwWC+Lj4xXr4+PjkZeX5/SYdevW4b333sO7777r0TXmz5+PiIgI6V9qaupFl7spOBuNWMecGyIiIq+1qqdnaWkpbr/9drz77ruIjY316JjZs2ejuLhY+nfy5MkmLmXjqJ3U0qhYcUNEROQ1v87KGBsbC41Gg/z8fMX6/Px8JCQkOOx/9OhRHD9+HBMnTpTWWa1WAIBWq8XBgwfRsWNHxTEGgwEGg6EJSu87ecVVOFfC0YiJiIh8wa/BjV6vx8CBA5GVlSV157ZarcjKysL06dMd9u/WrRv27NmjWPfUU0+htLQUr732WottcnLnyNlSjH1lrdOE4pzzFc1fICIiolbOr8ENAMyaNQtTpkzBoEGDMHjwYLz66qsoLy/HtGnTAAB33HEHkpOTMX/+fBiNRvTq1UtxfGRkJAA4rG8ttmRfcBrYAMCyPbl46g89mrdARERErZzfg5tJkybh3LlzmDt3LvLy8tCvXz+sWLFCSjLOycmBWt2qUoO8Yq5rVnMmNrRlN6cRERG1RCohhIt6g8BUUlKCiIgIFBcXIzzc/xNTvr3mKOb/+LvTbX1SIvD99OHNXCIiIqKWx5vnd+BWibQSpy5UutzWMymiGUtCREQUGBjc+FlksM7ltmeu7dmMJSEiIgoMfs+5aetMFmXOzfX9k/HKpH7+KQwREVEAYM2NH5VVm/H2GuVs4JxygYiI6OIwuPGj55cfcFjHUYmJiIguDoMbP9p09Ly/i0BERBRwGNz4kbM++Cqw6oaIiOhiMLjxo+yCcod1bJYiIiK6OAxuWpirejlOGEpERESeY3DTwlzWpZ2/i0BERNSqMbhpYVRslyIiIrooDG6IiIgooDC4aUGSI4P8XQQiIqJWj9MvtBAbnrgCCeFGfxeDiIio1WNw00IksdaGiIjIJ9gsRURERAGFwU0LcEl6lL+LQEREFDAY3PiJEPWTL7x120A/loSIiCiwMLjxE7O1PrjRqfljICIi8hU+Vf3EbKkPbrQaDtxHRETkKwxu/MRksUqvGdwQERH5DoMbPzHLghs2SxEREfkOn6p+UiNrllKrWXNDRETkKwxu/OTbnaf9XQQiIqKAxODGTzYdO+/vIhAREQUkBjd+YrZaG96JiIiIvMbgxk/2nCr2dxGIiIgCEoMbPympMvu7CERERAGJwQ0REREFFAY3REREFFAY3PhBXnGVv4tAREQUsBjc+MHfv98nvY4O0fuxJERERIGHwY0fnC+vll4/d10vP5aEiIgo8DC4aQZWq4DVWj/dgqh/CaNe44cSERERBS6tvwsQ6ArLTRjw7EoAwP/NHAkA2HbigrRdy3mliIiIfIrBTRPbkl0ovV68+ijySpTJxDUWjlRMRETkS2yWamIqWcVMUWWNw/aIICYUExER+RKDmyYmz68xWwW0Gt5yIiKipsRmqSYmnyBTq1ZByKMdADoNc26IiIh8idUITUyeUzMoPQoGrbJ3VO/kiOYuEhERUUBjcNPEaszKLuChxvrKMp1GBZWKNTdERES+xOCmieXKplqosVgVXb91zL8hIiLyOT5dm9grPx+SXlusAnpt/S1nnQ0REZHvMbhpRjUWgSDZiMQaDuBHRETkc+wt1YwsVquia3h6bIj/CkNERBSgGNw0oxqLwMebTkjLTCYmIiLyPTZLNaO1h8/BIptA87Iu7fxYGiIiosDE4KaJdUsIk14fO1cuvU6ODMKDl3f0R5GIiIgCGoObJhZicN7yd/uQ9g4D+hEREdHFY85NE/l443EcKyiH2cWs35UmSzOXiIiIqG1gcNNE5ny3z+32jUfPY+aVzVQYIiKiNoTNUn5itZtAk4iIiHyDwY2fsBc4ERFR02BwQ0RERAGFwU0TEB40Oc26smszlISIiKjtYXDTBOQD9Tlz44AUDOkY00ylISIialsY3DQBcwPBTVSwrplKQkRE1PYwuGkCDfWE0ml524mIiJoKn7JNoKGaG52Gt52IiKip8CnbBCwW98GNXsN+4ERERE2FwU0TsDTQLKVnsxQREVGT4VO2CTTUW0oF1twQERE1FQY3TcBZzo18RGIdm6WIiIiaDIObJmB1Etzcfml76XVKVHBzFoeIiKhNYXDTBMqqzQ7r1LKqGy1rboiIiJoMg5sm8N66bId1WnV9QMOu4ERERE2nRTxlFy1ahPT0dBiNRmRmZmLLli0u93333XcxYsQIREVFISoqCmPGjHG7vz/8b/cZh3UaWW2NPNAhIiIi3/J7cPP5559j1qxZmDdvHnbs2IG+ffti3LhxOHv2rNP9V69ejVtuuQWrVq3Cxo0bkZqairFjx+L06dPNXHLXnPWW0iiapfx+24mIiAKW35+yCxcuxD333INp06ahR48eWLx4MYKDg7FkyRKn+3/yySd44IEH0K9fP3Tr1g3//ve/YbVakZWV5XT/6upqlJSUKP41NWe9pTSKZinW3BARETUVvwY3JpMJ27dvx5gxY6R1arUaY8aMwcaNGz06R0VFBWpqahAdHe10+/z58xERESH9S01N9UnZ3bEfpO+G/slYe+ictKxV+z2mJCIiClh+fcoWFBTAYrEgPj5esT4+Ph55eXkenePxxx9HUlKSIkCSmz17NoqLi6V/J0+evOhyN6RfSiQA4M8jO2DRrQMwb2JPFJSZpO2suSEiImo6Wn8X4GIsWLAAS5cuxerVq2E0Gp3uYzAYYDAYmrVcVWYLAGBwRjRGd68N3OS1Ocy5ISIiajp+DW5iY2Oh0WiQn5+vWJ+fn4+EhAS3x7700ktYsGABfv75Z/Tp06cpi+m1304VAwCMOo20Ti8LaNhbioiIqOn4tQpBr9dj4MCBimRgW3LwkCFDXB73wgsv4Nlnn8WKFSswaNCg5iiqV2JD9QCUQUxEkE56HVO3nYiIiHzP781Ss2bNwpQpUzBo0CAMHjwYr776KsrLyzFt2jQAwB133IHk5GTMnz8fAPDPf/4Tc+fOxaeffor09HQpNyc0NBShoaF+ex9yts5S0SH1QYx8bqlgvd9vOxERUcDy+1N20qRJOHfuHObOnYu8vDz069cPK1askJKMc3JyoJb1LnrrrbdgMpnwxz/+UXGeefPm4e9//3tzFt0ls8UKQNn9W7ifKJyIiIh8xO/BDQBMnz4d06dPd7pt9erViuXjx483fYEukm0QP3mXbwFGN0RERM2B3XaagG0QP/mUC89c2wthRi2evLq7v4pFRETUJrSImptAU19zUx/cdE8Mx+65Y6FmTykiIqImxZobHxNC1Nfc2AUyDGyIiIiaHoMbH5NPmsnxbIiIiJofgxsfk0+aaV9zQ0RERE2PwY2PKWtueHuJiIiaG5++PsaaGyIiIv9icONjzLkhIiLyLwY3Pma21o5OrFKxdxQREZE/MLjxMWdj3BAREVHzYXDjY2aL8zFuiIiIqHkwuPExW82Njj2liIiI/IJPYB+z5dzI55UiIiKi5sPgxsfMzLkhIiLyK06c6UMv/vQ7Fq06CgAoKDP5uTRERERtE2tufOR8WbUU2BAREZH/MLjxkQqTxd9FICIiIjC48ZmqGgY3RERELQGDGx+pqrH6uwhEREQEBjc+U2VW1txEBev8VBIiIqK2jcGNj/ROjsCY7nHS8o45V/qxNERERG0XgxsfMeo0yIgNkZZVKo5zQ0RE5A8MbnwoWM9hg4iIiPyNwY0P/XFgClKjgzB1aLq/i0JERNRmsarBh1Kjg/HrY1f4uxhERERtGmtuiIiIKKAwuCEiIqKAwuCGiIiIAgqDGyIiIgooDG6IiIgooDC4ISIiooDC4IaIiIgCCoMbIiIiCigMboiIiCigMLghIiKigMLghoiIiAIKgxsiIiIKKAxuiIiIKKAwuCEiIqKAovV3AZqbEAIAUFJS4ueSEBERkadsz23bc9ydNhfclJaWAgBSU1P9XBIiIiLyVmlpKSIiItzuoxKehEABxGq14syZMwgLC4NKpWr0eUpKSpCamoqTJ08iPDzchyUke7zXzYf3unnxfjcf3uvm01T3WgiB0tJSJCUlQa12n1XT5mpu1Go1UlJSfHa+8PBw/qI0E97r5sN73bx4v5sP73XzaYp73VCNjQ0TiomIiCigMLghIiKigMLgppEMBgPmzZsHg8Hg76IEPN7r5sN73bx4v5sP73XzaQn3us0lFBMREVFgY80NERERBRQGN0RERBRQGNwQERFRQGFwQ0RERAGFwU0jLVq0COnp6TAajcjMzMSWLVv8XaQWbf78+bjkkksQFhaGuLg4XHfddTh48KBin6qqKjz44IOIiYlBaGgobrzxRuTn5yv2ycnJwYQJExAcHIy4uDg8+uijMJvNin1Wr16NAQMGwGAwoFOnTvjggw+a+u21aAsWLIBKpcJDDz0kreO99p3Tp0/jtttuQ0xMDIKCgtC7d29s27ZN2i6EwNy5c5GYmIigoCCMGTMGhw8fVpyjsLAQkydPRnh4OCIjI3HXXXehrKxMsc9vv/2GESNGwGg0IjU1FS+88EKzvL+WwmKxYM6cOcjIyEBQUBA6duyIZ599VjHPEO9146xduxYTJ05EUlISVCoV/vvf/yq2N+d9/fLLL9GtWzcYjUb07t0by5cvb9ybEuS1pUuXCr1eL5YsWSL27dsn7rnnHhEZGSny8/P9XbQWa9y4ceL9998Xe/fuFbt27RJXX321SEtLE2VlZdI+9913n0hNTRVZWVli27Zt4tJLLxVDhw6VtpvNZtGrVy8xZswYsXPnTrF8+XIRGxsrZs+eLe1z7NgxERwcLGbNmiX2798v3njjDaHRaMSKFSua9f22FFu2bBHp6emiT58+YsaMGdJ63mvfKCwsFO3btxdTp04VmzdvFseOHRM//fSTOHLkiLTPggULREREhPjvf/8rdu/eLa655hqRkZEhKisrpX2uuuoq0bdvX7Fp0ybx66+/ik6dOolbbrlF2l5cXCzi4+PF5MmTxd69e8Vnn30mgoKCxNtvv92s79efnnvuORETEyN++OEHkZ2dLb788ksRGhoqXnvtNWkf3uvGWb58uXjyySfFN998IwCIb7/9VrG9ue7r+vXrhUajES+88ILYv3+/eOqpp4ROpxN79uzx+j0xuGmEwYMHiwcffFBatlgsIikpScyfP9+PpWpdzp49KwCINWvWCCGEKCoqEjqdTnz55ZfSPgcOHBAAxMaNG4UQtb+AarVa5OXlSfu89dZbIjw8XFRXVwshhHjsscdEz549FdeaNGmSGDduXFO/pRantLRUdO7cWaxcuVJcdtllUnDDe+07jz/+uBg+fLjL7VarVSQkJIgXX3xRWldUVCQMBoP47LPPhBBC7N+/XwAQW7dulfb58ccfhUqlEqdPnxZCCPHmm2+KqKgo6d7brt21a1dfv6UWa8KECeLOO+9UrLvhhhvE5MmThRC8175iH9w05329+eabxYQJExTlyczMFH/+85+9fh9slvKSyWTC9u3bMWbMGGmdWq3GmDFjsHHjRj+WrHUpLi4GAERHRwMAtm/fjpqaGsV97datG9LS0qT7unHjRvTu3Rvx8fHSPuPGjUNJSQn27dsn7SM/h22ftvizefDBBzFhwgSH+8F77Tvff/89Bg0ahJtuuglxcXHo378/3n33XWl7dnY28vLyFPcpIiICmZmZinsdGRmJQYMGSfuMGTMGarUamzdvlvYZOXIk9Hq9tM+4ceNw8OBBXLhwoanfZoswdOhQZGVl4dChQwCA3bt3Y926dRg/fjwA3uum0pz31Zd/UxjceKmgoAAWi0XxRx8A4uPjkZeX56dStS5WqxUPPfQQhg0bhl69egEA8vLyoNfrERkZqdhXfl/z8vKc3nfbNnf7lJSUoLKysineTou0dOlS7NixA/Pnz3fYxnvtO8eOHcNbb72Fzp0746effsL999+Pv/71r/jwww8B1N8rd38v8vLyEBcXp9iu1WoRHR3t1c8j0D3xxBP405/+hG7dukGn06F///546KGHMHnyZAC8102lOe+rq30ac9/b3Kzg5H8PPvgg9u7di3Xr1vm7KAHp5MmTmDFjBlauXAmj0ejv4gQ0q9WKQYMG4fnnnwcA9O/fH3v37sXixYsxZcoUP5cusHzxxRf45JNP8Omnn6Jnz57YtWsXHnroISQlJfFekwPW3HgpNjYWGo3GoWdJfn4+EhIS/FSq1mP69On44YcfsGrVKqSkpEjrExISYDKZUFRUpNhffl8TEhKc3nfbNnf7hIeHIygoyNdvp0Xavn07zp49iwEDBkCr1UKr1WLNmjV4/fXXodVqER8fz3vtI4mJiejRo4diXffu3ZGTkwOg/l65+3uRkJCAs2fPKrabzWYUFhZ69fMIdI8++qhUe9O7d2/cfvvtmDlzplQ7yXvdNJrzvrrapzH3ncGNl/R6PQYOHIisrCxpndVqRVZWFoYMGeLHkrVsQghMnz4d3377LX755RdkZGQotg8cOBA6nU5xXw8ePIicnBzpvg4ZMgR79uxR/BKtXLkS4eHh0gNmyJAhinPY9mlLP5vRo0djz5492LVrl/Rv0KBBmDx5svSa99o3hg0b5jCkwaFDh9C+fXsAQEZGBhISEhT3qaSkBJs3b1bc66KiImzfvl3a55dffoHVakVmZqa0z9q1a1FTUyPts3LlSnTt2hVRUVFN9v5akoqKCqjVykeWRqOB1WoFwHvdVJrzvvr0b4rXKcgkli5dKgwGg/jggw/E/v37xb333isiIyMVPUtI6f777xcRERFi9erVIjc3V/pXUVEh7XPfffeJtLQ08csvv4ht27aJIUOGiCFDhkjbbd2Tx44dK3bt2iVWrFgh2rVr57R78qOPPioOHDggFi1a1Oa6Jzsj7y0lBO+1r2zZskVotVrx3HPPicOHD4tPPvlEBAcHi//85z/SPgsWLBCRkZHiu+++E7/99pu49tprnXaj7d+/v9i8ebNYt26d6Ny5s6IbbVFRkYiPjxe333672Lt3r1i6dKkIDg4O6O7J9qZMmSKSk5OlruDffPONiI2NFY899pi0D+9145SWloqdO3eKnTt3CgBi4cKFYufOneLEiRNCiOa7r+vXrxdarVa89NJL4sCBA2LevHnsCt7c3njjDZGWlib0er0YPHiw2LRpk7+L1KIBcPrv/fffl/aprKwUDzzwgIiKihLBwcHi+uuvF7m5uYrzHD9+XIwfP14EBQWJ2NhY8fDDD4uamhrFPqtWrRL9+vUTer1edOjQQXGNtso+uOG99p3//e9/olevXsJgMIhu3bqJd955R7HdarWKOXPmiPj4eGEwGMTo0aPFwYMHFfucP39e3HLLLSI0NFSEh4eLadOmidLSUsU+u3fvFsOHDxcGg0EkJyeLBQsWNPl7a0lKSkrEjBkzRFpamjAajaJDhw7iySefVHQt5r1unFWrVjn9+zxlyhQhRPPe1y+++EJ06dJF6PV60bNnT7Fs2bJGvSeVELLhHYmIiIhaOebcEBERUUBhcENEREQBhcENERERBRQGN0RERBRQGNwQERFRQGFwQ0RERAGFwQ0REREFFAY3REREFFAY3BBRi3H8+HGoVCrs2rWrya4xdepUXHfddU12fiLyPwY3ROQTU6dOhUqlcvh31VVXeXyO1NRU5ObmolevXk1YUt/aunUrkpKSAABnzpxBUFAQTCaTn0tF1LZp/V0AIgocV111Fd5//33FOoPB4PHxGo0GCQkJvi5Wk9q4cSOGDRsGAPj1118xaNAg6PV6P5eKqG1jzQ0R+YzBYEBCQoLiX1RUlLRdpVLhrbfewvjx4xEUFIQOHTrgq6++krbbN0tduHABkydPRrt27RAUFITOnTsrgqc9e/bgiiuuQFBQEGJiYnDvvfeirKxM2m6xWDBr1ixERkYiJiYGjz32GOyn07NarZg/fz4yMjIQFBSEvn37KsrUkA0bNkjBzbp166TXROQ/DG6IqFnNmTMHN954I3bv3o3JkyfjT3/6Ew4cOOBy3/379+PHH3/EgQMH8NZbbyE2NhYAUF5ejnHjxiEqKgpbt27Fl19+iZ9//hnTp0+Xjn/55ZfxwQcfYMmSJVi3bh0KCwvx7bffKq4xf/58fPTRR1i8eDH27duHmTNn4rbbbsOaNWtcvod169YhMjISkZGR+Oqrr/Dkk08iMjISixcvxuuvv47IyEgsWLDAB3eLiBqlUXOJExHZmTJlitBoNCIkJETx77nnnpP2ASDuu+8+xXGZmZni/vvvF0IIkZ2dLQCInTt3CiGEmDhxopg2bZrT673zzjsiKipKlJWVSeuWLVsm1Gq1yMvLE0IIkZiYKF544QVpe01NjUhJSRHXXnutEEKIqqoqERwcLDZs2KA491133SVuueUWl++1srJSZGdnix9//FFERUWJY8eOiW3btgm9Xi8OHDggsrOzxYULF9zfMCJqMsy5ISKfufzyy/HWW28p1kVHRyuWhwwZ4rDsqnfU/fffjxtvvBE7duzA2LFjcd1112Ho0KEAgAMHDqBv374ICQmR9h82bBisVisOHjwIo9GI3NxcZGZmStu1Wi0GDRokNU0dOXIEFRUVuPLKKxXXNZlM6N+/v8v3aTQakZ6eji+++ALjx49HRkYGNmzYgBEjRqBbt24ujyOi5sHghoh8JiQkBJ06dfLZ+caPH48TJ05g+fLlWLlyJUaPHo0HH3wQL730kk/Ob8vPWbZsGZKTkxXb3CVCh4aGAgCqq6uhVqvx3XffwWQyQQiB0NBQjBgxAj/++KNPykhE3mPODRE1q02bNjksd+/e3eX+7dq1w5QpU/Cf//wHr776Kt555x0AQPfu3bF7926Ul5dL+65fvx5qtRpdu3ZFREQEEhMTsXnzZmm72WzG9u3bpeUePXrAYDAgJycHnTp1UvxLTU11WaZdu3Zh27Zt0Gg0yMrKwq5duxATE4MvvvgCu3btwr///W+v7wsR+Q5rbojIZ6qrq5GXl6dYp9VqpSRgAPjyyy8xaNAgDB8+HJ988gm2bNmC9957z+n55s6di4EDB6Jnz56orq7GDz/8IAVCkydPxrx58zBlyhT8/e9/x7lz5/CXv/wFt99+O+Lj4wEAM2bMwIIFC9C5c2d069YNCxcuRFFRkXT+sLAwPPLII5g5cyasViuGDx+O4uJirF+/HuHh4ZgyZYrTcnXq1AmbNm1CfHw8hg8fjpycHJSWlmLixInQavlnlcjf+FtIRD6zYsUKJCYmKtZ17doVv//+u7T89NNPY+nSpXjggQeQmJiIzz77DD169HB6Pr1ej9mzZ+P48eMICgrCiBEjsHTpUgBAcHAwfvrpJ8yYMQOXXHIJgoODceONN2LhwoXS8Q8//DByc3MxZcoUqNVq3Hnnnbj++utRXFws7fPss8+iXbt2mD9/Po4dO4bIyEgMGDAAf/vb39y+19WrV2PkyJEAgDVr1mDIkCEMbIhaCJUQdoM+EBE1EZVKhW+//ZbTHxBRk2LODREREQUUBjdEREQUUNhATETNhq3gRNQcWHNDREREAYXBDREREQUUBjdEREQUUBjcEBERUUBhcENEREQBhcENERERBRQGN0RERBRQGNwQERFRQPl/XQHE33o6oLMAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "window = 100\n",
    "data = pd.DataFrame(returns)\n",
    "data = data.rolling(window).mean()\n",
    "plt.plot(data)\n",
    "plt.xlabel(\"Episode #\")\n",
    "plt.ylabel(\"Return\")\n",
    "plt.show()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Learned policy\n",
    "The learned policy can be studied by looking at the learned ranking and the probabilities with which ORLA chose each of those arguments:\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Learned ranking: [['nR'], ['nD'], ['L'], ['nL'], ['U'], ['nU'], ['D'], ['R']]\n",
      "    probabilities: tensor([0.9958, 0.9995, 0.9612, 0.5807, 0.6851, 0.9458, 0.7821, 1.0000],\n",
      "       grad_fn=<StackBackward0>)\n"
     ]
    }
   ],
   "source": [
    "ranking, probs = orla.decode_ranking(True)\n",
    "print(\"Learned ranking:\", ranking)\n",
    "print(\"    probabilities:\", probs)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### The learned policy in action\n",
    "The following cell creates a new FFL instance and renders an episode using the VAF derived from the learned ranking as the inference engine"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "env = FFL(argument_actions, n, p, render=True)\n",
    "_ = env.play(ranking)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "py39",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.13"
  },
  "orig_nbformat": 4
 },
 "nbformat": 4,
 "nbformat_minor": 2
}