Tutorial-ETK_thorn-IDMaxwellNRPy.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<script async src=\"https://www.googletagmanager.com/gtag/js?id=UA-59152712-8\"></script>\n",
    "<script>\n",
    "  window.dataLayer = window.dataLayer || [];\n",
    "  function gtag(){dataLayer.push(arguments);}\n",
    "  gtag('js', new Date());\n",
    "\n",
    "  gtag('config', 'UA-59152712-8');\n",
    "</script>\n",
    "\n",
    "# IDMaxwellNRPy: An Einstein Toolkit Initial Data Thorn for Maxwell's equations\n",
    "\n",
    "## Author: Zach Etienne\n",
    "### Formatting improvements courtesy Brandon Clark\n",
    "\n",
    "### NRPy+ Source Code for this module: [Maxwell/MaxwellCartesian_ID.py](../edit/Maxwell/MaxwellCartesian_ID.py) [\\[tutorial\\]](Tutorial-MaxwellCartesian.ipynb) Constructs these Maxwell initial data as SymPy expressions.\n",
    "\n",
    "## Introduction:\n",
    "In this part of the tutorial, we will construct an Einstein Toolkit (ETK) thorn (module) that will set up *initial data* for Maxwell's equations. In the [Tutorial-MaxwellCartesian](Tutorial-MaxwellCartesian.ipynb), tutorial notebook we used NRPy+ to contruct the SymPy expressions for plane-wave initial data. \n",
    "\n",
    "We will construct this thorn in two steps.\n",
    "\n",
    "1. Call on NRPy+ to convert the SymPy expressions for the initial data into one C-code kernel.\n",
    "2. Write the C code and linkages to the Einstein Toolkit infrastructure (i.e., the .ccl files) to complete this Einstein Toolkit module."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id='toc'></a>\n",
    "\n",
    "# Table of Contents\n",
    "$$\\label{toc}$$\n",
    "\n",
    "This notebook is organized as follows\n",
    "\n",
    "1. [Step 1](#initializenrpy): Call on NRPy+ to convert the SymPy expression for the Maxwell's equations initial data into a C-code kernel\n",
    "1. [Step 2](#einstein): Interfacing with the Einstein Toolkit\n",
    "    1. [Step 2.a](#einstein_c): Constructing the Einstein Toolkit C-code calling functions that include the C code kernels\n",
    "    1. [Step 2.b](#einstein_ccl): CCL files - Define how this module interacts and interfaces with the larger Einstein Toolkit infrastructure\n",
    "    1. [Step 2.c](#einstein_list): Add the C code to the Einstein Toolkit compilation list\n",
    "1. [Step 3](#latex_pdf_output): Output this notebook to $\\LaTeX$-formatted PDF\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id='initializenrpy'></a>\n",
    "\n",
    "# Step 1: Call on NRPy+ to convert the SymPy expression for the Maxwell's equations initial data into a C-code kernel \\[Back to [top](#toc)\\]\n",
    "$$\\label{initializenrpy}$$\n",
    "\n",
    "After importing the core modules, since we are writing an ETK thorn, we'll need to set `\"grid::GridFuncMemAccess\"` to `\"ETK\"`. SymPy expressions for plane wave initial data are written inside [Maxwell/MaxwellCartesian_ID.py](../edit/Maxwell/MaxwellCartesian_ID.py), and we simply import them for use here."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Step 1: Call on NRPy+ to convert the SymPy expression\n",
    "#         for the Maxwell's equations initial data into a C-code kernel\n",
    "\n",
    "# Step 1a: Import needed NRPy+ core modules:\n",
    "import NRPy_param_funcs as par   # NRPy+: Parameter interface\n",
    "import indexedexp as ixp         # NRPy+: Symbolic indexed expression (e.g., tensors, vectors, etc.) support\n",
    "import grid as gri               # NRPy+: Functions having to do with numerical grids\n",
    "import loop as lp                # NRPy+: Generate C code loops\n",
    "import sympy as sp               # SymPy: The Python computer algebra package upon which NRPy+ depends\n",
    "from outputC import lhrh         # NRPy+: Core C code output module\n",
    "import finite_difference as fin  # NRPy+: Finite difference C code generation module\n",
    "\n",
    "# Step 1b: This is an Einstein Toolkit (ETK) thorn. Here we\n",
    "#          tell NRPy+ that gridfunction memory access will\n",
    "#          therefore be in the \"ETK\" style.\n",
    "par.set_parval_from_str(\"grid::GridFuncMemAccess\",\"ETK\")\n",
    "#Set the spatial dimension parameter to 3.\n",
    "par.set_parval_from_str(\"grid::DIM\", 3)\n",
    "DIM = par.parval_from_str(\"grid::DIM\")\n",
    "\n",
    "import Maxwell.MaxwellCartesian_Evol as mwrhs # This declare the parameter System_to_use needed for MaxwellCartesian_ID\n",
    "# Step 1c: Call the MaxwellCartesian_ID() function from within the\n",
    "#          Maxwell/MaxwellCartesian_ID.py.py module.\n",
    "import Maxwell.MaxwellCartesian_ID as mwid\n",
    "# Step 3: Set up the plane wave initial data. This sets uu_ID and vv_ID.\n",
    "mwid.MaxwellCartesian_ID()\n",
    "\n",
    "# Step 4: Register gridfunctions so they can be written to by NRPy.\n",
    "# System I:\n",
    "AID = ixp.register_gridfunctions_for_single_rank1(\"EVOL\",\"AID\")\n",
    "EID = ixp.register_gridfunctions_for_single_rank1(\"EVOL\",\"EID\")\n",
    "psiI = gri.register_gridfunctions(\"EVOL\",\"psiI\")\n",
    "\n",
    "# Step 5: FIXME: ADD DESCRIPTION\n",
    "for i in range(DIM):\n",
    "    AID[i] = mwid.AidD[i]\n",
    "    EID[i] = mwid.EidD[i]\n",
    "psiI = mwid.psi_ID\n",
    "\n",
    "# Step 4: Register gridfunctions so they can be written to by NRPy.\n",
    "# System II:\n",
    "AIID = ixp.register_gridfunctions_for_single_rank1(\"EVOL\",\"AIID\")\n",
    "EIID = ixp.register_gridfunctions_for_single_rank1(\"EVOL\",\"EIID\")\n",
    "psiII = gri.register_gridfunctions(\"EVOL\",\"psiII\")\n",
    "Gamma = gri.register_gridfunctions(\"EVOL\",\"Gamma\")\n",
    "\n",
    "# Step 5: FIXME: ADD DESCRIPTION\n",
    "for i in range(DIM):\n",
    "    AIID[i] = mwid.AidD[i]\n",
    "    EIID[i] = mwid.EidD[i]\n",
    "psiII = mwid.psi_ID\n",
    "Gamma = mwid.Gamma_ID\n",
    "\n",
    "# Step 6: Create the C code output kernel.\n",
    "Maxwell_ID_to_print = [\\\n",
    "                        lhrh(lhs=gri.gfaccess(\"out_gfs\",\"AID0\"),rhs=AID[0]),\\\n",
    "                        lhrh(lhs=gri.gfaccess(\"out_gfs\",\"AID1\"),rhs=AID[1]),\\\n",
    "                        lhrh(lhs=gri.gfaccess(\"out_gfs\",\"AID2\"),rhs=AID[2]),\\\n",
    "                        lhrh(lhs=gri.gfaccess(\"out_gfs\",\"EID0\"),rhs=EID[0]),\\\n",
    "                        lhrh(lhs=gri.gfaccess(\"out_gfs\",\"EID1\"),rhs=EID[1]),\\\n",
    "                        lhrh(lhs=gri.gfaccess(\"out_gfs\",\"EID2\"),rhs=EID[2]),\\\n",
    "                        lhrh(lhs=gri.gfaccess(\"out_gfs\",\"psiI\"),rhs=psiI),\\\n",
    "                        lhrh(lhs=gri.gfaccess(\"out_gfs\",\"AIID0\"),rhs=AIID[0]),\\\n",
    "                        lhrh(lhs=gri.gfaccess(\"out_gfs\",\"AIID1\"),rhs=AIID[1]),\\\n",
    "                        lhrh(lhs=gri.gfaccess(\"out_gfs\",\"AIID2\"),rhs=AIID[2]),\\\n",
    "                        lhrh(lhs=gri.gfaccess(\"out_gfs\",\"EIID0\"),rhs=EIID[0]),\\\n",
    "                        lhrh(lhs=gri.gfaccess(\"out_gfs\",\"EIID1\"),rhs=EIID[1]),\\\n",
    "                        lhrh(lhs=gri.gfaccess(\"out_gfs\",\"EIID2\"),rhs=EIID[2]),\\\n",
    "                        lhrh(lhs=gri.gfaccess(\"out_gfs\",\"psiII\"),rhs=psiII),\\\n",
    "                        lhrh(lhs=gri.gfaccess(\"out_gfs\",\"Gamma\"),rhs=Gamma),]\n",
    "Maxwell_ID_CcodeKernel = fin.FD_outputC(\"returnstring\",Maxwell_ID_to_print)\n",
    "\n",
    "Maxwell_ID_looped = lp.loop([\"i2\",\"i1\",\"i0\"],[\"0\",\"0\",\"0\"],[\"cctk_lsh[2]\",\"cctk_lsh[1]\",\"cctk_lsh[0]\"],\\\n",
    "                            [\"1\",\"1\",\"1\"],[\"#pragma omp parallel for\",\"\",\"\"],\"\",\\\n",
    "                            Maxwell_ID_CcodeKernel.replace(\"time\",\"cctk_time\"))\n",
    "\n",
    "# Step 7: Create directories for the thorn if they don't exist.\n",
    "!mkdir MaxwellID     2>/dev/null # 2>/dev/null: Don't throw an error if the directory already exists.\n",
    "!mkdir MaxwellID/src 2>/dev/null # 2>/dev/null: Don't throw an error if the directory already exists.\n",
    "\n",
    "# Step 8: Write the C code kernel to file.\n",
    "with open(\"MaxwellID/src/Maxwell_ID.h\", \"w\") as file:\n",
    "    file.write(str(Maxwell_ID_looped))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id='einstein'></a>\n",
    "\n",
    "# Step 2: Interfacing with the Einstein Toolkit \\[Back to [top](#toc)\\]\n",
    "$$\\label{einstein}$$\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id='einstein_c'></a>\n",
    "\n",
    "## Step 2.a: Constructing the Einstein Toolkit C-code calling functions that include the C code kernels \\[Back to [top](#toc)\\]\n",
    "$$\\label{einstein_c}$$\n",
    "\n",
    "We will write another C file with the functions we need here."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Writing MaxwellID/src/InitialData.c\n"
     ]
    }
   ],
   "source": [
    "%%writefile MaxwellID/src/InitialData.c\n",
    "#include <math.h>\n",
    "#include <stdio.h>\n",
    "\n",
    "#include \"cctk.h\"\n",
    "#include \"cctk_Parameters.h\"\n",
    "#include \"cctk_Arguments.h\"\n",
    "\n",
    "void Maxwell_InitialData(CCTK_ARGUMENTS)\n",
    "{\n",
    "  DECLARE_CCTK_ARGUMENTS\n",
    "  DECLARE_CCTK_PARAMETERS\n",
    "\n",
    "  const CCTK_REAL *xGF = x;\n",
    "  const CCTK_REAL *yGF = y;\n",
    "  const CCTK_REAL *zGF = z;\n",
    "  const CCTK_REAL *gammaDD00GF = gxx;\n",
    "  const CCTK_REAL *gammaDD01GF = gxy;\n",
    "  const CCTK_REAL *gammaDD02GF = gxz;\n",
    "  const CCTK_REAL *gammaDD11GF = gyy;\n",
    "  const CCTK_REAL *gammaDD12GF = gyz;\n",
    "  const CCTK_REAL *gammaDD22GF = gzz;\n",
    "#include \"Maxwell_ID.h\"\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id='einstein_ccl'></a>\n",
    "\n",
    "## Step 2.b: CCL files - Define how this module interacts and interfaces with the larger Einstein Toolkit infrastructure \\[Back to [top](#toc)\\]\n",
    "$$\\label{einstein_ccl}$$\n",
    "\n",
    "Writing a module (\"thorn\") within the Einstein Toolkit requires that three \"ccl\" files be constructed, all in the root directory of the thorn:\n",
    "\n",
    "1. `interface.ccl`: defines the gridfunction groups needed, and provides keywords denoting what this thorn provides and what it should inherit from other thorns. Specifically, this file governs the interaction between this thorn and others; more information can be found in the [official Einstein Toolkit documentation](http://einsteintoolkit.org/usersguide/UsersGuidech12.html#x17-178000D2.2). \n",
    "With \"implements\", we give our thorn its unique name. By \"inheriting\" other thorns, we tell the Toolkit that we will rely on variables that exist and are declared \"public\" within those functions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Writing MaxwellID/interface.ccl\n"
     ]
    }
   ],
   "source": [
    "%%writefile MaxwellID/interface.ccl\n",
    "implements: MaxwellID\n",
    "inherits: admbase MaxwellEvol grid\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "2. `param.ccl`: specifies free parameters within the thorn, enabling them to be set at runtime. It is required to provide allowed ranges and default values for each parameter. More information on this file's syntax can be found in the [official Einstein Toolkit documentation](http://einsteintoolkit.org/usersguide/UsersGuidech12.html#x17-183000D2.3)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Writing MaxwellID/param.ccl\n"
     ]
    }
   ],
   "source": [
    "%%writefile MaxwellID/param.ccl\n",
    "shares: grid\n",
    "\n",
    "USES KEYWORD type\n",
    "\n",
    "restricted:\n",
    "CCTK_KEYWORD initial_data \"Type of initial data\"\n",
    "{\n",
    "  \"toroid\"      :: \"Toroidal Dipole field\"\n",
    "} \"toroid\"\n",
    "\n",
    "restricted:\n",
    "CCTK_REAL amp \"The amplitude of the initial wave packet\"\n",
    "{\n",
    " 0.0:* :: \"Should be positive\"\n",
    "} 1.0\n",
    "\n",
    "restricted:\n",
    "CCTK_REAL lam \"A parameter describing the size of the wave package\"\n",
    "{\n",
    " 0.0:* :: \"Should be positive\"\n",
    "} 1.0\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "3. `schedule.ccl`: allocates storage for gridfunctions, defines how the thorn's functions should be scheduled in a broader simulation, and specifies the regions of memory written to or read from gridfunctions. `schedule.ccl`'s official documentation may be found [here](http://einsteintoolkit.org/usersguide/UsersGuidech12.html#x17-186000D2.4). \n",
    "\n",
    "We specify here the standardized ETK \"scheduling bins\" in which we want each of our thorn's functions to run."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Writing MaxwellID/schedule.ccl\n"
     ]
    }
   ],
   "source": [
    "%%writefile MaxwellID/schedule.ccl\n",
    "\n",
    "schedule Maxwell_InitialData at CCTK_INITIAL as Maxwell_InitialData\n",
    "{\n",
    "  STORAGE:       MaxwellEvol::system_I[3]\n",
    "  STORAGE:       MaxwellEvol::system_II[3]\n",
    "  LANG:          C\n",
    "  READS: admbase::gxx(Everywhere)\n",
    "  READS: admbase::gxy(Everywhere)\n",
    "  READS: admbase::gxz(Everywhere)\n",
    "  READS: admbase::gyy(Everywhere)\n",
    "  READS: admbase::gyz(Everywhere)\n",
    "  READS: admbase::gzz(Everywhere)\n",
    "  READS: grid::x(Everywhere)\n",
    "  READS: grid::y(Everywhere)\n",
    "  READS: grid::y(Everywhere)\n",
    "  WRITES: AI0GF(Everywhere)\n",
    "  WRITES: AI1GF(Everywhere)\n",
    "  WRITES: AI2GF(Everywhere)\n",
    "  WRITES: EI0GF(Everywhere)\n",
    "  WRITES: EI1GF(Everywhere)\n",
    "  WRITES: EI2GF(Everywhere)\n",
    "  WRITES: psiIGF(Everywhere)\n",
    "  WRITES: AII0GF(Everywhere)\n",
    "  WRITES: AII1GF(Everywhere)\n",
    "  WRITES: AII2GF(Everywhere)\n",
    "  WRITES: EII0GF(Everywhere)\n",
    "  WRITES: EII1GF(Everywhere)\n",
    "  WRITES: EII2GF(Everywhere)\n",
    "  WRITES: psiIIGF(Everywhere)\n",
    "  WRITES: GammaGF(Everywhere)\n",
    "} \"Initial data for Maxwell's equations\"\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id='einstein_list'></a>\n",
    "\n",
    "## Step 2.c: Add the C code to the Einstein Toolkit compilation list \\[Back to [top](#toc)\\]\n",
    "$$\\label{einstein_list}$$\n",
    "\n",
    "We will also need `make.code.defn`, which indicates the list of files that need to be compiled. This thorn only has the one C file to compile."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Writing MaxwellID/src/make.code.defn\n"
     ]
    }
   ],
   "source": [
    "%%writefile MaxwellID/src/make.code.defn\n",
    "SRCS = InitialData.c"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id='latex_pdf_output'></a>\n",
    "\n",
    "# Step 3: Output this notebook to $\\LaTeX$-formatted PDF file \\[Back to [top](#toc)\\]\n",
    "$$\\label{latex_pdf_output}$$\n",
    "\n",
    "The following code cell converts this Jupyter notebook into a proper, clickable $\\LaTeX$-formatted PDF file. After the cell is successfully run, the generated PDF may be found in the root NRPy+ tutorial directory, with filename\n",
    "[Tutorial-ETK_thorn-IDMaxwellNRPy.pdf](Tutorial-ETK_thorn-IDMaxwellNRPy.pdf) (Note that clicking on this link may not work; you may need to open the PDF file through another means.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Created Tutorial-ETK_thorn-IDMaxwellNRPy.tex, and compiled LaTeX file to\n",
      "    PDF file Tutorial-ETK_thorn-IDMaxwellNRPy.pdf\n"
     ]
    }
   ],
   "source": [
    "import cmdline_helper as cmd    # NRPy+: Multi-platform Python command-line interface\n",
    "cmd.output_Jupyter_notebook_to_LaTeXed_PDF(\"Tutorial-ETK_thorn-IDMaxwellNRPy\")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}