cwpk-58-setting-up-remote-instance-web-page-server.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "CWPK \\#58: Setting Up a Remote Instance and Web Page Server\n",
    "=======================================\n",
    "\n",
    "The Beginning of Moving Our Environment into the Cloud\n",
    "--------------------------\n",
    "\n",
    "<div style=\"float: left; width: 305px; margin-right: 10px;\">\n",
    "\n",
    "<img src=\"http://kbpedia.org/cwpk-files/cooking-with-kbpedia-305.png\" title=\"Cooking with KBpedia\" width=\"305\" />\n",
    "\n",
    "</div>\n",
    "\n",
    "Today's installment in our [*Cooking with Python and KBpedia*](https://www.mkbergman.com/cooking-with-python-and-kbpedia/) series begins a three-part mini-series of moving our environment into the clouds. This first part begins with setting up a remote instance and Web page server using the mini-framework [Flask](https://en.wikipedia.org/wiki/Flask_(web_framework)). The next installment expands on this basis to begin adding a [SPARQL](https://en.wikipedia.org/wiki/SPARQL) [endpoint](https://en.wikipedia.org/wiki/Communication_endpoint) to the remote instance. And, then, in the concluding third part to this mini-series, we complete the endpoint and make the SPARQL service active.\n",
    "\n",
    "We undertake this mini-series because we want to make KBpedia more open to SPARQL queries from anywhere on the Web and we will be migrating our main KBpedia Web site content to Python. The three installments in this mini-series pave the way for us to achieve these aims.\n",
    "\n",
    "We begin today's installment by looking at the approach and alternatives for achieving these aims, and then we proceed to outline the steps needed to set up a Python instance on a remote (cloud) instance and to begin serving Web pages."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Approach and Alternatives\n",
    "We saw earlier in [**CWPK #25**](https://www.mkbergman.com/2358/cwpk-25-querying-kbpedia-with-sparql/) and [**CWPK #50**](https://www.mkbergman.com/2396/cwpk-50-querying-external-sources/) the usefulness of the SPARQL query language to interact with knowledge graphs. Our first objective, therefore, was to establish a similar facility for KBpedia. Though we have an alternate choice to set up some form of [RESTful API](https://en.wikipedia.org/wiki/Representational_state_transfer) to KBpedia (see further **Additional Documentation** below), and perhaps that may make sense at some point in time, our preference is to use SPARQL given its robustness and many query examples as earlier installments document.\n",
    "\n",
    "Further, we can foresee that our work with Python in KBpedia may warrant our moving our standard KBpedia Web site to that away from Clojure and its current Bootstrap-based Web page format. Though Python is not generally known for its Web-page serving capabilities, some exploration of that area may indicate whether we may go in that direction or not. Lastly, given our intent to make querying of KBpedia more broadly available, we also wanted to adhere to [linked data](https://en.wikipedia.org/wiki/Linked_data) standards. This latter objective is perhaps the most problematic of our aims as we will discuss in a couple of installments.\n",
    "\n",
    "Typically, and certainly the easiest path, when one wants to set up a SPARQL endpoint with linked data '[conneg](https://en.wikipedia.org/wiki/Content_negotiation)' (content negotiation) is to use an existing [triple store](https://en.wikipedia.org/wiki/Triplestore) that has these capabilities built in. We already use [Virtuoso](https://en.wikipedia.org/wiki/Virtuoso_Universal_Server) as a triple store, and there are a couple of Python [installation](https://www.openlinksw.com/blog/~kidehen/?id=1651) [guides](https://github.com/cltl/PythonVirtuosoInterface) already available for Virtuoso. Most of the other widely available triple stores have similar capabilities. \n",
    "\n",
    "Were we not interested in general Web page serving and were outside of the bounds of the objectives of this **CWPK** series, use of a triple store is the path we would recommend. However, since our aims are different, we needed to turn our attention to Python-based solutions.\n",
    "\n",
    "From the standpoint of Web-page serving, perhaps the best known and most widely installed Python option is [Django](https://en.wikipedia.org/wiki/Django_(web_framework)), a fully featured, open-source [Web framework](https://en.wikipedia.org/wiki/List_of_content_management_systems). Django has a similar scope and suite of capabilities to its PHP counterpart, [Drupal](https://en.wikipedia.org/wiki/Drupal). These are comprehensive and complicated frameworks, well suited to massive database-backed sites or ones with e-commerce or other specialty applications. We have had much experience with Drupal, and frankly did not want to tackle the complexity of a full framework. \n",
    "\n",
    "I was much more attracted to a simpler [microframework](https://en.wikipedia.org/wiki/Microframework). The two leading Python options are Flask and [Bottle](http://bottlepy.org/docs/dev/faq.html#common-problems-and-pitfalls) (though there is also [Falcon](https://falcon.readthedocs.io/en/stable/), which does not seem as developed). I was frankly more impressed with the documentation and growth and acceptance shown by Flask, and there appeared to be more analogous installations. Flask is based on the [Jinja](https://en.wikipedia.org/wiki/Jinja_(template_engine)) template engine and the [Werkzeug](https://werkzeug.palletsprojects.com/en/1.0.x/) [WSGI](https://en.wikipedia.org/wiki/Web_Server_Gateway_Interface) (Web-server Gateway Interface) utility library. It is fully based on Unicode.\n",
    "\n",
    "Another factor that needs to be considered is support for [RDFlib](https://github.com/RDFLib) the key package (and related) that we will be using for the SPARQL efforts. I first discussed this package in [**CWPK #24**](https://www.mkbergman.com/2357/cwpk-24-introduction-to-rdflib/), though it is featured in many installments."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Basic Installation\n",
    "We will be setting up these new endpoints on our cloud server, which is a large [EC2](https://docs.aws.amazon.com/ec2/index.html?nc2=h_ql_doc_ec2) instance on [Amazon Web Services](https://aws.amazon.com/free/) running [Ubuntu 18.04 LTS](https://ubuntu.com/blog/tag/ubuntu-18-04). Of course, this is only one of many cloud services. As a result, I will not discuss all of the preliminary steps to first securing an instance, or setting up an [SSH](https://en.wikipedia.org/wiki/Secure_Shell) client to access it, nor any of the initial other start-up issues. For EC2 on AWS, there are many such tutorials. Two that I have encountered in doing the research for this installment include [this one](https://ketakirk.wordpress.com/deploy-an-app-on-aws/) and [this other one](https://towardsdatascience.com/deploying-a-python-web-app-on-aws-57ed772b2319). There are multiple others, and ones applicable to other providers than AWS as well.\n",
    "\n",
    "So, we begin from the point that an instance exists and you know how to log in and do basic steps with it. Even with this simplification, I began my considerations with these questions in mind:\n",
    "\n",
    "1. Do I need a package manager, and if so, which one?\n",
    "1. Where should I place my Python projects within the remote instance directory structure?\n",
    "1. How do I also include the pip package environment?\n",
    "1. Should I use virtual environments?\n",
    "\n",
    "With regard to the first question, I was sure I wanted to maintain the [conda](https://docs.conda.io/en/latest/) package manager, but I was not convinced I needed the full GUI and set of packages with [Anaconda](https://www.anaconda.com/products/individual/get-started). I wanted to keep consistency with the set-up environment we put in place for our local instance (see [**CWPK #9**](https://www.mkbergman.com/2336/cwpk-9-installing-python/)). However, since I had gained much experience with Anaconda and conda, I felt comfortable not installing the full 4 GB full Anaconda distribution. This led me to adopt the minimal [miniconda](https://docs.conda.io/en/latest/miniconda.html) package, which has a much smaller footprint. True, I will need to install all specific Python packages and work from the command line (terminal), but that is easy given the experience I have gained.\n",
    "\n",
    "Second, in reviewing best practices information for directory structures on Ubuntu, I decided to create an analogous 'python-projects' master directory, similar to what I established for the local instance, under the standard user application directory. I thus decided and created this master directory: <code>usr/bin/python-projects</code>.\n",
    "\n",
    "So, having decided to use miniconda and knowing the master directory I wanted to use, I proceeded to begin to set up my remote installation. I followed a [useful source](https://www.digitalocean.com/community/tutorials/how-to-install-python-3-and-set-up-a-programming-environment-on-an-ubuntu-18-04-server) for this process, though importantly updated all references to the current versions. The first step was to navigate to my master directory and then to download the 64-bit Linux installer onto my remove instance, followed by executing the installation script:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh  * make sure you use updated version\n",
    "\n",
    "sh Miniconda3-latest-Linux-x86_64.sh    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The installation script first requires you to page through the license agreement (many lines!!) and then accept it. Upon acceptance, the Miniconda code is installed with some skeletal stubs in a new directory under your master directory, which in my case is <code>/usr/bin/python-projects/miniconda3</code>. The last step in the installation process asks where you want Miniconda installed. The default is to use <code>root/miniconda3</code>. Since I wanted to keep all of my Python project stuff isolated, I overrode this suggestion and used my standard location of <code>/usr/bin/python-projects/miniconda3</code>.\n",
    "\n",
    "After all of the appropriate files are copied, I agreed to initialize Miniconda. This completes the basic Miniconda installation.\n",
    "\n",
    " <div style=\"background-color:#ffecec; border:1px dotted #f5aca6; vertical-align:middle; margin:15px 60px; padding:8px;\"> \n",
    "  <span style=\"font-weight: bold;\">Important Note:</span> You will need to sign out and then sign back into your SSH shell at this point before the changes become active!\n",
    "  </div>\n",
    "\n",
    "After signing back in, I again navigated to my Python master directory and then installed the [pip package manager](https://pypi.org/project/pip/) since not all of the Python packages we use in this **CWPK** series and *cowpoke* are available through conda."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Virtual Environments and a Detour\n",
    "In the standard use of a Linux installation, one uses the distro's own installation and package management procedures. In the case of Ubuntu and related distros such as Debian, <code>apt</code> stands for '[advanced package tool](https://en.wikipedia.org/wiki/APT_(software))' and through commands such as <code>apt-get</code> is one means to install new capabilities to your remote instance. Other Linux distros may use [<code>yum</code>](https://en.wikipedia.org/wiki/Yum_(software)) or [<code>rpm</code>](https://en.wikipedia.org/wiki/RPM_Package_Manager) or similar to install new packages.\n",
    "\n",
    "Then, of course, within Python, one has the [<code>pip</code>](https://en.wikipedia.org/wiki/Pip_(package_manager)) package installer and the <code>conda</code> one we are using. Further, within a Linux installation, how one may install packages and where they may be applicable depends on the user rights of the installer. That is one reason why it is important to have [<code>sudo</code>](https://en.wikipedia.org/wiki/Sudo) (super user) rights as an administrator of the instance when one wants new packages to be available to all users.\n",
    "\n",
    "These package managers may conflict in rights and, if not used properly, may act to work at cross purposes. For example, in a standard AWS EC2 instance using Ubuntu, it comes packaged with its own Python version. This default is useful for the occasional app that needs Python, but does not conform to the segregation of packages that Python often requires using its '[virtual environments](https://en.wikipedia.org/wiki/Virtual_environment_software)'. \n",
    "\n",
    "On the face of it, the use of virtual environments seems to make sense for Python: we can keep projects separate with different dependencies and even different versions of Python. I took this viewpoint at face value when I began this transition to installing Python on a remote instance.\n",
    "\n",
    "Given this, it is important to never run <code>sudo pip install</code>. We need to know where and to what our Python packages apply, and generic (Linux-type) install routines confuse Linux conventions with Python ones. We are best in being explicit. There are conditions, then, where the idea of a Python virtual environment makes sense. These circumstances include, among others, a Python shop that has many projects; multiple developers with different versions and different applications; a mix of Python applications between Python 2 and 3 or where specific version dependencies create unworking conditions; and so forth. \n",
    "\n",
    "However, what I found in migrating to a remote instance is that virtual environments, combined with a remote instance and different operating system, added complexity that made finding and correcting problems difficult. After struggling for hours trying to get systems to work, not really knowing where the problem was occurring nor where to look for diagnostics, I learned some important things the hard way. I describe below some of these lessons."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### An Unfortunate Bad Fork\n",
    "So, I was convinced that a virtual environment made sense and set about following some [online](https://realpython.com/python-virtual-environments-a-primer/) [sources](https://www.codementor.io/@jqn/deploy-a-flask-app-on-aws-ec2-13hp1ilqy2) that documented this path. In general, these approaches used <code>virtualenv</code> (or <code>venv</code>), a pip-based environment manager, to set up this environment. Further, since I was using Ubuntu, AWS and Apache2, these aspects added to the constraints I needed to accommodate to pursue this path.\n",
    "\n",
    " <div style=\"background-color:#ffecec; border:1px dotted #f5aca6; vertical-align:middle; margin:15px 60px; padding:8px;\"> \n",
    "  <span style=\"font-weight: bold;\">Important Note:</span> For my configuration, this path is a dead end! If you have a similar configuration, do <strong>NOT</strong> follow this path. The section after this one presents the correct approach!</div>\n",
    "\n",
    "In implementing this path, I first installed <code>pip</code>:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sudo apt install -y python3-pip"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "I was now ready to tackle the fourth and last of my installation questions, namely to provide a virtual environment for my KBpedia-related Python projects. To do so, we first need to install the virtual environment package:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sudo apt install -y python3-venv"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Then, we make sure we are in the base directory where we want this virtual environment to reside. (In our case, <code>/usr/bin/python-projects/</code>. We also will name our virtual environment <code>kbpedia</code>. We first establish the virtual environment:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "python3.6 -m venv kbpedia"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "which pre-populates a directory with some skeletal capabilities. Then we activate it:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "source kbpedia/bin/activate"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The virtual environment is now active, and you can work with it as if you were are the standard command prompt, though that prompt does change in form to something like <code>(kbpedia) root@ip-xxx-xx-x-xx:/usr/bin/python-projects#</code>. You work with this directory as you normally would, adding test code next in our case. When done working with this environment, type <code>deactivate</code> to return from the virtual environment.\n",
    "\n",
    "The problem is, **none** of this worked for my circumstance, and likely never would. What I had neglected in taking this fork is that <code>conda</code> is **both** a package manager and a virtual environment manager. With the steps I had just taken, I had inadvertently set up multiple virtual environments, which were definitely in conflict."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### The Proper Installation\n",
    "Once I realized that my choice of <code>conda</code> meant I had already committed to a virtual environment manager, I needed to back off all of the installs I had undertaken with the bad fork. That meant I needed to uninstall all packages installed under that environment, remove the <code>venv</code> environment manager, remove all symbolic links, and so forth. I also needed to remove all Apache2 updates I had installed for working with <code>wsgi</code>. I had no confidence whatever I had installed had registered properly.\n",
    "\n",
    "The bad fork was a costly mistake, and it took me quite of bit of time to research and find the proper commands to remove the steps I had undertaken (which I do not document here). My intent was to get back to 'bare iron' for my remote installation so that I could pursue a clean install based on <code>conda</code>.\n",
    "\n",
    "#### Installing the mod-wsgi Apache Module\n",
    "\n",
    "After reclaiming the instance, my first step was to install the appropriate Apache2 modules to work with Python and <code>wsgi</code>. I began by installing the WSGI module to Apache2:\n",
    "\n",
    "<code>sudo apt-get install libapache2-mod-wsgi-py3</code>\n",
    "\n",
    "Which we then test to see if it was indeed installed properly:\n",
    "\n",
    "<code>apt-cache show libapache2-mod-wsgi-py3</code>\n",
    "\n",
    "Which displays:\n",
    "\n",
    "<pre>\n",
    "Package: libapache2-mod-wsgi-py3\n",
    "Architecture: amd64\n",
    "Version: 4.5.17-1ubuntu1\n",
    "Priority: optional\n",
    "Section: universe/httpd\n",
    "Source: mod-wsgi\n",
    "Origin: Ubuntu\n",
    "Maintainer: Ubuntu Developers <ubuntu-devel-discuss@lists.ubuntu.com>\n",
    "Original-Maintainer: Debian Python Modules Team <python-modules-team@lists.alioth.debian.org>\n",
    "Bugs: https://bugs.launchpad.net/ubuntu/+filebug\n",
    "Installed-Size: 271\n",
    "Provides: httpd-wsgi\n",
    "Depends: apache2-api-20120211, apache2-bin (>= 2.4.16), python3 (>= 3.6), python3 (<< 3.7), libc6 (>= 2.14), libpython3.6 (>= 3.6.5)\n",
    "Conflicts: libapache2-mod-wsgi\n",
    "Filename: pool/universe/m/mod-wsgi/libapache2-mod-wsgi-py3_4.5.17-1ubuntu1_amd64.deb\n",
    "Size: 88268\n",
    "MD5sum: 540669f9c5cc6d7a9980255656dd1273\n",
    "SHA1: 4130c072593fc7da07b2ff41a6eb7d8722afd9df\n",
    "SHA256: 6e443114d228c17f307736ee9145d6e6fcef74ff8f9ec872c645b951028f898b\n",
    "Homepage: http://www.modwsgi.org/\n",
    "Description-en: Python 3 WSGI adapter module for Apache\n",
    " The mod_wsgi adapter is an Apache module that provides a WSGI (Web Server\n",
    " Gateway Interface, a standard interface between web server software and\n",
    " web applications written in Python) compliant interface for hosting Python\n",
    " based web applications within Apache. The adapter provides significantly\n",
    " better performance than using existing WSGI adapters for mod_python or CGI.\n",
    " .\n",
    " This package provides module for Python 3.X.\n",
    "Description-md5: 9804c7965adca269cbc58c4a8eb236d8\n",
    "</pre>\n",
    "\n",
    "And next, we check to see if the module is properly registered:\n",
    "\n",
    "Our first test is:\n",
    "\n",
    "<code>sudo apachectl -t</code>\n",
    "\n",
    "and the second is:\n",
    "\n",
    "<code>sudo apachectl -M | grep wsgi</code>\n",
    "\n",
    "which gives us the correct answer:\n",
    "\n",
    "<code>wsgi_module (shared)</code>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Setting Up the Conda Virtual Environment\n",
    "\n",
    "We then proceed to create the 'kbpedia' virtual environment under <code>conda</code>:\n",
    "\n",
    "<code>conda create -n kbpedia python=3</code>\n",
    "\n",
    "Which we test with the Ubuntu path inquiry:\n",
    "\n",
    "<code>echo $PATH</code>\n",
    "\n",
    "which gives us the correct path registration (the first of the five listed):\n",
    "\n",
    "<pre>\n",
    "/usr/bin/python-projects/miniconda3/envs/kbpedia/bin:/usr/bin/python-projects/miniconda3/condabin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin\n",
    "</pre>\n",
    "\n",
    "and then we activate the 'kbpedia' environment:\n",
    "\n",
    "<code>conda activate kbpedia</code>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Installing Needed Packages\n",
    "\n",
    "Now that we are active in our virtual environment, we need to install our missing packages:\n",
    "\n",
    "<pre>\n",
    "conda install flask\n",
    "conda install pip\n",
    "</pre>\n",
    "\n",
    "#### Installing Needed Files\n",
    "For Flask to operate, we need two files. The first file is the basic calling application, <code>test_kbpedia.py</code>, that we place under the Web sites directory location, <code>/var/www/html/kbpedia/</code> (we create the <code>kbpedia</code> directory). We call up the editor for this new file:\n",
    "    \n",
    "<code>vi test_kbpedia.py</code>\n",
    "    \n",
    "and enter the following simple Python program:\n",
    "    \n",
    "<pre> \n",
    "from flask import Flask\n",
    "app = Flask(__name__)@app.route(\"/\")\n",
    "def hello():\n",
    "    return \"Hello KBpedia!\"\n",
    "</pre>\n",
    "\n",
    "It is important to know that Flask comes with its own minimal Web server, useful for only demo or test purposes, and one (because it can not be secured) should **NOT** be used in a production environment. Nonetheless, we can test this initial Flask specification by entering either of these two commands:\n",
    "\n",
    "<pre>\n",
    " wget http://127.0.0.1/ -O- \n",
    " \n",
    " curl http://127.0.0.1:5000\n",
    "</pre>\n",
    "\n",
    "Hmmm. These commands seem not to work. Something must not be correct in our Python file format. To get better feedback, we invoke Python and then:\n",
    "\n",
    "<code>flask run</code>\n",
    "\n",
    "This gives us the standard traceback listing that we have seen previously with Python programs. We get an error message that the name 'app' is not recognized. As we inspect the code more closely, we can see that one line in the example that we were following was inadvertenly truncated (denoted by the decorator '@' sign). We again edit <code>test_kbpedia.py</code> to now appear as follows:\n",
    "\n",
    "<pre> \n",
    "from flask import Flask\n",
    "app = Flask(__name__)\n",
    "@app.route(\"/\")\n",
    "def hello():\n",
    "    return \"Hello KBpedia!\"\n",
    "</pre>\n",
    "\n",
    "Great! That seems to fix the problem.\n",
    "    \n",
    "We next need to enter a second Python program that tells WSGI where to find this app. We call up the editor again in the same directory location:    \n",
    " \n",
    "<code>vi wsgi.py</code>\n",
    "  \n",
    "(We also may use the <code>*.wsgi</code> file extension if we so choose; some examples prefer this second option.)\n",
    "    \n",
    "and enter this simple program into our editor:\n",
    "    \n",
    "<pre>    \n",
    "import sys\n",
    "sys.path.insert(0, \"/var/www/html/kbpedia/\")\n",
    "from test_kbpedia import app as application\n",
    "</pre>\n",
    "    \n",
    "This program tells WSGI where to find the application, which we register via the <code>sys</code> package as the code states.\n",
    "    \n",
    "Lastly, to complete our needed chain of references, Apache2 needs instructions for where a 'kbpedia' reference in an incoming URI needs to point in order to find its corresponding resources. So, we go to the <code>/etc/apache2/sites-enabled</code> directory and then edit this file:    \n",
    "  \n",
    "<code>vi 000-default.conf</code>\n",
    "    \n",
    "Under <code>DocumentRoot</code> in this file, enter these instructions:\n",
    "\n",
    "<pre>\n",
    "        WSGIDaemonProcess kbpedia python-path=/usr/bin/python-projects/miniconda3/envs/kbpedia/lib/python3.8/site-packages\n",
    "        WSGIScriptAlias /kbpedia /var/www/html/kbpedia/wsgi.py\n",
    "        &lt;Directory /var/www/html/kbpedia&gt;\n",
    "            WSGIProcessGroup kbpedia\n",
    "            WSGIApplicationGroup %{GLOBAL}\n",
    "            Order deny,allow\n",
    "            Allow from all\n",
    "        &lt;/Directory&gt;\n",
    "</pre>   \n",
    "\n",
    "This provides the path for where to find the WSGI file and Python.\n",
    "\n",
    "We check to make sure all of these files are written correctly by entering this command:\n",
    "\n",
    "<code>sudo apachectl -t</code>\n",
    "\n",
    "When we first run this, while we get a 'Syntax OK' message, we also get a warning message that we need to \"Set the 'ServerName' directive globally to suppress this message\". While there is no problem to continue in this mode, in order to understand the suite of supporting files, we navigate to where the <code>ServerName</code> is set in the directory <code>/etc/apache2​</code> by editing this file:\n",
    "\n",
    "<code>vi apache2.conf</code>\n",
    "\n",
    "and enter this name:\n",
    "\n",
    "<code>ServerName localhost</code>\n",
    "\n",
    "Note anytime we make a change to an Apache2 configuration file, that we need to re-start the server to make sure the new configuration is now active. Here is the basic command:\n",
    "\n",
    "<code>sudo service apache2 restart</code>\n",
    "\n",
    "If there is no problem, the prompt is returned after entering the command.\n",
    "\n",
    "We are now complete with our initial inputs. To test whether we have properly set up Flask and our Web page serving, we use the IP for our instance and enter this command in a browser:\n",
    "\n",
    "<code>http://54.227.249.140/kbpedia</code>\n",
    "\n",
    "And, the browser returns a Web page stating:\n",
    "\n",
    "<code>Hello KBpedia!</code>\n",
    "\n",
    "Fantastic! We now are serving Web pages from our remote instance.\n",
    "\n",
    "Of course, this is only the simplest of examples, and we will need to invoke templates in order to use actual HTML and style sheets. These are topics we will undertake in the next installment. But, we have achieved a useful milestone in this three-part mini-series."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Some More conda Commands\n",
    "If you want to install libraries into an environment, or you want to use that environment, you have to activate the environment first by:\n",
    "\n",
    "<code>conda activate kbpedia</code>\n",
    "\n",
    "After invoking this command, we can install desired packages into the target environment, for example:\n",
    "\n",
    "<code>conda install package-name</code>\n",
    "\n",
    "But sometimes we need to use a different channel, in which case we first need to install that channel:\n",
    "\n",
    "<code>conda install --channel asmeurer</code>\n",
    "\n",
    "then, invoke it (say):\n",
    "\n",
    "<code>conda install -c conda-forge package-name</code>\n",
    "\n",
    "To see what packages are presently available to your conda environment, type:\n",
    "\n",
    "<code>conda list</code>\n",
    "\n",
    "And, to see what environments you have set up within <code>conda</code>, enter:\n",
    "\n",
    "<code>conda env list</code>\n",
    "\n",
    "which, in our current circumstance, give us this result:\n",
    "\n",
    "<pre>\n",
    "base                     /usr/bin/python-projects/miniconda3\n",
    "kbpedia               *  /usr/bin/python-projects/miniconda3/envs/kbpedia\n",
    "</pre>\n",
    "\n",
    "The environment shown with the asterisk (<code>*</code>) is the currently active one.\n",
    "\n",
    "Another useful command to know is to get full information on your currently active conda environment. To do so, type:\n",
    "\n",
    "<code>conda info</code>\n",
    "\n",
    "in our case, that produces the following output:\n",
    "\n",
    "<pre>\n",
    "\n",
    "     active environment : kbpedia\n",
    "    active env location : /usr/bin/python-projects/miniconda3/envs/kbpedia\n",
    "            shell level : 1\n",
    "       user config file : /root/.condarc\n",
    " populated config files :\n",
    "          conda version : 4.8.4\n",
    "    conda-build version : not installed\n",
    "         python version : 3.8.3.final.0\n",
    "       virtual packages : __glibc=2.27\n",
    "       base environment : /usr/bin/python-projects/miniconda3  (writable)\n",
    "           channel URLs : https://repo.anaconda.com/pkgs/main/linux-64\n",
    "                          https://repo.anaconda.com/pkgs/main/noarch\n",
    "                          https://repo.anaconda.com/pkgs/r/linux-64\n",
    "                          https://repo.anaconda.com/pkgs/r/noarch\n",
    "          package cache : /usr/bin/python-projects/miniconda3/pkgs\n",
    "                          /root/.conda/pkgs\n",
    "       envs directories : /usr/bin/python-projects/miniconda3/envs\n",
    "                          /root/.conda/envs\n",
    "               platform : linux-64\n",
    "             user-agent : conda/4.8.4 requests/2.24.0 CPython/3.8.3 Linux/4.15.0-115-generic ubuntu/16.04.1 glibc/2.27\n",
    "                UID:GID : 0:0\n",
    "             netrc file : None\n",
    "           offline mode : False\n",
    "\n",
    "</pre>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Additional Documentation\n",
    "Here are some additional resources touched upon in the prior discussions:\n",
    "\n",
    "#### Getting Oriented\n",
    "- [The Definitive Guide to Conda Environments](https://towardsdatascience.com/a-guide-to-conda-environments-bc6180fc533)\n",
    "- [Deploy Flask apps using Anaconda on Ubuntu Server](https://peteris.rocks/blog/deploy-flask-apps-using-anaconda-on-ubuntu-server/)\n",
    "- [Why You Need Python Environments and How to Manage Them with Conda](https://protostar.space/why-you-need-python-environments-and-how-to-manage-them-with-conda)\n",
    "- [Conda Best Practices](https://docs.csc.fi/support/tutorials/conda/).\n",
    "\n",
    "#### Flask on AWS\n",
    "Each of these cover some of the first steps needed to get set up on AWS, which we skip over herein:\n",
    "- [Deploy on AWS](https://www.codementor.io/@jqn/deploy-a-flask-app-on-aws-ec2-13hp1ilqy2) is a nice starting point, but not specific to our conda + AWS + Flask + Apache2 environment\n",
    "- [Flask and Apache and AWS](https://vishnut.me/blog/ec2-flask-apache-setup.html) is also pretty good, but lacks conda\n",
    "- [Another Deploy on AWS](https://ketakirk.wordpress.com/deploy-an-app-on-aws/) with the later steps useful, from about #7 forward.\n",
    "\n",
    "#### RESTful APIs\n",
    "- https://python-rest-framework.readthedocs.io/en/latest/tutorial/first_step.html (Python REST API) (a set of utilities based on werkzeug)\n",
    "- https://pypi.org/project/endpoints/\n",
    "- https://programminghistorian.org/en/lessons/creating-apis-with-python-and-flask\n",
    "- https://rapidapi.com/blog/how-to-use-an-api-with-python/\n",
    "- https://auth0.com/blog/developing-restful-apis-with-python-and-flask/\n",
    "- [Flask-RESTful](https://flask-restful.readthedocs.io/en/latest/)\n",
    "- [Another source](http://michal.karzynski.pl/blog/2016/06/19/building-beautiful-restful-apis-using-flask-swagger-ui-flask-restplus/) includes use of Swagger UI documenter and Flask-RESTplus.\n",
    "\n",
    "#### General Flask Resources\n",
    "- [A good, official introductory site to Flask](https://flask.palletsprojects.com/en/1.1.x/)\n",
    "  - [Using URL processors](https://flask.palletsprojects.com/en/1.1.x/patterns/urlprocessors/)\n",
    "  - [Using SQL3Lite with Flask](https://flask.palletsprojects.com/en/1.1.x/patterns/sqlite3/)\n",
    "  - [Quickstart](https://flask.palletsprojects.com/en/1.1.x/quickstart/)\n",
    "- [Useful reference on Flask as a Web framework](https://www.fullstackpython.com/flask.html)  \n",
    "- [Another good starting point](https://programminghistorian.org/en/lessons/creating-apis-with-python-and-flask) that also uses SQLite3 as an example backend database  \n",
    "- [The Mega Tutorial](https://blog.miguelgrinberg.com/post/the-flask-mega-tutorial-part-i-hello-world), and there is a [follow-up with emphasis on a RESTful Web API](https://blog.miguelgrinberg.com/post/designing-a-restful-api-with-python-and-flask). \n",
    "\n",
    "#### RDFlib\n",
    "Here is a [nice overview of RDFlib](https://www.wikiwand.com/en/RDFLib).\n",
    "\n",
    "#### General Remote Instance Set-up\n",
    "A [video on setting up an EC2 instance and Putty](https://www.youtube.com/watch?v=WE303yFWfV4); also deals with updating Python, Filezilla and crontab.\n",
    "\n",
    "#### Other Web Page Resources\n",
    "- [Ordia](http://www2.imm.dtu.dk/pubdb/edoc/imm7137.pdf) is a [Wikimedia tools](https://hay.toolforge.org/directory/#) app built on Flask that may have some aspects relevant to an eventual KBpedia Web site\n",
    "- [Converting Jupyter to WordPress HTML](https://indoml.com/2019/09/24/converting-jupyter-notebook-to-wordpress-com-html/)\n",
    "- We may also want to bring some graph visualizations to our Web pages. One interesting package is [Webweb](https://webwebpage.github.io/) and its related [paper](https://joss.theoj.org/papers/10.21105/joss.01458); another is [WebVOWL](http://www.visualdataweb.de/webvowl/).\n",
    "\n",
    "\n",
    " <div style=\"background-color:#ffecec; border:1px dotted #f5aca6; vertical-align:middle; margin:15px 60px; padding:8px;\"> \n",
    "  <span style=\"font-weight: bold;\">NOTE:</span> This article is part of the <a href=\"https://www.mkbergman.com/cooking-with-python-and-kbpedia/\" style=\"font-style: italic;\">Cooking with Python and KBpedia</a> series. See the <a href=\"https://www.mkbergman.com/cooking-with-python-and-kbpedia/\"><strong>CWPK</strong> listing</a> for other articles in the series. <a href=\"http://kbpedia.org/\">KBpedia</a> has its own Web site. The <em>cowpoke</em> Python <a href=\"https://github.com/Cognonto/cowpoke\">code listing covering the series</a> is also available from GitHub.\n",
    "  </div>\n",
    "\n",
    "<div style=\"background-color:#ebf8e2; border:1px dotted #71c837; vertical-align:middle; margin:15px 60px; padding:8px;\"> \n",
    "\n",
    "<span style=\"font-weight: bold;\">NOTE:</span> This <strong>CWPK \n",
    "installment</strong> is available both as an online interactive\n",
    "file <a href=\"https://mybinder.org/v2/gh/Cognonto/CWPK/master\" ><img src=\"https://mybinder.org/badge_logo.svg\" style=\"display:inline-block; vertical-align: middle;\" /></a> or as a <a href=\"https://github.com/Cognonto/CWPK\" title=\"CWPK notebook\" alt=\"CWPK notebook\">direct download</a> to use locally. Make sure and pick the correct installment number. For the online interactive option, pick the <code>*.ipynb</code> file. It may take a bit of time for the interactive option to load.</div>\n",
    "\n",
    "<div style=\"background-color:#feeedc; border:1px dotted #f7941d; vertical-align:middle; margin:15px 60px; padding:8px;\"> \n",
    "<div style=\"float: left; margin-right: 5px;\"><img src=\"http://kbpedia.org/cwpk-files/warning.png\" title=\"Caution!\" width=\"32\" /></div>I am at best an amateur with Python. There are likely more efficient methods for coding these steps than what I provide. I encourage you to experiment -- which is part of the fun of Python -- and to <a href=\"mailto:mike@mkbergman.com\">notify me</a> should you make improvements.    \n",
    "\n",
    "</div>"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}