cwpk-52-interactive-web-widgets-2.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "CWPK \\#52: Distributed Interactions via Web Widgets - II \n",
    "=======================================\n",
    "\n",
    "A Design for a Working Ecosystem of Existing Applications\n",
    "--------------------------\n",
    "\n",
    "<div style=\"float: left; width: 305px; margin-right: 10px;\">\n",
    "\n",
    "<img src=\"http://kbpedia.org/cwpk-files/cooking-with-kbpedia-305.png\" title=\"Cooking with KBpedia\" width=\"305\" />\n",
    "\n",
    "</div>\n",
    "\n",
    "In our last installment of the [*Cooking with Python and KBpedia*](https://www.mkbergman.com/cooking-with-python-and-kbpedia/) series, I discussed the rationale for using distributed Web widgets as one means to bring the use and management of a knowledge graph into closer alignment with existing content tasks. The idea is to embed small Web controls as either plug-ins, enhancements to existing application Web interfaces, or as simply accompanying Web pages. Under this design, ease-of-use and immediacy are paramount in order to facilitate the capture of new knowledge if and where it arises.\n",
    "\n",
    "More complicated knowledge graph tasks like initial builds, bulk changes, logic testing and the like are therefore kept as separate tasks. For our immediate knowledge purposes, knowledge workers should be able to issue suggestions for new concepts or modifications or deletions when they are discovered. These suggested modifications may be made to a working version of the knowledge graph that operates in parallel to a public (or staff-wide) release version. \n",
    "\n",
    "The basic concept behind this and the previous installment is that we would like to have the ability to use simple widgets -- embedded on any Web page for the applications we use -- as a means of capturing and communicating immediate new information of relevance to our knoweldge graphs. In a distributed manner, while working with production versions, we may want to communicate these updates to an interim version in the queue for review and promotion via a governance pipeline that might lead to a new production version.\n",
    "\n",
    "On some periodic basis, this modified working version could be inspected and tested by assigned managers with the responsibility to vet new, public versions. There is much governance and policy that may guide these workflows, which may also be captured by their own ontologies, that is a topic separate from the widget infrastructure to make this vision operational.\n",
    "\n",
    "While there are widgets available for analysis and visualization purposes, which we will touch upon in later installments, the ones we will emphasize in today's **CWPK** installment are focused on [CRUD](https://en.wikipedia.org/wiki/Create,_read,_update_and_delete) (*create - read - update - delete*) activities. CRUD is the means by which we manage our knowledge graphs in the immediate sense. We will be looking to develop these widgets in a manner directly useful to distributed applications."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Getting Started\n",
    "Remember from our last installment that we are basing our approach to code examples on the [Jupyter Notebook](https://en.wikipedia.org/wiki/Project_Jupyter#Jupyter_Notebook) [<code>ipywidgets</code>](https://ipywidgets.readthedocs.io/en/latest/) package. That package does not come pre-installed, so we need to install it in our system using <code>conda</code>:\n",
    "\n",
    "<pre>\n",
    "  conda install -c conda-forge ipywidgets\n",
    "</pre>\n",
    "\n",
    "Once done, we fire up our Notebook and begin with our standard start-up, only now including a new [<code>autoFill</code> module](https://github.com/germanesosa/ipywidget-autocomplete), which I explain next. This [module](https://github.com/germanesosa/ipywidget-autocomplete) is an example of an <code>ipywidgets</code> extension. Place this new <code>autoFill.py</code> page as a new module page within your [Spyder](https://en.wikipedia.org/wiki/Spyder_(software))  *cowpoke* project. Make sure and copy that file into the project before beginning the startup:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from cowpoke.__main__ import *\n",
    "from cowpoke.config import *\n",
    "from cowpoke.autoFill import *\n",
    "from owlready2 import *"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Some Basic Controls (Widgets)\n",
    "As noted, our basic controls for this installment revolve around CRUD, though we will not address them in that order. But before we can start manipulating individual objects, we need ways to discover what is already in the KBpedia (or your own) knowledge graph. Search and [auto-completion](https://en.wikipedia.org/wiki/Autocomplete) are two essential tools for this job, particularly given the fact that KBpedia has more than 58,000 concepts and 5,000 different properties."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Basic Search\n",
    "Recall in [**CWPK #23**](https://www.mkbergman.com/2356/cwpk-23-text-searching-kbpedia/) that we covered owlready2's basic search function and parameters. You may use the interactive commands shown in that documentation to search by type, subclasses, IRI, etc."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Auto-complete Helper\n",
    "We list the auto-completion helper next because it is leveraged by virtually every widget. Auto-completion is used in a text entry box where as characters are typed, a dropdown list provides matching items that satisfy the search string as entered so far. It is a useful tool to discover what exists in a system as well as to provide the exact spelling and capitalization, which may be necessary for the match. As suggested items appear in the dropdown, if the right one appears you click it to make your current selection.\n",
    "\n",
    "The added utility we found for this is the [ipywidget-autocomplete](https://github.com/germanesosa/ipywidget-autocomplete) module, which we have added as our own module to *cowpoke*. The module provides the underlying 'autofill' code that we import in the actual routine that we use within the Notebook. Here is an example of that Notebook code, with some explanations that follow below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "f4612d4d7fbf460eba7ae914b9787dae",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(VBox(children=(Box(children=(Text(value='', placeholder='Please type here...'),)), Box(children…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import cowpoke.autoFill as af\n",
    "from ipywidgets import *                                  # Note 1\n",
    "\n",
    "def open(value):\n",
    "    show.value=value.new\n",
    "\n",
    "strlist = []                                              # Note 2\n",
    "listing = list(kb.classes())                              # Note 3\n",
    "for item in listing:                                      # Note 4\n",
    "    item = str(item)\n",
    "    item = item.replace('rc.', '')\n",
    "    strlist.append(item)\n",
    "\n",
    "autofill = af.autoFill(strlist,callback=open)\n",
    "\n",
    "show = HTML(\"Begin typing substring characters to see 'auto-complete'!\")\n",
    "\n",
    "display(HBox([autofill,show]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Besides the 'autofill' component, we also are importing some of the basic <code>ipywidgets</code> code **(1)**. In this instance, we want to auto-complete on the 58 K concepts in KBpedia **(3)**, which we have to convert from a listing of classes to a listing of strings **(1)** and **(4)**. We can just as easily do an auto-complete on properties by changing one line **(3)**:\n",
    "\n",
    "<pre>\n",
    "  listing = list(kb.classes())\n",
    "</pre>\n",
    "\n",
    "or, of course, other specifications may be entered for the boundaries of our auto-complete.\n",
    "\n",
    "To convert the owlready2 listing of classes to strings occurs by looping over all items in the list, converting each item to a string, replacing the 'rc.' prefix, and then appending the new string item to a new <code>strlist</code> **(4)**. The 'callback' option relates the characters typed into the text box with this string listing. This particular expression will find string matches at any position (not just the beginning) and is case insensitive.\n",
    "\n",
    "When the item appears in the dropdown list that matches your interest, pick it. This value can then be retrieved with the following statement:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Mammal'"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "show.value"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Read Item\n",
    "Let's use a basic *record* format to show how individual property values may be obtained for a given reference concept (RC), as we just picked with the auto-complete helper. We could obviously expand this example to include any of the possible RC property values, but we will use the most prominent ones here:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "b295285c754f477ba1ed3005887fd424",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(VBox(children=(Text(value=\"['mammal']\", description='Preferred Label:', layout=Layout(width='76…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "r_val = show.value \n",
    "r_val = getattr(rc, r_val)                               # Note 1                                   \n",
    "a_val = str(r_val.prefLabel)                             # Note 2\n",
    "b_val = r_val.altLabel\n",
    "b_items = ''\n",
    "for index, item in enumerate(b_val):                     # Note 3 \n",
    "    item = str(item)\n",
    "    if index == [0]:        \n",
    "        b_items = item\n",
    "    else: \n",
    "        b_items = b_items + '||' + item\n",
    "b_val = b_items\n",
    "c_val = str(r_val.definition)\n",
    "d_val = r_val.subclasses()\n",
    "d_items = ''\n",
    "for index, item in enumerate(d_val):                     # Note 3\n",
    "    item = str(item)\n",
    "    if index == [0]:        \n",
    "        d_items = item\n",
    "    else: \n",
    "        d_items = d_items + '||' + item\n",
    "d_val = d_items\n",
    "e_val = str(r_val.wikidata_q_id)\n",
    "f_val = str(r_val.wikipedia_id)\n",
    "g_val = str(r_val.schema_org_id)\n",
    "                                                         # Note 4 \n",
    "a = widgets.Text(style={'description_width': '100px'}, layout=Layout(width='760px'), \n",
    "                 description='Preferred Label:', value = a_val)\n",
    "b = widgets.Text(style={'description_width': '100px'}, layout=Layout(width='760px'), \n",
    "                 description='AltLabel(s):', value = b_val)\n",
    "c = widgets.Textarea(style={'description_width': '100px'}, layout=Layout(width='760px'), \n",
    "                 description='Definition:', value = c_val)\n",
    "d = widgets.Textarea(style={'description_width': '100px'}, layout=Layout(width='760px'), \n",
    "                 description='subClasses:', value = d_val)\n",
    "e = widgets.Text(style={'description_width': '100px'}, layout=Layout(width='760px'), \n",
    "                 description='Q ID:', value = e_val)\n",
    "f = widgets.Text(style={'description_width': '100px'}, layout=Layout(width='760px'), \n",
    "                 description='Wikipedia ID:', value = f_val)\n",
    "g = widgets.Text(style={'description_width': '100px'}, layout=Layout(width='760px'), \n",
    "                 description='schema.org ID:', value = g_val)\n",
    "\n",
    "def f_out(a, b, c, d, e, f, g):\n",
    "    print(''.format(a, b, c, d, e, f, g))\n",
    "\n",
    "out = widgets.interactive_output(f_out, {'a': a, 'b': b, 'c': c, 'd': d, 'e': e, 'f': f, 'g': g})\n",
    "\n",
    "widgets.HBox([widgets.VBox([a, b, c, d, e, f, g]), out]) # Note 5 "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We begin by grabbing the <code>show.value</code> value **(1)** that came from our picking an item from the auto-complete list. We then start retrieving individual attribute values **(2)** for that resource, some of which we need to iterate over **(3)** because they return multiple items in a list. For display purposes, we need to convert all retrieved property values to strings.\n",
    "\n",
    "We can style our widgets **(4)**. 'Layout' applies to the entire widget, and 'style' applies to selected elements. Other values are possible, depending on the widget, which we can inspect with this statement (vary by widget type):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "text = Text(description='text box')\n",
    "print(text.style.keys)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Then, after defining a simple call procedure, we invoke the control on the Notebook page **(5)**.\n",
    "\n",
    "We could do more to clean up interim output values (such as removing brackets and quotes as we have done elsewhere), and can get fancier with grid layouts and such. In these regards, the Notebook widgets tend to work like and have parameter settings similar to other HTML widgets, though the amount of examples and degree of control is not as extensive as other widget libraries.\n",
    "\n",
    "Again, though, since so much of this in an actual deployment would need to be tailored for other Web frameworks and environments, we can simply state for now that quite a bit of control is available, depending on language, for bringing your knowledge graph information local to your current applications."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Modify (Update) Item\n",
    "Though strings in Python are what is known as 'immutable' (unchanging), it is possible through the <code>string.replace('old', 'new')</code> option to update or modify strings. For entire strings, 'old' may be brought into a text box via a variable name as shown above, altered, and then captured as <code>show.value</code> on a click event. The basic format of this approach can be patterned as follows (changing properties as appropriate):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "00f72bf133244a5a8481a67abd7e7490",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Text(value='This is the old input.', description='Value to modify:', layout=Layout(width='760px'), style=Descr…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from ipywidgets import widgets                      # Run the code cell to start\n",
    "\n",
    "old_text = widgets.Text(style={'description_width': '100px'}, layout=Layout(width='760px'), description = 'Value to modify:', value = 'This is the old input.')\n",
    "new_text = widgets.Text(description = 'New value:', value = 'This is the NEW value.')\n",
    "\n",
    "def bind_old_to_new(sender):\n",
    "    old_text.value = new_text.value\n",
    "    old_text.description = new_text.description\n",
    "old_text.on_submit(bind_old_to_new)\n",
    "\n",
    "old_text                                            # Click on the text box to invoke"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It is better to make modifications to an existing property or an existing class using something like [Protégé](https://en.wikipedia.org/wiki/Prot%C3%A9g%C3%A9_(software)), which is able to better account for internal cross-references. But, changes to attribute values may be pursued through the above approach.\n",
    "\n",
    "New class names or properties, or modifications to existing class names or properties, should never be allowed to production systems. These CRUD actions, like *create*, are best slipstreamed into a reserved cache or made against a review copy of the knowledge base that is segregated away from the production system. This separation allows review steps and the enforcement of governance standards before changes are made to an operating version of the knowledge graph."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Create Item\n",
    "Newly created items include new concepts and their attributes or subsumption relations, which deserve scrutiny before committing to a production version. Other items that perhaps deserve a lower degree of governance before adding include additional <code>altLabels</code> or mapping properties. \n",
    "\n",
    "In the case of new concepts, the form of addition (see also [**CWPK #39**](https://www.mkbergman.com/2378/cwpk-39-i-o-and-structural-ingest/), Note #6) has a number of forms, but one has worked best for KBpedia. \n",
    "\n",
    "On the other hand, the general form for adding a new attribute has the form:\n",
    "\n",
    "<pre>\n",
    "  CurResource.append(new_item)\n",
    "</pre>\n",
    "\n",
    "As with modifications, creating items should be limited to working, and not production copies."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Delete Item\n",
    "The form to delete an item is very simple, and takes the form:\n",
    "\n",
    "<pre>\n",
    "  destroy_entity(xxxName)\n",
    "</pre>\n",
    "\n",
    "Clearly, this is a function that should not be made available to casual users, and needs to be subject to governance policies and review. Again, working copies are the best versions to be the target of any such actions, and the acceptance of deletions should be an active subject of review. Forms to enable this are simple to build."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Not Part of the Package\n",
    "For the reasons of diversity of use and applications, none of these routines has been formally added to the *cowpoke* package. We see examples of how such can be implemented, and I have argued for the use of distributed Web widgets under governance and using working copies as the best means for utilizing these functions. But, given this diversity, it does not make sense to add widgets to the *cowpoke* package at this time.\n",
    "\n",
    "That being said, we also hope some of this discussion and related code commentary have encouraged you to think of how to apply workflows and life-cycle management to your own knowledge graphs. Embedding widgets into existing distributed applications is one means to bring the benefits and updates of knowledge graphs into immediate relevance."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Additional Documentation\n",
    "Here is additional documentation of working with the <code>ipywidgest</code> package:\n",
    "- [Official documentation](https://ipywidgets.readthedocs.io/en/latest/)\n",
    "  - The [<code>interact</code>](https://ipywidgets.readthedocs.io/en/latest/examples/Using%20Interact.html)\n",
    "  - A <code>ipywidgets</code> [tutorial](https://coderzcolumn.com/tutorials/python/interactive-widgets-in-jupyter-notebook-using-ipywidgets)\n",
    "  - Another good [tutorial](https://www.blog.pythonlibrary.org/2018/10/24/working-with-jupyter-notebook-widgets/)\n",
    "  - Explanation of <code>ipywidgets</code> [layouts](https://www.programcreek.com/python/example/90678/ipywidgets.Layout).\n",
    "  \n",
    "In addition, there are other interactive uses of Notebooks:  \n",
    "- The [Voilà](https://voila-gallery.org/) option for publishing interactive Jupyter Notebook pages  \n",
    "- [Creating interactive dashboards](https://jupyter-flex.extrapolations.dev/) with Jupyter.   \n",
    "      \n",
    "\n",
    " <div style=\"background-color:#ffecec; border:1px dotted #f5aca6; vertical-align:middle; margin:15px 60px; padding:8px;\"> \n",
    "  <span style=\"font-weight: bold;\">NOTE:</span> This article is part of the <a href=\"https://www.mkbergman.com/cooking-with-python-and-kbpedia/\" style=\"font-style: italic;\">Cooking with Python and KBpedia</a> series. See the <a href=\"https://www.mkbergman.com/cooking-with-python-and-kbpedia/\"><strong>CWPK</strong> listing</a> for other articles in the series. <a href=\"http://kbpedia.org/\">KBpedia</a> has its own Web site. The <em>cowpoke</em> Python <a href=\"https://github.com/Cognonto/cowpoke\">code listing covering the series</a> is also available from GitHub.\n",
    "  </div>\n",
    "\n",
    "<div style=\"background-color:#ebf8e2; border:1px dotted #71c837; vertical-align:middle; margin:15px 60px; padding:8px;\"> \n",
    "\n",
    "<span style=\"font-weight: bold;\">NOTE:</span> This <strong>CWPK \n",
    "installment</strong> is available both as an online interactive\n",
    "file <a href=\"https://mybinder.org/v2/gh/Cognonto/CWPK/master\" ><img src=\"https://mybinder.org/badge_logo.svg\" style=\"display:inline-block; vertical-align: middle;\" /></a> or as a <a href=\"https://github.com/Cognonto/CWPK\" title=\"CWPK notebook\" alt=\"CWPK notebook\">direct download</a> to use locally. Make sure and pick the correct installment number. For the online interactive option, pick the <code>*.ipynb</code> file. It may take a bit of time for the interactive option to load.</div>\n",
    "\n",
    "<div style=\"background-color:#feeedc; border:1px dotted #f7941d; vertical-align:middle; margin:15px 60px; padding:8px;\"> \n",
    "<div style=\"float: left; margin-right: 5px;\"><img src=\"http://kbpedia.org/cwpk-files/warning.png\" title=\"Caution!\" width=\"32\" /></div>I am at best an amateur with Python. There are likely more efficient methods for coding these steps than what I provide. I encourage you to experiment -- which is part of the fun of Python -- and to <a href=\"mailto:mike@mkbergman.com\">notify me</a> should you make improvements.    \n",
    "\n",
    "</div>"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}