16_reinforcement_learning.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Chapter 16 – Reinforcement Learning**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This notebook contains all the sample code and solutions to the exersices in chapter 16."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Setup"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First, let's make sure this notebook works well in both python 2 and 3, import a few common modules, ensure MatplotLib plots figures inline and prepare a function to save the figures:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# To support both python 2 and python 3\n",
    "from __future__ import division, print_function, unicode_literals\n",
    "\n",
    "# Common imports\n",
    "import numpy as np\n",
    "import os\n",
    "import sys\n",
    "\n",
    "# to make this notebook's output stable across runs\n",
    "def reset_graph(seed=42):\n",
    "    tf.reset_default_graph()\n",
    "    tf.set_random_seed(seed)\n",
    "    np.random.seed(seed)\n",
    "\n",
    "# To plot pretty figures and animations\n",
    "%matplotlib nbagg\n",
    "import matplotlib\n",
    "import matplotlib.animation as animation\n",
    "import matplotlib.pyplot as plt\n",
    "plt.rcParams['axes.labelsize'] = 14\n",
    "plt.rcParams['xtick.labelsize'] = 12\n",
    "plt.rcParams['ytick.labelsize'] = 12\n",
    "\n",
    "# Where to save the figures\n",
    "PROJECT_ROOT_DIR = \".\"\n",
    "CHAPTER_ID = \"rl\"\n",
    "\n",
    "def save_fig(fig_id, tight_layout=True):\n",
    "    path = os.path.join(PROJECT_ROOT_DIR, \"images\", CHAPTER_ID, fig_id + \".png\")\n",
    "    print(\"Saving figure\", fig_id)\n",
    "    if tight_layout:\n",
    "        plt.tight_layout()\n",
    "    plt.savefig(path, format='png', dpi=300)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note: there may be minor differences between the output of this notebook and the examples shown in the book. You can safely ignore these differences. They are mainly due to the fact that most of the environments provided by OpenAI gym have some randomness."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction to OpenAI gym"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this notebook we will be using [OpenAI gym](https://gym.openai.com/), a great toolkit for developing and comparing Reinforcement Learning algorithms. It provides many environments for your learning *agents* to interact with. Let's start by importing `gym`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import gym"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next we will load the MsPacman environment, version 0."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[2017-09-25 11:35:03,438] Making new env: MsPacman-v0\n"
     ]
    }
   ],
   "source": [
    "env = gym.make('MsPacman-v0')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's initialize the environment by calling is `reset()` method. This returns an observation:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "obs = env.reset()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Observations vary depending on the environment. In this case it is an RGB image represented as a 3D NumPy array of shape [width, height, channels] (with 3 channels: Red, Green and Blue). In other environments it may return different objects, as we will see later."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(210, 160, 3)"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "obs.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "An environment can be visualized by calling its `render()` method, and you can pick the rendering mode (the rendering options depend on the environment). In this example we will set `mode=\"rgb_array\"` to get an image of the environment as a NumPy array:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "img = env.render(mode=\"rgb_array\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's plot this image:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "application/javascript": [
       "/* Put everything inside the global mpl namespace */\n",
       "window.mpl = {};\n",
       "\n",
       "\n",
       "mpl.get_websocket_type = function() {\n",
       "    if (typeof(WebSocket) !== 'undefined') {\n",
       "        return WebSocket;\n",
       "    } else if (typeof(MozWebSocket) !== 'undefined') {\n",
       "        return MozWebSocket;\n",
       "    } else {\n",
       "        alert('Your browser does not have WebSocket support.' +\n",
       "              'Please try Chrome, Safari or Firefox ≥ 6. ' +\n",
       "              'Firefox 4 and 5 are also supported but you ' +\n",
       "              'have to enable WebSockets in about:config.');\n",
       "    };\n",
       "}\n",
       "\n",
       "mpl.figure = function(figure_id, websocket, ondownload, parent_element) {\n",
       "    this.id = figure_id;\n",
       "\n",
       "    this.ws = websocket;\n",
       "\n",
       "    this.supports_binary = (this.ws.binaryType != undefined);\n",
       "\n",
       "    if (!this.supports_binary) {\n",
       "        var warnings = document.getElementById(\"mpl-warnings\");\n",
       "        if (warnings) {\n",
       "            warnings.style.display = 'block';\n",
       "            warnings.textContent = (\n",
       "                \"This browser does not support binary websocket messages. \" +\n",
       "                    \"Performance may be slow.\");\n",
       "        }\n",
       "    }\n",
       "\n",
       "    this.imageObj = new Image();\n",
       "\n",
       "    this.context = undefined;\n",
       "    this.message = undefined;\n",
       "    this.canvas = undefined;\n",
       "    this.rubberband_canvas = undefined;\n",
       "    this.rubberband_context = undefined;\n",
       "    this.format_dropdown = undefined;\n",
       "\n",
       "    this.image_mode = 'full';\n",
       "\n",
       "    this.root = $('<div/>');\n",
       "    this._root_extra_style(this.root)\n",
       "    this.root.attr('style', 'display: inline-block');\n",
       "\n",
       "    $(parent_element).append(this.root);\n",
       "\n",
       "    this._init_header(this);\n",
       "    this._init_canvas(this);\n",
       "    this._init_toolbar(this);\n",
       "\n",
       "    var fig = this;\n",
       "\n",
       "    this.waiting = false;\n",
       "\n",
       "    this.ws.onopen =  function () {\n",
       "            fig.send_message(\"supports_binary\", {value: fig.supports_binary});\n",
       "            fig.send_message(\"send_image_mode\", {});\n",
       "            if (mpl.ratio != 1) {\n",
       "                fig.send_message(\"set_dpi_ratio\", {'dpi_ratio': mpl.ratio});\n",
       "            }\n",
       "            fig.send_message(\"refresh\", {});\n",
       "        }\n",
       "\n",
       "    this.imageObj.onload = function() {\n",
       "            if (fig.image_mode == 'full') {\n",
       "                // Full images could contain transparency (where diff images\n",
       "                // almost always do), so we need to clear the canvas so that\n",
       "                // there is no ghosting.\n",
       "                fig.context.clearRect(0, 0, fig.canvas.width, fig.canvas.height);\n",
       "            }\n",
       "            fig.context.drawImage(fig.imageObj, 0, 0);\n",
       "        };\n",
       "\n",
       "    this.imageObj.onunload = function() {\n",
       "        this.ws.close();\n",
       "    }\n",
       "\n",
       "    this.ws.onmessage = this._make_on_message_function(this);\n",
       "\n",
       "    this.ondownload = ondownload;\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._init_header = function() {\n",
       "    var titlebar = $(\n",
       "        '<div class=\"ui-dialog-titlebar ui-widget-header ui-corner-all ' +\n",
       "        'ui-helper-clearfix\"/>');\n",
       "    var titletext = $(\n",
       "        '<div class=\"ui-dialog-title\" style=\"width: 100%; ' +\n",
       "        'text-align: center; padding: 3px;\"/>');\n",
       "    titlebar.append(titletext)\n",
       "    this.root.append(titlebar);\n",
       "    this.header = titletext[0];\n",
       "}\n",
       "\n",
       "\n",
       "\n",
       "mpl.figure.prototype._canvas_extra_style = function(canvas_div) {\n",
       "\n",
       "}\n",
       "\n",
       "\n",
       "mpl.figure.prototype._root_extra_style = function(canvas_div) {\n",
       "\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._init_canvas = function() {\n",
       "    var fig = this;\n",
       "\n",
       "    var canvas_div = $('<div/>');\n",
       "\n",
       "    canvas_div.attr('style', 'position: relative; clear: both; outline: 0');\n",
       "\n",
       "    function canvas_keyboard_event(event) {\n",
       "        return fig.key_event(event, event['data']);\n",
       "    }\n",
       "\n",
       "    canvas_div.keydown('key_press', canvas_keyboard_event);\n",
       "    canvas_div.keyup('key_release', canvas_keyboard_event);\n",
       "    this.canvas_div = canvas_div\n",
       "    this._canvas_extra_style(canvas_div)\n",
       "    this.root.append(canvas_div);\n",
       "\n",
       "    var canvas = $('<canvas/>');\n",
       "    canvas.addClass('mpl-canvas');\n",
       "    canvas.attr('style', \"left: 0; top: 0; z-index: 0; outline: 0\")\n",
       "\n",
       "    this.canvas = canvas[0];\n",
       "    this.context = canvas[0].getContext(\"2d\");\n",
       "\n",
       "    var backingStore = this.context.backingStorePixelRatio ||\n",
       "\tthis.context.webkitBackingStorePixelRatio ||\n",
       "\tthis.context.mozBackingStorePixelRatio ||\n",
       "\tthis.context.msBackingStorePixelRatio ||\n",
       "\tthis.context.oBackingStorePixelRatio ||\n",
       "\tthis.context.backingStorePixelRatio || 1;\n",
       "\n",
       "    mpl.ratio = (window.devicePixelRatio || 1) / backingStore;\n",
       "\n",
       "    var rubberband = $('<canvas/>');\n",
       "    rubberband.attr('style', \"position: absolute; left: 0; top: 0; z-index: 1;\")\n",
       "\n",
       "    var pass_mouse_events = true;\n",
       "\n",
       "    canvas_div.resizable({\n",
       "        start: function(event, ui) {\n",
       "            pass_mouse_events = false;\n",
       "        },\n",
       "        resize: function(event, ui) {\n",
       "            fig.request_resize(ui.size.width, ui.size.height);\n",
       "        },\n",
       "        stop: function(event, ui) {\n",
       "            pass_mouse_events = true;\n",
       "            fig.request_resize(ui.size.width, ui.size.height);\n",
       "        },\n",
       "    });\n",
       "\n",
       "    function mouse_event_fn(event) {\n",
       "        if (pass_mouse_events)\n",
       "            return fig.mouse_event(event, event['data']);\n",
       "    }\n",
       "\n",
       "    rubberband.mousedown('button_press', mouse_event_fn);\n",
       "    rubberband.mouseup('button_release', mouse_event_fn);\n",
       "    // Throttle sequential mouse events to 1 every 20ms.\n",
       "    rubberband.mousemove('motion_notify', mouse_event_fn);\n",
       "\n",
       "    rubberband.mouseenter('figure_enter', mouse_event_fn);\n",
       "    rubberband.mouseleave('figure_leave', mouse_event_fn);\n",
       "\n",
       "    canvas_div.on(\"wheel\", function (event) {\n",
       "        event = event.originalEvent;\n",
       "        event['data'] = 'scroll'\n",
       "        if (event.deltaY < 0) {\n",
       "            event.step = 1;\n",
       "        } else {\n",
       "            event.step = -1;\n",
       "        }\n",
       "        mouse_event_fn(event);\n",
       "    });\n",
       "\n",
       "    canvas_div.append(canvas);\n",
       "    canvas_div.append(rubberband);\n",
       "\n",
       "    this.rubberband = rubberband;\n",
       "    this.rubberband_canvas = rubberband[0];\n",
       "    this.rubberband_context = rubberband[0].getContext(\"2d\");\n",
       "    this.rubberband_context.strokeStyle = \"#000000\";\n",
       "\n",
       "    this._resize_canvas = function(width, height) {\n",
       "        // Keep the size of the canvas, canvas container, and rubber band\n",
       "        // canvas in synch.\n",
       "        canvas_div.css('width', width)\n",
       "        canvas_div.css('height', height)\n",
       "\n",
       "        canvas.attr('width', width * mpl.ratio);\n",
       "        canvas.attr('height', height * mpl.ratio);\n",
       "        canvas.attr('style', 'width: ' + width + 'px; height: ' + height + 'px;');\n",
       "\n",
       "        rubberband.attr('width', width);\n",
       "        rubberband.attr('height', height);\n",
       "    }\n",
       "\n",
       "    // Set the figure to an initial 600x600px, this will subsequently be updated\n",
       "    // upon first draw.\n",
       "    this._resize_canvas(600, 600);\n",
       "\n",
       "    // Disable right mouse context menu.\n",
       "    $(this.rubberband_canvas).bind(\"contextmenu\",function(e){\n",
       "        return false;\n",
       "    });\n",
       "\n",
       "    function set_focus () {\n",
       "        canvas.focus();\n",
       "        canvas_div.focus();\n",
       "    }\n",
       "\n",
       "    window.setTimeout(set_focus, 100);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._init_toolbar = function() {\n",
       "    var fig = this;\n",
       "\n",
       "    var nav_element = $('<div/>')\n",
       "    nav_element.attr('style', 'width: 100%');\n",
       "    this.root.append(nav_element);\n",
       "\n",
       "    // Define a callback function for later on.\n",
       "    function toolbar_event(event) {\n",
       "        return fig.toolbar_button_onclick(event['data']);\n",
       "    }\n",
       "    function toolbar_mouse_event(event) {\n",
       "        return fig.toolbar_button_onmouseover(event['data']);\n",
       "    }\n",
       "\n",
       "    for(var toolbar_ind in mpl.toolbar_items) {\n",
       "        var name = mpl.toolbar_items[toolbar_ind][0];\n",
       "        var tooltip = mpl.toolbar_items[toolbar_ind][1];\n",
       "        var image = mpl.toolbar_items[toolbar_ind][2];\n",
       "        var method_name = mpl.toolbar_items[toolbar_ind][3];\n",
       "\n",
       "        if (!name) {\n",
       "            // put a spacer in here.\n",
       "            continue;\n",
       "        }\n",
       "        var button = $('<button/>');\n",
       "        button.addClass('ui-button ui-widget ui-state-default ui-corner-all ' +\n",
       "                        'ui-button-icon-only');\n",
       "        button.attr('role', 'button');\n",
       "        button.attr('aria-disabled', 'false');\n",
       "        button.click(method_name, toolbar_event);\n",
       "        button.mouseover(tooltip, toolbar_mouse_event);\n",
       "\n",
       "        var icon_img = $('<span/>');\n",
       "        icon_img.addClass('ui-button-icon-primary ui-icon');\n",
       "        icon_img.addClass(image);\n",
       "        icon_img.addClass('ui-corner-all');\n",
       "\n",
       "        var tooltip_span = $('<span/>');\n",
       "        tooltip_span.addClass('ui-button-text');\n",
       "        tooltip_span.html(tooltip);\n",
       "\n",
       "        button.append(icon_img);\n",
       "        button.append(tooltip_span);\n",
       "\n",
       "        nav_element.append(button);\n",
       "    }\n",
       "\n",
       "    var fmt_picker_span = $('<span/>');\n",
       "\n",
       "    var fmt_picker = $('<select/>');\n",
       "    fmt_picker.addClass('mpl-toolbar-option ui-widget ui-widget-content');\n",
       "    fmt_picker_span.append(fmt_picker);\n",
       "    nav_element.append(fmt_picker_span);\n",
       "    this.format_dropdown = fmt_picker[0];\n",
       "\n",
       "    for (var ind in mpl.extensions) {\n",
       "        var fmt = mpl.extensions[ind];\n",
       "        var option = $(\n",
       "            '<option/>', {selected: fmt === mpl.default_extension}).html(fmt);\n",
       "        fmt_picker.append(option)\n",
       "    }\n",
       "\n",
       "    // Add hover states to the ui-buttons\n",
       "    $( \".ui-button\" ).hover(\n",
       "        function() { $(this).addClass(\"ui-state-hover\");},\n",
       "        function() { $(this).removeClass(\"ui-state-hover\");}\n",
       "    );\n",
       "\n",
       "    var status_bar = $('<span class=\"mpl-message\"/>');\n",
       "    nav_element.append(status_bar);\n",
       "    this.message = status_bar[0];\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.request_resize = function(x_pixels, y_pixels) {\n",
       "    // Request matplotlib to resize the figure. Matplotlib will then trigger a resize in the client,\n",
       "    // which will in turn request a refresh of the image.\n",
       "    this.send_message('resize', {'width': x_pixels, 'height': y_pixels});\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.send_message = function(type, properties) {\n",
       "    properties['type'] = type;\n",
       "    properties['figure_id'] = this.id;\n",
       "    this.ws.send(JSON.stringify(properties));\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.send_draw_message = function() {\n",
       "    if (!this.waiting) {\n",
       "        this.waiting = true;\n",
       "        this.ws.send(JSON.stringify({type: \"draw\", figure_id: this.id}));\n",
       "    }\n",
       "}\n",
       "\n",
       "\n",
       "mpl.figure.prototype.handle_save = function(fig, msg) {\n",
       "    var format_dropdown = fig.format_dropdown;\n",
       "    var format = format_dropdown.options[format_dropdown.selectedIndex].value;\n",
       "    fig.ondownload(fig, format);\n",
       "}\n",
       "\n",
       "\n",
       "mpl.figure.prototype.handle_resize = function(fig, msg) {\n",
       "    var size = msg['size'];\n",
       "    if (size[0] != fig.canvas.width || size[1] != fig.canvas.height) {\n",
       "        fig._resize_canvas(size[0], size[1]);\n",
       "        fig.send_message(\"refresh\", {});\n",
       "    };\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_rubberband = function(fig, msg) {\n",
       "    var x0 = msg['x0'] / mpl.ratio;\n",
       "    var y0 = (fig.canvas.height - msg['y0']) / mpl.ratio;\n",
       "    var x1 = msg['x1'] / mpl.ratio;\n",
       "    var y1 = (fig.canvas.height - msg['y1']) / mpl.ratio;\n",
       "    x0 = Math.floor(x0) + 0.5;\n",
       "    y0 = Math.floor(y0) + 0.5;\n",
       "    x1 = Math.floor(x1) + 0.5;\n",
       "    y1 = Math.floor(y1) + 0.5;\n",
       "    var min_x = Math.min(x0, x1);\n",
       "    var min_y = Math.min(y0, y1);\n",
       "    var width = Math.abs(x1 - x0);\n",
       "    var height = Math.abs(y1 - y0);\n",
       "\n",
       "    fig.rubberband_context.clearRect(\n",
       "        0, 0, fig.canvas.width, fig.canvas.height);\n",
       "\n",
       "    fig.rubberband_context.strokeRect(min_x, min_y, width, height);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_figure_label = function(fig, msg) {\n",
       "    // Updates the figure title.\n",
       "    fig.header.textContent = msg['label'];\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_cursor = function(fig, msg) {\n",
       "    var cursor = msg['cursor'];\n",
       "    switch(cursor)\n",
       "    {\n",
       "    case 0:\n",
       "        cursor = 'pointer';\n",
       "        break;\n",
       "    case 1:\n",
       "        cursor = 'default';\n",
       "        break;\n",
       "    case 2:\n",
       "        cursor = 'crosshair';\n",
       "        break;\n",
       "    case 3:\n",
       "        cursor = 'move';\n",
       "        break;\n",
       "    }\n",
       "    fig.rubberband_canvas.style.cursor = cursor;\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_message = function(fig, msg) {\n",
       "    fig.message.textContent = msg['message'];\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_draw = function(fig, msg) {\n",
       "    // Request the server to send over a new figure.\n",
       "    fig.send_draw_message();\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_image_mode = function(fig, msg) {\n",
       "    fig.image_mode = msg['mode'];\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.updated_canvas_event = function() {\n",
       "    // Called whenever the canvas gets updated.\n",
       "    this.send_message(\"ack\", {});\n",
       "}\n",
       "\n",
       "// A function to construct a web socket function for onmessage handling.\n",
       "// Called in the figure constructor.\n",
       "mpl.figure.prototype._make_on_message_function = function(fig) {\n",
       "    return function socket_on_message(evt) {\n",
       "        if (evt.data instanceof Blob) {\n",
       "            /* FIXME: We get \"Resource interpreted as Image but\n",
       "             * transferred with MIME type text/plain:\" errors on\n",
       "             * Chrome.  But how to set the MIME type?  It doesn't seem\n",
       "             * to be part of the websocket stream */\n",
       "            evt.data.type = \"image/png\";\n",
       "\n",
       "            /* Free the memory for the previous frames */\n",
       "            if (fig.imageObj.src) {\n",
       "                (window.URL || window.webkitURL).revokeObjectURL(\n",
       "                    fig.imageObj.src);\n",
       "            }\n",
       "\n",
       "            fig.imageObj.src = (window.URL || window.webkitURL).createObjectURL(\n",
       "                evt.data);\n",
       "            fig.updated_canvas_event();\n",
       "            fig.waiting = false;\n",
       "            return;\n",
       "        }\n",
       "        else if (typeof evt.data === 'string' && evt.data.slice(0, 21) == \"data:image/png;base64\") {\n",
       "            fig.imageObj.src = evt.data;\n",
       "            fig.updated_canvas_event();\n",
       "            fig.waiting = false;\n",
       "            return;\n",
       "        }\n",
       "\n",
       "        var msg = JSON.parse(evt.data);\n",
       "        var msg_type = msg['type'];\n",
       "\n",
       "        // Call the  \"handle_{type}\" callback, which takes\n",
       "        // the figure and JSON message as its only arguments.\n",
       "        try {\n",
       "            var callback = fig[\"handle_\" + msg_type];\n",
       "        } catch (e) {\n",
       "            console.log(\"No handler for the '\" + msg_type + \"' message type: \", msg);\n",
       "            return;\n",
       "        }\n",
       "\n",
       "        if (callback) {\n",
       "            try {\n",
       "                // console.log(\"Handling '\" + msg_type + \"' message: \", msg);\n",
       "                callback(fig, msg);\n",
       "            } catch (e) {\n",
       "                console.log(\"Exception inside the 'handler_\" + msg_type + \"' callback:\", e, e.stack, msg);\n",
       "            }\n",
       "        }\n",
       "    };\n",
       "}\n",
       "\n",
       "// from http://stackoverflow.com/questions/1114465/getting-mouse-location-in-canvas\n",
       "mpl.findpos = function(e) {\n",
       "    //this section is from http://www.quirksmode.org/js/events_properties.html\n",
       "    var targ;\n",
       "    if (!e)\n",
       "        e = window.event;\n",
       "    if (e.target)\n",
       "        targ = e.target;\n",
       "    else if (e.srcElement)\n",
       "        targ = e.srcElement;\n",
       "    if (targ.nodeType == 3) // defeat Safari bug\n",
       "        targ = targ.parentNode;\n",
       "\n",
       "    // jQuery normalizes the pageX and pageY\n",
       "    // pageX,Y are the mouse positions relative to the document\n",
       "    // offset() returns the position of the element relative to the document\n",
       "    var x = e.pageX - $(targ).offset().left;\n",
       "    var y = e.pageY - $(targ).offset().top;\n",
       "\n",
       "    return {\"x\": x, \"y\": y};\n",
       "};\n",
       "\n",
       "/*\n",
       " * return a copy of an object with only non-object keys\n",
       " * we need this to avoid circular references\n",
       " * http://stackoverflow.com/a/24161582/3208463\n",
       " */\n",
       "function simpleKeys (original) {\n",
       "  return Object.keys(original).reduce(function (obj, key) {\n",
       "    if (typeof original[key] !== 'object')\n",
       "        obj[key] = original[key]\n",
       "    return obj;\n",
       "  }, {});\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.mouse_event = function(event, name) {\n",
       "    var canvas_pos = mpl.findpos(event)\n",
       "\n",
       "    if (name === 'button_press')\n",
       "    {\n",
       "        this.canvas.focus();\n",
       "        this.canvas_div.focus();\n",
       "    }\n",
       "\n",
       "    var x = canvas_pos.x * mpl.ratio;\n",
       "    var y = canvas_pos.y * mpl.ratio;\n",
       "\n",
       "    this.send_message(name, {x: x, y: y, button: event.button,\n",
       "                             step: event.step,\n",
       "                             guiEvent: simpleKeys(event)});\n",
       "\n",
       "    /* This prevents the web browser from automatically changing to\n",
       "     * the text insertion cursor when the button is pressed.  We want\n",
       "     * to control all of the cursor setting manually through the\n",
       "     * 'cursor' event from matplotlib */\n",
       "    event.preventDefault();\n",
       "    return false;\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._key_event_extra = function(event, name) {\n",
       "    // Handle any extra behaviour associated with a key event\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.key_event = function(event, name) {\n",
       "\n",
       "    // Prevent repeat events\n",
       "    if (name == 'key_press')\n",
       "    {\n",
       "        if (event.which === this._key)\n",
       "            return;\n",
       "        else\n",
       "            this._key = event.which;\n",
       "    }\n",
       "    if (name == 'key_release')\n",
       "        this._key = null;\n",
       "\n",
       "    var value = '';\n",
       "    if (event.ctrlKey && event.which != 17)\n",
       "        value += \"ctrl+\";\n",
       "    if (event.altKey && event.which != 18)\n",
       "        value += \"alt+\";\n",
       "    if (event.shiftKey && event.which != 16)\n",
       "        value += \"shift+\";\n",
       "\n",
       "    value += 'k';\n",
       "    value += event.which.toString();\n",
       "\n",
       "    this._key_event_extra(event, name);\n",
       "\n",
       "    this.send_message(name, {key: value,\n",
       "                             guiEvent: simpleKeys(event)});\n",
       "    return false;\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.toolbar_button_onclick = function(name) {\n",
       "    if (name == 'download') {\n",
       "        this.handle_save(this, null);\n",
       "    } else {\n",
       "        this.send_message(\"toolbar_button\", {name: name});\n",
       "    }\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.toolbar_button_onmouseover = function(tooltip) {\n",
       "    this.message.textContent = tooltip;\n",
       "};\n",
       "mpl.toolbar_items = [[\"Home\", \"Reset original view\", \"fa fa-home icon-home\", \"home\"], [\"Back\", \"Back to  previous view\", \"fa fa-arrow-left icon-arrow-left\", \"back\"], [\"Forward\", \"Forward to next view\", \"fa fa-arrow-right icon-arrow-right\", \"forward\"], [\"\", \"\", \"\", \"\"], [\"Pan\", \"Pan axes with left mouse, zoom with right\", \"fa fa-arrows icon-move\", \"pan\"], [\"Zoom\", \"Zoom to rectangle\", \"fa fa-square-o icon-check-empty\", \"zoom\"], [\"\", \"\", \"\", \"\"], [\"Download\", \"Download plot\", \"fa fa-floppy-o icon-save\", \"download\"]];\n",
       "\n",
       "mpl.extensions = [\"eps\", \"jpeg\", \"pdf\", \"png\", \"ps\", \"raw\", \"svg\", \"tif\"];\n",
       "\n",
       "mpl.default_extension = \"png\";var comm_websocket_adapter = function(comm) {\n",
       "    // Create a \"websocket\"-like object which calls the given IPython comm\n",
       "    // object with the appropriate methods. Currently this is a non binary\n",
       "    // socket, so there is still some room for performance tuning.\n",
       "    var ws = {};\n",
       "\n",
       "    ws.close = function() {\n",
       "        comm.close()\n",
       "    };\n",
       "    ws.send = function(m) {\n",
       "        //console.log('sending', m);\n",
       "        comm.send(m);\n",
       "    };\n",
       "    // Register the callback with on_msg.\n",
       "    comm.on_msg(function(msg) {\n",
       "        //console.log('receiving', msg['content']['data'], msg);\n",
       "        // Pass the mpl event to the overriden (by mpl) onmessage function.\n",
       "        ws.onmessage(msg['content']['data'])\n",
       "    });\n",
       "    return ws;\n",
       "}\n",
       "\n",
       "mpl.mpl_figure_comm = function(comm, msg) {\n",
       "    // This is the function which gets called when the mpl process\n",
       "    // starts-up an IPython Comm through the \"matplotlib\" channel.\n",
       "\n",
       "    var id = msg.content.data.id;\n",
       "    // Get hold of the div created by the display call when the Comm\n",
       "    // socket was opened in Python.\n",
       "    var element = $(\"#\" + id);\n",
       "    var ws_proxy = comm_websocket_adapter(comm)\n",
       "\n",
       "    function ondownload(figure, format) {\n",
       "        window.open(figure.imageObj.src);\n",
       "    }\n",
       "\n",
       "    var fig = new mpl.figure(id, ws_proxy,\n",
       "                           ondownload,\n",
       "                           element.get(0));\n",
       "\n",
       "    // Call onopen now - mpl needs it, as it is assuming we've passed it a real\n",
       "    // web socket which is closed, not our websocket->open comm proxy.\n",
       "    ws_proxy.onopen();\n",
       "\n",
       "    fig.parent_element = element.get(0);\n",
       "    fig.cell_info = mpl.find_output_cell(\"<div id='\" + id + \"'></div>\");\n",
       "    if (!fig.cell_info) {\n",
       "        console.error(\"Failed to find cell for figure\", id, fig);\n",
       "        return;\n",
       "    }\n",
       "\n",
       "    var output_index = fig.cell_info[2]\n",
       "    var cell = fig.cell_info[0];\n",
       "\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_close = function(fig, msg) {\n",
       "    var width = fig.canvas.width/mpl.ratio\n",
       "    fig.root.unbind('remove')\n",
       "\n",
       "    // Update the output cell to use the data from the current canvas.\n",
       "    fig.push_to_output();\n",
       "    var dataURL = fig.canvas.toDataURL();\n",
       "    // Re-enable the keyboard manager in IPython - without this line, in FF,\n",
       "    // the notebook keyboard shortcuts fail.\n",
       "    IPython.keyboard_manager.enable()\n",
       "    $(fig.parent_element).html('<img src=\"' + dataURL + '\" width=\"' + width + '\">');\n",
       "    fig.close_ws(fig, msg);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.close_ws = function(fig, msg){\n",
       "    fig.send_message('closing', msg);\n",
       "    // fig.ws.close()\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.push_to_output = function(remove_interactive) {\n",
       "    // Turn the data on the canvas into data in the output cell.\n",
       "    var width = this.canvas.width/mpl.ratio\n",
       "    var dataURL = this.canvas.toDataURL();\n",
       "    this.cell_info[1]['text/html'] = '<img src=\"' + dataURL + '\" width=\"' + width + '\">';\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.updated_canvas_event = function() {\n",
       "    // Tell IPython that the notebook contents must change.\n",
       "    IPython.notebook.set_dirty(true);\n",
       "    this.send_message(\"ack\", {});\n",
       "    var fig = this;\n",
       "    // Wait a second, then push the new image to the DOM so\n",
       "    // that it is saved nicely (might be nice to debounce this).\n",
       "    setTimeout(function () { fig.push_to_output() }, 1000);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._init_toolbar = function() {\n",
       "    var fig = this;\n",
       "\n",
       "    var nav_element = $('<div/>')\n",
       "    nav_element.attr('style', 'width: 100%');\n",
       "    this.root.append(nav_element);\n",
       "\n",
       "    // Define a callback function for later on.\n",
       "    function toolbar_event(event) {\n",
       "        return fig.toolbar_button_onclick(event['data']);\n",
       "    }\n",
       "    function toolbar_mouse_event(event) {\n",
       "        return fig.toolbar_button_onmouseover(event['data']);\n",
       "    }\n",
       "\n",
       "    for(var toolbar_ind in mpl.toolbar_items){\n",
       "        var name = mpl.toolbar_items[toolbar_ind][0];\n",
       "        var tooltip = mpl.toolbar_items[toolbar_ind][1];\n",
       "        var image = mpl.toolbar_items[toolbar_ind][2];\n",
       "        var method_name = mpl.toolbar_items[toolbar_ind][3];\n",
       "\n",
       "        if (!name) { continue; };\n",
       "\n",
       "        var button = $('<button class=\"btn btn-default\" href=\"#\" title=\"' + name + '\"><i class=\"fa ' + image + ' fa-lg\"></i></button>');\n",
       "        button.click(method_name, toolbar_event);\n",
       "        button.mouseover(tooltip, toolbar_mouse_event);\n",
       "        nav_element.append(button);\n",
       "    }\n",
       "\n",
       "    // Add the status bar.\n",
       "    var status_bar = $('<span class=\"mpl-message\" style=\"text-align:right; float: right;\"/>');\n",
       "    nav_element.append(status_bar);\n",
       "    this.message = status_bar[0];\n",
       "\n",
       "    // Add the close button to the window.\n",
       "    var buttongrp = $('<div class=\"btn-group inline pull-right\"></div>');\n",
       "    var button = $('<button class=\"btn btn-mini btn-primary\" href=\"#\" title=\"Stop Interaction\"><i class=\"fa fa-power-off icon-remove icon-large\"></i></button>');\n",
       "    button.click(function (evt) { fig.handle_close(fig, {}); } );\n",
       "    button.mouseover('Stop Interaction', toolbar_mouse_event);\n",
       "    buttongrp.append(button);\n",
       "    var titlebar = this.root.find($('.ui-dialog-titlebar'));\n",
       "    titlebar.prepend(buttongrp);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._root_extra_style = function(el){\n",
       "    var fig = this\n",
       "    el.on(\"remove\", function(){\n",
       "\tfig.close_ws(fig, {});\n",
       "    });\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._canvas_extra_style = function(el){\n",
       "    // this is important to make the div 'focusable\n",
       "    el.attr('tabindex', 0)\n",
       "    // reach out to IPython and tell the keyboard manager to turn it's self\n",
       "    // off when our div gets focus\n",
       "\n",
       "    // location in version 3\n",
       "    if (IPython.notebook.keyboard_manager) {\n",
       "        IPython.notebook.keyboard_manager.register_events(el);\n",
       "    }\n",
       "    else {\n",
       "        // location in version 2\n",
       "        IPython.keyboard_manager.register_events(el);\n",
       "    }\n",
       "\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._key_event_extra = function(event, name) {\n",
       "    var manager = IPython.notebook.keyboard_manager;\n",
       "    if (!manager)\n",
       "        manager = IPython.keyboard_manager;\n",
       "\n",
       "    // Check for shift+enter\n",
       "    if (event.shiftKey && event.which == 13) {\n",
       "        this.canvas_div.blur();\n",
       "        // select the cell after this one\n",
       "        var index = IPython.notebook.find_cell_index(this.cell_info[0]);\n",
       "        IPython.notebook.select(index + 1);\n",
       "    }\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_save = function(fig, msg) {\n",
       "    fig.ondownload(fig, null);\n",
       "}\n",
       "\n",
       "\n",
       "mpl.find_output_cell = function(html_output) {\n",
       "    // Return the cell and output element which can be found *uniquely* in the notebook.\n",
       "    // Note - this is a bit hacky, but it is done because the \"notebook_saving.Notebook\"\n",
       "    // IPython event is triggered only after the cells have been serialised, which for\n",
       "    // our purposes (turning an active figure into a static one), is too late.\n",
       "    var cells = IPython.notebook.get_cells();\n",
       "    var ncells = cells.length;\n",
       "    for (var i=0; i<ncells; i++) {\n",
       "        var cell = cells[i];\n",
       "        if (cell.cell_type === 'code'){\n",
       "            for (var j=0; j<cell.output_area.outputs.length; j++) {\n",
       "                var data = cell.output_area.outputs[j];\n",
       "                if (data.data) {\n",
       "                    // IPython >= 3 moved mimebundle to data attribute of output\n",
       "                    data = data.data;\n",
       "                }\n",
       "                if (data['text/html'] == html_output) {\n",
       "                    return [cell, data, j];\n",
       "                }\n",
       "            }\n",
       "        }\n",
       "    }\n",
       "}\n",
       "\n",
       "// Register the function which deals with the matplotlib target/channel.\n",
       "// The kernel may be null if the page has been refreshed.\n",
       "if (IPython.notebook.kernel != null) {\n",
       "    IPython.notebook.kernel.comm_manager.register_target('matplotlib', mpl.mpl_figure_comm);\n",
       "}\n"
      ],
      "text/plain": [
       "<IPython.core.display.Javascript object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<img src=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAfQAAAGQCAYAAABYs5LGAAAgAElEQVR4Xu3dsY5sx3afceoxLnByJYYDAQacyHAiwIEzJY78CnR4H+GGPq/gyAkzBwaUGFZiwIACQYlyAvcxaDTJocaHvfdMVa9atarqx4g4Z1fVqu+/ur7eu2f6/MV3/kMAAQQQQACB5Qn8xfI7sAEEEEAAAQQQ+I7QNQECCCCAAAIbECD0DUK0BQQQQAABBAhdDyCAAAIIILABAULfIERbQAABBBBAgND1AAIIIIAAAhsQIPQNQrQFBBBAAAEECF0PIIAAAgggsAEBQt8gRFtAAAEEEECA0PUAAggggAACGxAg9A1CtAUEEEAAAQQIXQ8ggAACCCCwAQFC3yBEW0AAAQQQQIDQ9QACCCCAAAIbECD0DUK0BQQQQAABBAhdDyCAAAIIILABAULfIERbQAABBBBAgND1AAIIIIAAAhsQIPQNQrQFBBBAAAEECF0PIIAAAgggsAEBQt8gxE9u4adPXucyBBDYk4Dzfs9cf9uVgDcP+G17P37/PaEfkrVtIvCMwJevX533m7eGgDcP+COhf/nhD0cQ+PFv/3y5z1MYnBC0nL/77ooBoe//CiD0/TP+eYdXd+inyMxBf0ajy5nQz+j057sk9EPSJ3R36Ce0OqET+gl9frVHQj8kfUIn9BNandAJ/YQ+J/STU/bI/fJzxUdbnPKxwwkvAUIn9BP6nNBPTpnQCf2Q/id0Qj+k1Z9u0yP3Q9L3yN0j9xNandAJ/YQ+d4d+csru0N2hH9L/hE7oh7S6O/STg+65Q7/8fdab313PGNNzaM8ec/c5fQazu97PWr91ndmZ3f18Retefv7V0YvvQkjrDV8ss70CPHLfPuJfNkjo7Y/cI4WSdmh3fFHQdNFc1BzJv0fOPWMq5OyLZQ451J9sk9APyZ7QCf2q1Qk9rjcI/ZADteg2Cb1oMNFl9Qg9uoaZ8/Xc7c2s19p9BOTsM/S+ztljFKHvkeOHuyD09ruwD6G6oBwBQif0ck2ZWBChJ8KeuRShE/rM/stam9AJPavXKq5D6BVTGVAToRP6gLYqNyWhE3q5pkwsiNATYc9citAJfWb/Za1N6ISe1WsV1yH0iqkMqInQCX1AW5WbktAJvVxTJhZE6ImwZy5F6IQ+s/+y1iZ0Qs/qtYrrEHrFVAbUROiEPqCtyk1J6IRerikTCyL0RNgzl8oS+t2B2rr/yH/WNOugj9x/K69Vrz8958j9P3rAN8Wt+kp4vW5Cf53hEjMQes4dOqG3vxwihbbiG7fI/RN6e//tNILQd0rzZi89Qo/8StAezJEH3YoHfQ+zFcecnnPk/gl9xVdAXM2EHsey9EyE7g69aoNGCm3FN26R+yf0ql2eUxeh53CevgqhE/r0JrwoIFJohO4z9Kp9nlEXoWdQLrAGoRN6gTZ8WgKh/yE0Gj8UF4pzqckIfam4+osldELv756xIwmd0Md22DmzE/ohWRM6oVdtdUIn9Kq9uVpdhL5aYp317iT0ns9JK4zpjG77YVdCr5BZT22tgUW+ofFDca3097qe0PfK83I3hN5+hx4tlENarXmbPdKsPKYVAKG3EnP9FQFCP6Q3CJ3Qq7Z6ZTn31NbKmdBbibme0A/vAUIn9KovgR5pVh7TypnQW4m5ntAP74GdhN4TZc/j8+h1euY7YUyk0FbMOXL/PkM/4RVzvUeP3A/Jv0foPWgiv8s88qBb8aDv4b/imNNzjtw/oa/4CoirmdDjWJaeidDbH7n3BBr5hqZn/RXHRAptxTdukfsn9BVfAXE1E3ocy9IzETqhV23QSKERuq9+rdrnGXURegblAmsQOqEXaMOnJRC6L5ap2pur1UXoqyXWWS+hE3pn6wwfRuiEPrzJDlmA0A8JmtAJvWqrEzqhV+3N1eoi9NUS66yX0Am9s3WGDyN0Qh/eZIcsQOiHBJ0l9ENw2iYCZQn451PLRjO8MEIfjrjGAoReIwdVIDCaAKGPJlx3fkKvm01oZYQeitNkCJQlQOhloxleGKEPR1xjAUKvkYMqEBhNgNBHE647P6HXzSa0MkIPxWkyBMoSIPSy0QwvjNCHI66xAKHXyEEVCIwmQOijCdedn9DrZhNaGaGH4jQZAmUJEHrZaIYXRujDEddYgNBr5KAKBEYTIPTRhOvOT+h1swmtrEfolwfDD9dfhJExpucf4Jg95u7LUzKY3TVT1vqt68zO7MHsKrfWvTzmmj7m61fnfeipWm8yAdfLZEhFhN7+TXGRQiH0dqFF8u+Rc8+YCjm7Qx9yhC4xKaEvEdPrRRI6oV910fQ7x4snPoTe/ibo9kmAO/TXD9LiMxB68YCiyusRetTa5kEAgTwC7tDzWFdbidCrJTKoHkIfBNa0CBQjQOjFAkksh9ATYc9citBn0rc2AnkECD2PdbWVCL1aIoPqIfRBYE2LQDEChF4skMRyCD0R9sylCH0mfWsjkEeA0PNYV1uJ0KslMqgeQh8E1rQIFCNA6MUCSSyH0BNhz1yK0GfStzYCeQQIPY91tZUIvVoig+oh9EFgTYtAMQKEXiyQxHIIPRH2zKWuhN5T0923YfXM1/PFJq3r3H1JSetcj+sjGUTX1rOfqDGVuWTUFrnGI5PI3vjii2Wi2rzsPIReNprYwgj9+pviekhHHtyRh3bPXiLHVOaSUVvkGoQe2ZlnzEXoZ+T8HaETekarRwot+o1ORm2RaxB6RsfutQah75Xn5W4IndAzWj1SaITukXtGz+60BqHvlObNXgid0DNandCv/2nhHv6Rb2p8ht6TwFpjCH2tvLqrJXRC726ehoGETugN7eLSYAKEHgy06nSETugZvUnohJ7RZ9Z4ToDQD+kMQif0jFYndELP6DNrEPrRPZAl9J7fKW8dc/e54pVQIj+LfDRS6zp3oouubWaj9+yzlWXv/nrWaR0Tuf/HPiN7w2fovZ2zzjh36Otk9VKlhJ5zh9765iT60H6pSQIGRwotUmY9b8J6xkTuP7o3CD2gwYtPQejFA4oqj9AJPaqX7uaJFBqhu0PP6Nmd1iD0ndK82UuW0Htw9tzVtq6TJYfWuqLvwnrWjxzjM3SfoUf2k7naCBB6G69lryb0nDv0ngaJfrPRU0PUGEIn9KheMk87AUJvZ7bkCEIn9IzGJXRCz+gzazwnQOiHdAahE3pGqxM6oWf0mTUI/egeIHRCz3gBEDqhZ/SZNQj96B4gdELPeAEQOqFn9Jk1CP3oHiB0Qs94ARA6oWf0mTUI/egeIHRCz3gBEDqhZ/SZNQj97B748qefqgLI+D30qntX15oEluzZH//oh6DXbLdPVy3gT6Na/EJCXzxA5VciQOiV0lDLGwFCP6UXCP2UpH/b5//5V//rcs//9p/+/XE8IjdM6JE0zRVFgNCjSFafh9CrJxRa353M3xYi9X7khN7PzshxBAh9HNtaMxN6rTwGV0PoYwET+li+Zu8jQOh93NYbRejrZdZR8ZvI3999P/7s27vx98J3p94OmtDbmRkxngChj2dcYwVCr5HD4CoIfTDgX6cn9BzOVmkjQOhtvNa9mtDXza6h8md33p/9s4Zljr+U0I9vgZIACL1kLAOK6hB6z6GVMebunxu9+mKT2WPuvnAlktmP//zPPzdPyyP3L3/5l08bLqvmqpk9oLTWNpvZ7cnh99AHHKy1piT0WnmMq4bQL9m2HtqVD3pCv/5GwJ1y7vpGPkIfd74WmZnQiwQxvAxCP0roLf3kDj3uTYA79JbOc200AUKPJlp1vg6hZ22l55FzVm2rrfOZX1f7dk9+yr095SV71h16e9CLjSD0xQLrLpfQu9GtNJDQc9Ii9BzOVmkjQOhtvNa9mtDXza6hckJvgPXCpYT+AjxDhxEg9GFoi01M6MUCGVMOoY/h+u2shJ7D2SptBAi9jde6VxP6utk1VE7oDbBeuJTQX4Bn6DAChD4MbbGJCb1YIGPLef91r9/+/2Plxw/CPftWubFV7TM7oe+T5U47IfSd0rzbC6GfkvTP+yT0sXET+li+Zu8jQOh93NYbRejrZabisgQIvWw0RxdG6IfE/+P33/8UtdWub6m6Wfzua1mjao6eJ5LBivu/4nk6l8j9PxhH9saXr1+d99EHQbH5BFwskFHlEHos2ciDO/LQjt1l+2ync4ncP6G399/pIwj9kA4g9NigIw9uQn+ezYpcIvuC0GNfsyfMRugnpPx4dOeRe2jSkQf3iuLyyP05gci+IPTQl+wRkxH6ETETenTMkQc3obtDv+rPyN7wGXr0KVBvPkKvl8mQityhx2Il9PF3qJEyi03/erbIvnCHnpXaPusQ+j5Z3u6E0GODjjy4VxSXR+7j39AQeuxr9oTZCP2ElH2GHp4yoY8X2opvdCL7gtDDX7bbT0jo20f8ywbdoccGHXlwrygud+jj39AQeuxr9oTZCP2ElBOF3vMNWisK7UrokfuPfNMQ3eYZ+9ypL+7kfJdzJAM/FBf9Kqg3H6HXy2RIRVl36JEH/RAQQZMS+p+fkuyRUyvLoAiHTBO5f3foQyLaelJC3zref9kcoccG3Sqh6IM+djfts0W+cWtl2V5t3ojonN2h52W3w0qEvkOKn9hDltA/UcrvLok8tHrW7xkT+Ti8R449NUeOyaj59L5whx7ZsWfMRehn5OyH4oJzJvT2R+6tERC6f5yltWdOv57QD+kAd+ixQRM6oT/rqMi+cIce+5o9YTZCPyHlxJ9y78F5+p1YxuPrnlzuxmTUfHpfEHp01+4/H6Hvn/HPO3SHHht05J1Yhhxjd3/9KDiDS/ReIueL3D+hRyZzxlyEfkbOhB6cc+TBTejPw3GH7jP04Jft9tMR+vYR/7JBd+ixQRO6z9B9hh77mjLb6wQI/XWGS8xA6LExETqhE3rsa8psrxMg9NcZrjHDlz/91Froio+CW/dY4foVOa9Yc4WsW2sI5fzjH533rQEsdr2AFwusu1xC70Y3emDooT262F/nX7HmJDShy4RyJvTQbCpORugVUxlRE6GPoBoyZ+ihHVLRx5OsWPPHu6p3RShnQq8XcHBFhB4MtOx0hF42mtBDO2mXK9achCZ0mVDOhB6aTcXJCL1iKiNqIvQRVEPmDD20Qyr6eJIVa/54V/WuCOVM6PUCDq6I0IOBlp2O0MtGE3poJ+1yxZqT0IQuE8qZ0EOzqTgZoVdMZURNhD6CasicoYd2SEUfT7JizR/vqt4VoZwJvV7AwRURejDQstMRetloQg/tpF2uWHMSmtBlQjkTemg2FScj9IqpjKgpSeg9B1DrmLuvBL36wpfZY+6+iKZ1/yPao3XOnppbx8zO7MGktZ+ic25ldpsjobe2+XLXE/pykXUWTOiX4FoP7QoHfWcXhA3rEU3rGEIP/kdwCD2s/6tOROhVk4mui9AJPbCnWuX8WLp1DKG3M3OHHtjkC05F6AuG1lVyktC7ajt8UKvoPovrf/zn/3556X/8b//ps9M8vW5UzS8VteHgUM7u0DfskP9/S4S+fcS/bpDQyyYdemi/2yWhl43804WF9gahf5r7qhcS+qrJtdZN6K3E0q4PPbR/rfpO5m8be+UufUTNacAXWiiUM6EvlHxfqYTex229UYReNrPIQ/tN5O9l/fizb+X9Xvg9Yo+suWwwBQoL5UzoBRIdWwKhj+VbZ3ZCr5PFN5VEHtqEXjbmrsIie+M7Qu/KYKVBhL5SWq/USuiv0Bs6NvLQfnbn/dk/a9lkZM0t6552bShnQt++fQh9+4h/3SChl0068tB2h1425q7CInvDHXpXBEsNIvSl4nqhWEJ/Ad7YoZGHNqGPzSp79sjeIPTs9PLXI/R85lNW/PH773+KWvju6y171rj7ApGe+TLGRDKIPLQ/89Pt3/Kp+kNxp/fFI6dIBl++fnXeZxwOE9cQ8ET4mUsTeixtQv/zU6AZXGKTjJ0tcv+EHpvNCbMR+gkpP97pu0MPTTry4HaH/jyayLvT0PBvJovsC0LPSm2fdQh9nyxvd0LosUFHHtyETuhX3Rn5psYj99gzoOJshF4xlQE1EXosVEL3yP1ZR0X2hTv02NfsCbMR+gkpe+QennLkwe0O3R26O/Twl+iRExL6IbG7Q48NuqrQ33b5/utev/3/xzWPn2x/9itun6UU+SYkQ2af3der10X2hTv0V9M4bzyhH5I5occGHXlwj5Ajocfm/dnZIvuC0D9L3XVvBAj9kF7IEnqPnCJ/8CcrzquDO3L/0XKIZJOxz5364k7OdzlHMvBDcZGvgJpzEXrNXMKrIvRYpITe/kNxrW8CImUWm/71bD1y7hnTsx9C76G21hhCXyuv7moJvRvd04GETujPGqNHzj1jerqZ0HuorTWG0NfKq7vaLKH3FLjbnVgrgxX3f7XHyI8JVuQSuX+fobe+klxP6If0AKHHBh15cK8oLkJ/TiCyLwg99jV7wmyEfkLKfg89POXIg5vQn8ezIpfIviD08Jft9hMS+vYR/7JBd+ixQUce3CuKyx26O/TYV5TZIggQegTFBeYg9NiQCH280FZ8oxPZF+7QY1+zJ8xG6Cek7A49POXIg3tFcblDH/+GhtDDX7bbT0jo20fskfuIiAl9vNBWfKMT2ReEPuKVu/echL53vr/tziP32KAjD+4VxeUOffwbGkKPfc2eMBuhn5DyY49f/vRT1a22foNY1X2o6xwCS/bsj3903m/eogLePODftkfopyRtnwkECD0BsiWaCRB6M7JFBxD6osH1l/2Pf/rfvxv8r//4737+s8ffvf1//wrnjiT0c7OvvHNCr5xOZG2EHkmz/FzPZP6+6IfM364h9vY4Cb2dmRHjCRD6eMY1ViD0GjkkVUHoY0ET+li+Zu8jQOh93NYbRejrZfZCxd8K/f2j9rdp3Zn3Ayb0fnZGjiNA6OPY1pqZ0GvlMbgaQh8LmNDH8jV7HwFC7+O23ihCXy+zoIqvHr+7Q+8HTOj97IwcR4DQx7GtNXOH0HsOrYwxd1/EcvWFL7PH3H0RTQazt2Z8L/c3oWet37rO7MwezFr7qUrOTw8fv4de60weUA2hD4BackpCv4yl9dBe+aAn9N+3QeQbB0IvefodUxShnxI1oRP6r79//gbCHfovJAj9lENw/30S+v4Z/7LDDqFnoWl9FJtV1y7r+Aw9Pskle9Yj9/hGKDYjoRcLZFg5hD4MbfWJCT0+IUKPZ2rG1wkQ+usM15iB0NfIKajKj75Y5rGMn3Lvh03o/eyMHEeA0MexrTUzodfKY3A1hD4WMKGP5Wv2PgKE3sdtvVGEvl5mL1T8kdDdnb8A9+YH6e5+yv21FQNG+ww9AGLtKQi9dj5x1RF6HMtFZrr719YW2ULZMt2hl43m6MII/ZT4Cf2UpH/bJ6GPi5zQx7E1cz8BQu9nt9ZIQl8rL9WWJkDopeM5tjhCPyV6Qj8laftMIEDoCZAt0UyA0JuRrTngx++//ymq8ugf/Mk4HHu+DayHV8ZeHnXd7aen7mdj5HxNcsWcv3z96ryPenEUnUfARYOJLovQ/3yJNFJcKx70V2Aiudy9CYlcxxu365OD0KNP1XrzEXq9TIZUROiE3tpYkaIl9Fb6v1wf+SSG0PsyWGkUoa+U1gu1Ejqht7YPoXvk3tozrp9LgNDn8k9bndAJvbXZCJ3QW3vG9XMJEPpc/mmrEzqhtzYboRN6a8+4fi4BQp/LP211Qif01mYjdEJv7RnXzyVA6HP5p61O6ITe2myETuitPeP6uQQIfS7/tNWzhN7za1utY3p+NWn2mDs5tu7/0TSRP/181YRZNV+tMzuzB5fW2rKY9Rwcfsq9h9paYwh9rby6qyX09jv0SKFUPugJPa43KudM6N3H5zIDCX2ZqF4rlNDjDu3d7twIPa43CP21c8ro1wgQ+mv8lhmdJfQeID2PnFvX6bnbbl3j7lF41ufRPTX3CL1nHTm3U4v8aMUdejv/1UYQ+mqJddZL6O13YT2oM6R198ahp2ZCb6e2Ys6E3p7zaiMIfbXEOusldEJvbZ2spwqR63gSc50yobe+Ata7ntDXy6yrYkIn9NbGiRRt1scRhE7orX2+0/WEvlOaN3shdEJvbXVCvybmkXtrN7k+gwChZ1AusAahE3prGxI6obf2jOvnEiD0ufzTVid0Qm9tNkIn9Naecf1cAoQ+l3/a6oRO6K3NRuiE3tozrp9LgNDn8k9b/Uro0Yd22oYaF9rth6Uat1/i8ozPnbNyLgH0oohLzl+/Ou8rBxdQm4ADIK4wBaHvdYe+Qs99WyOh56RG6DmcK65C6BVTGVAToRP6gLZqmpLQm3B1X0zo3eiWH0joy0f4uQ0QOqF/rlPGXUXo49i+n5nQczhXXIXQK6YyoCZCJ/QBbdU0JaE34eq+mNC70S0/kNCXj/BzGyB0Qv9cp4y7itDHsXWHnsO2+iqEXj2hoPoIndCDWql7GkLvRtc00B16E66tLib0reK83gyhE/rsVif0nAQIPYdzxVUIvWIqA2rqEXrPAZwxpud3jWePuft9/wxmdy2VtX7rOrMzezC7yq11L4+5po/xe+gDTtZaUxJ6rTyGVUPo7XfokUIh9HahRfLvkXPPmAo5u0MfdoyWn5jQy0cUUyChE/pVJ02/c/zhD09LI/T2N0G3TwLcocccpoVnIfTC4USW1iP0yPVnz9Ujh56ae+TYs86KYzLYZOVcmb879MrpjK2N0MfyLTM7obffofeElyGtnroqjMlgQ+g3d/Xu0Cu8DIbWQOhD8daZnNAJfXY3EnpOAu7QczhXXIXQK6YyoCZCJ/QBbdU0JaE34eq+mNC70S0/kNCXj/BzGyB0Qv9cp4y7itDHsX0/M6HncK64CqFXTGVATYRO6APaqmlKQm/C1X0xoXejW34goS8f4ec2QOiE/rlOGXcVoY9j6w49h231VQi9ekJB9RE6oQe1Uvc0hN6NrmmgO/QmXFtdTOhbxXm9mSyh3/3aUCvqu2/dap1rt19niuR8xTKSf2tevddn5RzJP5ozofd2z/rjCH39DD+1A0LPuUP/VBgBF0UKhdDbA4nkT+jt/I14ToDQD+kMQif01laPFk3r+j3Xu0P3xTI9fbPLGELfJckP9kHohN7a6oR+8xHW317302zOHrm3JrDP9YS+T5a3OyF0Qm9tdUIn9Naecf1cAoQ+l3/a6oRO6K3NRuiE3tozrp9LgNDn8k9bndAJvbXZCJ3QW3vG9XMJEPpc/mmrEzqhtzYboRN6a8+4fi4BQp/LP231HqH3fBFIxq/z9Pwk8+wxd3KczfmqCbNqvlpndmYPLj21tb6o0zj751Nbo1nuekJfLrK+ggm9/Q49UihZh3ZfdzwflVVzjzQrj2nNII0zobdGs9z1hL5cZH0FEzqht3ZOmmh++MPT0iLfUPXebfe8cSjLmdBbo1nuekJfLrK+gnuE3rNSxiP36Lp2+6y4h8+zMbtxidxP1T5/5Oj30KNeAevNQ+jrZdZVMaG336F3gU4aFCmUq5IjBZiE5VJmd3foPbVF8o/mTOg9ie4xhtD3yPHDXRA6oX/YJN9cEC2a1vV7ru95TB+9Tut80ZwJvTWBfa4n9H2yvN0JoRN6a6tHi6Z1/Z7rCd0j956+2WUMoe+S5Af7IHRCb211Qr8m5pF7aze5PoMAoWdQLrAGoRN6axsSOqG39ozr5xIg9Ln801YndEJvbTZCJ/TWnnH9XAKEPpd/2uqETuitzUbohN7aM66fS4DQ5/JPW53Q4/796p7QouUY+Rnu1X5WrPkum8j9RPKPrOuxfz/l3vMK3WMMoe+R44e7IHRC/7BJvrkgSzStdfVeH7kfQu9NwbiRBAh9JN1Cc/cIffY/GlL1AO6JNXIvd3dhPbW5Q2+nRujtzIwYT4DQxzMusQKhu0NvbcQV34R45O6Re2uf73Q9oe+U5s1eCJ3QW1ud0K+JuUNv7SbXZxAg9AzKBdYgdEJvbUNCJ/TWnnH9XAKEPpd/2uqETuitzUbohN7aM66fS4DQ5/JPW53QCb212Qid0Ft7xvVzCRD6XP5pq+8k9J5/gCPyM8+e0O7kOPu3Ca72s2LNd9lc7Wd2P2W9cfry9avzvufFu9AYAS8U1iulEnrdO3RCf6WzPz+W0An9892y5pWEvmZuzVUTOqG3No079O++63kTEMm5da7H9b4profaHmMIfY8cP9wFoRP6h03yzQWETuitPeP6uQQIfS7/tNV3EnoPtMqfoVfdT9Znuz377xkTuZ/Ifoqsyx16T2fsM4bQ98nydic9Qu9BU/Wgi6yrh0vWod1T29WYFWu+23/kfiL7KbIuQo98Baw3F6Gvl1lXxYRe95F7T6CRQiH09gQi+RN6O38jnhMg9EM6g9AJvbXVs0TTWlfv9ZH7IfTeFIwbSYDQR9ItNDehE3prO0YK8O5RcGtdvddH7ofQe1MwbiQBQh9Jt9DchE7ore0YKUBCv6afxdkXy7S+Ata7ntDXy6yrYkIn9NbGyRJNa12910fuxx16bwrGjSRA6CPpFpqb0Am9tR0jBegO3R16a/+5vp0AobczW3JEltCXhKNoBDYi4JviNgqzcSuE3ghs1csJfdXk1I1AGwFCb+O109WEvlOaN3sh9EOCts3jCRD6uS1A6IdkT+iHBG2bxxMg9HNbgNAPyZ7QDwnaNo8nQOjntgChH5I9oR8StG0eT4DQz20BQj8ke0I/JGjbPJ4AoZ/bAoR+SPaEfkjQtnk8AUI/twUI/ZDse4R+eTD88IdLahlj7r6l6+rLUGaPufuSlgxmd22etX7rOrMzezBr7acKORP6IYf6k20S+iHZE/r1N8W1HtqrHvRXrd4q2sc8GWMIPZjz16/O+83PewFvHvDb9gid0An9+ZOlyDcO7tAPOVCLbpPQiwYTXVaP0KNrMB8CCIwn4JH7eMZVVyD0qskE10XowUBNh0BRAoReNJiEsgg9AXKFJQi9QgpqQGA8AUIfz7jqCoReNZngugg9GKjpEChKgNCLBpNQFqEnQDGqnHwAAAjuSURBVK6wBKFXSEENCIwnQOjjGVddgdCrJhNcF6EHAzUdAkUJEHrRYBLKIvQEyBWWIPQKKagBgfEECH0846orEHrVZILrIvRgoKZDoCgBQi8aTEJZhJ4AucISV0KvUJsaEEBgPIEvviluPOTJKxD65ACylif0LNLWQaAmAUKvmUtkVYQeSbPwXIReOBylIZBAgNATIE9egtAnB5C4/E+Ja1kKAQTqEXDe18sktCIBh+IsPRmhl45HcQgMJ+C8H4547gICnss/c/Vwof/d3/2b7/7mb/7vz3t4/P+3/7393aubzFrn1TqN34fA//yrv/ptM//hH/5hl40573dJ8mIfAt484HfbCxP6m7wfwn4m8iixt6wT9ebhnHaw02cE3kT+XuIbyd15v3nbC3jzgAn9nIDt9HUChP46QzPMI0Do89hnrxx2h/5W+Pu785GP3r9dZ+Tj/exQrFefwHvJPxN+/R38VqHzfqGwekoVcA+1NccMFfp7JNGPv68e60evs2asqh5N4O6ufbHP1533o5tl8vwCnhxA4vKEngjbUvsQIPR9stx9J4S+e8L/sr+hQh95t/zs0f45sdnpTAIPmV/9gNyjLnfoM9Ox9rcECP2cniD0c7K20yAChB4E0jQpBAg9BXOJRQi9RAyKWIHARz/t/n4PC92lO+9XaL4XahTwC/AWGzpU6O9ZRD9+90Nxi3XaBuU++93z939G6BuEvOEWCH3DUC+2ROjnZG2nLxIg9BcBGj6FAKFPwT5l0TShf7u7V+/YR34b3ZQkLFqWwNVd+KPgx6P1b/9+ocftjy0478t2XkxhAo7huMIs4UJ/bDpLth+t8+qbhhUCVON4Ah8JfXwFQ1dw3g/FO39yAc/PIKsCQs8ibZ1lCRD6stEp3COYo3pgiNDfCGZ9HWvWOkd1hs3+jsBG/yDL+725gdu81wW8ecDvtkfo52Rtpy8SIPQXARo+hQChT8E+ZdGhQp+yI4sigEALAed9C60FrxXwgqF1lkzoneAMQ2ATAs77TYK82oaANw8465H7ORjtFIFlCTjvl43uc4UL+HOcdrjKHfoOKdoDAv0EnPf97JYYKeAlYgopktBDMJoEgWUJOO+Xje5zhQv4c5x2uIrQd0jRHhDoJ+C872e3xEgBLxFTSJGEHoLRJJ8h8Pf/9a9/d9lf/5e//8xQ14wj4Lwfx7bEzAIuEUNKEYSegtkiDwKEXrIPnPclY4krSsBxLKvPROjVE9qgvvcif7sjf/ZnG2x1xS0471dMraFmATfAWvxSQl88wOrlfyTuj/6++v42qM95v0GId1sQ8OYBv9seoZ+T9ZSdfiTsj/5+StFnLeq83zxvAW8eMKGfE/DsnX4k7I/+fnb9B6zvvN88ZAFvHjChnxPw7J1+JOyP/n52/Qes77zfPGQBbx4woZ8T8OydfiTsj/5+dv0HrO+83zxkAW8eMKGfE3CFnfop9wopXNbgvC8dz+vFCfh1hqvM4IfiVklq4ToJvXR4zvvS8bxenIBfZ7jKDIS+SlIb1OmLZUqG6LwvGUtcUQKOY1l9JkKvnpD6EBhLwHk/lu/02QU8PYK0Agg9DbWFEChJwHlfMpa4ogQcx7L6TIRePSH1ITCWgPN+LN/pswt4egRpBRB6GmoLIVCSgPO+ZCxxRQk4jmX1mQi9ekLqQ2AsAef9WL7TZxfw9AjSCiD0NNQWQqAkAed9yVjiihJwHMvqMxF69YTUh8BYAs77sXynzy7g6RGkFUDoaagthEBJAs77krHEFSXgOJbVZyL06gmpD4GxBJz3Y/lOn13A0yNIK4DQ01BbCIGSBJz3JWOJK0rAcSyrz0To1RNSHwJjCTjvx/KdPruAp0eQVgChp6G2EAIlCTjvS8YSV5SA41hWn4nQqyekPgTGEnDej+U7fXYBT48grQBCT0NtIQRKEnDel4wlrigBx7GsPhOhV09IfQiMJeC8H8t3+uwCnh5BWgGEnobaQgiUJOC8LxlLXFECjmNZfSZCr56Q+hAYS8B5P5bv9NkFPD2CtAIIPQ21hRAoScB5XzKWuKIEHMey+kyEXj0h9SEwloDzfizf6bMLeHoEaQUQehpqCyFQkoDzvmQscUUJOI5l9ZkIvXpC6kNgLAHn/Vi+02cX8PQI0gog9DTUFkKgJAHnfclY4ooScBzL6jMRevWE1IfAWALO+7F8p88u4OkRpBVA6GmoLYRASQLO+5KxxBUl4DiW1Wci9OoJqQ+BsQSc92P5Tp9dwNMjSCuA0NNQWwiBkgSc9yVjiStKwHEszYQAAggggMA0AoQ+Db2FEUAAAQQQiCNA6HEszYQAAggggMA0AoQ+Db2FEUAAAQQQiCNA6HEszYQAAggggMA0AoQ+Db2FEUAAAQQQiCNA6HEszYQAAggggMA0AoQ+Db2FEUAAAQQQiCNA6HEszYQAAggggMA0AoQ+Db2FEUAAAQQQiCNA6HEszYQAAggggMA0AoQ+Db2FEUAAAQQQiCNA6HEszYQAAggggMA0AoQ+Db2FEUAAAQQQiCNA6HEszYQAAggggMA0AoQ+Db2FEUAAAQQQiCNA6HEszYQAAggggMA0AoQ+Db2FEUAAAQQQiCNA6HEszYQAAggggMA0AoQ+Db2FEUAAAQQQiCNA6HEszYQAAggggMA0AoQ+Db2FEUAAAQQQiCNA6HEszYQAAggggMA0AoQ+Db2FEUAAAQQQiCNA6HEszYQAAggggMA0AoQ+Db2FEUAAAQQQiCNA6HEszYQAAggggMA0AoQ+Db2FEUAAAQQQiCNA6HEszYQAAggggMA0AoQ+Db2FEUAAAQQQiCNA6HEszYQAAggggMA0AoQ+Db2FEUAAAQQQiCNA6HEszYQAAggggMA0AoQ+Db2FEUAAAQQQiCNA6HEszYQAAggggMA0AoQ+Db2FEUAAAQQQiCNA6HEszYQAAggggMA0AoQ+Db2FEUAAAQQQiCNA6HEszYQAAggggMA0AoQ+Db2FEUAAAQQQiCNA6HEszYQAAggggMA0AoQ+Db2FEUAAAQQQiCNA6HEszYQAAggggMA0Av8PnhpJCBEjUxkAAAAASUVORK5CYII=\" width=\"500\">"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Saving figure MsPacman\n"
     ]
    }
   ],
   "source": [
    "plt.figure(figsize=(5,4))\n",
    "plt.imshow(img)\n",
    "plt.axis(\"off\")\n",
    "save_fig(\"MsPacman\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Welcome back to the 1980s! :)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this environment, the rendered image is simply equal to the observation (but in many environments this is not the case):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "(img == obs).all()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's create a little helper function to plot an environment:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def plot_environment(env, figsize=(5,4)):\n",
    "    plt.close()  # or else nbagg sometimes plots in the previous cell\n",
    "    plt.figure(figsize=figsize)\n",
    "    img = env.render(mode=\"rgb_array\")\n",
    "    plt.imshow(img)\n",
    "    plt.axis(\"off\")\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's see how to interact with an environment. Your agent will need to select an action from an \"action space\" (the set of possible actions). Let's see what this environment's action space looks like:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Discrete(9)"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "env.action_space"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`Discrete(9)` means that the possible actions are integers 0 through 8, which represents the 9 possible positions of the joystick (0=center, 1=up, 2=right, 3=left, 4=down, 5=upper-right, 6=upper-left, 7=lower-right, 8=lower-left)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next we need to tell the environment which action to play, and it will compute the next step of the game. Let's go left for 110 steps, then lower left for 40 steps:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "env.reset()\n",
    "for step in range(110):\n",
    "    env.step(3) #left\n",
    "for step in range(40):\n",
    "    env.step(8) #lower-left"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Where are we now?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/javascript": [
       "/* Put everything inside the global mpl namespace */\n",
       "window.mpl = {};\n",
       "\n",
       "\n",
       "mpl.get_websocket_type = function() {\n",
       "    if (typeof(WebSocket) !== 'undefined') {\n",
       "        return WebSocket;\n",
       "    } else if (typeof(MozWebSocket) !== 'undefined') {\n",
       "        return MozWebSocket;\n",
       "    } else {\n",
       "        alert('Your browser does not have WebSocket support.' +\n",
       "              'Please try Chrome, Safari or Firefox ≥ 6. ' +\n",
       "              'Firefox 4 and 5 are also supported but you ' +\n",
       "              'have to enable WebSockets in about:config.');\n",
       "    };\n",
       "}\n",
       "\n",
       "mpl.figure = function(figure_id, websocket, ondownload, parent_element) {\n",
       "    this.id = figure_id;\n",
       "\n",
       "    this.ws = websocket;\n",
       "\n",
       "    this.supports_binary = (this.ws.binaryType != undefined);\n",
       "\n",
       "    if (!this.supports_binary) {\n",
       "        var warnings = document.getElementById(\"mpl-warnings\");\n",
       "        if (warnings) {\n",
       "            warnings.style.display = 'block';\n",
       "            warnings.textContent = (\n",
       "                \"This browser does not support binary websocket messages. \" +\n",
       "                    \"Performance may be slow.\");\n",
       "        }\n",
       "    }\n",
       "\n",
       "    this.imageObj = new Image();\n",
       "\n",
       "    this.context = undefined;\n",
       "    this.message = undefined;\n",
       "    this.canvas = undefined;\n",
       "    this.rubberband_canvas = undefined;\n",
       "    this.rubberband_context = undefined;\n",
       "    this.format_dropdown = undefined;\n",
       "\n",
       "    this.image_mode = 'full';\n",
       "\n",
       "    this.root = $('<div/>');\n",
       "    this._root_extra_style(this.root)\n",
       "    this.root.attr('style', 'display: inline-block');\n",
       "\n",
       "    $(parent_element).append(this.root);\n",
       "\n",
       "    this._init_header(this);\n",
       "    this._init_canvas(this);\n",
       "    this._init_toolbar(this);\n",
       "\n",
       "    var fig = this;\n",
       "\n",
       "    this.waiting = false;\n",
       "\n",
       "    this.ws.onopen =  function () {\n",
       "            fig.send_message(\"supports_binary\", {value: fig.supports_binary});\n",
       "            fig.send_message(\"send_image_mode\", {});\n",
       "            if (mpl.ratio != 1) {\n",
       "                fig.send_message(\"set_dpi_ratio\", {'dpi_ratio': mpl.ratio});\n",
       "            }\n",
       "            fig.send_message(\"refresh\", {});\n",
       "        }\n",
       "\n",
       "    this.imageObj.onload = function() {\n",
       "            if (fig.image_mode == 'full') {\n",
       "                // Full images could contain transparency (where diff images\n",
       "                // almost always do), so we need to clear the canvas so that\n",
       "                // there is no ghosting.\n",
       "                fig.context.clearRect(0, 0, fig.canvas.width, fig.canvas.height);\n",
       "            }\n",
       "            fig.context.drawImage(fig.imageObj, 0, 0);\n",
       "        };\n",
       "\n",
       "    this.imageObj.onunload = function() {\n",
       "        this.ws.close();\n",
       "    }\n",
       "\n",
       "    this.ws.onmessage = this._make_on_message_function(this);\n",
       "\n",
       "    this.ondownload = ondownload;\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._init_header = function() {\n",
       "    var titlebar = $(\n",
       "        '<div class=\"ui-dialog-titlebar ui-widget-header ui-corner-all ' +\n",
       "        'ui-helper-clearfix\"/>');\n",
       "    var titletext = $(\n",
       "        '<div class=\"ui-dialog-title\" style=\"width: 100%; ' +\n",
       "        'text-align: center; padding: 3px;\"/>');\n",
       "    titlebar.append(titletext)\n",
       "    this.root.append(titlebar);\n",
       "    this.header = titletext[0];\n",
       "}\n",
       "\n",
       "\n",
       "\n",
       "mpl.figure.prototype._canvas_extra_style = function(canvas_div) {\n",
       "\n",
       "}\n",
       "\n",
       "\n",
       "mpl.figure.prototype._root_extra_style = function(canvas_div) {\n",
       "\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._init_canvas = function() {\n",
       "    var fig = this;\n",
       "\n",
       "    var canvas_div = $('<div/>');\n",
       "\n",
       "    canvas_div.attr('style', 'position: relative; clear: both; outline: 0');\n",
       "\n",
       "    function canvas_keyboard_event(event) {\n",
       "        return fig.key_event(event, event['data']);\n",
       "    }\n",
       "\n",
       "    canvas_div.keydown('key_press', canvas_keyboard_event);\n",
       "    canvas_div.keyup('key_release', canvas_keyboard_event);\n",
       "    this.canvas_div = canvas_div\n",
       "    this._canvas_extra_style(canvas_div)\n",
       "    this.root.append(canvas_div);\n",
       "\n",
       "    var canvas = $('<canvas/>');\n",
       "    canvas.addClass('mpl-canvas');\n",
       "    canvas.attr('style', \"left: 0; top: 0; z-index: 0; outline: 0\")\n",
       "\n",
       "    this.canvas = canvas[0];\n",
       "    this.context = canvas[0].getContext(\"2d\");\n",
       "\n",
       "    var backingStore = this.context.backingStorePixelRatio ||\n",
       "\tthis.context.webkitBackingStorePixelRatio ||\n",
       "\tthis.context.mozBackingStorePixelRatio ||\n",
       "\tthis.context.msBackingStorePixelRatio ||\n",
       "\tthis.context.oBackingStorePixelRatio ||\n",
       "\tthis.context.backingStorePixelRatio || 1;\n",
       "\n",
       "    mpl.ratio = (window.devicePixelRatio || 1) / backingStore;\n",
       "\n",
       "    var rubberband = $('<canvas/>');\n",
       "    rubberband.attr('style', \"position: absolute; left: 0; top: 0; z-index: 1;\")\n",
       "\n",
       "    var pass_mouse_events = true;\n",
       "\n",
       "    canvas_div.resizable({\n",
       "        start: function(event, ui) {\n",
       "            pass_mouse_events = false;\n",
       "        },\n",
       "        resize: function(event, ui) {\n",
       "            fig.request_resize(ui.size.width, ui.size.height);\n",
       "        },\n",
       "        stop: function(event, ui) {\n",
       "            pass_mouse_events = true;\n",
       "            fig.request_resize(ui.size.width, ui.size.height);\n",
       "        },\n",
       "    });\n",
       "\n",
       "    function mouse_event_fn(event) {\n",
       "        if (pass_mouse_events)\n",
       "            return fig.mouse_event(event, event['data']);\n",
       "    }\n",
       "\n",
       "    rubberband.mousedown('button_press', mouse_event_fn);\n",
       "    rubberband.mouseup('button_release', mouse_event_fn);\n",
       "    // Throttle sequential mouse events to 1 every 20ms.\n",
       "    rubberband.mousemove('motion_notify', mouse_event_fn);\n",
       "\n",
       "    rubberband.mouseenter('figure_enter', mouse_event_fn);\n",
       "    rubberband.mouseleave('figure_leave', mouse_event_fn);\n",
       "\n",
       "    canvas_div.on(\"wheel\", function (event) {\n",
       "        event = event.originalEvent;\n",
       "        event['data'] = 'scroll'\n",
       "        if (event.deltaY < 0) {\n",
       "            event.step = 1;\n",
       "        } else {\n",
       "            event.step = -1;\n",
       "        }\n",
       "        mouse_event_fn(event);\n",
       "    });\n",
       "\n",
       "    canvas_div.append(canvas);\n",
       "    canvas_div.append(rubberband);\n",
       "\n",
       "    this.rubberband = rubberband;\n",
       "    this.rubberband_canvas = rubberband[0];\n",
       "    this.rubberband_context = rubberband[0].getContext(\"2d\");\n",
       "    this.rubberband_context.strokeStyle = \"#000000\";\n",
       "\n",
       "    this._resize_canvas = function(width, height) {\n",
       "        // Keep the size of the canvas, canvas container, and rubber band\n",
       "        // canvas in synch.\n",
       "        canvas_div.css('width', width)\n",
       "        canvas_div.css('height', height)\n",
       "\n",
       "        canvas.attr('width', width * mpl.ratio);\n",
       "        canvas.attr('height', height * mpl.ratio);\n",
       "        canvas.attr('style', 'width: ' + width + 'px; height: ' + height + 'px;');\n",
       "\n",
       "        rubberband.attr('width', width);\n",
       "        rubberband.attr('height', height);\n",
       "    }\n",
       "\n",
       "    // Set the figure to an initial 600x600px, this will subsequently be updated\n",
       "    // upon first draw.\n",
       "    this._resize_canvas(600, 600);\n",
       "\n",
       "    // Disable right mouse context menu.\n",
       "    $(this.rubberband_canvas).bind(\"contextmenu\",function(e){\n",
       "        return false;\n",
       "    });\n",
       "\n",
       "    function set_focus () {\n",
       "        canvas.focus();\n",
       "        canvas_div.focus();\n",
       "    }\n",
       "\n",
       "    window.setTimeout(set_focus, 100);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._init_toolbar = function() {\n",
       "    var fig = this;\n",
       "\n",
       "    var nav_element = $('<div/>')\n",
       "    nav_element.attr('style', 'width: 100%');\n",
       "    this.root.append(nav_element);\n",
       "\n",
       "    // Define a callback function for later on.\n",
       "    function toolbar_event(event) {\n",
       "        return fig.toolbar_button_onclick(event['data']);\n",
       "    }\n",
       "    function toolbar_mouse_event(event) {\n",
       "        return fig.toolbar_button_onmouseover(event['data']);\n",
       "    }\n",
       "\n",
       "    for(var toolbar_ind in mpl.toolbar_items) {\n",
       "        var name = mpl.toolbar_items[toolbar_ind][0];\n",
       "        var tooltip = mpl.toolbar_items[toolbar_ind][1];\n",
       "        var image = mpl.toolbar_items[toolbar_ind][2];\n",
       "        var method_name = mpl.toolbar_items[toolbar_ind][3];\n",
       "\n",
       "        if (!name) {\n",
       "            // put a spacer in here.\n",
       "            continue;\n",
       "        }\n",
       "        var button = $('<button/>');\n",
       "        button.addClass('ui-button ui-widget ui-state-default ui-corner-all ' +\n",
       "                        'ui-button-icon-only');\n",
       "        button.attr('role', 'button');\n",
       "        button.attr('aria-disabled', 'false');\n",
       "        button.click(method_name, toolbar_event);\n",
       "        button.mouseover(tooltip, toolbar_mouse_event);\n",
       "\n",
       "        var icon_img = $('<span/>');\n",
       "        icon_img.addClass('ui-button-icon-primary ui-icon');\n",
       "        icon_img.addClass(image);\n",
       "        icon_img.addClass('ui-corner-all');\n",
       "\n",
       "        var tooltip_span = $('<span/>');\n",
       "        tooltip_span.addClass('ui-button-text');\n",
       "        tooltip_span.html(tooltip);\n",
       "\n",
       "        button.append(icon_img);\n",
       "        button.append(tooltip_span);\n",
       "\n",
       "        nav_element.append(button);\n",
       "    }\n",
       "\n",
       "    var fmt_picker_span = $('<span/>');\n",
       "\n",
       "    var fmt_picker = $('<select/>');\n",
       "    fmt_picker.addClass('mpl-toolbar-option ui-widget ui-widget-content');\n",
       "    fmt_picker_span.append(fmt_picker);\n",
       "    nav_element.append(fmt_picker_span);\n",
       "    this.format_dropdown = fmt_picker[0];\n",
       "\n",
       "    for (var ind in mpl.extensions) {\n",
       "        var fmt = mpl.extensions[ind];\n",
       "        var option = $(\n",
       "            '<option/>', {selected: fmt === mpl.default_extension}).html(fmt);\n",
       "        fmt_picker.append(option)\n",
       "    }\n",
       "\n",
       "    // Add hover states to the ui-buttons\n",
       "    $( \".ui-button\" ).hover(\n",
       "        function() { $(this).addClass(\"ui-state-hover\");},\n",
       "        function() { $(this).removeClass(\"ui-state-hover\");}\n",
       "    );\n",
       "\n",
       "    var status_bar = $('<span class=\"mpl-message\"/>');\n",
       "    nav_element.append(status_bar);\n",
       "    this.message = status_bar[0];\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.request_resize = function(x_pixels, y_pixels) {\n",
       "    // Request matplotlib to resize the figure. Matplotlib will then trigger a resize in the client,\n",
       "    // which will in turn request a refresh of the image.\n",
       "    this.send_message('resize', {'width': x_pixels, 'height': y_pixels});\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.send_message = function(type, properties) {\n",
       "    properties['type'] = type;\n",
       "    properties['figure_id'] = this.id;\n",
       "    this.ws.send(JSON.stringify(properties));\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.send_draw_message = function() {\n",
       "    if (!this.waiting) {\n",
       "        this.waiting = true;\n",
       "        this.ws.send(JSON.stringify({type: \"draw\", figure_id: this.id}));\n",
       "    }\n",
       "}\n",
       "\n",
       "\n",
       "mpl.figure.prototype.handle_save = function(fig, msg) {\n",
       "    var format_dropdown = fig.format_dropdown;\n",
       "    var format = format_dropdown.options[format_dropdown.selectedIndex].value;\n",
       "    fig.ondownload(fig, format);\n",
       "}\n",
       "\n",
       "\n",
       "mpl.figure.prototype.handle_resize = function(fig, msg) {\n",
       "    var size = msg['size'];\n",
       "    if (size[0] != fig.canvas.width || size[1] != fig.canvas.height) {\n",
       "        fig._resize_canvas(size[0], size[1]);\n",
       "        fig.send_message(\"refresh\", {});\n",
       "    };\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_rubberband = function(fig, msg) {\n",
       "    var x0 = msg['x0'] / mpl.ratio;\n",
       "    var y0 = (fig.canvas.height - msg['y0']) / mpl.ratio;\n",
       "    var x1 = msg['x1'] / mpl.ratio;\n",
       "    var y1 = (fig.canvas.height - msg['y1']) / mpl.ratio;\n",
       "    x0 = Math.floor(x0) + 0.5;\n",
       "    y0 = Math.floor(y0) + 0.5;\n",
       "    x1 = Math.floor(x1) + 0.5;\n",
       "    y1 = Math.floor(y1) + 0.5;\n",
       "    var min_x = Math.min(x0, x1);\n",
       "    var min_y = Math.min(y0, y1);\n",
       "    var width = Math.abs(x1 - x0);\n",
       "    var height = Math.abs(y1 - y0);\n",
       "\n",
       "    fig.rubberband_context.clearRect(\n",
       "        0, 0, fig.canvas.width, fig.canvas.height);\n",
       "\n",
       "    fig.rubberband_context.strokeRect(min_x, min_y, width, height);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_figure_label = function(fig, msg) {\n",
       "    // Updates the figure title.\n",
       "    fig.header.textContent = msg['label'];\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_cursor = function(fig, msg) {\n",
       "    var cursor = msg['cursor'];\n",
       "    switch(cursor)\n",
       "    {\n",
       "    case 0:\n",
       "        cursor = 'pointer';\n",
       "        break;\n",
       "    case 1:\n",
       "        cursor = 'default';\n",
       "        break;\n",
       "    case 2:\n",
       "        cursor = 'crosshair';\n",
       "        break;\n",
       "    case 3:\n",
       "        cursor = 'move';\n",
       "        break;\n",
       "    }\n",
       "    fig.rubberband_canvas.style.cursor = cursor;\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_message = function(fig, msg) {\n",
       "    fig.message.textContent = msg['message'];\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_draw = function(fig, msg) {\n",
       "    // Request the server to send over a new figure.\n",
       "    fig.send_draw_message();\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_image_mode = function(fig, msg) {\n",
       "    fig.image_mode = msg['mode'];\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.updated_canvas_event = function() {\n",
       "    // Called whenever the canvas gets updated.\n",
       "    this.send_message(\"ack\", {});\n",
       "}\n",
       "\n",
       "// A function to construct a web socket function for onmessage handling.\n",
       "// Called in the figure constructor.\n",
       "mpl.figure.prototype._make_on_message_function = function(fig) {\n",
       "    return function socket_on_message(evt) {\n",
       "        if (evt.data instanceof Blob) {\n",
       "            /* FIXME: We get \"Resource interpreted as Image but\n",
       "             * transferred with MIME type text/plain:\" errors on\n",
       "             * Chrome.  But how to set the MIME type?  It doesn't seem\n",
       "             * to be part of the websocket stream */\n",
       "            evt.data.type = \"image/png\";\n",
       "\n",
       "            /* Free the memory for the previous frames */\n",
       "            if (fig.imageObj.src) {\n",
       "                (window.URL || window.webkitURL).revokeObjectURL(\n",
       "                    fig.imageObj.src);\n",
       "            }\n",
       "\n",
       "            fig.imageObj.src = (window.URL || window.webkitURL).createObjectURL(\n",
       "                evt.data);\n",
       "            fig.updated_canvas_event();\n",
       "            fig.waiting = false;\n",
       "            return;\n",
       "        }\n",
       "        else if (typeof evt.data === 'string' && evt.data.slice(0, 21) == \"data:image/png;base64\") {\n",
       "            fig.imageObj.src = evt.data;\n",
       "            fig.updated_canvas_event();\n",
       "            fig.waiting = false;\n",
       "            return;\n",
       "        }\n",
       "\n",
       "        var msg = JSON.parse(evt.data);\n",
       "        var msg_type = msg['type'];\n",
       "\n",
       "        // Call the  \"handle_{type}\" callback, which takes\n",
       "        // the figure and JSON message as its only arguments.\n",
       "        try {\n",
       "            var callback = fig[\"handle_\" + msg_type];\n",
       "        } catch (e) {\n",
       "            console.log(\"No handler for the '\" + msg_type + \"' message type: \", msg);\n",
       "            return;\n",
       "        }\n",
       "\n",
       "        if (callback) {\n",
       "            try {\n",
       "                // console.log(\"Handling '\" + msg_type + \"' message: \", msg);\n",
       "                callback(fig, msg);\n",
       "            } catch (e) {\n",
       "                console.log(\"Exception inside the 'handler_\" + msg_type + \"' callback:\", e, e.stack, msg);\n",
       "            }\n",
       "        }\n",
       "    };\n",
       "}\n",
       "\n",
       "// from http://stackoverflow.com/questions/1114465/getting-mouse-location-in-canvas\n",
       "mpl.findpos = function(e) {\n",
       "    //this section is from http://www.quirksmode.org/js/events_properties.html\n",
       "    var targ;\n",
       "    if (!e)\n",
       "        e = window.event;\n",
       "    if (e.target)\n",
       "        targ = e.target;\n",
       "    else if (e.srcElement)\n",
       "        targ = e.srcElement;\n",
       "    if (targ.nodeType == 3) // defeat Safari bug\n",
       "        targ = targ.parentNode;\n",
       "\n",
       "    // jQuery normalizes the pageX and pageY\n",
       "    // pageX,Y are the mouse positions relative to the document\n",
       "    // offset() returns the position of the element relative to the document\n",
       "    var x = e.pageX - $(targ).offset().left;\n",
       "    var y = e.pageY - $(targ).offset().top;\n",
       "\n",
       "    return {\"x\": x, \"y\": y};\n",
       "};\n",
       "\n",
       "/*\n",
       " * return a copy of an object with only non-object keys\n",
       " * we need this to avoid circular references\n",
       " * http://stackoverflow.com/a/24161582/3208463\n",
       " */\n",
       "function simpleKeys (original) {\n",
       "  return Object.keys(original).reduce(function (obj, key) {\n",
       "    if (typeof original[key] !== 'object')\n",
       "        obj[key] = original[key]\n",
       "    return obj;\n",
       "  }, {});\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.mouse_event = function(event, name) {\n",
       "    var canvas_pos = mpl.findpos(event)\n",
       "\n",
       "    if (name === 'button_press')\n",
       "    {\n",
       "        this.canvas.focus();\n",
       "        this.canvas_div.focus();\n",
       "    }\n",
       "\n",
       "    var x = canvas_pos.x * mpl.ratio;\n",
       "    var y = canvas_pos.y * mpl.ratio;\n",
       "\n",
       "    this.send_message(name, {x: x, y: y, button: event.button,\n",
       "                             step: event.step,\n",
       "                             guiEvent: simpleKeys(event)});\n",
       "\n",
       "    /* This prevents the web browser from automatically changing to\n",
       "     * the text insertion cursor when the button is pressed.  We want\n",
       "     * to control all of the cursor setting manually through the\n",
       "     * 'cursor' event from matplotlib */\n",
       "    event.preventDefault();\n",
       "    return false;\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._key_event_extra = function(event, name) {\n",
       "    // Handle any extra behaviour associated with a key event\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.key_event = function(event, name) {\n",
       "\n",
       "    // Prevent repeat events\n",
       "    if (name == 'key_press')\n",
       "    {\n",
       "        if (event.which === this._key)\n",
       "            return;\n",
       "        else\n",
       "            this._key = event.which;\n",
       "    }\n",
       "    if (name == 'key_release')\n",
       "        this._key = null;\n",
       "\n",
       "    var value = '';\n",
       "    if (event.ctrlKey && event.which != 17)\n",
       "        value += \"ctrl+\";\n",
       "    if (event.altKey && event.which != 18)\n",
       "        value += \"alt+\";\n",
       "    if (event.shiftKey && event.which != 16)\n",
       "        value += \"shift+\";\n",
       "\n",
       "    value += 'k';\n",
       "    value += event.which.toString();\n",
       "\n",
       "    this._key_event_extra(event, name);\n",
       "\n",
       "    this.send_message(name, {key: value,\n",
       "                             guiEvent: simpleKeys(event)});\n",
       "    return false;\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.toolbar_button_onclick = function(name) {\n",
       "    if (name == 'download') {\n",
       "        this.handle_save(this, null);\n",
       "    } else {\n",
       "        this.send_message(\"toolbar_button\", {name: name});\n",
       "    }\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.toolbar_button_onmouseover = function(tooltip) {\n",
       "    this.message.textContent = tooltip;\n",
       "};\n",
       "mpl.toolbar_items = [[\"Home\", \"Reset original view\", \"fa fa-home icon-home\", \"home\"], [\"Back\", \"Back to  previous view\", \"fa fa-arrow-left icon-arrow-left\", \"back\"], [\"Forward\", \"Forward to next view\", \"fa fa-arrow-right icon-arrow-right\", \"forward\"], [\"\", \"\", \"\", \"\"], [\"Pan\", \"Pan axes with left mouse, zoom with right\", \"fa fa-arrows icon-move\", \"pan\"], [\"Zoom\", \"Zoom to rectangle\", \"fa fa-square-o icon-check-empty\", \"zoom\"], [\"\", \"\", \"\", \"\"], [\"Download\", \"Download plot\", \"fa fa-floppy-o icon-save\", \"download\"]];\n",
       "\n",
       "mpl.extensions = [\"eps\", \"jpeg\", \"pdf\", \"png\", \"ps\", \"raw\", \"svg\", \"tif\"];\n",
       "\n",
       "mpl.default_extension = \"png\";var comm_websocket_adapter = function(comm) {\n",
       "    // Create a \"websocket\"-like object which calls the given IPython comm\n",
       "    // object with the appropriate methods. Currently this is a non binary\n",
       "    // socket, so there is still some room for performance tuning.\n",
       "    var ws = {};\n",
       "\n",
       "    ws.close = function() {\n",
       "        comm.close()\n",
       "    };\n",
       "    ws.send = function(m) {\n",
       "        //console.log('sending', m);\n",
       "        comm.send(m);\n",
       "    };\n",
       "    // Register the callback with on_msg.\n",
       "    comm.on_msg(function(msg) {\n",
       "        //console.log('receiving', msg['content']['data'], msg);\n",
       "        // Pass the mpl event to the overriden (by mpl) onmessage function.\n",
       "        ws.onmessage(msg['content']['data'])\n",
       "    });\n",
       "    return ws;\n",
       "}\n",
       "\n",
       "mpl.mpl_figure_comm = function(comm, msg) {\n",
       "    // This is the function which gets called when the mpl process\n",
       "    // starts-up an IPython Comm through the \"matplotlib\" channel.\n",
       "\n",
       "    var id = msg.content.data.id;\n",
       "    // Get hold of the div created by the display call when the Comm\n",
       "    // socket was opened in Python.\n",
       "    var element = $(\"#\" + id);\n",
       "    var ws_proxy = comm_websocket_adapter(comm)\n",
       "\n",
       "    function ondownload(figure, format) {\n",
       "        window.open(figure.imageObj.src);\n",
       "    }\n",
       "\n",
       "    var fig = new mpl.figure(id, ws_proxy,\n",
       "                           ondownload,\n",
       "                           element.get(0));\n",
       "\n",
       "    // Call onopen now - mpl needs it, as it is assuming we've passed it a real\n",
       "    // web socket which is closed, not our websocket->open comm proxy.\n",
       "    ws_proxy.onopen();\n",
       "\n",
       "    fig.parent_element = element.get(0);\n",
       "    fig.cell_info = mpl.find_output_cell(\"<div id='\" + id + \"'></div>\");\n",
       "    if (!fig.cell_info) {\n",
       "        console.error(\"Failed to find cell for figure\", id, fig);\n",
       "        return;\n",
       "    }\n",
       "\n",
       "    var output_index = fig.cell_info[2]\n",
       "    var cell = fig.cell_info[0];\n",
       "\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_close = function(fig, msg) {\n",
       "    var width = fig.canvas.width/mpl.ratio\n",
       "    fig.root.unbind('remove')\n",
       "\n",
       "    // Update the output cell to use the data from the current canvas.\n",
       "    fig.push_to_output();\n",
       "    var dataURL = fig.canvas.toDataURL();\n",
       "    // Re-enable the keyboard manager in IPython - without this line, in FF,\n",
       "    // the notebook keyboard shortcuts fail.\n",
       "    IPython.keyboard_manager.enable()\n",
       "    $(fig.parent_element).html('<img src=\"' + dataURL + '\" width=\"' + width + '\">');\n",
       "    fig.close_ws(fig, msg);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.close_ws = function(fig, msg){\n",
       "    fig.send_message('closing', msg);\n",
       "    // fig.ws.close()\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.push_to_output = function(remove_interactive) {\n",
       "    // Turn the data on the canvas into data in the output cell.\n",
       "    var width = this.canvas.width/mpl.ratio\n",
       "    var dataURL = this.canvas.toDataURL();\n",
       "    this.cell_info[1]['text/html'] = '<img src=\"' + dataURL + '\" width=\"' + width + '\">';\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.updated_canvas_event = function() {\n",
       "    // Tell IPython that the notebook contents must change.\n",
       "    IPython.notebook.set_dirty(true);\n",
       "    this.send_message(\"ack\", {});\n",
       "    var fig = this;\n",
       "    // Wait a second, then push the new image to the DOM so\n",
       "    // that it is saved nicely (might be nice to debounce this).\n",
       "    setTimeout(function () { fig.push_to_output() }, 1000);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._init_toolbar = function() {\n",
       "    var fig = this;\n",
       "\n",
       "    var nav_element = $('<div/>')\n",
       "    nav_element.attr('style', 'width: 100%');\n",
       "    this.root.append(nav_element);\n",
       "\n",
       "    // Define a callback function for later on.\n",
       "    function toolbar_event(event) {\n",
       "        return fig.toolbar_button_onclick(event['data']);\n",
       "    }\n",
       "    function toolbar_mouse_event(event) {\n",
       "        return fig.toolbar_button_onmouseover(event['data']);\n",
       "    }\n",
       "\n",
       "    for(var toolbar_ind in mpl.toolbar_items){\n",
       "        var name = mpl.toolbar_items[toolbar_ind][0];\n",
       "        var tooltip = mpl.toolbar_items[toolbar_ind][1];\n",
       "        var image = mpl.toolbar_items[toolbar_ind][2];\n",
       "        var method_name = mpl.toolbar_items[toolbar_ind][3];\n",
       "\n",
       "        if (!name) { continue; };\n",
       "\n",
       "        var button = $('<button class=\"btn btn-default\" href=\"#\" title=\"' + name + '\"><i class=\"fa ' + image + ' fa-lg\"></i></button>');\n",
       "        button.click(method_name, toolbar_event);\n",
       "        button.mouseover(tooltip, toolbar_mouse_event);\n",
       "        nav_element.append(button);\n",
       "    }\n",
       "\n",
       "    // Add the status bar.\n",
       "    var status_bar = $('<span class=\"mpl-message\" style=\"text-align:right; float: right;\"/>');\n",
       "    nav_element.append(status_bar);\n",
       "    this.message = status_bar[0];\n",
       "\n",
       "    // Add the close button to the window.\n",
       "    var buttongrp = $('<div class=\"btn-group inline pull-right\"></div>');\n",
       "    var button = $('<button class=\"btn btn-mini btn-primary\" href=\"#\" title=\"Stop Interaction\"><i class=\"fa fa-power-off icon-remove icon-large\"></i></button>');\n",
       "    button.click(function (evt) { fig.handle_close(fig, {}); } );\n",
       "    button.mouseover('Stop Interaction', toolbar_mouse_event);\n",
       "    buttongrp.append(button);\n",
       "    var titlebar = this.root.find($('.ui-dialog-titlebar'));\n",
       "    titlebar.prepend(buttongrp);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._root_extra_style = function(el){\n",
       "    var fig = this\n",
       "    el.on(\"remove\", function(){\n",
       "\tfig.close_ws(fig, {});\n",
       "    });\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._canvas_extra_style = function(el){\n",
       "    // this is important to make the div 'focusable\n",
       "    el.attr('tabindex', 0)\n",
       "    // reach out to IPython and tell the keyboard manager to turn it's self\n",
       "    // off when our div gets focus\n",
       "\n",
       "    // location in version 3\n",
       "    if (IPython.notebook.keyboard_manager) {\n",
       "        IPython.notebook.keyboard_manager.register_events(el);\n",
       "    }\n",
       "    else {\n",
       "        // location in version 2\n",
       "        IPython.keyboard_manager.register_events(el);\n",
       "    }\n",
       "\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._key_event_extra = function(event, name) {\n",
       "    var manager = IPython.notebook.keyboard_manager;\n",
       "    if (!manager)\n",
       "        manager = IPython.keyboard_manager;\n",
       "\n",
       "    // Check for shift+enter\n",
       "    if (event.shiftKey && event.which == 13) {\n",
       "        this.canvas_div.blur();\n",
       "        // select the cell after this one\n",
       "        var index = IPython.notebook.find_cell_index(this.cell_info[0]);\n",
       "        IPython.notebook.select(index + 1);\n",
       "    }\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_save = function(fig, msg) {\n",
       "    fig.ondownload(fig, null);\n",
       "}\n",
       "\n",
       "\n",
       "mpl.find_output_cell = function(html_output) {\n",
       "    // Return the cell and output element which can be found *uniquely* in the notebook.\n",
       "    // Note - this is a bit hacky, but it is done because the \"notebook_saving.Notebook\"\n",
       "    // IPython event is triggered only after the cells have been serialised, which for\n",
       "    // our purposes (turning an active figure into a static one), is too late.\n",
       "    var cells = IPython.notebook.get_cells();\n",
       "    var ncells = cells.length;\n",
       "    for (var i=0; i<ncells; i++) {\n",
       "        var cell = cells[i];\n",
       "        if (cell.cell_type === 'code'){\n",
       "            for (var j=0; j<cell.output_area.outputs.length; j++) {\n",
       "                var data = cell.output_area.outputs[j];\n",
       "                if (data.data) {\n",
       "                    // IPython >= 3 moved mimebundle to data attribute of output\n",
       "                    data = data.data;\n",
       "                }\n",
       "                if (data['text/html'] == html_output) {\n",
       "                    return [cell, data, j];\n",
       "                }\n",
       "            }\n",
       "        }\n",
       "    }\n",
       "}\n",
       "\n",
       "// Register the function which deals with the matplotlib target/channel.\n",
       "// The kernel may be null if the page has been refreshed.\n",
       "if (IPython.notebook.kernel != null) {\n",
       "    IPython.notebook.kernel.comm_manager.register_target('matplotlib', mpl.mpl_figure_comm);\n",
       "}\n"
      ],
      "text/plain": [
       "<IPython.core.display.Javascript object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<img src=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAfQAAAGQCAYAAABYs5LGAAAgAElEQVR4Xu3dQapkWXKn8dAyEmINPRD0RKAeCAQ9L9A2soe1hBp2bEOQ8waBBi3QpEGDXkNCLSMbj6hX/TLkHs/tutm9x+z8NFJFnmvX7PsfP5+7P3/+/uaT/0MAAQQQQACB9gT+pv0EBkAAAQQQQACBT4RuEyCAAAIIIDCAAKEPCNEICCCAAAIIELo9gAACCCCAwAAChD4gRCMggAACCCBA6PYAAggggAACAwgQ+oAQjYAAAggggACh2wMIIIAAAggMIEDoA0I0AgIIIIAAAoRuDyCAAAIIIDCAAKEPCNEICCCAAAIIELo9gAACCCCAwAAChD4gRCMggAACCCBA6PYAAggggAACAwgQ+oAQjYAAAggggACh2wMIIIAAAggMIEDoA0I0AgIIIIAAAoRuDyCAAAIIIDCAAKEPCNEICCCAAAIIELo9gAACCCCAwAAChD4gRCMggAACCCBA6PYAAggggAACAwgQ+oAQjYAAAggggACh2wMIIIAAAggMIEDoA0I0AgIIIIAAAoRuDyCAAAIIIDCAAKEPCNEICCCAAAIIELo9gAACCCCAwAAChD4gRCMggAACCCBA6PYAAggggAACAwgQ+oAQjYAAAggggACh2wMIIIAAAggMIEDoA0I0AgIIIIAAAoRuDyCAAAIIIDCAAKEPCNEICCCAAAIIELo9gAACCCCAwAAChD4gRCMggAACCCBA6PYAAggggAACAwgQ+oAQjYAAAggggACh2wMIIIAAAggMIEDoA0I0AgIIIIAAAoRuDyCAAAIIIDCAAKEPCNEICCCAAAIIELo9gAACCCCAwAAChD4gRCMggAACCCBA6PYAAggggAACAwgQ+oAQjYAAAggggACh2wMRAr9FFluLAAKXEXC2X4b+uhsL/Tr27e78688/E3q71DS8I4HPX7442zcMXugbhn50ZEI/Ss51CJxLgNDP5b3K3Qh9lSQa9PFI6J9/+alB97EWf/3Dn+9eMHHWGJn1V++U3cNZvUJff6MWdEjoBVCnliT0T58Iff3dTeifPnmFvv4+reiQ0CuoDq1J6ITeYWsTOqF32KcVPRJ6BdWhNQmd0DtsbUIn9A77tKJHQq+gOrQmoRN6h61N6ITeYZ9W9EjoFVSH1iR0Qu+wtQmd0Dvs04oeCb2C6tCahE7oHbY2oRN6h31a0SOhV1AdWjMq9MjBGll7w7vS+pV6OcLm3nbtMNOj3zjQO6EPPYI/HIvQP0RkwRsBQr//Cr2DQG4ZRn7lrsNMhP6DJ7Z+D33Lg5vQt4z92NCETugrSXSlXn70hOnek6Py3gn92CHX/CpCbx7gme1HhX5mb9n3ir5Czb6/escJ7JSdb4o7vk8mXknoE1MtmonQY29bF8Wg7AcECN3P0Hd9kBD6rskfmJvQCf3Atjn9EkIn9NM33SI3JPRFgujQBqETeot9utEf1vGWe4cdeV6PhH4e6/Z3InRC77CJvUL3Cr3DPq3okdArqA6tSeiE3mFrEzqhd9inFT0SegXVoTUJndA7bG1CJ/QO+7SiR0KvoDq0JqETeoetTeiE3mGfVvRI6BVUh9asFPqjQziKMvJtaD+qXSmFrFmjbDqsz8ivMrsbw4z8Mub8US+ffbFMh+2e3iOhpyOdWzAq9MjBmnFI3siXH5S//PRywFmzvtzIggUy8ovsuyMIMvLLmJPQj6Q3+xpCn51v6nSEnvOEIUMIqcEuVCxDdITuLfeFtvSprRD6qbh734zQCb16BxN6jLDfQ4/xmr6a0KcnnDgfoRN64na6W4rQY4QJPcZr+mpCn55w4nyETuiJ24nQE2ASegLEQSUIfVCY1aMQOqFX7zGv0GOECT3Ga/pqQp+ecOJ8nYUe/aBUZH1k7S0OH4p7vCkjfyc8svZ2x+r1kYdaxhOXH+0lv7YWSWPOWkKfk2X5JIR+XwqEnrf1ItKNrCX0vIxUWpcAoa+bzXKdETqhV2/KiKQjawm9Ojn1VyBA6Cuk0KSHzkKPIo6+6o7U95Z7/C33DL7Vb3NHeqzuxVvukTTmrCX0OVmWTxIVeqShLMmVH5S+KS4Sa3htRn6VT8Z+9HPryLAZc/6oF0KPpDFnLaHPybJ8EkL3KffqTZYhOkL3TXHV+3TV+oS+ajIL9kXohF69LQk9RtivrcV4TV9N6NMTTpyP0Ak9cTvdLUXoMcKEHuM1fTWhT084cT5CJ/TE7UToCTAJPQHioBKEPijM6lEIndCr95hX6DHChB7jNX01oU9POHG+SqEntqkUAtsQIPRton5qUEJ/CpNFNwKEbh8gsBYBQl8rj6u7IfSrE2h0f0JvFJZWtyBA6FvE/PSQhP40KgsJ3R5AYC0ChL5WHld3Q+hXJ9Do/oTeKCytbkGA0LeI+ekhCf1pVBYSuj2AwFoECH2tPK7uhtCvTqDR/Qm9UVha3YIAoW8R89NDEvrTqCyMCj3yndqRtbckVlq/Ui9H2Nzb2R1mWu3Pp0b6iay95RNe/+WLs33DI1voG4Z+dGRCr/176OFD+8FffovKmND/fPchUZlHZe2vTwAI/egx1/o6Qm8d37nNEzqhl4so8CRlpV6ir6LLeyf0cw/HRe5G6IsE0aGNqNA7zKRHBDoT8DP0zunl907o+UzHViT0sdEarCkBQm8aXFHbhF4EdmJZQp+Yqpk6EyD0zunl907o+UzHViT0sdEarCkBQm8aXFHbhF4EdmJZQp+Yqpk6EyD0zunl907o+UzHViT0sdEarCkBQm8aXFHbhF4EdmJZQp+Yqpk6EyD0zunl907o+UzHViT0sdEarCkBQm8aXFHbhF4EdmLZR0KPzvroSzUidTK+De1H93tUP9Jjxpy3+2X0Euk7c+1KDKp7yaiflbVvisvcxX1qEXqfrC7vlNBjEWQc8IT+jXmG6KrzyKifMeeNF6HHHqtTVhP6lCRPmIPQY5AzDvgsmcU6z1u9EoPqXjLqE3re3tuxEqHvmPrBmQk9Bi7jgCd0r9Bju+7baq/Qj1Drfw2h98/wtAkIPYaa0B//2c8YSW+5R3kRepTYjPWEPiPHU6Yg9BhmQif02I7JeeLiFXqU+pz1hD4ny/JJCD2GmNAJPbZjCD3Ky/rfEyB0O+JpApVCj/4a2lXrn4Z1+zlm4G97f31VFVwf6eWqtdGZousjc0VrV66P1o7M6RV6lNac9YQ+J8vySQg9hjh6aEfXx7q5ZnV0puj6yFTR2pXro7UjcxJ6lNac9YQ+J8vySQg9hjh6aEfXx7q5ZnV0puj6yFTR2pXro7UjcxJ6lNac9YQ+J8vySSqFHm0++pZ7Vv1IHT9D9zP0yH65rfV76FFi1r8nQOj2w9MECP1pVF8XEvpaDKrzyKhP6LHHmNW/J0DodsTTBAj9aVSE/hdUGZLLeuVa3UtGfUKPPcasJnR74CABQo+Byzjgs2QW6zxv9UoMqnvJqE/oeXtvx0peoe+Y+sGZCT0GLuOAJ/RvzDNEV51HRv2MOb/+uOfLF2d77OE6YrXQR8R4zhCEHuOcccBnySzWed7qlRhU95JRn9Dz9t6OlQh9x9QPzkzoMXAZBzyhe4Ue23XfVnuFfoRa/2sIvX+G503w+U+/nXezH9+p+tfWVplTH+sRaLH3fv2js329rVPekdDLEQ+6AaEPCtMoRwkQ+lFyrqsmQOjVhCfVJ/RJaZrlIAFCPwjOZeUECL0c8aAbEPqgMI1ylAChHyXnumoChF5NeFJ9Qp+UplkOEiD0g+BcVk6A0MsRD7oBoQ8K0yhHCRD6UXKuqyZA6NWEJ9Un9ElpmuUgAUI/CM5l5QQIvRzxoBsQ+qAwjXKUAKEfJee6agKEXk14Uv2g0CMHX2TtDelK61fq5Qibe1u0w0zRvym+0vqsXh4eL34PfdLJ+/QshP40Kgs/EfrdP4naQX633Rv55roOM2VJ8YpZs3ondOfyewKEbj88T4DQCf2Xn+7ul85S7Nw7oT9/fO2wktB3SDlrxqDQs26b8ZZwZS9q70Ug+gTgEjrecr8E+9U3JfSrE+h0f0LvlJZeiwgQehFYZV8mQOgvI9yoAKFvFLZRHxEgdHtjVQKEvmoyK/ZF6CumoqeTCRD6ycDd7mkChP40Kgujn3KvJNbiUK0EoPZlBFrsPT9Dv2x/XHljQr+Sfrd7e4XeLTH9FhAg9AKoSqYQIPQUjJsUIfRNgjbmjwgQuv2xKgFCXzWZBfv69eeff8toK/IFJ4/u9+hQzegvq0bGnLdeOsz6iNlODDJmzcr685cvzvasB3KjOkJvFNbVrRJ6LIGMA57QvzHPEl0swdjqjLyz5iT0WHZTVhP6lCRPmIPQY5AzDvguMvMKPfbVutXvPBF67LE6ZTWhT0nyhDkIPQaZ0HMk1+VJTUbeXqHHHmNW/54AodsRTxMg9KdRfV2YccB3kZlX6Dl5E3rsMWY1odsDBwkQegwcoedIrsuTmoy8CT32GLOa0O2BgwQIPQYu44DvIjOv0HOevBB67DFmNaHbAwcJVAo9+ru9WQffQRRPXZb1N6+jbJ5qLnlRtMes9cljvFQukndk7ZGmfCjuCLX+1/gZev8MT5uA0GOoo4d21vpYlzmrswQdZZDTfU6VSO+RtUe6I/Qj1PpfQ+j9MzxtAkKPoY4e2lnrY13eX/0P//R3d//Dv/7zv9/9d0J//Jb7PTbRrKOZEnqU2Iz1hD4jx1OmqBR6dIDOb7lnzZr1M/rv+3kk8/frvhd7VOhZDKJ1Ktdn5JG1rwm9Mul1axP6utks1xmhxyLJOOBvd6yWJaHHcn20OiNvQs/JYtcqhL5r8gfmJvQYtIwD/kyhv39l/vYK/Pt/u7fmjB6zRBdLMLY6I++sOb1Cj2U3ZTWhT0nyhDkIPQY544A/Q5ZvUz2S9du/v3+b/fZv7/939bsIWaKLJRhbnZF31pyEHstuympCn5LkCXMQegxyxgFP6N+YZ4kulmBsdUbeWXMSeiy7KasJfUqSJ8xB6DHIGQf8VUJ/ZlKv0H9PKSNvQn9m51nziACh2xtPEyD0p1F9XZhxwBO6V+ixXfdttVfoR6j1v4bQ+2d42gSEHkNN6PVPamKJ1K7OyNsr9NqMplcn9OkJZ873+U+/RcpVf1Aq0kvntWdxXPlT7p3zi/SelvWvf3S2R8APWSv0IUGeMgahn4L5+5ukHfIfdP/+0+wffbL97E+5XwL+gpumZU3oF6R3/S0J/foM+nRA6JdklXbIPyn0Z4c880Nxz/bUfV1a1oTefSsc6p/QD2Hb9CJCvyT4tEOe0C/JL3LTtKwJPYJ9zFpCHxPlCYMQ+gmQ//Mt0g75J4V+e+X96O33e2/F38qe1eMlAZx40zSOhH5iauvcitDXyWL9Tgj9kozSDvnC7jv0WDh+Wuk0joSelkmnQoTeKa2reyX0SxJIO+QLu+/QY+H4aaXTOBJ6WiadChF6p7Su7rVQ6NGDbKX1K/Vy1Ra5gkH0b4qvtD6rl4d5E/pVD4VL70vol+JvdnNCv/vtb1fI7LZzMr7IJGsHXsEgS4qdeyf0rB08ow6hz8jxnCkIndAf7LTOUuzcO6Gfc/R1uQuhd0lqhT4Lhb7CeKv2EBXOFXN06PEKLtF7pnH0lnsU/Yj1hD4ixpOGIPSTQP/+NmmHfGH3HXosHD+tdBpHQk/LpFMhQu+U1tW9EvolCaQd8oXdd+ixcPy00mkcCT0tk06FCL1TWlf3SuiXJJB2yBd236HHwvHTSqdxJPS0TDoVIvROaV3dK6FfkkDaIV/YfYceC8dPK53GkdDTMulUiNA7pXV1r4R+SQJph3xh9x16LBw/rXQaR0JPy6RTIULvlNbVvRL6JQmkHfKF3XfosXD8tNJpHAk9LZNOhQi9U1oX9/rrzz//ltFCxheiPDr4MvrLqpEx562XtEM+a7A7dap73CXvrDk/f/nibC/c76uWFvqqySzYF6HHQiH0vG+zyxJdLMHY6oy8s+Yk9Fh2U1YT+pQkT5iD0GOQMw54r9C/Mc8SXSzB2OqMvLPmJPRYdlNWE/qUJE+Yg9BjkDMOeEIn9Niu+7aa0I9Q638NoffP8LQJCD2GmtC95R7bMXnvRBB6lPyM9YQ+I8dTpiD0GGZCJ/TYjiH0KC/rf0+A0O2IpwkQ+tOovr3t+ctPsQserK7+BHlGk9U9Zv1sOWPWRzUy8s6a0yv0yqTXrU3o62azXGeVQo8KIevgq4R81d/rrpzpUe2s/KLMrpg1KvR7bKrnJPSVdsZ5vRD6eazb34nQYxFGD+2s9bEuc1YT+uN3ZAg9Z4+p8jEBQv+YkRV/IUDosa2QJeioLGNd5qyO9pi1Pqf7nCqRvCNrj3TnFfoRav2vIfT+GZ42QaXQo0N0fst94qzRt6EnMvAz9Giq1mcTIPRsooPrEXos3IwD/nbHDk9eCD3nQ5BZWXuFHnusTllN6FOSPGEOQo9BJvQcyXV5UpORN6HHHmNW/54AodsRTxMg9KdRfV2YccB3kZlX6Dl5E3rsMWY1odsDBwkQegwcoedIrsuTmoy8CT32GLOa0O2BgwQIPQYu44DvIjOv0HOevBB67DFmNaHbAwcJEHoMHKHnSK7Lk5qMvAk99hizmtDtgYMECD0GLuOA7yIzr9BznrwQeuwxZjWh2wNHCXz+029HL82+LvrFJNn3V29fAi323q9/9IHnDbeo0DcM/fDIhH4YnQvnECD0OVlOm4TQpyVaOQ+hV9JVuwkBQm8S1IZtEvqGoR8emdAPo3PhHAKEPifLaZMQ+rREK+ch9Eq6ajchQOhNgtqwTULfMPTDIxP6YXQunEOA0OdkOW0SQp+WaOU8hF5JV+0mBAi9SVAbtknoG4Z+eOSg0CMHX2Ttrf/o+sMzb3phlO8V66N/U3yl9Vm9PNyefm1ty0cuoW8Z+8GhCf0guH6XXSHoG6WI6CJro7Wr12f1Tuj9HluVHRN6Jd1ptQl9WqKPffCHP9/9b1kiynjCsFIv0ScAWb0T+jYPyacGJfSnMFn0lUBQ6JXUokKo7EXtvQi02Hvect9rU/5lWkLfMvaDQxP6QXAum0SA0CelOWsWQp+VZ+00hF7LV/UWBAi9RUxbNknoW8Z+cGhCPwjOZZMIEPqkNGfNQuiz8qydhtBr+areggCht4hpyyYJfcvYDw5N6AfBuWwSAUKflOasWQh9Vp610xB6LV/VWxAg9BYxbdkkoW8Z+7Ghf/3559+OXfn7qx79Dm6kdvWhWlm/svaN4aP6Eb6P1mZk96MeM+pfxbey92h2n798cbZHoQ1YL/QBIZ41AqE//iazSAZXCSfSI6E/plWZX9aTMULP2O39ahB6v8wu65jQCT3jVahX6PEnC9EHPaFHic1YT+gzcjxlCkIndEKv/cNAXqGfcpSNvQmhj402fzBCJ3RCJ/T8k0XFLAKEnkVygzqETuiETugbHHVtRyT0ttGd3zihEzqhE/r5J487PkuA0J8lZd0nQid0Qid0R+G6BAh93WyW66xS6NFfBYquj8KM1I+sPfIJ76z6UQb31mf9He+MmVbq5cYq0k9k7ZHcfMr9CLX+1xB6/wxPm4DQ7x/aGXKKCuHI+oyNEhVR5frK2kf4RvqJrD2SG6Efodb/GkLvn+FpExA6oUdFVLm+sjahn3asuFEiAUJPhDm9VKXQo+yir4pXqn9V71EGkbfco7UrGVTWPvIjkwgbv4ceoWXt9wQI3Z54mgCh+1CcD8X5UNzTB4aFpxMg9NOR970hoRM6oRN63xNsfueEPj/jtAkJndAJndDTDhSF0gkQejrSuQUJndAJndDnnnD9JyP0/hmeNgGhEzqhE/ppB44bhQkQehjZvhcQOqETOqHvewKuPzmhr5/RMh0+EnrWIb/MoJ+uObQncnyUaeWvllXWXmmP3np5OOuXL8721cI6oR+hnwB5yi0Ivccr9A77rVK6lbVXY0voqyVybT+Efi3/VncndELP2rCV0q2snTV/Vh1CzyI5ow6hz8jxlCkIndCzNlqldCtrZ82fVYfQs0jOqEPoM3I8ZQpCJ/SsjVYp3craWfNn1SH0LJIz6hD6jBxPmYLQCT1ro1VKt7J21vxZdQg9i+SMOoQ+I8dTpiB0Qs/aaJXSraydNX9WHULPIjmjDqHPyPGUKQi9959PjfxaXFSKV6zf6c+nhmf1a2unnImr3YTQV0tk4X4IndDDYvnlp7s7OuMJwEq93IaM9BNZG639dT2hL3yS1rVG6HVsx1XuLPR/+Ke/u5vHv/7zv78snAw5HTq0k2R5D0CHmcqlmMT3Hsvy3gl93Pn7zECE/gwla74S6Cr0RzJ/i/We1KNCi2yRytqRPq5cW8mgsvaVzEJPvAh9tahO6YfQT8E84yaE7kNxWTu5UrqVtbPmz6rjQ3FZJGfUIfQZOZ4yRTehv39lfnsV/v3/vkG7929f3434w5/vMo18sOxRKJW1T9kICTepZFBZO2H01BKEnoqzfTFCbx/heQN0F/obqZvE37/N/ib19/9WKYXK2ufthtfuVMmgsvZrU+dfTej5TDtXJPTO6Z3cO6F7yz1ry1VKt7J21vxZdQg9i+SMOoQ+I8dTpugs9GcAeYX+DKWcNZXSraydM31eFULPYzmhEqFPSPGkGQjdK/SsrVYp3craWfNn1SH0LJIz6hD6jBxPmYLQCT1ro1VKt7J21vxZdQg9i+SMOoQ+I8dTpqgU+qODKTrY+0+hr/op9+hM0fVZLO/dN+NT/tF5ouurhZ7BN4sjoUd3x+z1hD4739Tpugr97VfWHn2y/exPuaeGcqdYhnAe9ZglokoGhO6rXyv318q1CX3ldBbrravQn8V41ofinu3n6DpCr/sOgVsmGXyznhh5hX70UTLzOkKfmWvJVISe8zP0knDeFc0Qjlfoj1PK4Evo1Y+CPesT+p65H5q6q9C/f8v9/RfLfP8lM29gqt+2PRTAkxdlCIfQCf3J7WbZQgQIfaEwVm+lm9Bf4Uno9+llvbJ8JZuPrq3OLuMJUxZHb7l/tBv2+u+EvlfeL01L6N5yzxLRSxvxg4sJ3YfiKvfXyrUJfeV0Fuuts9Cjh3xkfWTtLdKr1mdsp/K/4x34G+Qr9RJlW967P58ajWTEekIfEeM5QxD6/VfoVwk6KoWMXRK9Z+X6yto3VtH6Eb7R2uH1hB6JY8xaQh8TZf0ghE7oYbEEXnFHJbpSL9FHX3nvhB6NZMR6Qh8R4zlDdBZ6lFD0VXe0fuX6jA9tPerPz9D9Hnrl3lX7NQKE/hq/ra4mdB+KI3RC3+rQazYsoTcL7Mp2CZ3QCZ3QrzyD3PvHBAjdDnmaAKETOqET+tMHhoWnEyD005H3vSGhEzqhE3rfE2x+54Q+P+O0CQmd0Amd0NMOFIXSCRB6OtK5BQmd0Amd0OeecP0nI/T+GZ42AaHXos6SZYdfW6vs8VFKK/Gt7uWz30OvfbAuWp3QFw1mxbYIvTaV6kM+o/sOPRK673LP2OsdaxB6x9Qu6pnQa8F3kGWHHgmd0GsfqetWJ/R1s1muM0KvjaSDLDv0SOiEXvtIXbc6oa+bzXKdEXptJB1k2aFHQif02kfqutUJfd1sluuM0Gsj6SDLDj0SOqHXPlLXrU7o62azXGeEXhtJB1l26JHQCb32kbpudUJfN5vlOuss9OhfT1vp16pW6r36z35WbvqVeq9+YuTX1ip30rq1CX3dbJbrjNBrI1lJONFXuSs96ejQO6HXPpZ2rU7ouyZ/YG5CPwAtcAmhB2AdWLoSX0I/EKBLPiRA6B8isuCNQGehR1Nc6S33lXqvFlF01sj6lXqv7sVb7pGdMWctoc/JsnwSQq9FXH3IZ3TfocfoW+5RLhlP9qo5Eno01RnrCX1GjqdMQei1mKsP+YzuO/RI6D7lnrHXO9Yg9I6pXdQzodeC7yDLDj0SOqHXPlLXrU7o62azXGeEXhtJB1l26JHQCb32kbpudUJfN5vlOiP02kg6yLJDj4RO6LWP1HWrE/q62SzXGaHXRtJBlh16JHRCr32krlud0NfNZrnOCL02kg6y7NAjoRN67SN13eqEvm42y3VWKfTlhtUQAg0IPPyGvi9fnO0N8stuUejZRAfXiwr9//7pf/+Oxn/543/7+r9v//72/w/GZTQEygkQejniVjcg9FZxXdssoV/L390R+J4AodsT7wkQuv3wNIFnhf79K/Pvb3B7df62xiv1p/FbiMB/IkDoNgWh2wOHCDwj9I9k/nZjIj8UgYsQ+B0BQrchCN0eOESA0A9hcxECZQQIvQxty8Lecm8Z2zVNvyp0H4q7Jjd3nUuA0Odme2QyQj9CbdNrXhX6e2zfv+X+8GD65ae7tFdav1IvN1jRfu4Bjta4Yn313zevrF9Z+5anv7a25yFN6HvmfmhqQv/06d5BfIXMvh7aSU92CP3Pdx8PWXzv7Y/K2oR+6HgbcRGhj4jxnCEIndDLRRR4krJSL9EnWOW9+2KZcw7Fxe5C6IsFsnI7lUJfeW69IbAqAT9DXzWZa/oi9Gu4t7wrobeMTdODCRD64HAPjEboB6Dtegmh75q8uVclQOirJnNNX4R+DfeWd31G6LfBPvpyGV8q0zJ+TS9IgNAXDOXClgj9Qvjdbv2s0N/mevTHWbrNrV8EViVA6Ksmc01fhH4N95Z3JfSWsWl6MAFCHxzugdEI/QC0XS+JCn1XTuZG4CwChH4W6R73IfQeOS3R5SOhL9GcJhBA4K8EfFPcnpuB0PfM/dDUhH4Im4sQOJ0AoZ+OfIkbEvoSMfRogtB75KRLBAh9zz1A6HvmfnTq345e6DoEEDiVgLP9VNxr3Ezoa+TQpQtC75KUPncn4GzfcAcIfcPQXxg5Tej/8i//9Wsb//iP/+fT2///1tft3179v1vNtzoV9V/tz/VrE/hff/u3Xxv87//xH2s3+rg7Z3vX5F7oW+gvwNvw0hSh/0i275keEfuPnii8WnvDvLcb+XuRv/3vhnJ3tm+3ez99EvqGob8wMqG/AM+l6630X7kAAAdESURBVBMg9PUz0qG3ZeyBHAIpQr+18v5t8Htvu7+9HX+07Y/qH3n1f7QX1/Ul8F7wzd6G92Kt77Y73LnQD6Pb8sISoVe8Ff79z81ffYKwZdqG/nRP4rd/a/CzdWf7hvtX6BuG/sLIhP4CPJf2I0Do/TLbuWNC3zn9+OwlQq94+/v7t9zjo7pidwLfvxJv9gE5Z/uGG1joG4b+wsiE/gI8l/Yh8COZv59i4bfene19tltap0JPQ7lFIULfImZDEro90JEAoXdM7bqeS4T+fpyst999KO66TTLhzj/69TWv0CckPHMGQp+Za9VU5ULPkvs9oWfVroKr7vUE3v+c/Htx+xn69fno4McECN0OiRBIE/rbTX8k3ldfrVfWjkCztg+BHwm9zxRfO3W2Nwsso12hZ1Dcpwah75P1lpMS+paxjxma0MdEecog6UK/dV39x1Oq659C3k1OJdDsW+HusXG2n7pj1riZ0NfIoUsXJULvMrw+EWhEwNneKKysVoWeRXKPOoS+R86m7E/A2d4/w/AEQg8j2/oCQt86fsM3IuBsbxRWVqtCzyK5Rx1C3yNnU/Yn4Gzvn2F4AqGHkW19AaFvHb/hGxFwtjcKK6tVoWeR3KMOoe+Rsyn7E3C2988wPIHQw8i2voDQB8f/b//z7/863d//j3+7O+n7NfcW3Lvu3jWP6g/Ge/ZozvaziS9wP6EvEEKjFgi9UVjRVgk9Smzp9c72peOpaU7oNVynViX0ocl+/yr6o1fa3//3R08Gvv/3Z540DEV89ljO9rOJL3A/oS8QQqMWCL1RWNFWP5Ltj/77vf/29m/RJwfRvq2/S8DZvuHGEPqGob8wMqG/AG/1Swl99YRC/TnbQ7hmLBb6jBzPmoLQzyJ9wX0+EvqtpciH4rxCvyDE/39LZ/ul+K+5udCv4d71roTeNbkn+v5I6N5yfwLiOkuc7etkcVonQj8N9YgbEfqIGO8PQeijwnW2j4rzuWGE/hwnq74RIPTBOyFb6N+/RX/7cNxH9xiM9+zRnO1nE1/gfkJfIIRGLRB6o7CeafWjn4nfarz/lPpH6z/6RPtbT75Y5pl0XlrjbH8JX8+Lhd4zt6u6JvSryBfd9yNBE3oR+PqyzvZ6xsvdQejLRbJ0Q4S+dDyaQ+CvBJztG24GoW8Y+gsjE/oL8FyKwIkEnO0nwl7lVkJfJYkefRB6j5x0iYCzfcM9IPQNQ39hZEJ/AZ5LETiRgLP9RNir3EroqyTRow9C75GTLhFwtm+4B4S+YegvjEzoL8BzKQInEnC2nwh7lVsJfZUkevRB6D1y0iUCzvYN94DQNwz9hZEJ/QV4LkXgRALO9hNhr3Iroa+SRI8+CL1HTrpEwNm+4R4Q+oahvzAyob8Az6UInEjA2X4i7FVuJfRVkujRB6H3yEmXCDjbN9wDQt8w9BdGJvQX4LkUgRMJONtPhL3KrYS+ShI9+iD0HjnpEgFn+4Z7QOgbhv7CyIT+AjyXInAiAWf7ibBXuZXQV0miRx+E3iMnXSLgbN9wDwh9w9BfGJnQX4DnUgROJOBsPxH2KrcS+ipJ9OiD0HvkpEsEnO0b7gGhbxj6CyMT+gvwXIrAiQSc7SfCXuVWQl8liR59EHqPnHSJgLN9wz0g9A1Df2FkQn8BnksROJGAs/1E2KvcSuirJNGjD0LvkZMuEXC2b7gHhL5h6C+MTOgvwHMpAicScLafCHuVWwl9lSR69EHoPXLSJQLO9g33gNA3DN3ICCCAAALzCBD6vExNhAACCCCwIQFC3zB0IyOAAAIIzCNA6PMyNRECCCCAwIYECH3D0I2MAAIIIDCPAKHPy9RECCCAAAIbEiD0DUM3MgIIIIDAPAKEPi9TEyGAAAIIbEiA0DcM3cgIIIAAAvMIEPq8TE2EAAIIILAhAULfMHQjI4AAAgjMI0Do8zI1EQIIIIDAhgQIfcPQjYwAAgggMI8Aoc/L1EQIIIAAAhsSIPQNQzcyAggggMA8AoQ+L1MTIYAAAghsSIDQNwzdyAgggAAC8wgQ+rxMTYQAAgggsCEBQt8wdCMjgAACCMwjQOjzMjURAggggMCGBAh9w9CNjAACCCAwjwChz8vURAgggAACGxIg9A1DNzICCCCAwDwChD4vUxMhgAACCGxIgNA3DN3ICCCAAALzCBD6vExNhAACCCCwIQFC3zB0IyOAAAIIzCNA6PMyNRECCCCAwIYECH3D0I2MAAIIIDCPAKHPy9RECCCAAAIbEiD0DUM3MgIIIIDAPAKEPi9TEyGAAAIIbEiA0DcM3cgIIIAAAvMIEPq8TE2EAAIIILAhAULfMHQjI4AAAgjMI0Do8zI1EQIIIIDAhgQIfcPQjYwAAgggMI8Aoc/L1EQIIIAAAhsSIPQNQzcyAggggMA8AoQ+L1MTIYAAAghsSIDQNwzdyAgggAAC8wgQ+rxMTYQAAgggsCEBQt8wdCMjgAACCMwjQOjzMjURAggggMCGBP4flqAx6n0hLZMAAAAASUVORK5CYII=\" width=\"500\">"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_environment(env)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `step()` function actually returns several important objects:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "obs, reward, done, info = env.step(0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The observation tells the agent what the environment looks like, as discussed earlier. This is a 210x160 RGB image:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(210, 160, 3)"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "obs.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The environment also tells the agent how much reward it got during the last step:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.0"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "reward"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When the game is over, the environment returns `done=True`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "done"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, `info` is an environment-specific dictionary that can provide some extra information about the internal state of the environment. This is useful for debugging, but your agent should not use this information for learning (it would be cheating)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'ale.lives': 3}"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "info"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's play one full game (with 3 lives), by moving in random directions for 10 steps at a time, recording each frame:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "frames = []\n",
    "\n",
    "n_max_steps = 1000\n",
    "n_change_steps = 10\n",
    "\n",
    "obs = env.reset()\n",
    "for step in range(n_max_steps):\n",
    "    img = env.render(mode=\"rgb_array\")\n",
    "    frames.append(img)\n",
    "    if step % n_change_steps == 0:\n",
    "        action = env.action_space.sample() # play randomly\n",
    "    obs, reward, done, info = env.step(action)\n",
    "    if done:\n",
    "        break"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now show the animation (it's a bit jittery within Jupyter):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def update_scene(num, frames, patch):\n",
    "    patch.set_data(frames[num])\n",
    "    return patch,\n",
    "\n",
    "def plot_animation(frames, repeat=False, interval=40):\n",
    "    plt.close()  # or else nbagg sometimes plots in the previous cell\n",
    "    fig = plt.figure()\n",
    "    patch = plt.imshow(frames[0])\n",
    "    plt.axis('off')\n",
    "    return animation.FuncAnimation(fig, update_scene, fargs=(frames, patch), frames=len(frames), repeat=repeat, interval=interval)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/javascript": [
       "/* Put everything inside the global mpl namespace */\n",
       "window.mpl = {};\n",
       "\n",
       "\n",
       "mpl.get_websocket_type = function() {\n",
       "    if (typeof(WebSocket) !== 'undefined') {\n",
       "        return WebSocket;\n",
       "    } else if (typeof(MozWebSocket) !== 'undefined') {\n",
       "        return MozWebSocket;\n",
       "    } else {\n",
       "        alert('Your browser does not have WebSocket support.' +\n",
       "              'Please try Chrome, Safari or Firefox ≥ 6. ' +\n",
       "              'Firefox 4 and 5 are also supported but you ' +\n",
       "              'have to enable WebSockets in about:config.');\n",
       "    };\n",
       "}\n",
       "\n",
       "mpl.figure = function(figure_id, websocket, ondownload, parent_element) {\n",
       "    this.id = figure_id;\n",
       "\n",
       "    this.ws = websocket;\n",
       "\n",
       "    this.supports_binary = (this.ws.binaryType != undefined);\n",
       "\n",
       "    if (!this.supports_binary) {\n",
       "        var warnings = document.getElementById(\"mpl-warnings\");\n",
       "        if (warnings) {\n",
       "            warnings.style.display = 'block';\n",
       "            warnings.textContent = (\n",
       "                \"This browser does not support binary websocket messages. \" +\n",
       "                    \"Performance may be slow.\");\n",
       "        }\n",
       "    }\n",
       "\n",
       "    this.imageObj = new Image();\n",
       "\n",
       "    this.context = undefined;\n",
       "    this.message = undefined;\n",
       "    this.canvas = undefined;\n",
       "    this.rubberband_canvas = undefined;\n",
       "    this.rubberband_context = undefined;\n",
       "    this.format_dropdown = undefined;\n",
       "\n",
       "    this.image_mode = 'full';\n",
       "\n",
       "    this.root = $('<div/>');\n",
       "    this._root_extra_style(this.root)\n",
       "    this.root.attr('style', 'display: inline-block');\n",
       "\n",
       "    $(parent_element).append(this.root);\n",
       "\n",
       "    this._init_header(this);\n",
       "    this._init_canvas(this);\n",
       "    this._init_toolbar(this);\n",
       "\n",
       "    var fig = this;\n",
       "\n",
       "    this.waiting = false;\n",
       "\n",
       "    this.ws.onopen =  function () {\n",
       "            fig.send_message(\"supports_binary\", {value: fig.supports_binary});\n",
       "            fig.send_message(\"send_image_mode\", {});\n",
       "            if (mpl.ratio != 1) {\n",
       "                fig.send_message(\"set_dpi_ratio\", {'dpi_ratio': mpl.ratio});\n",
       "            }\n",
       "            fig.send_message(\"refresh\", {});\n",
       "        }\n",
       "\n",
       "    this.imageObj.onload = function() {\n",
       "            if (fig.image_mode == 'full') {\n",
       "                // Full images could contain transparency (where diff images\n",
       "                // almost always do), so we need to clear the canvas so that\n",
       "                // there is no ghosting.\n",
       "                fig.context.clearRect(0, 0, fig.canvas.width, fig.canvas.height);\n",
       "            }\n",
       "            fig.context.drawImage(fig.imageObj, 0, 0);\n",
       "        };\n",
       "\n",
       "    this.imageObj.onunload = function() {\n",
       "        this.ws.close();\n",
       "    }\n",
       "\n",
       "    this.ws.onmessage = this._make_on_message_function(this);\n",
       "\n",
       "    this.ondownload = ondownload;\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._init_header = function() {\n",
       "    var titlebar = $(\n",
       "        '<div class=\"ui-dialog-titlebar ui-widget-header ui-corner-all ' +\n",
       "        'ui-helper-clearfix\"/>');\n",
       "    var titletext = $(\n",
       "        '<div class=\"ui-dialog-title\" style=\"width: 100%; ' +\n",
       "        'text-align: center; padding: 3px;\"/>');\n",
       "    titlebar.append(titletext)\n",
       "    this.root.append(titlebar);\n",
       "    this.header = titletext[0];\n",
       "}\n",
       "\n",
       "\n",
       "\n",
       "mpl.figure.prototype._canvas_extra_style = function(canvas_div) {\n",
       "\n",
       "}\n",
       "\n",
       "\n",
       "mpl.figure.prototype._root_extra_style = function(canvas_div) {\n",
       "\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._init_canvas = function() {\n",
       "    var fig = this;\n",
       "\n",
       "    var canvas_div = $('<div/>');\n",
       "\n",
       "    canvas_div.attr('style', 'position: relative; clear: both; outline: 0');\n",
       "\n",
       "    function canvas_keyboard_event(event) {\n",
       "        return fig.key_event(event, event['data']);\n",
       "    }\n",
       "\n",
       "    canvas_div.keydown('key_press', canvas_keyboard_event);\n",
       "    canvas_div.keyup('key_release', canvas_keyboard_event);\n",
       "    this.canvas_div = canvas_div\n",
       "    this._canvas_extra_style(canvas_div)\n",
       "    this.root.append(canvas_div);\n",
       "\n",
       "    var canvas = $('<canvas/>');\n",
       "    canvas.addClass('mpl-canvas');\n",
       "    canvas.attr('style', \"left: 0; top: 0; z-index: 0; outline: 0\")\n",
       "\n",
       "    this.canvas = canvas[0];\n",
       "    this.context = canvas[0].getContext(\"2d\");\n",
       "\n",
       "    var backingStore = this.context.backingStorePixelRatio ||\n",
       "\tthis.context.webkitBackingStorePixelRatio ||\n",
       "\tthis.context.mozBackingStorePixelRatio ||\n",
       "\tthis.context.msBackingStorePixelRatio ||\n",
       "\tthis.context.oBackingStorePixelRatio ||\n",
       "\tthis.context.backingStorePixelRatio || 1;\n",
       "\n",
       "    mpl.ratio = (window.devicePixelRatio || 1) / backingStore;\n",
       "\n",
       "    var rubberband = $('<canvas/>');\n",
       "    rubberband.attr('style', \"position: absolute; left: 0; top: 0; z-index: 1;\")\n",
       "\n",
       "    var pass_mouse_events = true;\n",
       "\n",
       "    canvas_div.resizable({\n",
       "        start: function(event, ui) {\n",
       "            pass_mouse_events = false;\n",
       "        },\n",
       "        resize: function(event, ui) {\n",
       "            fig.request_resize(ui.size.width, ui.size.height);\n",
       "        },\n",
       "        stop: function(event, ui) {\n",
       "            pass_mouse_events = true;\n",
       "            fig.request_resize(ui.size.width, ui.size.height);\n",
       "        },\n",
       "    });\n",
       "\n",
       "    function mouse_event_fn(event) {\n",
       "        if (pass_mouse_events)\n",
       "            return fig.mouse_event(event, event['data']);\n",
       "    }\n",
       "\n",
       "    rubberband.mousedown('button_press', mouse_event_fn);\n",
       "    rubberband.mouseup('button_release', mouse_event_fn);\n",
       "    // Throttle sequential mouse events to 1 every 20ms.\n",
       "    rubberband.mousemove('motion_notify', mouse_event_fn);\n",
       "\n",
       "    rubberband.mouseenter('figure_enter', mouse_event_fn);\n",
       "    rubberband.mouseleave('figure_leave', mouse_event_fn);\n",
       "\n",
       "    canvas_div.on(\"wheel\", function (event) {\n",
       "        event = event.originalEvent;\n",
       "        event['data'] = 'scroll'\n",
       "        if (event.deltaY < 0) {\n",
       "            event.step = 1;\n",
       "        } else {\n",
       "            event.step = -1;\n",
       "        }\n",
       "        mouse_event_fn(event);\n",
       "    });\n",
       "\n",
       "    canvas_div.append(canvas);\n",
       "    canvas_div.append(rubberband);\n",
       "\n",
       "    this.rubberband = rubberband;\n",
       "    this.rubberband_canvas = rubberband[0];\n",
       "    this.rubberband_context = rubberband[0].getContext(\"2d\");\n",
       "    this.rubberband_context.strokeStyle = \"#000000\";\n",
       "\n",
       "    this._resize_canvas = function(width, height) {\n",
       "        // Keep the size of the canvas, canvas container, and rubber band\n",
       "        // canvas in synch.\n",
       "        canvas_div.css('width', width)\n",
       "        canvas_div.css('height', height)\n",
       "\n",
       "        canvas.attr('width', width * mpl.ratio);\n",
       "        canvas.attr('height', height * mpl.ratio);\n",
       "        canvas.attr('style', 'width: ' + width + 'px; height: ' + height + 'px;');\n",
       "\n",
       "        rubberband.attr('width', width);\n",
       "        rubberband.attr('height', height);\n",
       "    }\n",
       "\n",
       "    // Set the figure to an initial 600x600px, this will subsequently be updated\n",
       "    // upon first draw.\n",
       "    this._resize_canvas(600, 600);\n",
       "\n",
       "    // Disable right mouse context menu.\n",
       "    $(this.rubberband_canvas).bind(\"contextmenu\",function(e){\n",
       "        return false;\n",
       "    });\n",
       "\n",
       "    function set_focus () {\n",
       "        canvas.focus();\n",
       "        canvas_div.focus();\n",
       "    }\n",
       "\n",
       "    window.setTimeout(set_focus, 100);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._init_toolbar = function() {\n",
       "    var fig = this;\n",
       "\n",
       "    var nav_element = $('<div/>')\n",
       "    nav_element.attr('style', 'width: 100%');\n",
       "    this.root.append(nav_element);\n",
       "\n",
       "    // Define a callback function for later on.\n",
       "    function toolbar_event(event) {\n",
       "        return fig.toolbar_button_onclick(event['data']);\n",
       "    }\n",
       "    function toolbar_mouse_event(event) {\n",
       "        return fig.toolbar_button_onmouseover(event['data']);\n",
       "    }\n",
       "\n",
       "    for(var toolbar_ind in mpl.toolbar_items) {\n",
       "        var name = mpl.toolbar_items[toolbar_ind][0];\n",
       "        var tooltip = mpl.toolbar_items[toolbar_ind][1];\n",
       "        var image = mpl.toolbar_items[toolbar_ind][2];\n",
       "        var method_name = mpl.toolbar_items[toolbar_ind][3];\n",
       "\n",
       "        if (!name) {\n",
       "            // put a spacer in here.\n",
       "            continue;\n",
       "        }\n",
       "        var button = $('<button/>');\n",
       "        button.addClass('ui-button ui-widget ui-state-default ui-corner-all ' +\n",
       "                        'ui-button-icon-only');\n",
       "        button.attr('role', 'button');\n",
       "        button.attr('aria-disabled', 'false');\n",
       "        button.click(method_name, toolbar_event);\n",
       "        button.mouseover(tooltip, toolbar_mouse_event);\n",
       "\n",
       "        var icon_img = $('<span/>');\n",
       "        icon_img.addClass('ui-button-icon-primary ui-icon');\n",
       "        icon_img.addClass(image);\n",
       "        icon_img.addClass('ui-corner-all');\n",
       "\n",
       "        var tooltip_span = $('<span/>');\n",
       "        tooltip_span.addClass('ui-button-text');\n",
       "        tooltip_span.html(tooltip);\n",
       "\n",
       "        button.append(icon_img);\n",
       "        button.append(tooltip_span);\n",
       "\n",
       "        nav_element.append(button);\n",
       "    }\n",
       "\n",
       "    var fmt_picker_span = $('<span/>');\n",
       "\n",
       "    var fmt_picker = $('<select/>');\n",
       "    fmt_picker.addClass('mpl-toolbar-option ui-widget ui-widget-content');\n",
       "    fmt_picker_span.append(fmt_picker);\n",
       "    nav_element.append(fmt_picker_span);\n",
       "    this.format_dropdown = fmt_picker[0];\n",
       "\n",
       "    for (var ind in mpl.extensions) {\n",
       "        var fmt = mpl.extensions[ind];\n",
       "        var option = $(\n",
       "            '<option/>', {selected: fmt === mpl.default_extension}).html(fmt);\n",
       "        fmt_picker.append(option)\n",
       "    }\n",
       "\n",
       "    // Add hover states to the ui-buttons\n",
       "    $( \".ui-button\" ).hover(\n",
       "        function() { $(this).addClass(\"ui-state-hover\");},\n",
       "        function() { $(this).removeClass(\"ui-state-hover\");}\n",
       "    );\n",
       "\n",
       "    var status_bar = $('<span class=\"mpl-message\"/>');\n",
       "    nav_element.append(status_bar);\n",
       "    this.message = status_bar[0];\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.request_resize = function(x_pixels, y_pixels) {\n",
       "    // Request matplotlib to resize the figure. Matplotlib will then trigger a resize in the client,\n",
       "    // which will in turn request a refresh of the image.\n",
       "    this.send_message('resize', {'width': x_pixels, 'height': y_pixels});\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.send_message = function(type, properties) {\n",
       "    properties['type'] = type;\n",
       "    properties['figure_id'] = this.id;\n",
       "    this.ws.send(JSON.stringify(properties));\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.send_draw_message = function() {\n",
       "    if (!this.waiting) {\n",
       "        this.waiting = true;\n",
       "        this.ws.send(JSON.stringify({type: \"draw\", figure_id: this.id}));\n",
       "    }\n",
       "}\n",
       "\n",
       "\n",
       "mpl.figure.prototype.handle_save = function(fig, msg) {\n",
       "    var format_dropdown = fig.format_dropdown;\n",
       "    var format = format_dropdown.options[format_dropdown.selectedIndex].value;\n",
       "    fig.ondownload(fig, format);\n",
       "}\n",
       "\n",
       "\n",
       "mpl.figure.prototype.handle_resize = function(fig, msg) {\n",
       "    var size = msg['size'];\n",
       "    if (size[0] != fig.canvas.width || size[1] != fig.canvas.height) {\n",
       "        fig._resize_canvas(size[0], size[1]);\n",
       "        fig.send_message(\"refresh\", {});\n",
       "    };\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_rubberband = function(fig, msg) {\n",
       "    var x0 = msg['x0'] / mpl.ratio;\n",
       "    var y0 = (fig.canvas.height - msg['y0']) / mpl.ratio;\n",
       "    var x1 = msg['x1'] / mpl.ratio;\n",
       "    var y1 = (fig.canvas.height - msg['y1']) / mpl.ratio;\n",
       "    x0 = Math.floor(x0) + 0.5;\n",
       "    y0 = Math.floor(y0) + 0.5;\n",
       "    x1 = Math.floor(x1) + 0.5;\n",
       "    y1 = Math.floor(y1) + 0.5;\n",
       "    var min_x = Math.min(x0, x1);\n",
       "    var min_y = Math.min(y0, y1);\n",
       "    var width = Math.abs(x1 - x0);\n",
       "    var height = Math.abs(y1 - y0);\n",
       "\n",
       "    fig.rubberband_context.clearRect(\n",
       "        0, 0, fig.canvas.width, fig.canvas.height);\n",
       "\n",
       "    fig.rubberband_context.strokeRect(min_x, min_y, width, height);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_figure_label = function(fig, msg) {\n",
       "    // Updates the figure title.\n",
       "    fig.header.textContent = msg['label'];\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_cursor = function(fig, msg) {\n",
       "    var cursor = msg['cursor'];\n",
       "    switch(cursor)\n",
       "    {\n",
       "    case 0:\n",
       "        cursor = 'pointer';\n",
       "        break;\n",
       "    case 1:\n",
       "        cursor = 'default';\n",
       "        break;\n",
       "    case 2:\n",
       "        cursor = 'crosshair';\n",
       "        break;\n",
       "    case 3:\n",
       "        cursor = 'move';\n",
       "        break;\n",
       "    }\n",
       "    fig.rubberband_canvas.style.cursor = cursor;\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_message = function(fig, msg) {\n",
       "    fig.message.textContent = msg['message'];\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_draw = function(fig, msg) {\n",
       "    // Request the server to send over a new figure.\n",
       "    fig.send_draw_message();\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_image_mode = function(fig, msg) {\n",
       "    fig.image_mode = msg['mode'];\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.updated_canvas_event = function() {\n",
       "    // Called whenever the canvas gets updated.\n",
       "    this.send_message(\"ack\", {});\n",
       "}\n",
       "\n",
       "// A function to construct a web socket function for onmessage handling.\n",
       "// Called in the figure constructor.\n",
       "mpl.figure.prototype._make_on_message_function = function(fig) {\n",
       "    return function socket_on_message(evt) {\n",
       "        if (evt.data instanceof Blob) {\n",
       "            /* FIXME: We get \"Resource interpreted as Image but\n",
       "             * transferred with MIME type text/plain:\" errors on\n",
       "             * Chrome.  But how to set the MIME type?  It doesn't seem\n",
       "             * to be part of the websocket stream */\n",
       "            evt.data.type = \"image/png\";\n",
       "\n",
       "            /* Free the memory for the previous frames */\n",
       "            if (fig.imageObj.src) {\n",
       "                (window.URL || window.webkitURL).revokeObjectURL(\n",
       "                    fig.imageObj.src);\n",
       "            }\n",
       "\n",
       "            fig.imageObj.src = (window.URL || window.webkitURL).createObjectURL(\n",
       "                evt.data);\n",
       "            fig.updated_canvas_event();\n",
       "            fig.waiting = false;\n",
       "            return;\n",
       "        }\n",
       "        else if (typeof evt.data === 'string' && evt.data.slice(0, 21) == \"data:image/png;base64\") {\n",
       "            fig.imageObj.src = evt.data;\n",
       "            fig.updated_canvas_event();\n",
       "            fig.waiting = false;\n",
       "            return;\n",
       "        }\n",
       "\n",
       "        var msg = JSON.parse(evt.data);\n",
       "        var msg_type = msg['type'];\n",
       "\n",
       "        // Call the  \"handle_{type}\" callback, which takes\n",
       "        // the figure and JSON message as its only arguments.\n",
       "        try {\n",
       "            var callback = fig[\"handle_\" + msg_type];\n",
       "        } catch (e) {\n",
       "            console.log(\"No handler for the '\" + msg_type + \"' message type: \", msg);\n",
       "            return;\n",
       "        }\n",
       "\n",
       "        if (callback) {\n",
       "            try {\n",
       "                // console.log(\"Handling '\" + msg_type + \"' message: \", msg);\n",
       "                callback(fig, msg);\n",
       "            } catch (e) {\n",
       "                console.log(\"Exception inside the 'handler_\" + msg_type + \"' callback:\", e, e.stack, msg);\n",
       "            }\n",
       "        }\n",
       "    };\n",
       "}\n",
       "\n",
       "// from http://stackoverflow.com/questions/1114465/getting-mouse-location-in-canvas\n",
       "mpl.findpos = function(e) {\n",
       "    //this section is from http://www.quirksmode.org/js/events_properties.html\n",
       "    var targ;\n",
       "    if (!e)\n",
       "        e = window.event;\n",
       "    if (e.target)\n",
       "        targ = e.target;\n",
       "    else if (e.srcElement)\n",
       "        targ = e.srcElement;\n",
       "    if (targ.nodeType == 3) // defeat Safari bug\n",
       "        targ = targ.parentNode;\n",
       "\n",
       "    // jQuery normalizes the pageX and pageY\n",
       "    // pageX,Y are the mouse positions relative to the document\n",
       "    // offset() returns the position of the element relative to the document\n",
       "    var x = e.pageX - $(targ).offset().left;\n",
       "    var y = e.pageY - $(targ).offset().top;\n",
       "\n",
       "    return {\"x\": x, \"y\": y};\n",
       "};\n",
       "\n",
       "/*\n",
       " * return a copy of an object with only non-object keys\n",
       " * we need this to avoid circular references\n",
       " * http://stackoverflow.com/a/24161582/3208463\n",
       " */\n",
       "function simpleKeys (original) {\n",
       "  return Object.keys(original).reduce(function (obj, key) {\n",
       "    if (typeof original[key] !== 'object')\n",
       "        obj[key] = original[key]\n",
       "    return obj;\n",
       "  }, {});\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.mouse_event = function(event, name) {\n",
       "    var canvas_pos = mpl.findpos(event)\n",
       "\n",
       "    if (name === 'button_press')\n",
       "    {\n",
       "        this.canvas.focus();\n",
       "        this.canvas_div.focus();\n",
       "    }\n",
       "\n",
       "    var x = canvas_pos.x * mpl.ratio;\n",
       "    var y = canvas_pos.y * mpl.ratio;\n",
       "\n",
       "    this.send_message(name, {x: x, y: y, button: event.button,\n",
       "                             step: event.step,\n",
       "                             guiEvent: simpleKeys(event)});\n",
       "\n",
       "    /* This prevents the web browser from automatically changing to\n",
       "     * the text insertion cursor when the button is pressed.  We want\n",
       "     * to control all of the cursor setting manually through the\n",
       "     * 'cursor' event from matplotlib */\n",
       "    event.preventDefault();\n",
       "    return false;\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._key_event_extra = function(event, name) {\n",
       "    // Handle any extra behaviour associated with a key event\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.key_event = function(event, name) {\n",
       "\n",
       "    // Prevent repeat events\n",
       "    if (name == 'key_press')\n",
       "    {\n",
       "        if (event.which === this._key)\n",
       "            return;\n",
       "        else\n",
       "            this._key = event.which;\n",
       "    }\n",
       "    if (name == 'key_release')\n",
       "        this._key = null;\n",
       "\n",
       "    var value = '';\n",
       "    if (event.ctrlKey && event.which != 17)\n",
       "        value += \"ctrl+\";\n",
       "    if (event.altKey && event.which != 18)\n",
       "        value += \"alt+\";\n",
       "    if (event.shiftKey && event.which != 16)\n",
       "        value += \"shift+\";\n",
       "\n",
       "    value += 'k';\n",
       "    value += event.which.toString();\n",
       "\n",
       "    this._key_event_extra(event, name);\n",
       "\n",
       "    this.send_message(name, {key: value,\n",
       "                             guiEvent: simpleKeys(event)});\n",
       "    return false;\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.toolbar_button_onclick = function(name) {\n",
       "    if (name == 'download') {\n",
       "        this.handle_save(this, null);\n",
       "    } else {\n",
       "        this.send_message(\"toolbar_button\", {name: name});\n",
       "    }\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.toolbar_button_onmouseover = function(tooltip) {\n",
       "    this.message.textContent = tooltip;\n",
       "};\n",
       "mpl.toolbar_items = [[\"Home\", \"Reset original view\", \"fa fa-home icon-home\", \"home\"], [\"Back\", \"Back to  previous view\", \"fa fa-arrow-left icon-arrow-left\", \"back\"], [\"Forward\", \"Forward to next view\", \"fa fa-arrow-right icon-arrow-right\", \"forward\"], [\"\", \"\", \"\", \"\"], [\"Pan\", \"Pan axes with left mouse, zoom with right\", \"fa fa-arrows icon-move\", \"pan\"], [\"Zoom\", \"Zoom to rectangle\", \"fa fa-square-o icon-check-empty\", \"zoom\"], [\"\", \"\", \"\", \"\"], [\"Download\", \"Download plot\", \"fa fa-floppy-o icon-save\", \"download\"]];\n",
       "\n",
       "mpl.extensions = [\"eps\", \"jpeg\", \"pdf\", \"png\", \"ps\", \"raw\", \"svg\", \"tif\"];\n",
       "\n",
       "mpl.default_extension = \"png\";var comm_websocket_adapter = function(comm) {\n",
       "    // Create a \"websocket\"-like object which calls the given IPython comm\n",
       "    // object with the appropriate methods. Currently this is a non binary\n",
       "    // socket, so there is still some room for performance tuning.\n",
       "    var ws = {};\n",
       "\n",
       "    ws.close = function() {\n",
       "        comm.close()\n",
       "    };\n",
       "    ws.send = function(m) {\n",
       "        //console.log('sending', m);\n",
       "        comm.send(m);\n",
       "    };\n",
       "    // Register the callback with on_msg.\n",
       "    comm.on_msg(function(msg) {\n",
       "        //console.log('receiving', msg['content']['data'], msg);\n",
       "        // Pass the mpl event to the overriden (by mpl) onmessage function.\n",
       "        ws.onmessage(msg['content']['data'])\n",
       "    });\n",
       "    return ws;\n",
       "}\n",
       "\n",
       "mpl.mpl_figure_comm = function(comm, msg) {\n",
       "    // This is the function which gets called when the mpl process\n",
       "    // starts-up an IPython Comm through the \"matplotlib\" channel.\n",
       "\n",
       "    var id = msg.content.data.id;\n",
       "    // Get hold of the div created by the display call when the Comm\n",
       "    // socket was opened in Python.\n",
       "    var element = $(\"#\" + id);\n",
       "    var ws_proxy = comm_websocket_adapter(comm)\n",
       "\n",
       "    function ondownload(figure, format) {\n",
       "        window.open(figure.imageObj.src);\n",
       "    }\n",
       "\n",
       "    var fig = new mpl.figure(id, ws_proxy,\n",
       "                           ondownload,\n",
       "                           element.get(0));\n",
       "\n",
       "    // Call onopen now - mpl needs it, as it is assuming we've passed it a real\n",
       "    // web socket which is closed, not our websocket->open comm proxy.\n",
       "    ws_proxy.onopen();\n",
       "\n",
       "    fig.parent_element = element.get(0);\n",
       "    fig.cell_info = mpl.find_output_cell(\"<div id='\" + id + \"'></div>\");\n",
       "    if (!fig.cell_info) {\n",
       "        console.error(\"Failed to find cell for figure\", id, fig);\n",
       "        return;\n",
       "    }\n",
       "\n",
       "    var output_index = fig.cell_info[2]\n",
       "    var cell = fig.cell_info[0];\n",
       "\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_close = function(fig, msg) {\n",
       "    var width = fig.canvas.width/mpl.ratio\n",
       "    fig.root.unbind('remove')\n",
       "\n",
       "    // Update the output cell to use the data from the current canvas.\n",
       "    fig.push_to_output();\n",
       "    var dataURL = fig.canvas.toDataURL();\n",
       "    // Re-enable the keyboard manager in IPython - without this line, in FF,\n",
       "    // the notebook keyboard shortcuts fail.\n",
       "    IPython.keyboard_manager.enable()\n",
       "    $(fig.parent_element).html('<img src=\"' + dataURL + '\" width=\"' + width + '\">');\n",
       "    fig.close_ws(fig, msg);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.close_ws = function(fig, msg){\n",
       "    fig.send_message('closing', msg);\n",
       "    // fig.ws.close()\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.push_to_output = function(remove_interactive) {\n",
       "    // Turn the data on the canvas into data in the output cell.\n",
       "    var width = this.canvas.width/mpl.ratio\n",
       "    var dataURL = this.canvas.toDataURL();\n",
       "    this.cell_info[1]['text/html'] = '<img src=\"' + dataURL + '\" width=\"' + width + '\">';\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.updated_canvas_event = function() {\n",
       "    // Tell IPython that the notebook contents must change.\n",
       "    IPython.notebook.set_dirty(true);\n",
       "    this.send_message(\"ack\", {});\n",
       "    var fig = this;\n",
       "    // Wait a second, then push the new image to the DOM so\n",
       "    // that it is saved nicely (might be nice to debounce this).\n",
       "    setTimeout(function () { fig.push_to_output() }, 1000);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._init_toolbar = function() {\n",
       "    var fig = this;\n",
       "\n",
       "    var nav_element = $('<div/>')\n",
       "    nav_element.attr('style', 'width: 100%');\n",
       "    this.root.append(nav_element);\n",
       "\n",
       "    // Define a callback function for later on.\n",
       "    function toolbar_event(event) {\n",
       "        return fig.toolbar_button_onclick(event['data']);\n",
       "    }\n",
       "    function toolbar_mouse_event(event) {\n",
       "        return fig.toolbar_button_onmouseover(event['data']);\n",
       "    }\n",
       "\n",
       "    for(var toolbar_ind in mpl.toolbar_items){\n",
       "        var name = mpl.toolbar_items[toolbar_ind][0];\n",
       "        var tooltip = mpl.toolbar_items[toolbar_ind][1];\n",
       "        var image = mpl.toolbar_items[toolbar_ind][2];\n",
       "        var method_name = mpl.toolbar_items[toolbar_ind][3];\n",
       "\n",
       "        if (!name) { continue; };\n",
       "\n",
       "        var button = $('<button class=\"btn btn-default\" href=\"#\" title=\"' + name + '\"><i class=\"fa ' + image + ' fa-lg\"></i></button>');\n",
       "        button.click(method_name, toolbar_event);\n",
       "        button.mouseover(tooltip, toolbar_mouse_event);\n",
       "        nav_element.append(button);\n",
       "    }\n",
       "\n",
       "    // Add the status bar.\n",
       "    var status_bar = $('<span class=\"mpl-message\" style=\"text-align:right; float: right;\"/>');\n",
       "    nav_element.append(status_bar);\n",
       "    this.message = status_bar[0];\n",
       "\n",
       "    // Add the close button to the window.\n",
       "    var buttongrp = $('<div class=\"btn-group inline pull-right\"></div>');\n",
       "    var button = $('<button class=\"btn btn-mini btn-primary\" href=\"#\" title=\"Stop Interaction\"><i class=\"fa fa-power-off icon-remove icon-large\"></i></button>');\n",
       "    button.click(function (evt) { fig.handle_close(fig, {}); } );\n",
       "    button.mouseover('Stop Interaction', toolbar_mouse_event);\n",
       "    buttongrp.append(button);\n",
       "    var titlebar = this.root.find($('.ui-dialog-titlebar'));\n",
       "    titlebar.prepend(buttongrp);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._root_extra_style = function(el){\n",
       "    var fig = this\n",
       "    el.on(\"remove\", function(){\n",
       "\tfig.close_ws(fig, {});\n",
       "    });\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._canvas_extra_style = function(el){\n",
       "    // this is important to make the div 'focusable\n",
       "    el.attr('tabindex', 0)\n",
       "    // reach out to IPython and tell the keyboard manager to turn it's self\n",
       "    // off when our div gets focus\n",
       "\n",
       "    // location in version 3\n",
       "    if (IPython.notebook.keyboard_manager) {\n",
       "        IPython.notebook.keyboard_manager.register_events(el);\n",
       "    }\n",
       "    else {\n",
       "        // location in version 2\n",
       "        IPython.keyboard_manager.register_events(el);\n",
       "    }\n",
       "\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._key_event_extra = function(event, name) {\n",
       "    var manager = IPython.notebook.keyboard_manager;\n",
       "    if (!manager)\n",
       "        manager = IPython.keyboard_manager;\n",
       "\n",
       "    // Check for shift+enter\n",
       "    if (event.shiftKey && event.which == 13) {\n",
       "        this.canvas_div.blur();\n",
       "        // select the cell after this one\n",
       "        var index = IPython.notebook.find_cell_index(this.cell_info[0]);\n",
       "        IPython.notebook.select(index + 1);\n",
       "    }\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_save = function(fig, msg) {\n",
       "    fig.ondownload(fig, null);\n",
       "}\n",
       "\n",
       "\n",
       "mpl.find_output_cell = function(html_output) {\n",
       "    // Return the cell and output element which can be found *uniquely* in the notebook.\n",
       "    // Note - this is a bit hacky, but it is done because the \"notebook_saving.Notebook\"\n",
       "    // IPython event is triggered only after the cells have been serialised, which for\n",
       "    // our purposes (turning an active figure into a static one), is too late.\n",
       "    var cells = IPython.notebook.get_cells();\n",
       "    var ncells = cells.length;\n",
       "    for (var i=0; i<ncells; i++) {\n",
       "        var cell = cells[i];\n",
       "        if (cell.cell_type === 'code'){\n",
       "            for (var j=0; j<cell.output_area.outputs.length; j++) {\n",
       "                var data = cell.output_area.outputs[j];\n",
       "                if (data.data) {\n",
       "                    // IPython >= 3 moved mimebundle to data attribute of output\n",
       "                    data = data.data;\n",
       "                }\n",
       "                if (data['text/html'] == html_output) {\n",
       "                    return [cell, data, j];\n",
       "                }\n",
       "            }\n",
       "        }\n",
       "    }\n",
       "}\n",
       "\n",
       "// Register the function which deals with the matplotlib target/channel.\n",
       "// The kernel may be null if the page has been refreshed.\n",
       "if (IPython.notebook.kernel != null) {\n",
       "    IPython.notebook.kernel.comm_manager.register_target('matplotlib', mpl.mpl_figure_comm);\n",
       "}\n"
      ],
      "text/plain": [
       "<IPython.core.display.Javascript object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<img src=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAoAAAAHgCAYAAAA10dzkAAAgAElEQVR4Xu3dP7IkyZEf4N5jjFmfgcLquzJ1HKQp4ggQ2QcZnTJXX4FnGDMcY2hvsAM0Zl9VZXile4RHfFRoJDIywj/36vxNvn//8sX/IUCAAAECBAgQOErgX46qVrEECBAgQIAAAQJfBEBDQIAAAQIECBA4TEAAPKzhyiVAgAABAgQICIBmgAABAgQIECBwmIAAeFjDlUuAAAECBAgQEADNAAECBAgQIEDgMAEB8LCGK5cAAQIECBAgIACaAQIECBAgQIDAYQIC4GENVy4BAgQIECBAQAA0AwQIECBAgACBwwQEwMMarlwCBAgQIECAgABoBggQIECAAAEChwkIgIc1XLkECBAgQIAAAQHQDBAgQIAAAQIEDhMQAA9ruHIJECBAgAABAgKgGSBAgAABAgQIHCYgAB7WcOUSIECAAAECBARAM0CAAAECBAgQOExAADys4colQIAAAQIECAiAZoAAAQIECBAgcJiAAHhYw5VLgAABAgQIEBAAzQABAgQIECBA4DABAfCwhiuXAAECBAgQICAAmgECBAgQIECAwGECAuBhDVcuAQIECBAgQEAANAMECBAgQIAAgcMEBMDDGq5cAgQIECBAgIAAaAYIECBAgAABAocJCICHNVy5BAgQIECAAAEB0AwQIECAAAECBA4TEAAPa7hyCRAgQIAAAQICoBkgQIAAAQIECBwmIAAe1nDlEiBAgAABAgQEQDNAgAABAgQIEDhMQAA8rOHKJUCAAAECBAgIgGaAAAECBAgQIHCYgAB4WMOVS4AAAQIECBAQAM0AAQIECBAgQOAwAQHwsIYrlwABAgQIECAgAJoBAgQIECBAgMBhAgLgYQ1XLgECBAgQIEBAADQDBAgQIECAAIHDBATAwxquXAIECBAgQICAAGgGCBAgQIAAAQKHCQiAhzVcuQQIECBAgAABAdAMECBAgAABAgQOExAAD2u4cgkQIECAAAECAqAZIECAAAECBAgcJiAAHtZw5RIgQIAAAQIEBEAzQIAAAQIECBA4TEAAPKzhyiVAgAABAgQICIBmgAABAgQIECBwmIAAeFjDlUuAAAECBAgQEADNAAECBAgQIEDgMAEB8LCGK5cAAQIECBAgIACaAQIECBAgQIDAYQIC4GENVy4BAgQIECBAQAA0AycJ/HpSsWolQGCKgOfqFHabjgoY1FEx17cV+OXbNwGwbfccnEAPga/fv3uu9mjV8ac0qMePwDkAAuA5vVYpgVkCAuAsefuOCgiAo2KubyvwLAB+/fmntnW9e/Bf/vTXp7c42eZd287rn83F6TPx1MYbwM5jf9TZBcCj2n12sQLg5/0XAM/+XDyqXgB8PBcCoM/MDgIC4A5dVMMlAQFQALw0KC76TUAAFAB9FPYWEAD37q/qfhAQAAVAH4jrAgKgAHh9WlzZUUAA7Ng1Zw4JCIACYGhwDl0kAAqAh47+MWULgMe0WqECoADoU3BdQAAUAK9Piys7CgiAHbvmzCEBAVAADA3OoYsEQAHw0NE/pmwB8JhWK1QAFAB9Cq4LCIAC4PVpcWVHAQGwY9ecOSQQDYDRB+EO6z6gn/3Otx1qjP5Ou91rj9b3MTPRtVus83sAQ/8+W1QvIADWm9txkoAAeP8bwC0e2MFfAr577dH6BEB/Cm7SP/G2HRQQAAfBXN5XQAAUAD8T8AZwfC5emUXD4xbrvAHs+5A47OQC4GENP7lcAXD8Qe9LwMHvA3vyVnGLkPPirenuNfpLICc/SfapXQDcp5cqeSEQDYC7w/pTcLt3OFZfNMTFduu1SgDs1S+n/VxAADQZxwgIgPe/ATxmeA4sVAAMvv31JeADPy09SxYAe/bNqQMCAqAAGBibY5cIgALgscN/SOEC4CGNVuaXLwKgAOhzcF1AABQAr0+LKzsKCIAdu+bMIQEBUAAMDc6hiwRAAfDQ0T+mbAHwmFYrVAAUAH0KrgsIgALg9WlxZUcBAbBj15w5JCAACoChwTl0kQAoAB46+seULQAe02qFCoACoE/BdQEBUAC8Pi2u7CggAHbsmjOHBARAATA0OIcuEgAFwENH/5iyBcBjWq3Q6gD46hcsRzry6k9wRe756pwz9ozUcfKa6h5l7PfRv1ezGOlxxln9IuhIJ6xZTUAAXK0jzpMmEA2A0TchOzzMPppR/QBNG4CNb1zdo4z9BMCNB1RpSwoIgEu2xaEyBATAdb4EnBGOM2amyz0zAln0P3zeMcuYi3IbfwnknRGwtlBAACzEttVcAQFQAJw7gXm7l4ecn39KKUYATGF1UwKfCgiABuMYAQFQANx12AXAx50tt/EGcNeP2XZ1CYDbtVRBjwQEQAFw109HecjxBvDhKH0VAHf9mG1XlwC4XUsVJACOzcCrL7tVh4ux07v6Q6C6Rxn7fdTxahYj3c44q58CjnTCmtUEBMDVOuI8aQLeAHoDmDZck29cHnK8AfQGcPLM2/59AQHwfUN3aCKwewCM/tTmq7cuz8JF1p5NRmqZY1b36FXgXGkuXp010kRvACNq1qwmIACu1hHnSRMQAO9/A7jSgz5tcBrcWAB83CQBsMEAO+IUAQFwCrtNZwgIgALgjLmr2FMAFAAr5sweewkIgHv1UzVPBARAAXDXD4gAKADuOtvqyhMQAPNs3Xkxgd0DYJT7ne8BzNozet9T15V/mdMPgTwcNb8G5tRPYb+6BcB+PXPioIAAeP8bwGArUn7dR/QsO6wTAL0B3GGO1VArIADWetttokA0AEaP/OrNWuS+1Q/6jzPO2DNic/Ka6h5l7PfRvx0+M94AnvxJ7FW7ANirX077hoAA6A3gG+Oz9NKMQBb9Ce93oATAd/SsJTAmIACOebm6sYAAKAA2Ht+nRxcAfQl419lWV56AAJhn686LCQiAAuBiI3nbcQRAAfC2YXKjYwQEwGNarVABUADc9VMgAAqAu862uvIEBMA8W3deTEAAFAAXG8nbjiMACoC3DZMbHSMgAB7TaoUKgALgrp8CAVAA3HW21ZUnIADm2brzYgLVAXCx8h2HAIGbBJ7+hPT3756rNzm7Ta6AQc31dfeFBATAhZrhKAQaCwiAjZvn6H8XEAANwzECAuAxrVYogVQBATCV182LBATAImjbzBcQAOf3wAkI7CAgAO7QRTUIgGbgGAEB8JhWK5RAqoAAmMrr5kUCAmARtG3mCwiA83vgBAR2EBAAd+iiGgRAM3CMgAB4TKsVSiBVQABM5XXzIgEBsAjaNvMFBMD5PXACAjsICIA7dFENAqAZOEZAADym1QolkCogAKbyunmRgABYBG2b+QLRAPj0H/uff3pY2A7rPop79lcmdqgx+lc0dq89Wt/HzETXbrHOL4Ke/4+9E1wSEAAvMbloBwEB8PMuPnvoCoCPJ3+LsJLwHzACoL8EssPz4oQaBMATuqzG3wQEQAHwMwFvAMfn4pXZ0eHYG0BPnCYCAmCTRjnm+wLRAPj+zu5AgMBOAr4HcKdunluLAHhu74+rXAA8ruUKJpAiIACmsLppsYAAWAxuu3kCAuA8ezsT2ElAANypm+fWIgCe2/vjKhcAj2u5ggmkCAiAKaxuWiwgABaD226egAA4z97OBHYSEAB36ua5tQiA5/b+uMoFwONarmACKQICYAqrmxYLCIDF4LabJyAAzrO3M4GdBATAnbp5bi0C4Lm9P65yAfC4liuYQIqAAJjC6qbFAgJgMbjt5gkIgPPs7UxgJwEBcKdunluLAHhu74+r/FkAjGK8+osIkftG/4pCZK+PNa/+FFz0vtU20XOutK6LWcY5X81ixp4Zs//VXwJZ6SPlLE8EBEDjcYyAAPh5qzMegh87dXlgr/QB6GKWcU4BcKVJdJYTBATAE7qsxt8EBEABcPWPQkawygj4GecUAFefTufbTUAA3K2j6nkoIAAKgKt/PDKClQD4uOspNr4EvPrHzPn+S0AANArHCAiAAuDqwy4A/vVhi9rYCICrf8ycTwA0A6cJCIAC4Ooz3ybk/PxTCuUOPwDlh0BSRsNNEwS8AUxAdcs1BQRAAXDNyfzHqQRAbwBXn1Hn20dAANynlyp5ISAACoCrf0gEQAFw9Rl1vn0EBMB9eqkSATA0AxnfCP9xkC5hJoSWtKiLWcY5P0h9CThpsNyWwCcCAqCxOEag+g1g9GG20rp3huNZSMio8Z2zrrK2i9mrAJjR3zY2fghklY+Tc7wQEACNyDECAuA6XwLOCAg7DHKbkPPih0Ay+tvGRgDc4aN4RA0C4BFtVuRvX1769u3XuyWqH0rV+73j1ems79R559ouZt4APu66nwK+8xPhXpkCAmCmrnsvJVAdAKPFR9+eZOwXvefHulchIXLvrO9XjJwlY00Xs4xz/vYfaX/yQyAZc+WeBD4TEADNxTECAuA6XwKODp0AOC6XYSYAegM4PolWrCYgAK7WEedJExAABcC04brpxhnBSgB83JwUG98DeNOnwW2yBQTAbGH3X0ZAABQAlxnGBwcRAH0JePUZdb59BATAfXqpkhcCAqAAuPqHRAAUAFefUefbR0AA3KeXKhEAQzOQ8WWwj4N0CTMhtKRFXcwyzvlB6odAkgbLbQl8IiAAGotjBLwB9AZw9WHPCFYZAT/jnALg6tPpfLsJCIC7dVQ9DwUEQAFw9Y9HRrASAB93PcXGD4Gs/jFzvv8SEACNwjECAqAAuPqwC4C+B3D1GXW+fQQEwH16qZJXAl//cvtfAnm1ZeR/r/4+qMgZrSGQIbDF7P/yZ8/VjOFwz9sFDOrtpG64rIAAuGxrHIzAh4AAaA4I1AkIgHXWdpotIADO7oD9CTwVEAANCIE6AQGwztpOswUEwNkdsD8BAdAMEFhEQABcpBGOUSAgABYg24JAXMAbwLidlQRGBQTAUTHX9xUQAPv2zsmPEBAAj2izIhcREAAXaYRjFAgIgAXItiAQFxAA43ZWEhgVEABHxVzfV0AA7Ns7Jz9CQAA8os2KXERAAFykEY5RICAAFiDbgkBcQACM21lJYFRAABwVc31fgWAAjD6Udlj30exnf51ihxqjf31j99qj9X3MTHRtl3VP/xH0i6D7PiMOO7kAeFjDjy5XAPy0/a/+HqoA+PmnpktYqT6nAOgvgRz9nGlUvADYqFmO+qaAACgAfiLgDeC9AVcAFADf/Jfa8iIBAbAI2jYLCAQDYPXJo29sqs9pPwJ3C2wx+74EfPdYuF+SgACYBOu2CwoIgAs2xZEI/ENAADQNBOoEBMA6azvNFhAAZ3fA/gSeCgiABoRAnYAAWGdtp9kCAuDsDtifgABoBggsIiAALtIIxygQEAALkG1BIC7gDWDczkoCowIC4KiY6/sKCIB9e+fkRwgIgEe0WZGLCAiAizTCMQoEBMACZFsQiAsIgHE7KwmMCgiAo2Ku7ysgAPbtnZMfISAAHtFmRS4iIAAu0gjHKBAQAAuQbUEgLiAAxu2sJDAqIACOirm+rcAv3779evfho39F4tk5Xv1ptrtryLofm3FZZo/Nuth8/f7dc3V89K2YIGBQJ6Dbco6AAFjr3uWBXavyfDdmAuBK8+gsewsIgHv3V3U/CAiAteMgzIx7MxMAx6fGCgIxAQEw5mZVQwEBsLZpwsy4NzMBcHxqrCAQExAAY25WNRQQAGubJsyMezMTAMenxgoCMQEBMOZmVUMBAbC2acLMuDczAXB8aqwgEBMQAGNuVjUUEABrmybMjHszEwDHp8YKAjEBATDmZlVDAQGwtmnCzLg3MwFwfGqsIBATEABjblY1FBAAa5smzIx7MxMAx6fGCgIxAQEw5mZVQ4HqABj9qwYn/CLoDJuM8JQx5tW1m6efHrYxw8Yvgs741LhnhoAAmKHqnksKCIC1bXkWyKpDUG3lz3errj0j5MzwrJ6naI0CYFTOumoBAbBa3H7TBATAWvrqB7Y3gJ/3VwD0BrD2k2+3LgICYJdOOefbAgLg24RDNxAAxwNZtdlQQydf3MXGG8DJg2L7ywIC4GUqF3YXqA6AUa8T3thk2Oz+BjDDLHrPGesy+pvxWRMAZ0yHPSMCAmBEzZqWAgJgbduqH9gZ+2WIRb8HMHqWjJATPcs76zL6m2EjAL7TZWsrBQTASm17TRUQAGv5qx/YGftliAmAMdWM/gqAsV5YtYeAALhHH1VxQUAAvIB04yXVD+yM/W7k+PutBMCYakZ/BcBYL6zaQ0AA3KOPqrggIABeQLrxkuoHdsZ+N3IIgG9iZvRXAHyzKZa3FhAAW7fP4UcEBMARrfevrX5gZ+z3vsJ/v4M3gDHVjP4KgLFeWLWHgAC4Rx9VcUFAALyAdOMl1Q/sjP1u5PAG8E3MjP4KgG82xfLWAgJg6/Y5/IiAADii9f611Q/sjP3eV/AG8C7DjP4KgHd1x306CgiAHbvmzDGBr3/5NbKw+kt2kTOesmaHXuxQwy7zltKLX/7subrLgGxeh0HdvMHK+0FAAGw/DikP7GKVHWooJkvbLqUXAmBav9z4XgEB8F5Pd1tZQABcuTuXzpbywL60830X7VDDfRpz75TSCwFwblPtfllAALxM5cL2AgJg+xamPLCLVXaooZgsbbuUXgiAaf1y43sFBMB7Pd1tZQEBcOXuXDpbygP70s73XbRDDfdpzL1TSi8EwLlNtftlAQHwMpUL2wsIgO1bmPLALlbZoYZisrTtUnohAKb1y43vFRAA7/V0t5UFBMCVu3PpbCkP7Es733fRDjXcpzH3Tim9EADnNtXulwUEwMtULmwvIAC2b2HKA7tYZYcaisnStkvphQCY1i83vldAALzX091WFhAAV+7OpbOlPLAv7XzfRTvUcJ/G3Dul9EIAnNtUu18WEAAvU7mwvYAA2L6FKQ/sYpVoDV3WfXB2OWv0nE9HRgAs/kTZLiogAEblrOsnIAD269kfTpzywC5WidbQZZ0A6C+BFH+kbBcUEACDcJY1FBAAGzbtn48cDUErFR6tocs6AVAAXOnz5iyPBQRA03GOgADYvtfRELRS4dEauqwTAAXAlT5vziIAmgECX74IgO2nIBqCVip8hxpW8nznLCm98D2A77TE2kIBbwALsW01WUAAnNyA97dPeWC/f6yhO+xQw1DBC1+c0gsBcOGOO9qPAgKgeThHQABs3+uUB3axyg41FJOlbZfSCwEwrV9ufK+AAHivp7utLCAArtydS2dLeWBf2vm+i3ao4T6NuXdK6YUAOLepdr8sIABepnJhewEBsH0LUx7YxSo71FBMlrZdSi8EwLR+ufG9AgLgvZ7utrKAALhydy6dLeWBfWnn+y7aoYb7NObeKaUXAuDcptr9soAAeJnKhe0FBMD2LUx5YBer7FBDMVnadim9EADT+uXG9woIgPd6utvKAgLgyt25dLaUB/alne+7aIca7tOYe6eUXgiAc5tq98sCAuBlKhe2FxAA27cw5YFdrLJDDcVkadul9EIATOuXG98rIADe6+luCwv88u3br3cf7+vPP919yy/PHkq3b5Z4w2qbjP0yeFJCx5ODmqfHOBk2X79/91zN+OC45+0CBvV2UjdcVUAArO1MRiCrDk8ZYtU1ZIScDJdX96yep1fnefS/C4BROeuqBQTAanH7TRMQAGvpqx/YGftliAmAMdWM/maEYwEw1l+r6gUEwHpzO04SEABr4asf2Bn7ZYgJgDHVjP4KgLFeWLWHgAC4Rx9VcUFAALyAdOMl1Q/sjP1u5Pj7rQTAmGpGfwXAWC+s2kNAANyjj6q4ICAAXkC68ZLqB3bGfjdyCIBvYmb0VwB8symWtxYQAFu3z+FHBATAEa33r61+YGfs977Cf7+DN4Ax1Yz+CoCxXli1h4AAuEcfVXFBQAC8gHTjJdUP7Iz9buTwBvBNzIz+CoBvNsXy1gICYOv2OfyIQHUAjL7pyXgojTjddW31Aztjv7ssfrxPxlw8q/2EecowjfbeTwFH5ayrFhAAq8XtN01AAKylzwhk0Qd9beXPd4vWkLFuJZdXZ4mG3Oi6V+d59L8LgFE566oFBMBqcftNExAAa+kFwM+9M4JcdcipnaS/7RatMbouWqMAGJWzrlpAAKwWt980AQGwll4AFADvnLhokIuui55dAIzKWVctIABWi9tvmkB1AIwWesL3bJ1u8/DLh/629MPRqP4PiuiMCoBROeuqBQTAanH7TRMQAGvpuzywa1We78bssU8XGwFwpU+UszwTEADNxzECAmBtq7s8sGtVBMCod5d5EgCjHbauWkAArBa33zQBAbCWvssDu1ZFAIx6d5knATDaYeuqBQTAanH7TRMQAGvpuzywa1UEwKh3l3kSAKMdtq5aQACsFrffNAEBsJa+ywO7VkUAjHp3mScBMNph66oFBMBqcftNExAAa+m7PLBrVQTAqHeXeRIAox22rlpAAKwWt980AQGwlr7LA7tWRQCMeneZJwEw2mHrqgUEwGpx+00TEABr6bs8sGtVBMCod5d5EgCjHbauWkAArBa33zyBr3/5dd7m13eO/qmw6zu4ksCaAlvM/i9/9lxdc7yc6g8CBtVInCMgAJ7T64JK/99f/u9vu/yPP//7b//37//vH7f+/X8rOM4WWwiAW7RREU0EBMAmjXLMGwQEwBsQ3eJ3AQHw/lkQAO83dUcCjwQEQLNxjoAAeE6vEyu9Evz+uL03gdcaIgBec3IVgTsEBMA7FN2jh4AA2KNPi59SAMxrkACYZ+vOBP4oIACaiXMEBMBzel1Q6R+/5+/jLd9n3wf4cRRvAK81RAC85uQqAncICIB3KLpHDwEBsEefmpxSALy/UQLg/abuSOCRgABoNs4READP6XVBpY/e9nnjF8cXAON2VhIYFRAAR8Vc31cgGACjD6XqdX0b0/Pkn70B7FnJOqeu/sxE93sq5vcArjNQTvJUQAA0IOcICIDn9LqgUgHwfuRoIKteJwDe33t3rBcQAOvN7ThLQACcJb/lvgLg/W2tDnLR/QTA+3vvjvUCAmC9uR1nCQiAs+S33FcAvL+t0UBWvU4AvL/37lgvIADWm9txlkAwAFYfN/owqz7n6fv5IZD7J2CL2fc9gPcPhjumCAiAKaxuuqSAALhkW7oeSgC8v3MC4P2m7kjgkYAAaDbOERAAz+l1QaXPAuAft/eLoK81RAC85uQqAncICIB3KLpHDwEBsEefmpxSALy/UQLg/abuSMAbQDNAQAA0AwkCV4KgN4DX4AXAa06uInCHgDeAdyi6Rw8BAbBHn5qe8rMgKPiNNVMAHPNyNYF3BATAd/Ss7SUgAPbqV7PTCoDvN0wAfN/QHQhcFRAAr0q5rr+AANi/hyrYWkAA3Lq9iltMQABcrCGOkyggACbiujWB9wUEwPcN3YHAVQEB8KqU69oL/PLt2693F/H155/uvuWX6ofgs/0+ijuhxrubuLtZRn0fPVht9iNz8fX7d8/VCJw15QIGtZzchrMEBMDP5QXA+ycyIyCtFI4y6hMA759DdyTwTEAANB/HCAiAAmDVsGcEJAGw9m17dFa8AYzKWVctIABWi9tvmoAAKABWDZ8AGJNeKeTGKvjyRQCMyllXLSAAVovbb5qAACgAVg2fABiTFgBjblYRiAgIgBE1a1oKCIACYNXgCoAxaQEw5mYVgYiAABhRs6algAAoAFYNrgAYkxYAY25WEYgICIARNWtaCgiAAmDV4AqAMWkBMOZmFYGIgAAYUbOmpYAAKABWDa4AGJMWAGNuVhGICAiAETVrWgpUB8Dow2yldR+N3j3MZAzzM7OV+ptxznc8V7KJ1uGngKNy1lULCIDV4vabJiAAegNYNXwZwWqlcJTxHwUfvVmpxuisCIBROeuqBQTAanH7TRMQAAXAquETAGPSAmDMzSoCEQEBMKJmTUuB6gAYRYo+BDP28yXgmGrGG7KV5iKjvnfeAMa69PyNY/Se3gBG5ayrFhAAq8XtN01AAPQGsGr4MgKSAOhPwVXNr33OEBAAz+izKj++v+jbt1/vhtj9Qe8NYGxidp+LjPq8AYzNmlUEogICYFTOunYCAqA3gFVDmxGQvAH0BrBqfu1zhoAAeEafVekN4MMZeBYsvAGMfXQEwJjbSiE3VsGXL74HMCpnXbWAAFgtbr9pAt4AegNYNXwCYExaAIy5WUUgIiAARtSsaSkgAAqAVYMrAMakBcCYm1UEIgICYETNmpYCAqAAWDW4AmBMWgCMuVlFICIgAEbUrGkpIAAKgFWDKwDGpAXAmJtVBCICAmBEzZqWAs8CYMYDuwvSCT8E0qUXz865Ujg6+fPy0aOnvfj+3XN1hw/cATUY1AOarMS/CQiA574B3OEzIACu00UBcJ1eOElcQACM21nZTEAAFACbjew/HVcAXKd7AuA6vXCSuIAAGLezspmAACgANhtZAXDRhgmAizbGsYYEBMAhLhd3FhAABcDW8/unvz48fsb35FW/cezUGwGwU7ec9ZGAAGg2jhEQAAXAzsNeHciq9+vUGwGwU7ecVQA0A8cLCIACYOcPQXUgq96vU28EwE7dclYB0AwcLyAACoCdPwTVgax6v069EQA7dctZBUAzcLyAACgAdv4QVAey6v069UYA7NQtZxUAzcDxAtEAGH0Q7rDuY2h2+AGDaC+efWii99x93Tv/0Gxh4xdBvzMC1hYK+CGQQmxbzRUQAM99AxgNFgLg5wIZ/1HwsVO0T0utEwDn/kNv98sCAuBlKhd2FxAABcDPBKJhZqnQ8fNPDz+e1ed859+J6rOm7CcAvjMC1hYKCICF2LaaKxANgHNPnb/7s4fgLl8CzlfM3yEaVqInq94ves4Z63wP4Ax1e94tIADeLep+ywoIgOe+AVx2KAcOVh3IqvcboJh+qQA4vQUOcIOAAHgDolv0EBAABcAekzrep+iXsZ95CICPdQTAzp8kZ/9dQAA0C8cICIDjweJjhXCxxkekOpBV77eG8rVTCIDXnFy1toAAuHZ/nO5GAQFQALxxnMpvVR3Iqmlp/EkAABKFSURBVPcrB31jQwHwDTxLlxEQAJdphYNkCwiAAmD2jGXevzqQVe+XaXf3vQXAu0Xdb4aAADhD3Z5TBARAAXDK4N20aXUgq97vJqaS2wiAJcw2SRYQAJOB3X4dAQFQAFxnGsdPUh3IqvcbF5m3QgCcZ2/n+wQEwPss3WlxAQFQAFx8RJ8erzqQVe/XqTcCYKduOesjAQHQbBwjUB0AX/2C5Qh89U/kfpwxY89I7e+syejFs/PsbpZVX0afMs4qAL7zabR2FQEBcJVOOEe6gAC4zhvA9Gb/YYOMYCEA3t/FjD4JgPf3yR33EBAA9+ijKi4ICIAC4IUxueWSjNBxy8EGbjLjS8AC4ECDXErgTQEB8E1Ay/sICIACYNW0CoAxaQEw5mYVgYiAABhRs6algAAoAFYNrgAYkxYAY25WEYgICIARNWtaCgiAAmDV4AqAMWkBMOZmFYGIgAAYUbOmpYAAKABWDa4AGJMWAGNuVhGICAiAETVrWgoIgAJg1eAKgDFpATDmZhWBiIAAGFGzpqWAACgAVg2uABiTFgBjblYRiAgIgBE1a1oKRANg9NdhVD/Mss65e5jJGOZnZhl9Wmm/dzx3+Mx8/f7dc/WdIbC2TMCgllHbaLaAAOgNYNUMrhTIqgPnO8YC4Dt61hIYExAAx7xc3VhAABQAq8ZXAIxJC4AxN6sIRAQEwIiaNS0FogEwWmz1wyzrnL4EPC67u1lWfTt8ZnwJePzzYsUcAQFwjrtdJwgIgN4AVo1dVkCqOv/HPtEvHb9zRgHwHT1rCYwJCIBjXq5uLCAACoBV4ysAxqQFwJibVQQiAgJgRM2algICoABYNbgCYExaAIy5WUUgIiAARtSsaSkgAAqAVYMrAMakBcCYm1UEIgICYETNmpYCAqAAWDW4AmBMWgCMuVlFICIgAEbUrGkpIAAKgFWDKwDGpAXAmJtVBCICAmBEzZqWAgKgAFg1uAJgTFoAjLlZRSAiIABG1KxpKSAACoBVgysAxqQFwJibVQQiAgJgRM2algICYCwAVjc7IzxlBItnLjvUUF3fx34ZfaruhV8EXf0vhv2iAgJgVM66dgICoABYNbTVoaOqrt/3yahPAKzuov1OFxAAT5+Ag+qPBsDoX0TY4W3GjPHICBcZvah+Q1ZdQ3V9AuCMT5s9TxYQAE/u/mG1C4DeAFaN/A4hVgB8LPD0Pwq/f/dcrfqg2ectAYP6Fp/FnQQEQAGwal4FwJh0xlvO6l74HsBY762qFxAA683tOElAABQAq0avOnRU1fX7Phn1+RJwdRftd7qAAHj6BBxUvwAoAFaNe0ZAyng7FvXIqE8AjHbDOgIxAQEw5mZVQwEBUACsGtuMgCQAxrpX3QtfAo71yap6AQGw3tyOkwQEQAGwavSqQ0dVXb4E/DcBPwRSPXH2yxAQADNU3XNJgd0D4Eq/ruadAXgWnrrUuEMNz3qYEXBfBavoTGWcVQCMdsO6lQQEwJW64SypAgJg/zeAAmDqR+TyzTNClQB4md+FBG4REABvYXSTDgICoABYNafeAMakM77PMSOsegMY669VawkIgGv1w2kSBQRAATBxvP7p1gJgTFoAjLlZRSAiIABG1KxpKbB7AIw2JeOhGz3Lx7rqNzbvnPXR2h1qeOaSUd/HfhmzmHFWbwAzPjXuWS0gAFaL22+aQDQARg+8w8MsWvs766of2O+cVQC8V2+Hz4xfA3PvTLhbnoAAmGfrzosJCICfNyTjoftO6wXA9fuU0SNvAN/51FhLYFxAABw3s6KpgAC4frD4OGFGuKgOuTvU8OxjnlGfANj0H1bHbisgALZtnYOPCgiAAuDozESvzwhI1SFWAHws4HsAo58M61YSEABX6oazpAoIgAJg6oD9cHMBMCadEXKre+F7AGO9t6peQACsN7fjJAEBUACsGr3q0FFV1+/7ZNTnS8DVXbTf6QIC4OkTcFD9AqAAWDXuGQEp4+1Y1COjPgEw2g3rCMQEBMCYm1UNBaoDYEMiRyZA4IKA7wG8gOSS5QUEwOVb5IB3CQiAd0m6D4GzBQTAs/u/S/UC4C6dVMdLAQHwJZELCBC4ICAAXkByyfICAuDyLXLAuwQEwLsk3YfA2QIC4Nn936V6AXCXTqrjpYAA+JLIBQQIXBAQAC8guWR5AQFw+RY54F0CAuBdku5D4GwBAfDs/u9SvQC4SyfV8VJAAHxJ5AICBC4ICIAXkFyyvIAAuHyLHPAuAQHwLkn3IXC2gAB4dv93qV4A3KWT6ngpIAC+JHIBAQIXBATAC0guWV5AAFy+RQ54l0A0AD79x/7nnx4eb4d1H8U9+6sPO9QY/asWXWq/6/Oz431Sevj9u+fqjsOyYU0GdcOmKulzAQHwgcuf/vp0ZATAcbeVzPx78FhAADQdJwsIgCd3/7DaBcDxIOMNYHF4SHijfNjHfKhcAXCIy8WbCQiAmzVUOU8e2N++/frof13pjU3KQykYLARAAXDnf1NSPmu+BLzzyGxVmwC4VTsV80wg+gaQKgECBH4U8EMg5mEHAQFwhy6q4ZKAAHiJyUUECLwQEACNyA4CAuAOXVTDJQEB8BKTiwgQEADNwAECAuABTVbi3wQEQJNAgMAdAt4A3qHoHrMFBMDZHbB/mYAAWEZtIwJbCwiAW7f3mOIEwGNarVAB0AwQIHCHgAB4h6J7zBYQAGd3wP5lAgJgGbWNCGwtIABu3d5jihMAj2m1QgVAM0CAwB0CAuAdiu4xW0AAnN0B+5cJCIBl1DYisLWAALh1e48pTgA8ptUKfRYA6RAgQOAOga/+EsgdjO5RICAAFiDbYg0BAXCNPjgFgZ0FBMCdu7tXbQLgXv1UzRMBAdB4ECCQLSAAZgu7/10CAuBdku7TQeDXDod0RgIEWgt4rrZu3zmHN6jn9FqlX74IgKaAAIFsAc/VbGH3v0XAoN7C6CZNBATAJo1yzP0E/s+//uvfi/qf//mf+xX4j4o8V3fu7ka1GdSNmqmUlwIC4EsiFxDIERAAc1zdlUBUQACMylnXUUAA7Ng1Z24t8Hvw+/Gt349h8KO4zd4Ieq62nthzDm9Qz+m1Sn0PoBkgsITAH0PhZyFxiYPGDuG5GnOzqljAoBaD226qgDeAU/ltTuBvAgKgSSAwX0AAnN8DJ6gTEADrrO1E4KHAozd+H///G3w52HPV7LcQMKgt2uSQNwkIgDdBug2BdwQEwHf0rCVwj4AAeI+ju/QQEAB79MkpNxb47C3fZj8h7Lm68fzuVJpB3ambanklIAC+EvK/E0gWEACTgd2ewEUBAfAilMu2EBAAt2ijIjoKXPl1MD/W1fh7AT1XOw7ogWc2qAc2/eCSBcCDm6/0uQKffZn3j78PUACc2yO7nyUgAJ7V79OrFQBPnwD1TxMQAKfR25jApwICoME4SUAAPKnbal1C4MpbPj8EskSrHOIwAQHwsIYfXq4AePgAKL9eQACsN7cjgSsCAuAVJdfsIiAA7tJJdbQRuBIA2xRz7aCeq9ecXDVZwKBOboDtSwUEwFJumxH4x599+8yi8U/6Pmut56rBbyFgUFu0ySFvEhAAb4J0GwKjApt9n58AODoArl9OQABcriUOlCggACbiujUBAr8JeK4ahBYCBrVFmxzyJgEB8CZItyFA4KGA56rhaCFgUFu0ySFvEhAAb4J0GwIEBEAz0FtAAOzdP6cfExAAx7xcTYDAuIDn6riZFRMEDOoEdFtOExAAp9HbmMAxAp6rx7S6d6EGtXf/nH5MQAAc83I1AQLjAp6r42ZWTBAwqBPQbTlNQACcRm9jAscIeK4e0+rehRrU3v1z+jEBAXDM69ir/+N//9vl2v/tf/3H36+NrvvjZp/d58d9Lh/OhTMEPFdnqNtzWMCgDpNZ0FhAAGzcvMqjR4NcdJ0AWNnd9L08V9OJbXCHgEG9Q9E9uggIgF061eCcfwx7H2/oPvv/Gykl454j+7v2FgHP1VsY3SRbwKBmC7v/SgIC4ErdaH6WjLCWcc/mzB2P77nasWsHntmgHtj0g0sWAA9u/t2l3xnWrrw5vHLN3TW6X0jAczXEZlG1gEGtFrffTAEBcKb+ZnsLgJs19L5yPFfvs3SnRAGDmojr1ssJCIDLtWTtA135oY47fgr4ytu9K9esrXnM6TxXj2l170INau/+Of2YgAA45nX81QLg8SMQAfBcjahZUy5gUMvJbThRQACciL/b1tE3ctEvHUf32829QT2eqw2a5IhfvhhUU3CSgAB4UreTa40GsmfrouEwuVS3HxPwXB3zcvUkAYM6Cd62UwQEwCnse24qAO7Z1xuq8ly9AdEt8gUMar6xHdYREADX6cXSJ7nyvX+/F3DHD4H8EcOfglt6PF4dznP1lZD/fQkBg7pEGxyiSEAALILuvo0A2L2DU8/vuTqV3+ZXBQzqVSnX7SAgAO7QRTUQWFvAc3Xt/jjdfwkYVKNwkoAAeFK31UpgjoDn6hx3uw4KGNRBMJe3FhAAW7fP4Qm0EPBcbdEmhzSoZuAkAQHwpG6rlcAcAc/VOe52HRQwqINgLm8tIAC2bp/DE2gh4Lnaok0OaVDNwEkCAuBJ3VYrgTkCnqtz3O06KGBQB8Fc3lpAAGzdPocn0ELAc7VFmxzSoJqBkwQEwJO6rVYCcwQ8V+e423VQwKAOgrm8tYAA2Lp9Dk+ghYDnaos2OaRBNQMnCQiAJ3VbrQTmCHiuznG366CAQR0Ec3lrAQGwdfscnkALAc/VFm1ySINqBk4SEABP6rZaCcwR8Fyd427XQQGDOgjm8tYCAmDr9jk8gRYCnqst2uSQBtUMnCQgAJ7UbbUSmCPguTrH3a6DAgZ1EMzlrQUEwNbtc3gCLQQ8V1u0ySENqhk4SUAAPKnbaiUwR8BzdY67XQcFDOogmMtbCwiArdvn8ARaCHiutmiTQxpUM3CSgAB4UrfVSmCOgOfqHHe7DgoY1EEwl7cWEABbt8/hCbQQ8Fxt0SaHNKhm4CQBAfCkbquVwBwBz9U57nYdFDCog2Auby0gALZun8MTaCHgudqiTQ5pUM3ASQIC4EndViuBOQKeq3Pc7TooYFAHwVzeWkAAbN0+hyfQQsBztUWbHNKgmoGTBATAk7qtVgJzBDxX57jbdVDAoA6Cuby1gADYun0OT6CFgOdqizY5pEE1AycJCIAndVutBOYIeK7OcbfroIBBHQRzeWsBAbB1+xyeQAsBz9UWbXJIg2oGThIQAE/qtloJzBHwXJ3jbtdBAYM6COZyAgQIECBAgEB3AQGwewednwABAgQIECAwKCAADoK5nAABAgQIECDQXUAA7N5B5ydAgAABAgQIDAoIgINgLidAgAABAgQIdBcQALt30PkJECBAgAABAoMCAuAgmMsJECBAgAABAt0FBMDuHXR+AgQIECBAgMCggAA4COZyAgQIECBAgEB3AQGwewednwABAgQIECAwKCAADoK5nAABAgQIECDQXUAA7N5B5ydAgAABAgQIDAoIgINgLidAgAABAgQIdBcQALt30PkJECBAgAABAoMCAuAgmMsJECBAgAABAt0FBMDuHXR+AgQIECBAgMCggAA4COZyAgQIECBAgEB3AQGwewednwABAgQIECAwKCAADoK5nAABAgQIECDQXUAA7N5B5ydAgAABAgQIDAoIgINgLidAgAABAgQIdBcQALt30PkJECBAgAABAoMCAuAgmMsJECBAgAABAt0FBMDuHXR+AgQIECBAgMCggAA4COZyAgQIECBAgEB3AQGwewednwABAgQIECAwKCAADoK5nAABAgQIECDQXUAA7N5B5ydAgAABAgQIDAoIgINgLidAgAABAgQIdBcQALt30PkJECBAgAABAoMCAuAgmMsJECBAgAABAt0FBMDuHXR+AgQIECBAgMCggAA4COZyAgQIECBAgEB3AQGwewednwABAgQIECAwKCAADoK5nAABAgQIECDQXUAA7N5B5ydAgAABAgQIDAoIgINgLidAgAABAgQIdBcQALt30PkJECBAgAABAoMCAuAgmMsJECBAgAABAt0FBMDuHXR+AgQIECBAgMCggAA4COZyAgQIECBAgEB3AQGwewednwABAgQIECAwKCAADoK5nAABAgQIECDQXUAA7N5B5ydAgAABAgQIDAoIgINgLidAgAABAgQIdBcQALt30PkJECBAgAABAoMCAuAgmMsJECBAgAABAt0FBMDuHXR+AgQIECBAgMCggAA4COZyAgQIECBAgEB3AQGwewednwABAgQIECAwKCAADoK5nAABAgQIECDQXUAA7N5B5ydAgAABAgQIDAoIgINgLidAgAABAgQIdBf4/yhUfFggyYeZAAAAAElFTkSuQmCC\" width=\"640\">"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "video = plot_animation(frames)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once you have finished playing with an environment, you should close it to free up resources:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "env.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To code our first learning agent, we will be using a simpler environment: the Cart-Pole. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# A simple environment: the Cart-Pole"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The Cart-Pole is a very simple environment composed of a cart that can move left or right, and pole placed vertically on top of it. The agent must move the cart left or right to keep the pole upright."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[2017-09-25 11:36:20,830] Making new env: CartPole-v0\n"
     ]
    }
   ],
   "source": [
    "env = gym.make(\"CartPole-v0\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "obs = env.reset()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 0.01109035,  0.01262894, -0.01916945,  0.04726117])"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "obs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The observation is a 1D NumPy array composed of 4 floats: they represent the cart's horizontal position, its velocity, the angle of the pole (0 = vertical), and the angular velocity. Let's render the environment... unfortunately we need to fix an annoying rendering issue first."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Fixing the rendering issue"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Some environments (including the Cart-Pole) require access to your display, which opens up a separate window, even if you specify the `rgb_array` mode. In general you can safely ignore that window. However, if Jupyter is running on a headless server (ie. without a screen) it will raise an exception. One way to avoid this is to install a fake X server like Xvfb. You can start Jupyter using the `xvfb-run` command:\n",
    "\n",
    "    $ xvfb-run -s \"-screen 0 1400x900x24\" jupyter notebook\n",
    "\n",
    "If Jupyter is running on a headless server but you don't want to worry about Xvfb, then you can just use the following rendering function for the Cart-Pole:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from PIL import Image, ImageDraw\n",
    "\n",
    "try:\n",
    "    from pyglet.gl import gl_info\n",
    "    openai_cart_pole_rendering = True   # no problem, let's use OpenAI gym's rendering function\n",
    "except Exception:\n",
    "    openai_cart_pole_rendering = False  # probably no X server available, let's use our own rendering function\n",
    "\n",
    "def render_cart_pole(env, obs):\n",
    "    if openai_cart_pole_rendering:\n",
    "        # use OpenAI gym's rendering function\n",
    "        return env.render(mode=\"rgb_array\")\n",
    "    else:\n",
    "        # rendering for the cart pole environment (in case OpenAI gym can't do it)\n",
    "        img_w = 600\n",
    "        img_h = 400\n",
    "        cart_w = img_w // 12\n",
    "        cart_h = img_h // 15\n",
    "        pole_len = img_h // 3.5\n",
    "        pole_w = img_w // 80 + 1\n",
    "        x_width = 2\n",
    "        max_ang = 0.2\n",
    "        bg_col = (255, 255, 255)\n",
    "        cart_col = 0x000000 # Blue Green Red\n",
    "        pole_col = 0x669acc # Blue Green Red\n",
    "\n",
    "        pos, vel, ang, ang_vel = obs\n",
    "        img = Image.new('RGB', (img_w, img_h), bg_col)\n",
    "        draw = ImageDraw.Draw(img)\n",
    "        cart_x = pos * img_w // x_width + img_w // x_width\n",
    "        cart_y = img_h * 95 // 100\n",
    "        top_pole_x = cart_x + pole_len * np.sin(ang)\n",
    "        top_pole_y = cart_y - cart_h // 2 - pole_len * np.cos(ang)\n",
    "        draw.line((0, cart_y, img_w, cart_y), fill=0)\n",
    "        draw.rectangle((cart_x - cart_w // 2, cart_y - cart_h // 2, cart_x + cart_w // 2, cart_y + cart_h // 2), fill=cart_col) # draw cart\n",
    "        draw.line((cart_x, cart_y - cart_h // 2, top_pole_x, top_pole_y), fill=pole_col, width=pole_w) # draw pole\n",
    "        return np.array(img)\n",
    "\n",
    "def plot_cart_pole(env, obs):\n",
    "    plt.close()  # or else nbagg sometimes plots in the previous cell\n",
    "    img = render_cart_pole(env, obs)\n",
    "    plt.imshow(img)\n",
    "    plt.axis(\"off\")\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/javascript": [
       "/* Put everything inside the global mpl namespace */\n",
       "window.mpl = {};\n",
       "\n",
       "\n",
       "mpl.get_websocket_type = function() {\n",
       "    if (typeof(WebSocket) !== 'undefined') {\n",
       "        return WebSocket;\n",
       "    } else if (typeof(MozWebSocket) !== 'undefined') {\n",
       "        return MozWebSocket;\n",
       "    } else {\n",
       "        alert('Your browser does not have WebSocket support.' +\n",
       "              'Please try Chrome, Safari or Firefox ≥ 6. ' +\n",
       "              'Firefox 4 and 5 are also supported but you ' +\n",
       "              'have to enable WebSockets in about:config.');\n",
       "    };\n",
       "}\n",
       "\n",
       "mpl.figure = function(figure_id, websocket, ondownload, parent_element) {\n",
       "    this.id = figure_id;\n",
       "\n",
       "    this.ws = websocket;\n",
       "\n",
       "    this.supports_binary = (this.ws.binaryType != undefined);\n",
       "\n",
       "    if (!this.supports_binary) {\n",
       "        var warnings = document.getElementById(\"mpl-warnings\");\n",
       "        if (warnings) {\n",
       "            warnings.style.display = 'block';\n",
       "            warnings.textContent = (\n",
       "                \"This browser does not support binary websocket messages. \" +\n",
       "                    \"Performance may be slow.\");\n",
       "        }\n",
       "    }\n",
       "\n",
       "    this.imageObj = new Image();\n",
       "\n",
       "    this.context = undefined;\n",
       "    this.message = undefined;\n",
       "    this.canvas = undefined;\n",
       "    this.rubberband_canvas = undefined;\n",
       "    this.rubberband_context = undefined;\n",
       "    this.format_dropdown = undefined;\n",
       "\n",
       "    this.image_mode = 'full';\n",
       "\n",
       "    this.root = $('<div/>');\n",
       "    this._root_extra_style(this.root)\n",
       "    this.root.attr('style', 'display: inline-block');\n",
       "\n",
       "    $(parent_element).append(this.root);\n",
       "\n",
       "    this._init_header(this);\n",
       "    this._init_canvas(this);\n",
       "    this._init_toolbar(this);\n",
       "\n",
       "    var fig = this;\n",
       "\n",
       "    this.waiting = false;\n",
       "\n",
       "    this.ws.onopen =  function () {\n",
       "            fig.send_message(\"supports_binary\", {value: fig.supports_binary});\n",
       "            fig.send_message(\"send_image_mode\", {});\n",
       "            if (mpl.ratio != 1) {\n",
       "                fig.send_message(\"set_dpi_ratio\", {'dpi_ratio': mpl.ratio});\n",
       "            }\n",
       "            fig.send_message(\"refresh\", {});\n",
       "        }\n",
       "\n",
       "    this.imageObj.onload = function() {\n",
       "            if (fig.image_mode == 'full') {\n",
       "                // Full images could contain transparency (where diff images\n",
       "                // almost always do), so we need to clear the canvas so that\n",
       "                // there is no ghosting.\n",
       "                fig.context.clearRect(0, 0, fig.canvas.width, fig.canvas.height);\n",
       "            }\n",
       "            fig.context.drawImage(fig.imageObj, 0, 0);\n",
       "        };\n",
       "\n",
       "    this.imageObj.onunload = function() {\n",
       "        this.ws.close();\n",
       "    }\n",
       "\n",
       "    this.ws.onmessage = this._make_on_message_function(this);\n",
       "\n",
       "    this.ondownload = ondownload;\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._init_header = function() {\n",
       "    var titlebar = $(\n",
       "        '<div class=\"ui-dialog-titlebar ui-widget-header ui-corner-all ' +\n",
       "        'ui-helper-clearfix\"/>');\n",
       "    var titletext = $(\n",
       "        '<div class=\"ui-dialog-title\" style=\"width: 100%; ' +\n",
       "        'text-align: center; padding: 3px;\"/>');\n",
       "    titlebar.append(titletext)\n",
       "    this.root.append(titlebar);\n",
       "    this.header = titletext[0];\n",
       "}\n",
       "\n",
       "\n",
       "\n",
       "mpl.figure.prototype._canvas_extra_style = function(canvas_div) {\n",
       "\n",
       "}\n",
       "\n",
       "\n",
       "mpl.figure.prototype._root_extra_style = function(canvas_div) {\n",
       "\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._init_canvas = function() {\n",
       "    var fig = this;\n",
       "\n",
       "    var canvas_div = $('<div/>');\n",
       "\n",
       "    canvas_div.attr('style', 'position: relative; clear: both; outline: 0');\n",
       "\n",
       "    function canvas_keyboard_event(event) {\n",
       "        return fig.key_event(event, event['data']);\n",
       "    }\n",
       "\n",
       "    canvas_div.keydown('key_press', canvas_keyboard_event);\n",
       "    canvas_div.keyup('key_release', canvas_keyboard_event);\n",
       "    this.canvas_div = canvas_div\n",
       "    this._canvas_extra_style(canvas_div)\n",
       "    this.root.append(canvas_div);\n",
       "\n",
       "    var canvas = $('<canvas/>');\n",
       "    canvas.addClass('mpl-canvas');\n",
       "    canvas.attr('style', \"left: 0; top: 0; z-index: 0; outline: 0\")\n",
       "\n",
       "    this.canvas = canvas[0];\n",
       "    this.context = canvas[0].getContext(\"2d\");\n",
       "\n",
       "    var backingStore = this.context.backingStorePixelRatio ||\n",
       "\tthis.context.webkitBackingStorePixelRatio ||\n",
       "\tthis.context.mozBackingStorePixelRatio ||\n",
       "\tthis.context.msBackingStorePixelRatio ||\n",
       "\tthis.context.oBackingStorePixelRatio ||\n",
       "\tthis.context.backingStorePixelRatio || 1;\n",
       "\n",
       "    mpl.ratio = (window.devicePixelRatio || 1) / backingStore;\n",
       "\n",
       "    var rubberband = $('<canvas/>');\n",
       "    rubberband.attr('style', \"position: absolute; left: 0; top: 0; z-index: 1;\")\n",
       "\n",
       "    var pass_mouse_events = true;\n",
       "\n",
       "    canvas_div.resizable({\n",
       "        start: function(event, ui) {\n",
       "            pass_mouse_events = false;\n",
       "        },\n",
       "        resize: function(event, ui) {\n",
       "            fig.request_resize(ui.size.width, ui.size.height);\n",
       "        },\n",
       "        stop: function(event, ui) {\n",
       "            pass_mouse_events = true;\n",
       "            fig.request_resize(ui.size.width, ui.size.height);\n",
       "        },\n",
       "    });\n",
       "\n",
       "    function mouse_event_fn(event) {\n",
       "        if (pass_mouse_events)\n",
       "            return fig.mouse_event(event, event['data']);\n",
       "    }\n",
       "\n",
       "    rubberband.mousedown('button_press', mouse_event_fn);\n",
       "    rubberband.mouseup('button_release', mouse_event_fn);\n",
       "    // Throttle sequential mouse events to 1 every 20ms.\n",
       "    rubberband.mousemove('motion_notify', mouse_event_fn);\n",
       "\n",
       "    rubberband.mouseenter('figure_enter', mouse_event_fn);\n",
       "    rubberband.mouseleave('figure_leave', mouse_event_fn);\n",
       "\n",
       "    canvas_div.on(\"wheel\", function (event) {\n",
       "        event = event.originalEvent;\n",
       "        event['data'] = 'scroll'\n",
       "        if (event.deltaY < 0) {\n",
       "            event.step = 1;\n",
       "        } else {\n",
       "            event.step = -1;\n",
       "        }\n",
       "        mouse_event_fn(event);\n",
       "    });\n",
       "\n",
       "    canvas_div.append(canvas);\n",
       "    canvas_div.append(rubberband);\n",
       "\n",
       "    this.rubberband = rubberband;\n",
       "    this.rubberband_canvas = rubberband[0];\n",
       "    this.rubberband_context = rubberband[0].getContext(\"2d\");\n",
       "    this.rubberband_context.strokeStyle = \"#000000\";\n",
       "\n",
       "    this._resize_canvas = function(width, height) {\n",
       "        // Keep the size of the canvas, canvas container, and rubber band\n",
       "        // canvas in synch.\n",
       "        canvas_div.css('width', width)\n",
       "        canvas_div.css('height', height)\n",
       "\n",
       "        canvas.attr('width', width * mpl.ratio);\n",
       "        canvas.attr('height', height * mpl.ratio);\n",
       "        canvas.attr('style', 'width: ' + width + 'px; height: ' + height + 'px;');\n",
       "\n",
       "        rubberband.attr('width', width);\n",
       "        rubberband.attr('height', height);\n",
       "    }\n",
       "\n",
       "    // Set the figure to an initial 600x600px, this will subsequently be updated\n",
       "    // upon first draw.\n",
       "    this._resize_canvas(600, 600);\n",
       "\n",
       "    // Disable right mouse context menu.\n",
       "    $(this.rubberband_canvas).bind(\"contextmenu\",function(e){\n",
       "        return false;\n",
       "    });\n",
       "\n",
       "    function set_focus () {\n",
       "        canvas.focus();\n",
       "        canvas_div.focus();\n",
       "    }\n",
       "\n",
       "    window.setTimeout(set_focus, 100);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._init_toolbar = function() {\n",
       "    var fig = this;\n",
       "\n",
       "    var nav_element = $('<div/>')\n",
       "    nav_element.attr('style', 'width: 100%');\n",
       "    this.root.append(nav_element);\n",
       "\n",
       "    // Define a callback function for later on.\n",
       "    function toolbar_event(event) {\n",
       "        return fig.toolbar_button_onclick(event['data']);\n",
       "    }\n",
       "    function toolbar_mouse_event(event) {\n",
       "        return fig.toolbar_button_onmouseover(event['data']);\n",
       "    }\n",
       "\n",
       "    for(var toolbar_ind in mpl.toolbar_items) {\n",
       "        var name = mpl.toolbar_items[toolbar_ind][0];\n",
       "        var tooltip = mpl.toolbar_items[toolbar_ind][1];\n",
       "        var image = mpl.toolbar_items[toolbar_ind][2];\n",
       "        var method_name = mpl.toolbar_items[toolbar_ind][3];\n",
       "\n",
       "        if (!name) {\n",
       "            // put a spacer in here.\n",
       "            continue;\n",
       "        }\n",
       "        var button = $('<button/>');\n",
       "        button.addClass('ui-button ui-widget ui-state-default ui-corner-all ' +\n",
       "                        'ui-button-icon-only');\n",
       "        button.attr('role', 'button');\n",
       "        button.attr('aria-disabled', 'false');\n",
       "        button.click(method_name, toolbar_event);\n",
       "        button.mouseover(tooltip, toolbar_mouse_event);\n",
       "\n",
       "        var icon_img = $('<span/>');\n",
       "        icon_img.addClass('ui-button-icon-primary ui-icon');\n",
       "        icon_img.addClass(image);\n",
       "        icon_img.addClass('ui-corner-all');\n",
       "\n",
       "        var tooltip_span = $('<span/>');\n",
       "        tooltip_span.addClass('ui-button-text');\n",
       "        tooltip_span.html(tooltip);\n",
       "\n",
       "        button.append(icon_img);\n",
       "        button.append(tooltip_span);\n",
       "\n",
       "        nav_element.append(button);\n",
       "    }\n",
       "\n",
       "    var fmt_picker_span = $('<span/>');\n",
       "\n",
       "    var fmt_picker = $('<select/>');\n",
       "    fmt_picker.addClass('mpl-toolbar-option ui-widget ui-widget-content');\n",
       "    fmt_picker_span.append(fmt_picker);\n",
       "    nav_element.append(fmt_picker_span);\n",
       "    this.format_dropdown = fmt_picker[0];\n",
       "\n",
       "    for (var ind in mpl.extensions) {\n",
       "        var fmt = mpl.extensions[ind];\n",
       "        var option = $(\n",
       "            '<option/>', {selected: fmt === mpl.default_extension}).html(fmt);\n",
       "        fmt_picker.append(option)\n",
       "    }\n",
       "\n",
       "    // Add hover states to the ui-buttons\n",
       "    $( \".ui-button\" ).hover(\n",
       "        function() { $(this).addClass(\"ui-state-hover\");},\n",
       "        function() { $(this).removeClass(\"ui-state-hover\");}\n",
       "    );\n",
       "\n",
       "    var status_bar = $('<span class=\"mpl-message\"/>');\n",
       "    nav_element.append(status_bar);\n",
       "    this.message = status_bar[0];\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.request_resize = function(x_pixels, y_pixels) {\n",
       "    // Request matplotlib to resize the figure. Matplotlib will then trigger a resize in the client,\n",
       "    // which will in turn request a refresh of the image.\n",
       "    this.send_message('resize', {'width': x_pixels, 'height': y_pixels});\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.send_message = function(type, properties) {\n",
       "    properties['type'] = type;\n",
       "    properties['figure_id'] = this.id;\n",
       "    this.ws.send(JSON.stringify(properties));\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.send_draw_message = function() {\n",
       "    if (!this.waiting) {\n",
       "        this.waiting = true;\n",
       "        this.ws.send(JSON.stringify({type: \"draw\", figure_id: this.id}));\n",
       "    }\n",
       "}\n",
       "\n",
       "\n",
       "mpl.figure.prototype.handle_save = function(fig, msg) {\n",
       "    var format_dropdown = fig.format_dropdown;\n",
       "    var format = format_dropdown.options[format_dropdown.selectedIndex].value;\n",
       "    fig.ondownload(fig, format);\n",
       "}\n",
       "\n",
       "\n",
       "mpl.figure.prototype.handle_resize = function(fig, msg) {\n",
       "    var size = msg['size'];\n",
       "    if (size[0] != fig.canvas.width || size[1] != fig.canvas.height) {\n",
       "        fig._resize_canvas(size[0], size[1]);\n",
       "        fig.send_message(\"refresh\", {});\n",
       "    };\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_rubberband = function(fig, msg) {\n",
       "    var x0 = msg['x0'] / mpl.ratio;\n",
       "    var y0 = (fig.canvas.height - msg['y0']) / mpl.ratio;\n",
       "    var x1 = msg['x1'] / mpl.ratio;\n",
       "    var y1 = (fig.canvas.height - msg['y1']) / mpl.ratio;\n",
       "    x0 = Math.floor(x0) + 0.5;\n",
       "    y0 = Math.floor(y0) + 0.5;\n",
       "    x1 = Math.floor(x1) + 0.5;\n",
       "    y1 = Math.floor(y1) + 0.5;\n",
       "    var min_x = Math.min(x0, x1);\n",
       "    var min_y = Math.min(y0, y1);\n",
       "    var width = Math.abs(x1 - x0);\n",
       "    var height = Math.abs(y1 - y0);\n",
       "\n",
       "    fig.rubberband_context.clearRect(\n",
       "        0, 0, fig.canvas.width, fig.canvas.height);\n",
       "\n",
       "    fig.rubberband_context.strokeRect(min_x, min_y, width, height);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_figure_label = function(fig, msg) {\n",
       "    // Updates the figure title.\n",
       "    fig.header.textContent = msg['label'];\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_cursor = function(fig, msg) {\n",
       "    var cursor = msg['cursor'];\n",
       "    switch(cursor)\n",
       "    {\n",
       "    case 0:\n",
       "        cursor = 'pointer';\n",
       "        break;\n",
       "    case 1:\n",
       "        cursor = 'default';\n",
       "        break;\n",
       "    case 2:\n",
       "        cursor = 'crosshair';\n",
       "        break;\n",
       "    case 3:\n",
       "        cursor = 'move';\n",
       "        break;\n",
       "    }\n",
       "    fig.rubberband_canvas.style.cursor = cursor;\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_message = function(fig, msg) {\n",
       "    fig.message.textContent = msg['message'];\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_draw = function(fig, msg) {\n",
       "    // Request the server to send over a new figure.\n",
       "    fig.send_draw_message();\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_image_mode = function(fig, msg) {\n",
       "    fig.image_mode = msg['mode'];\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.updated_canvas_event = function() {\n",
       "    // Called whenever the canvas gets updated.\n",
       "    this.send_message(\"ack\", {});\n",
       "}\n",
       "\n",
       "// A function to construct a web socket function for onmessage handling.\n",
       "// Called in the figure constructor.\n",
       "mpl.figure.prototype._make_on_message_function = function(fig) {\n",
       "    return function socket_on_message(evt) {\n",
       "        if (evt.data instanceof Blob) {\n",
       "            /* FIXME: We get \"Resource interpreted as Image but\n",
       "             * transferred with MIME type text/plain:\" errors on\n",
       "             * Chrome.  But how to set the MIME type?  It doesn't seem\n",
       "             * to be part of the websocket stream */\n",
       "            evt.data.type = \"image/png\";\n",
       "\n",
       "            /* Free the memory for the previous frames */\n",
       "            if (fig.imageObj.src) {\n",
       "                (window.URL || window.webkitURL).revokeObjectURL(\n",
       "                    fig.imageObj.src);\n",
       "            }\n",
       "\n",
       "            fig.imageObj.src = (window.URL || window.webkitURL).createObjectURL(\n",
       "                evt.data);\n",
       "            fig.updated_canvas_event();\n",
       "            fig.waiting = false;\n",
       "            return;\n",
       "        }\n",
       "        else if (typeof evt.data === 'string' && evt.data.slice(0, 21) == \"data:image/png;base64\") {\n",
       "            fig.imageObj.src = evt.data;\n",
       "            fig.updated_canvas_event();\n",
       "            fig.waiting = false;\n",
       "            return;\n",
       "        }\n",
       "\n",
       "        var msg = JSON.parse(evt.data);\n",
       "        var msg_type = msg['type'];\n",
       "\n",
       "        // Call the  \"handle_{type}\" callback, which takes\n",
       "        // the figure and JSON message as its only arguments.\n",
       "        try {\n",
       "            var callback = fig[\"handle_\" + msg_type];\n",
       "        } catch (e) {\n",
       "            console.log(\"No handler for the '\" + msg_type + \"' message type: \", msg);\n",
       "            return;\n",
       "        }\n",
       "\n",
       "        if (callback) {\n",
       "            try {\n",
       "                // console.log(\"Handling '\" + msg_type + \"' message: \", msg);\n",
       "                callback(fig, msg);\n",
       "            } catch (e) {\n",
       "                console.log(\"Exception inside the 'handler_\" + msg_type + \"' callback:\", e, e.stack, msg);\n",
       "            }\n",
       "        }\n",
       "    };\n",
       "}\n",
       "\n",
       "// from http://stackoverflow.com/questions/1114465/getting-mouse-location-in-canvas\n",
       "mpl.findpos = function(e) {\n",
       "    //this section is from http://www.quirksmode.org/js/events_properties.html\n",
       "    var targ;\n",
       "    if (!e)\n",
       "        e = window.event;\n",
       "    if (e.target)\n",
       "        targ = e.target;\n",
       "    else if (e.srcElement)\n",
       "        targ = e.srcElement;\n",
       "    if (targ.nodeType == 3) // defeat Safari bug\n",
       "        targ = targ.parentNode;\n",
       "\n",
       "    // jQuery normalizes the pageX and pageY\n",
       "    // pageX,Y are the mouse positions relative to the document\n",
       "    // offset() returns the position of the element relative to the document\n",
       "    var x = e.pageX - $(targ).offset().left;\n",
       "    var y = e.pageY - $(targ).offset().top;\n",
       "\n",
       "    return {\"x\": x, \"y\": y};\n",
       "};\n",
       "\n",
       "/*\n",
       " * return a copy of an object with only non-object keys\n",
       " * we need this to avoid circular references\n",
       " * http://stackoverflow.com/a/24161582/3208463\n",
       " */\n",
       "function simpleKeys (original) {\n",
       "  return Object.keys(original).reduce(function (obj, key) {\n",
       "    if (typeof original[key] !== 'object')\n",
       "        obj[key] = original[key]\n",
       "    return obj;\n",
       "  }, {});\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.mouse_event = function(event, name) {\n",
       "    var canvas_pos = mpl.findpos(event)\n",
       "\n",
       "    if (name === 'button_press')\n",
       "    {\n",
       "        this.canvas.focus();\n",
       "        this.canvas_div.focus();\n",
       "    }\n",
       "\n",
       "    var x = canvas_pos.x * mpl.ratio;\n",
       "    var y = canvas_pos.y * mpl.ratio;\n",
       "\n",
       "    this.send_message(name, {x: x, y: y, button: event.button,\n",
       "                             step: event.step,\n",
       "                             guiEvent: simpleKeys(event)});\n",
       "\n",
       "    /* This prevents the web browser from automatically changing to\n",
       "     * the text insertion cursor when the button is pressed.  We want\n",
       "     * to control all of the cursor setting manually through the\n",
       "     * 'cursor' event from matplotlib */\n",
       "    event.preventDefault();\n",
       "    return false;\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._key_event_extra = function(event, name) {\n",
       "    // Handle any extra behaviour associated with a key event\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.key_event = function(event, name) {\n",
       "\n",
       "    // Prevent repeat events\n",
       "    if (name == 'key_press')\n",
       "    {\n",
       "        if (event.which === this._key)\n",
       "            return;\n",
       "        else\n",
       "            this._key = event.which;\n",
       "    }\n",
       "    if (name == 'key_release')\n",
       "        this._key = null;\n",
       "\n",
       "    var value = '';\n",
       "    if (event.ctrlKey && event.which != 17)\n",
       "        value += \"ctrl+\";\n",
       "    if (event.altKey && event.which != 18)\n",
       "        value += \"alt+\";\n",
       "    if (event.shiftKey && event.which != 16)\n",
       "        value += \"shift+\";\n",
       "\n",
       "    value += 'k';\n",
       "    value += event.which.toString();\n",
       "\n",
       "    this._key_event_extra(event, name);\n",
       "\n",
       "    this.send_message(name, {key: value,\n",
       "                             guiEvent: simpleKeys(event)});\n",
       "    return false;\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.toolbar_button_onclick = function(name) {\n",
       "    if (name == 'download') {\n",
       "        this.handle_save(this, null);\n",
       "    } else {\n",
       "        this.send_message(\"toolbar_button\", {name: name});\n",
       "    }\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.toolbar_button_onmouseover = function(tooltip) {\n",
       "    this.message.textContent = tooltip;\n",
       "};\n",
       "mpl.toolbar_items = [[\"Home\", \"Reset original view\", \"fa fa-home icon-home\", \"home\"], [\"Back\", \"Back to  previous view\", \"fa fa-arrow-left icon-arrow-left\", \"back\"], [\"Forward\", \"Forward to next view\", \"fa fa-arrow-right icon-arrow-right\", \"forward\"], [\"\", \"\", \"\", \"\"], [\"Pan\", \"Pan axes with left mouse, zoom with right\", \"fa fa-arrows icon-move\", \"pan\"], [\"Zoom\", \"Zoom to rectangle\", \"fa fa-square-o icon-check-empty\", \"zoom\"], [\"\", \"\", \"\", \"\"], [\"Download\", \"Download plot\", \"fa fa-floppy-o icon-save\", \"download\"]];\n",
       "\n",
       "mpl.extensions = [\"eps\", \"jpeg\", \"pdf\", \"png\", \"ps\", \"raw\", \"svg\", \"tif\"];\n",
       "\n",
       "mpl.default_extension = \"png\";var comm_websocket_adapter = function(comm) {\n",
       "    // Create a \"websocket\"-like object which calls the given IPython comm\n",
       "    // object with the appropriate methods. Currently this is a non binary\n",
       "    // socket, so there is still some room for performance tuning.\n",
       "    var ws = {};\n",
       "\n",
       "    ws.close = function() {\n",
       "        comm.close()\n",
       "    };\n",
       "    ws.send = function(m) {\n",
       "        //console.log('sending', m);\n",
       "        comm.send(m);\n",
       "    };\n",
       "    // Register the callback with on_msg.\n",
       "    comm.on_msg(function(msg) {\n",
       "        //console.log('receiving', msg['content']['data'], msg);\n",
       "        // Pass the mpl event to the overriden (by mpl) onmessage function.\n",
       "        ws.onmessage(msg['content']['data'])\n",
       "    });\n",
       "    return ws;\n",
       "}\n",
       "\n",
       "mpl.mpl_figure_comm = function(comm, msg) {\n",
       "    // This is the function which gets called when the mpl process\n",
       "    // starts-up an IPython Comm through the \"matplotlib\" channel.\n",
       "\n",
       "    var id = msg.content.data.id;\n",
       "    // Get hold of the div created by the display call when the Comm\n",
       "    // socket was opened in Python.\n",
       "    var element = $(\"#\" + id);\n",
       "    var ws_proxy = comm_websocket_adapter(comm)\n",
       "\n",
       "    function ondownload(figure, format) {\n",
       "        window.open(figure.imageObj.src);\n",
       "    }\n",
       "\n",
       "    var fig = new mpl.figure(id, ws_proxy,\n",
       "                           ondownload,\n",
       "                           element.get(0));\n",
       "\n",
       "    // Call onopen now - mpl needs it, as it is assuming we've passed it a real\n",
       "    // web socket which is closed, not our websocket->open comm proxy.\n",
       "    ws_proxy.onopen();\n",
       "\n",
       "    fig.parent_element = element.get(0);\n",
       "    fig.cell_info = mpl.find_output_cell(\"<div id='\" + id + \"'></div>\");\n",
       "    if (!fig.cell_info) {\n",
       "        console.error(\"Failed to find cell for figure\", id, fig);\n",
       "        return;\n",
       "    }\n",
       "\n",
       "    var output_index = fig.cell_info[2]\n",
       "    var cell = fig.cell_info[0];\n",
       "\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_close = function(fig, msg) {\n",
       "    var width = fig.canvas.width/mpl.ratio\n",
       "    fig.root.unbind('remove')\n",
       "\n",
       "    // Update the output cell to use the data from the current canvas.\n",
       "    fig.push_to_output();\n",
       "    var dataURL = fig.canvas.toDataURL();\n",
       "    // Re-enable the keyboard manager in IPython - without this line, in FF,\n",
       "    // the notebook keyboard shortcuts fail.\n",
       "    IPython.keyboard_manager.enable()\n",
       "    $(fig.parent_element).html('<img src=\"' + dataURL + '\" width=\"' + width + '\">');\n",
       "    fig.close_ws(fig, msg);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.close_ws = function(fig, msg){\n",
       "    fig.send_message('closing', msg);\n",
       "    // fig.ws.close()\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.push_to_output = function(remove_interactive) {\n",
       "    // Turn the data on the canvas into data in the output cell.\n",
       "    var width = this.canvas.width/mpl.ratio\n",
       "    var dataURL = this.canvas.toDataURL();\n",
       "    this.cell_info[1]['text/html'] = '<img src=\"' + dataURL + '\" width=\"' + width + '\">';\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.updated_canvas_event = function() {\n",
       "    // Tell IPython that the notebook contents must change.\n",
       "    IPython.notebook.set_dirty(true);\n",
       "    this.send_message(\"ack\", {});\n",
       "    var fig = this;\n",
       "    // Wait a second, then push the new image to the DOM so\n",
       "    // that it is saved nicely (might be nice to debounce this).\n",
       "    setTimeout(function () { fig.push_to_output() }, 1000);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._init_toolbar = function() {\n",
       "    var fig = this;\n",
       "\n",
       "    var nav_element = $('<div/>')\n",
       "    nav_element.attr('style', 'width: 100%');\n",
       "    this.root.append(nav_element);\n",
       "\n",
       "    // Define a callback function for later on.\n",
       "    function toolbar_event(event) {\n",
       "        return fig.toolbar_button_onclick(event['data']);\n",
       "    }\n",
       "    function toolbar_mouse_event(event) {\n",
       "        return fig.toolbar_button_onmouseover(event['data']);\n",
       "    }\n",
       "\n",
       "    for(var toolbar_ind in mpl.toolbar_items){\n",
       "        var name = mpl.toolbar_items[toolbar_ind][0];\n",
       "        var tooltip = mpl.toolbar_items[toolbar_ind][1];\n",
       "        var image = mpl.toolbar_items[toolbar_ind][2];\n",
       "        var method_name = mpl.toolbar_items[toolbar_ind][3];\n",
       "\n",
       "        if (!name) { continue; };\n",
       "\n",
       "        var button = $('<button class=\"btn btn-default\" href=\"#\" title=\"' + name + '\"><i class=\"fa ' + image + ' fa-lg\"></i></button>');\n",
       "        button.click(method_name, toolbar_event);\n",
       "        button.mouseover(tooltip, toolbar_mouse_event);\n",
       "        nav_element.append(button);\n",
       "    }\n",
       "\n",
       "    // Add the status bar.\n",
       "    var status_bar = $('<span class=\"mpl-message\" style=\"text-align:right; float: right;\"/>');\n",
       "    nav_element.append(status_bar);\n",
       "    this.message = status_bar[0];\n",
       "\n",
       "    // Add the close button to the window.\n",
       "    var buttongrp = $('<div class=\"btn-group inline pull-right\"></div>');\n",
       "    var button = $('<button class=\"btn btn-mini btn-primary\" href=\"#\" title=\"Stop Interaction\"><i class=\"fa fa-power-off icon-remove icon-large\"></i></button>');\n",
       "    button.click(function (evt) { fig.handle_close(fig, {}); } );\n",
       "    button.mouseover('Stop Interaction', toolbar_mouse_event);\n",
       "    buttongrp.append(button);\n",
       "    var titlebar = this.root.find($('.ui-dialog-titlebar'));\n",
       "    titlebar.prepend(buttongrp);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._root_extra_style = function(el){\n",
       "    var fig = this\n",
       "    el.on(\"remove\", function(){\n",
       "\tfig.close_ws(fig, {});\n",
       "    });\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._canvas_extra_style = function(el){\n",
       "    // this is important to make the div 'focusable\n",
       "    el.attr('tabindex', 0)\n",
       "    // reach out to IPython and tell the keyboard manager to turn it's self\n",
       "    // off when our div gets focus\n",
       "\n",
       "    // location in version 3\n",
       "    if (IPython.notebook.keyboard_manager) {\n",
       "        IPython.notebook.keyboard_manager.register_events(el);\n",
       "    }\n",
       "    else {\n",
       "        // location in version 2\n",
       "        IPython.keyboard_manager.register_events(el);\n",
       "    }\n",
       "\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._key_event_extra = function(event, name) {\n",
       "    var manager = IPython.notebook.keyboard_manager;\n",
       "    if (!manager)\n",
       "        manager = IPython.keyboard_manager;\n",
       "\n",
       "    // Check for shift+enter\n",
       "    if (event.shiftKey && event.which == 13) {\n",
       "        this.canvas_div.blur();\n",
       "        // select the cell after this one\n",
       "        var index = IPython.notebook.find_cell_index(this.cell_info[0]);\n",
       "        IPython.notebook.select(index + 1);\n",
       "    }\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_save = function(fig, msg) {\n",
       "    fig.ondownload(fig, null);\n",
       "}\n",
       "\n",
       "\n",
       "mpl.find_output_cell = function(html_output) {\n",
       "    // Return the cell and output element which can be found *uniquely* in the notebook.\n",
       "    // Note - this is a bit hacky, but it is done because the \"notebook_saving.Notebook\"\n",
       "    // IPython event is triggered only after the cells have been serialised, which for\n",
       "    // our purposes (turning an active figure into a static one), is too late.\n",
       "    var cells = IPython.notebook.get_cells();\n",
       "    var ncells = cells.length;\n",
       "    for (var i=0; i<ncells; i++) {\n",
       "        var cell = cells[i];\n",
       "        if (cell.cell_type === 'code'){\n",
       "            for (var j=0; j<cell.output_area.outputs.length; j++) {\n",
       "                var data = cell.output_area.outputs[j];\n",
       "                if (data.data) {\n",
       "                    // IPython >= 3 moved mimebundle to data attribute of output\n",
       "                    data = data.data;\n",
       "                }\n",
       "                if (data['text/html'] == html_output) {\n",
       "                    return [cell, data, j];\n",
       "                }\n",
       "            }\n",
       "        }\n",
       "    }\n",
       "}\n",
       "\n",
       "// Register the function which deals with the matplotlib target/channel.\n",
       "// The kernel may be null if the page has been refreshed.\n",
       "if (IPython.notebook.kernel != null) {\n",
       "    IPython.notebook.kernel.comm_manager.register_target('matplotlib', mpl.mpl_figure_comm);\n",
       "}\n"
      ],
      "text/plain": [
       "<IPython.core.display.Javascript object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<img src=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAoAAAAHgCAYAAAA10dzkAAAgAElEQVR4Xu3d260lSREFUK4buAFu4AZjBsIMcAM3wA3caDRCIF6F4ioyT8VjzXdlnYi182OrNN399Qv/ESBAgAABAgQIrBL4WrWtZQkQIECAAAECBH6hALoEBAgQIECAAIFlAgrgssCtS4AAAQIECBBQAN0BAgQIECBAgMAyAQVwWeDWJUCAAAECBAgogO4AAQIECBAgQGCZgAK4LHDrEiBAgAABAgQUQHeAAAECBAgQILBMQAFcFrh1CRAgQIAAAQIKoDtAgAABAgQIEFgmoAAuC9y6BAgQIECAAAEF0B0gQIAAAQIECCwTUACXBW5dAgQIECBAgIAC6A4QIECAAAECBJYJKIDLArcuAQIECBAgQEABdAcIECBAgAABAssEFMBlgVuXAAECBAgQIKAAugMECBAgQIAAgWUCCuCywK1LgAABAgQIEFAA3QECBAgQIECAwDIBBXBZ4NYlQIAAAQIECCiA7gABAgQIECBAYJmAArgscOsSIECAAAECBBRAd4AAAQIECBAgsExAAVwWuHUJECBAgAABAgqgO0CAAAECBAgQWCagAC4L3LoECBAgQIAAAQXQHSBAgAABAgQILBNQAJcFbl0CBAgQIECAgALoDhAgQIAAAQIElgkogMsCty4BAgQIECBAQAF0BwgQIECAAAECywQUwGWBW5cAAQIECBAgoAC6AwQIECBAgACBZQIK4LLArUuAAAECBAgQUADdAQIECBAgQIDAMgEFcFng1iVAgAABAgQIKIDuAAECBAgQIEBgmYACuCxw6xIgQIAAAQIEFEB3gAABAgQIECCwTEABXBa4dQkQIECAAAECCqA7QIAAAQIECBBYJqAALgvcugQIECBAgAABBdAdIECAAAECBAgsE1AAlwVuXQIECBAgQICAAugOECBAgAABAgSWCSiAywK3LgECBAgQIEBAAXQHCBAgQIAAAQLLBBTAZYFblwABAgQIECCgALoDBAgQIECAAIFlAgrgssCtS4AAAQIECBBQAN0BAgQIECBAgMAyAQVwWeDWJUCAAAECBAgogO4AAQIECBAgQGCZgAK4LHDrEiBAgAABAgQUQHeAAAECBAgQILBMQAFcFrh1CRAgQIAAAQIKoDtAgAABAgQIEFgmoAAuC9y6BAgQIECAAAEF0B0gQIAAAQIECCwTUACXBW5dAgQIECBAgIAC6A4QIECAAAECBJYJKIDLArcuAQIECBAgQEABdAcIECBAgAABAssEFMBlgVuXAAECBAgQIKAAugMECBAgQIAAgWUCCuCywK1LgAABAgQIEFAA3QECBAgQIECAwDIBBXBZ4NYlQIAAAQIECCiA7gABAgQIECBAYJmAArgscOsSIECAAAECBBRAd4AAAQIECBAgsExAAVwWuHUJECBAgAABAgqgO0CAAAECBAgQWCagAC4L3LoECBAgQIAAAQXQHSBAgAABAgQILBNQAJOB//jx40fyFY4TIECAAAECHxb4+vpa3YFWL3/irimAJxS9gwABAgQIfFZAAfys97hfUwDHRWohAgQIEFggoAAuCPnmigrgTV3vJkCAAAECdwQUwDuua96qAK6J2qIECBAgMEhAARwU5hurKIBvqPtNAgQIECCQE1AAc37rTyuA668AAAIECBBoKKAANgyt0sgKYKU0zEKAAAECBGICCmDMyVMPAgqgq0GAAAECBPoJKID9Mis1sQJYKg7DECBAgACBkIACGGLy0JOAAuhuECBAgACBfgIKYL/MSk2sAJaKwzAECBAgQCAkoACGmDzkC6A7QIAAAQIE5ggogHOyfGUTXwBfYfejBAgQIEAgJaAApvgcVgDdAQIECBAg0E9AAeyXWamJFcBScRiGAAECBAiEBBTAEJOHngQUQHeDAAECBAj0E1AA+2VWamIFsFQchiFAgAABAiEBBTDE5CFfAN0BAgQIECAwR0ABnJPlK5v4AvgKux8lQIAAAQIpAQUwxeewAugOECBAgACBfgIKYL/MSk2sAJaKwzAECBAgQCAkoACGmDz0JKAAuhsECBAgQKCfgALYL7NSEyuApeIwDAECBAgQCAkogCEmD/kC6A4QIECAAIE5AgrgnCxf2cQXwFfY/SgBAgQIEEgJKIApPocVQHeAAAECBAj0E1AA+2VWamIFsFQchiFAgAABAiEBBTDE5KEnAQXQ3SBAgAABAv0EFMB+mZWaWAEsFYdhCBAgQIBASEABDDF5yBdAd4AAAQIECMwRUADnZPnKJr4AvsLuRwkQIECAQEpAAUzxOawAugMECBAgQKCfgALYL7NSEyuApeIwDAECBAgQCAkogCEmDz0JKIDuBgECBAgQ6CegAPbLrNTECmCpOAxDgAABAgRCAgpgiMlDvgC6AwQIECBAYI6AAjgny1c28QXwFXY/SoAAAQIEUgIKYIrPYQXQHSBAgAABAv0EFMB+mZWaWAEsFYdhCBAgQIBASEABDDF56ElAAXQ3CBAgQIBAPwEFsF9mpSZWAEvFYRgCBAgQIBASUABDTB7yBdAdIECAAAECcwQUwDlZvrKJL4CvsPtRAgQIECCQElAAU3wOK4DuAAECBAgQ6CegAPbLrNTECmCpOAxDgAABAgRCAgpgiMlDTwIKoLtBgAABAgT6CSiA/TIrNbECWCoOwxAgQIAAgZCAAhhi8pAvgO4AAQIECBCYI6AAzsnylU18AXyF3Y8SIECAAIGUgAKY4nNYAXQHCBAgQIBAPwEFsF9mpSZWAEvFYRgCBAgQIBASUABDTB56ElAA3Q0CBAgQINBPQAHsl1mpiRXAUnEYhgABAgQIhAQUwBCTh3wBdAcIECBAgMAcAQVwTpavbOIL4CvsfpQAAQIECKQEFMAUn8MKoDtAgAABAgT6CSiA/TIrNbECWCoOwxAgQIAAgZCAAhhi8tCTgALobhAgQIAAgX4CCmC/zEpNrACWisMwBAgQIEAgJKAAhpg85AugO0CAAAECBOYIKIBzsnxlE18AX2H3owQIECBAICWgAKb4HFYA3QECBAgQINBPQAHsl1mpiRXAUnEYhgABAgQIhAQUwBCTh54EFEB3gwABAgQI9BNQAPtlVmpiBbBUHIYhQIAAAQIhAQUwxOQhXwDdAQIECBAgMEdAAZyT5Sub+AL4CrsfJUCAAAECKQEFMMXnsALoDhAgQIAAgX4CCmC/zEpNrACWisMwBAgQIEAgJKAAhpg89CSgALobBAgQIECgn4AC2C+zUhMrgKXiMAwBAgQIEAgJKIAhJg/5AugOECBAgACBOQIK4JwsX9nEF8BX2P0oAQIECBBICSiAKT6HFUB3gAABAgQI9BNQAPtlVmpiBbBUHIYhQIAAAQIhAQUwxOShJwEF0N0gQIAAAQL9BBTAfpmVmlgBLBWHYQgQIECAQEhAAQwxecgXQHeAAAECBAjMEVAA52T5yia+AL7C7kcJECBAgEBKQAFM8TmsALoDBAgQIECgn4AC2C+zUhMrgKXiMAwBAgQIEAgJKIAhJg89CSiA7gYBAgQIEOgnoAD2y6zUxApgqTgMQ4AAAQIEQgIKYIjJQ74AugMECBAgQGCOgAI4J8tXNvEF8BV2P0qAAAECBFICCmCKz2EF0B0gQIAAAQL9BBTAfpmVmlgBLBWHYQgQIECAQEhAAQwxeehJQAF0NwgQIECAQD8BBbBfZqUmVgBLxWEYAgQIECAQElAAQ0we8gXQHSBAgAABAnMEFMA5Wb6yiS+Ar7D7UQIECBAgkBJQAFN8DiuA7gABAgQIEOgnoAD2y6zUxApgqTgMQ4AAAQIEQgIKYIjJQ08CCqC7QYAAAQIE+gkogP0yKzWxAlgqDsMQIECAAIGQgAIYYvKQL4DuAAECBAgQmCOgAM7J8pVNfAF8hd2PEiBAgACBlIACmOJzWAF0BwgQIECAQD8BBbBfZqUmVgBLxWEYAgQIECAQElAAQ0weehJQAN0NAgQIECDQT0AB7JdZqYkVwFJxGIYAAQIECIQEFMAQk4d8AXQHCBAgQIDAHAEFcE6Wr2ziC+Ar7H6UAAECBAikBBTAFJ/DCqA7QIAAAQIE+gkogP0yKzWxAlgqDsMQIECAAIGQgAIYYvLQk4AC6G4QIECAAIF+Agpgv8xKTawAlorDMAQIECBAICSgAIaYPOQLoDtAgAABAgTmCCiAc7J8ZRNfAF9h96MECBAgQCAloACm+BxWAN0BAgQIECDQT0AB7JdZqYkVwFJxGIYAAQIECIQEFMAQk4eeBBRAd4MAAQIECPQTUAD7ZVZqYgWwVByGIUCAAAECIQEFMMTkIV8A3QECBAgQIDBHQAGck+Urm/gC+Aq7HyVAgAABAikBBTDF57AC6A4QIECAAIF+Agpgv8xKTawAlorDMAQIECBAICSgAIaYPPQkoAC6GwQIECBAoJ+AAtgvs1ITK4Cl4jAMAQIECBAICSiAISYP+QLoDhAgQIAAgTkCCuCcLF/ZxBfAV9j9KAECBAgQSAkogCk+hxVAd4AAAQIECPQTUAD7ZVZqYgWwVByGIUCAAAECIQEFMMTkoScBBdDdIECAAAEC/QQUwH6ZlZpYASwVh2EIECBAgEBIQAEMMXnIF0B3gAABAgQIzBFQAOdk+comvgC+wu5HCRAgQIBASkABTPE5rAC6AwQIECBAoJ+AAtgvs1ITK4Cl4jAMAQIECBAICSiAISYPPQkogO4GAQIECBDoJ6AA9sus1MQKYKk4DEOAAAECBEICCmCIyUO+ALoDBAgQIEBgjoACOCfLVzbxBfAVdj9KgAABAgRSAgpgis9hBdAdIECAAAEC/QQUwH6ZlZpYASwVh2EIECBAgEBIQAEMMXnoSUABdDcIECBAgEA/AQWwX2alJlYAS8VhGAIECBAgEBJQAENMHvIF0B0gQIAAAQJzBBTAOVm+sokvgK+w+1ECBAgQIJASUABTfA4rgO4AAQIECBDoJ6AA9sus1MQKYKk4DEOAAAECBEICCmCIyUNPAgqgu0FgrsBf/vjTP5f71W//MHdRmxFYKKAALgz95MoK4ElN7yLwnsC/lr3/NYUC+F42fpnADQEF8IbqoncqgIvCtupoAQVwdLyWI/BfAgqgS5ESUABTfA4TKCOgAJaJwiAEPiKgAH6Eee6PKIBzs7XZLgEFcFfetiWgALoDKQEFMMXnMIEyAgpgmSgMQuAjAgrgR5jn/ogCODdbm+0SUAB35W1bAgqgO5ASUABTfA4TKCOgAJaJwiAEPiKgAH6Eee6PKIBzs7XZLgEFcFfetiWgALoDKQEFMMXnMIEyAgpgmSgMQuAjAgrgR5jn/ogCODdbm+0SUAB35W1bAgqgO5ASUABTfA4TKCOgAJaJwiAEPiKgAH6Eee6PKIBzs7XZLgEFcFfetiWgALoDKQEFMMXnMIEyAgpgmSgMQuAjAgrgR5jn/ogCODdbm+0SUAB35W1bAgqgO5ASUABTfA4TKCOgAJaJwiAEPiKgAH6Eee6PKIBzs7XZLgEFcFfetiWgALoDKQEFMMXnMIEyAgpgmSgMQuAjAgrgR5jn/ogCODdbm+0SUAB35W1bAgqgO5ASUABTfA4TKCWgBJaKwzAErgoogFd5579cAZyfsQ33CCiAe7K2KQEF0B1ICSiAKT6HCZQSUABLxWEYAlcFFMCrvPNfrgDOz9iGewQUwD1Z25SAAugOpAQUwBSfwwRKCSiApeIwDIGrAgrgVd75L1cA52dswz0CCuCerG1KQAF0B1ICCmCKz2ECpQQUwFJxGIbAVQEF8Crv/JcrgPMztuEeAQVwT9Y2JaAAugMpAQUwxecwgVICCmCpOAxD4KqAAniVd/7LFcD5Gdtwj4ACuCdrmxJQAN2BlIACmOJzmEApAQWwVByGIXBVQAG8yjv/5Qrg/IxtuEdAAdyTtU0JKIDuQEpAAUzxOUyglIACWCoOwxC4KqAAXuWd/3IFcH7GNtwjoADuydqmBBRAdyAloACm+BwmUEpAASwVh2EIXBVQAK/yzn+5Ajg/YxvuEVAA92RtUwIKoDuQElAAU3wOEygloACWisMwBK4KKIBXeee/XAGcn7EN9wgogHuytikBBdAdSAkogCk+hwmUElAAS8VhGAJXBRTAq7zzX64Azs/YhnsEFMA9WduUgALoDqQEFMAUn8MESgkogKXiMAyBqwIK4FXe+S9XAOdnbMM9AgrgnqxtSkABdAdSAgpgis9hAqUEFMBScRiGwFUBBfAq7/yXK4DzM7bhHgEFcE/WNiWgALoDKQEFMMXnMIFSAgpgqTgMQ+CqgAJ4lXf+yxXA+RnbcI+AArgna5sSUADdgZSAApjic5hAKQEFsFQchiFwVUABvMo7/+UK4PyMbbhHQAHck7VNCSiA7kBKQAFM8TlMoJSAAlgqDsMQuCqgAF7lnf9yBXB+xjbcI6AA7snapgQUQHcgJaAApvgcJlBKQAEsFYdhCFwVUACv8s5/uQI4P2Mb7hFQAPdkbVMCCqA7kBJQAFN8DhMoJaAAlorDMASuCiiAV3nnv1wBnJ+xDfcIKIB7srYpAQXQHUgJKIApPocJlBJQAEvFYRgCVwUUwKu881+uAM7P2IZ7BBTAPVnblIAC6A6kBBTAFJ/DBEoJKICl4jAMgasCCuBV3vkvVwDnZ2zDPQIK4J6sbUpAAXQHUgIKYIrPYQKlBBTAUnEYhsBVAQXwKu/8lyuA8zO24R4BBXBP1jYloAC6AykBBTDF5zCBUgIKYKk4DEPgqoACeJV3/ssVwPkZ23CPgAK4J2ubElAA3YGUgAKY4nOYQCkBBbBUHIYhcFVAAbzKO//lCuD8jG24S0AJ3JW3bfcKKIB7sz+yuQJ4hNFLCJQRUADLRGEQAlcFFMCrvPNfrgDOz9iGuwQUwF1523avgAK4N/sjmyuARxi9hEAZAQWwTBQGIXBVQAG8yjv/5Qrg/IxtuEtAAdyVt233CiiAe7M/srkCeITRSwiUEVAAy0RhEAJXBRTAq7zzX64Azs/YhrsEFMBdedt2r4ACuDf7I5srgEcYvYRAGQEFsEwUBiFwVUABvMo7/+UK4PyMbbhLQAHclbdt9woogHuzP7K5AniE0UsIlBFQAMtEYRACVwUUwKu881+uAM7P2Ia7BBTAXXnbdq+AArg3+yObK4BHGL2EQBkBBbBMFAYhcFVAAbzKO//lCuD8jG24S0AB3JW3bfcKKIB7sz+yuQJ4hNFLCJQRUADLRGEQAlcFFMCrvPNfrgDOz9iGuwQUwF1523avgAK4N/sjmyuARxi9hEAZAQWwTBQGIXBVQAG8yjv/5Qrg/IxtuEtAAdyVt233CiiAe7M/srkCeITRSwiUEVAAy0RhEAJXBRTAq7zzX64Azs/YhrsEFMBdedt2r4ACuDf7I5srgEcYvYRAGQEFsEwUBiFwVUABvMo7/+UK4PyMbbhLQAHclbdt9woogHuzP7K5AniE0UsIlBFQAMtEYRACVwUUwKu881+uAM7P2Ia7BBTAXXnbdq+AArg3+yObK4BHGL2EQBkBBbBMFAYhcFVAAbzKO//lCuD8jG24S0AB3JW3bfcKKIB7sz+yuQJ4hNFLCJQRUADLRGEQAlcFFMCrvPNfrgDOz9iGuwQUwF1523avgAK4N/sjmyuARxi9hEAZAQWwTBQGIXBVQAG8yjv/5Qrg/IxtuEtAAdyVt233CiiAe7M/srkCeITRSwiUEVAAy0RhEAJXBRTAq7zzX64Azs/YhrsEFMBdedt2r4ACuDf7I5srgEcYvYRAGQEFsEwUBiFwVUABvMo7/+UK4PyMbdhH4OvrKz3sn//w2//7jl//9MfUb/z48SN13mECBM4IKIBnHNe+RQFcG73FCwoogAVDMRKBogIKYNFguoylAHZJypwbBE4VwD/99b+/Av7ml3//8ucL4IabZMcNAgrghpQv7qgAXsT1agLfFLhZAH8e5ecSqAB+MxSPEygqoAAWDabLWApgl6TMuUHgdgH82fD3v/91itL/A5jic5jAMQEF8BjlzhcpgDtzt3VNgRMF8He/+/P/XU4BrJm9qQh8V0AB/K6Y5/9NQAF0IQjUEVAA62RhEgLVBRTA6gkVn08BLB6Q8VYJnCiAP/81MP/rD4H8A9IXwFVXyrKDBRTAweF+YjUF8BPKfoNATOB2AfSHQGI5eIpABwEFsENKhWdUAAuHY7R1AjcLoL8GZt11svBwAQVweMC311MAbwt7P4G4wKkC+K+/mP1rX/5zen8KOJ6nJwncFFAAb+oueLcCuCBkK7YROFEAby+rAN4W9n4CMQEFMObkqWcB/7Cn20GAAAECBPoJ5P/x8H47/3Pi1cufyM0XwBOK3kHgjIAvgGccvYXABgFfADekfHFHBfAirlcT+KaAAvhNMI8TWCygAC4O/8TqCuAJRe8gcEZAATzj6C0ENggogBtSvrijAngR16sJfFNAAfwmmMcJLBZQABeHf2J1BfCEoncQOCOgAJ5x9BYCGwQUwA0pX9xRAbyI69UEvimgAH4TzOMEFgsogIvDP7G6AnhC0TsInBFQAM84eguBDQIK4IaUL+6oAF7E9WoC3xRQAL8J5nECiwUUwMXhn1hdATyh6B0EzggogGccvYXABgEFcEPKF3dUAC/iejWBbwoogN8E8ziBxQIK4OLwT6yuAJ5Q9A4CZwQUwDOO3kJgg4ACuCHlizsqgBdxvZrANwUUwG+CeZzAYgEFcHH4J1ZXAE8oegcBAgQIEPisgAL4We9xv6YAjovUQgQIECCwQEABXBDyzRUVwJu63k2AAAECBO4IKIB3XNe8VQFcE7VFCRAgQGCQgAI4KMw3VlEA31D3mwQIECBAICegAOb81p9WANdfAQAECBAg0FBAAWwYWqWRFcBKaZiFAAECBAjEBBTAmJOnHgQUQFeDAAECBAj0E1AA+2VWamIFsFQchiFAgAABAiEBBTDE5KEnAQXQ3SBAgAABAv0EFMB+mZWaWAEsFYdhCBAgQIBASEABDDF5yBdAd4AAAQIECMwRUADnZPnKJr4AvsLuRwkQIECAQEpAAUzxOawAugMECBAgQKCfgALYL7NSEyuApeIwDAECBAgQCAkogCEmDz0JKIDuBgECBAgQ6CegAPbLrNTECmCpOAxDgAABAgRCAgpgiMlDvgC6AwQIECBAYI6AAjgny1c28QXwFXY/SoAAAQIEUgIKYIrPYQXQHSBAgAABAv0EFMB+mZWaWAEsFYdhCBAgQIBASEABDDF56ElAAXQ3CBAgQIBAPwEFsF9mpSZWAEvFYRgCBAgQIBASUABDTB7yBdAdIECAAAECcwQUwDlZvrKJL4CvsPtRAgQIECCQElAAU3wOK4DuAAECBAgQ6CegAPbLrNTECmCpOAxDgAABAgRCAgpgiMlDTwIKoLtBgAABAgT6CSiA/TIrNbECWCoOwxAgQIAAgZCAAhhi8pAvgO4AAQIECBCYI6AAzsnylU18AXyF3Y8SIECAAIGUgAKY4nNYAXQHCBAgQIBAPwEFsF9mpSZWAEvFYRgCBAgQIBASUABDTB56ElAA3Q0CBAgQINBPQAHsl1mpiRXAUnEYhgABAgQIhAQUwBCTh3wBdAcIECBAgMAcAQVwTpavbOIL4CvsfpQAAQIECKQEFMAUn8MKoDtAgAABAgT6CSiA/TIrNbECWCoOwxAgQIAAgZCAAhhi8tCTgALobhAgQIAAgX4CCmC/zEpNrACWisMwBAgQIEAgJKAAhpg85AugO0CAAAECBOYIKIBzsnxlE18AX2H3owQIECBAICWgAKb4HFYA3QECBAgQINBPQAHsl1mpiRXAUnEYhgABAgQIhAQUwBCTh54EFEB3gwABAgQI9BNQAPtlVmpiBbBUHIYhQIAAAQIhAQUwxOQhXwDdAQIECBAgMEdAAZyT5Sub+AL4CrsfJUCAAAECKQEFMMXnsALoDhAgQIAAgX4CCmC/zEpNrACWisMwBAgQIEAgJKAAhpg89CSgALobBAgQIECgn4AC2C+zUhMrgKXiMAwBAgQIEAgJKIAhJg/5AugOECBAgACBOQIK4JwsX9nEF8BX2P0oAQIECBBICSiAKT6HFUB3gAABAgQI9BNQAPtlVmpiBbBUHIYhQIAAAQIhAQUwxOShJwEF0N0gQIAAAQL9BBTAfpmVmlgBLBWHYQgQIECAQEhAAQwxecgXQHeAAAECBAjMEVAA52T5yia+AL7C7kcJECBAgEBKQAFM8TmsALoDBAgQIECgn4AC2C+zUhMrgKXiMAwBAgQIEAgJKIAhJg89CSiA7gYBAgQIEOgnoAD2y6zUxApgqTgMQ4AAAQIEQgIKYIjJQ74AugMECBAgQGCOgAI4J8tXNvEF8BV2P0qAAAECBFICCmCKz2EF0B0gQIAAAQL9BBTAfpmVmlgBLBWHYQgQIECAQEhAAQwxeehJQAF0NwgQIECAQD8BBbBfZiYmQIAAAQIECBBICHwlzjpKgAABAgQIECDQUEABbBiakQkQIECAAAECGQEFMKPnLAECBAgQIECgoYAC2DA0IxMgQIAAAQIEMgIKYEbPWQIECBAgQIBAQwEFsGFoRiZAgAABAgQIZAQUwIyeswQIECBAgACBhgIKYMPQjEyAAAECBAgQyAgogBk9ZwkQIECAAAECDQUUwIahGZkAAQIECBAgkBFQADN6zhIgQIAAAQIEGgoogA1DMzIBAgQIECBAICOgAGb0nCVAgAABAgQINBRQABuGZmQCBAgQIECAQEZAAczoOUuAAAECBAgQaCigADYMzcgECBAgQIAAgYyAApjRc5YAAQIECBAg0FBAAWwYmpEJECBAgAABAhkBBTCj5ywBAgQIECBAoKGAAtgwNCMTIECAAAECBDICCmBGz1kCBAgQIECAQEMBBbBhaEYmQIAAAQIECGQEFMCMnrMECBAgQIAAgYYCCmDD0IxMgAABAgQIEMgIKIAZPWcJECBAgAABAg0FFMCGoRmZAAECBDG/p5wAAAKxSURBVAgQIJARUAAzes4SIECAAAECBBoKKIANQzMyAQIECBAgQCAjoABm9JwlQIAAAQIECDQUUAAbhmZkAgQIECBAgEBGQAHM6DlLgAABAgQIEGgooAA2DM3IBAgQIECAAIGMgAKY0XOWAAECBAgQINBQQAFsGJqRCRAgQIAAAQIZAQUwo+csAQIECBAgQKChgALYMDQjEyBAgAABAgQyAgpgRs9ZAgQIECBAgEBDAQWwYWhGJkCAAAECBAhkBBTAjJ6zBAgQIECAAIGGAgpgw9CMTIAAAQIECBDICCiAGT1nCRAgQIAAAQINBRTAhqEZmQABAgQIECCQEVAAM3rOEiBAgAABAgQaCiiADUMzMgECBAgQIEAgI6AAZvScJUCAAAECBAg0FFAAG4ZmZAIECBAgQIBARkABzOg5S4AAAQIECBBoKKAANgzNyAQIECBAgACBjIACmNFzlgABAgQIECDQUEABbBiakQkQIECAAAECGQEFMKPnLAECBAgQIECgoYAC2DA0IxMgQIAAAQIEMgIKYEbPWQIECBAgQIBAQwEFsGFoRiZAgAABAgQIZAQUwIyeswQIECBAgACBhgIKYMPQjEyAAAECBAgQyAgogBk9ZwkQIECAAAECDQUUwIahGZkAAQIECBAgkBFQADN6zhIgQIAAAQIEGgoogA1DMzIBAgQIECBAICOgAGb0nCVAgAABAgQINBRQABuGZmQCBAgQIECAQEZAAczoOUuAAAECBAgQaCigADYMzcgECBAgQIAAgYyAApjRc5YAAQIECBAg0FBAAWwYmpEJECBAgAABAhkBBTCj5ywBAgQIECBAoKGAAtgwNCMTIECAAAECBDICCmBGz1kCBAgQIECAQEMBBbBhaEYmQIAAAQIECGQEFMCMnrMECBAgQIAAgYYCfwO3Sb47Cu9haQAAAABJRU5ErkJggg==\" width=\"640\">"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_cart_pole(env, obs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's look at the action space:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Discrete(2)"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "env.action_space"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Yep, just two possible actions: accelerate towards the left or towards the right. Let's push the cart left until the pole falls:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "obs = env.reset()\n",
    "while True:\n",
    "    obs, reward, done, info = env.step(0)\n",
    "    if done:\n",
    "        break"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/javascript": [
       "/* Put everything inside the global mpl namespace */\n",
       "window.mpl = {};\n",
       "\n",
       "\n",
       "mpl.get_websocket_type = function() {\n",
       "    if (typeof(WebSocket) !== 'undefined') {\n",
       "        return WebSocket;\n",
       "    } else if (typeof(MozWebSocket) !== 'undefined') {\n",
       "        return MozWebSocket;\n",
       "    } else {\n",
       "        alert('Your browser does not have WebSocket support.' +\n",
       "              'Please try Chrome, Safari or Firefox ≥ 6. ' +\n",
       "              'Firefox 4 and 5 are also supported but you ' +\n",
       "              'have to enable WebSockets in about:config.');\n",
       "    };\n",
       "}\n",
       "\n",
       "mpl.figure = function(figure_id, websocket, ondownload, parent_element) {\n",
       "    this.id = figure_id;\n",
       "\n",
       "    this.ws = websocket;\n",
       "\n",
       "    this.supports_binary = (this.ws.binaryType != undefined);\n",
       "\n",
       "    if (!this.supports_binary) {\n",
       "        var warnings = document.getElementById(\"mpl-warnings\");\n",
       "        if (warnings) {\n",
       "            warnings.style.display = 'block';\n",
       "            warnings.textContent = (\n",
       "                \"This browser does not support binary websocket messages. \" +\n",
       "                    \"Performance may be slow.\");\n",
       "        }\n",
       "    }\n",
       "\n",
       "    this.imageObj = new Image();\n",
       "\n",
       "    this.context = undefined;\n",
       "    this.message = undefined;\n",
       "    this.canvas = undefined;\n",
       "    this.rubberband_canvas = undefined;\n",
       "    this.rubberband_context = undefined;\n",
       "    this.format_dropdown = undefined;\n",
       "\n",
       "    this.image_mode = 'full';\n",
       "\n",
       "    this.root = $('<div/>');\n",
       "    this._root_extra_style(this.root)\n",
       "    this.root.attr('style', 'display: inline-block');\n",
       "\n",
       "    $(parent_element).append(this.root);\n",
       "\n",
       "    this._init_header(this);\n",
       "    this._init_canvas(this);\n",
       "    this._init_toolbar(this);\n",
       "\n",
       "    var fig = this;\n",
       "\n",
       "    this.waiting = false;\n",
       "\n",
       "    this.ws.onopen =  function () {\n",
       "            fig.send_message(\"supports_binary\", {value: fig.supports_binary});\n",
       "            fig.send_message(\"send_image_mode\", {});\n",
       "            if (mpl.ratio != 1) {\n",
       "                fig.send_message(\"set_dpi_ratio\", {'dpi_ratio': mpl.ratio});\n",
       "            }\n",
       "            fig.send_message(\"refresh\", {});\n",
       "        }\n",
       "\n",
       "    this.imageObj.onload = function() {\n",
       "            if (fig.image_mode == 'full') {\n",
       "                // Full images could contain transparency (where diff images\n",
       "                // almost always do), so we need to clear the canvas so that\n",
       "                // there is no ghosting.\n",
       "                fig.context.clearRect(0, 0, fig.canvas.width, fig.canvas.height);\n",
       "            }\n",
       "            fig.context.drawImage(fig.imageObj, 0, 0);\n",
       "        };\n",
       "\n",
       "    this.imageObj.onunload = function() {\n",
       "        this.ws.close();\n",
       "    }\n",
       "\n",
       "    this.ws.onmessage = this._make_on_message_function(this);\n",
       "\n",
       "    this.ondownload = ondownload;\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._init_header = function() {\n",
       "    var titlebar = $(\n",
       "        '<div class=\"ui-dialog-titlebar ui-widget-header ui-corner-all ' +\n",
       "        'ui-helper-clearfix\"/>');\n",
       "    var titletext = $(\n",
       "        '<div class=\"ui-dialog-title\" style=\"width: 100%; ' +\n",
       "        'text-align: center; padding: 3px;\"/>');\n",
       "    titlebar.append(titletext)\n",
       "    this.root.append(titlebar);\n",
       "    this.header = titletext[0];\n",
       "}\n",
       "\n",
       "\n",
       "\n",
       "mpl.figure.prototype._canvas_extra_style = function(canvas_div) {\n",
       "\n",
       "}\n",
       "\n",
       "\n",
       "mpl.figure.prototype._root_extra_style = function(canvas_div) {\n",
       "\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._init_canvas = function() {\n",
       "    var fig = this;\n",
       "\n",
       "    var canvas_div = $('<div/>');\n",
       "\n",
       "    canvas_div.attr('style', 'position: relative; clear: both; outline: 0');\n",
       "\n",
       "    function canvas_keyboard_event(event) {\n",
       "        return fig.key_event(event, event['data']);\n",
       "    }\n",
       "\n",
       "    canvas_div.keydown('key_press', canvas_keyboard_event);\n",
       "    canvas_div.keyup('key_release', canvas_keyboard_event);\n",
       "    this.canvas_div = canvas_div\n",
       "    this._canvas_extra_style(canvas_div)\n",
       "    this.root.append(canvas_div);\n",
       "\n",
       "    var canvas = $('<canvas/>');\n",
       "    canvas.addClass('mpl-canvas');\n",
       "    canvas.attr('style', \"left: 0; top: 0; z-index: 0; outline: 0\")\n",
       "\n",
       "    this.canvas = canvas[0];\n",
       "    this.context = canvas[0].getContext(\"2d\");\n",
       "\n",
       "    var backingStore = this.context.backingStorePixelRatio ||\n",
       "\tthis.context.webkitBackingStorePixelRatio ||\n",
       "\tthis.context.mozBackingStorePixelRatio ||\n",
       "\tthis.context.msBackingStorePixelRatio ||\n",
       "\tthis.context.oBackingStorePixelRatio ||\n",
       "\tthis.context.backingStorePixelRatio || 1;\n",
       "\n",
       "    mpl.ratio = (window.devicePixelRatio || 1) / backingStore;\n",
       "\n",
       "    var rubberband = $('<canvas/>');\n",
       "    rubberband.attr('style', \"position: absolute; left: 0; top: 0; z-index: 1;\")\n",
       "\n",
       "    var pass_mouse_events = true;\n",
       "\n",
       "    canvas_div.resizable({\n",
       "        start: function(event, ui) {\n",
       "            pass_mouse_events = false;\n",
       "        },\n",
       "        resize: function(event, ui) {\n",
       "            fig.request_resize(ui.size.width, ui.size.height);\n",
       "        },\n",
       "        stop: function(event, ui) {\n",
       "            pass_mouse_events = true;\n",
       "            fig.request_resize(ui.size.width, ui.size.height);\n",
       "        },\n",
       "    });\n",
       "\n",
       "    function mouse_event_fn(event) {\n",
       "        if (pass_mouse_events)\n",
       "            return fig.mouse_event(event, event['data']);\n",
       "    }\n",
       "\n",
       "    rubberband.mousedown('button_press', mouse_event_fn);\n",
       "    rubberband.mouseup('button_release', mouse_event_fn);\n",
       "    // Throttle sequential mouse events to 1 every 20ms.\n",
       "    rubberband.mousemove('motion_notify', mouse_event_fn);\n",
       "\n",
       "    rubberband.mouseenter('figure_enter', mouse_event_fn);\n",
       "    rubberband.mouseleave('figure_leave', mouse_event_fn);\n",
       "\n",
       "    canvas_div.on(\"wheel\", function (event) {\n",
       "        event = event.originalEvent;\n",
       "        event['data'] = 'scroll'\n",
       "        if (event.deltaY < 0) {\n",
       "            event.step = 1;\n",
       "        } else {\n",
       "            event.step = -1;\n",
       "        }\n",
       "        mouse_event_fn(event);\n",
       "    });\n",
       "\n",
       "    canvas_div.append(canvas);\n",
       "    canvas_div.append(rubberband);\n",
       "\n",
       "    this.rubberband = rubberband;\n",
       "    this.rubberband_canvas = rubberband[0];\n",
       "    this.rubberband_context = rubberband[0].getContext(\"2d\");\n",
       "    this.rubberband_context.strokeStyle = \"#000000\";\n",
       "\n",
       "    this._resize_canvas = function(width, height) {\n",
       "        // Keep the size of the canvas, canvas container, and rubber band\n",
       "        // canvas in synch.\n",
       "        canvas_div.css('width', width)\n",
       "        canvas_div.css('height', height)\n",
       "\n",
       "        canvas.attr('width', width * mpl.ratio);\n",
       "        canvas.attr('height', height * mpl.ratio);\n",
       "        canvas.attr('style', 'width: ' + width + 'px; height: ' + height + 'px;');\n",
       "\n",
       "        rubberband.attr('width', width);\n",
       "        rubberband.attr('height', height);\n",
       "    }\n",
       "\n",
       "    // Set the figure to an initial 600x600px, this will subsequently be updated\n",
       "    // upon first draw.\n",
       "    this._resize_canvas(600, 600);\n",
       "\n",
       "    // Disable right mouse context menu.\n",
       "    $(this.rubberband_canvas).bind(\"contextmenu\",function(e){\n",
       "        return false;\n",
       "    });\n",
       "\n",
       "    function set_focus () {\n",
       "        canvas.focus();\n",
       "        canvas_div.focus();\n",
       "    }\n",
       "\n",
       "    window.setTimeout(set_focus, 100);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._init_toolbar = function() {\n",
       "    var fig = this;\n",
       "\n",
       "    var nav_element = $('<div/>')\n",
       "    nav_element.attr('style', 'width: 100%');\n",
       "    this.root.append(nav_element);\n",
       "\n",
       "    // Define a callback function for later on.\n",
       "    function toolbar_event(event) {\n",
       "        return fig.toolbar_button_onclick(event['data']);\n",
       "    }\n",
       "    function toolbar_mouse_event(event) {\n",
       "        return fig.toolbar_button_onmouseover(event['data']);\n",
       "    }\n",
       "\n",
       "    for(var toolbar_ind in mpl.toolbar_items) {\n",
       "        var name = mpl.toolbar_items[toolbar_ind][0];\n",
       "        var tooltip = mpl.toolbar_items[toolbar_ind][1];\n",
       "        var image = mpl.toolbar_items[toolbar_ind][2];\n",
       "        var method_name = mpl.toolbar_items[toolbar_ind][3];\n",
       "\n",
       "        if (!name) {\n",
       "            // put a spacer in here.\n",
       "            continue;\n",
       "        }\n",
       "        var button = $('<button/>');\n",
       "        button.addClass('ui-button ui-widget ui-state-default ui-corner-all ' +\n",
       "                        'ui-button-icon-only');\n",
       "        button.attr('role', 'button');\n",
       "        button.attr('aria-disabled', 'false');\n",
       "        button.click(method_name, toolbar_event);\n",
       "        button.mouseover(tooltip, toolbar_mouse_event);\n",
       "\n",
       "        var icon_img = $('<span/>');\n",
       "        icon_img.addClass('ui-button-icon-primary ui-icon');\n",
       "        icon_img.addClass(image);\n",
       "        icon_img.addClass('ui-corner-all');\n",
       "\n",
       "        var tooltip_span = $('<span/>');\n",
       "        tooltip_span.addClass('ui-button-text');\n",
       "        tooltip_span.html(tooltip);\n",
       "\n",
       "        button.append(icon_img);\n",
       "        button.append(tooltip_span);\n",
       "\n",
       "        nav_element.append(button);\n",
       "    }\n",
       "\n",
       "    var fmt_picker_span = $('<span/>');\n",
       "\n",
       "    var fmt_picker = $('<select/>');\n",
       "    fmt_picker.addClass('mpl-toolbar-option ui-widget ui-widget-content');\n",
       "    fmt_picker_span.append(fmt_picker);\n",
       "    nav_element.append(fmt_picker_span);\n",
       "    this.format_dropdown = fmt_picker[0];\n",
       "\n",
       "    for (var ind in mpl.extensions) {\n",
       "        var fmt = mpl.extensions[ind];\n",
       "        var option = $(\n",
       "            '<option/>', {selected: fmt === mpl.default_extension}).html(fmt);\n",
       "        fmt_picker.append(option)\n",
       "    }\n",
       "\n",
       "    // Add hover states to the ui-buttons\n",
       "    $( \".ui-button\" ).hover(\n",
       "        function() { $(this).addClass(\"ui-state-hover\");},\n",
       "        function() { $(this).removeClass(\"ui-state-hover\");}\n",
       "    );\n",
       "\n",
       "    var status_bar = $('<span class=\"mpl-message\"/>');\n",
       "    nav_element.append(status_bar);\n",
       "    this.message = status_bar[0];\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.request_resize = function(x_pixels, y_pixels) {\n",
       "    // Request matplotlib to resize the figure. Matplotlib will then trigger a resize in the client,\n",
       "    // which will in turn request a refresh of the image.\n",
       "    this.send_message('resize', {'width': x_pixels, 'height': y_pixels});\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.send_message = function(type, properties) {\n",
       "    properties['type'] = type;\n",
       "    properties['figure_id'] = this.id;\n",
       "    this.ws.send(JSON.stringify(properties));\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.send_draw_message = function() {\n",
       "    if (!this.waiting) {\n",
       "        this.waiting = true;\n",
       "        this.ws.send(JSON.stringify({type: \"draw\", figure_id: this.id}));\n",
       "    }\n",
       "}\n",
       "\n",
       "\n",
       "mpl.figure.prototype.handle_save = function(fig, msg) {\n",
       "    var format_dropdown = fig.format_dropdown;\n",
       "    var format = format_dropdown.options[format_dropdown.selectedIndex].value;\n",
       "    fig.ondownload(fig, format);\n",
       "}\n",
       "\n",
       "\n",
       "mpl.figure.prototype.handle_resize = function(fig, msg) {\n",
       "    var size = msg['size'];\n",
       "    if (size[0] != fig.canvas.width || size[1] != fig.canvas.height) {\n",
       "        fig._resize_canvas(size[0], size[1]);\n",
       "        fig.send_message(\"refresh\", {});\n",
       "    };\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_rubberband = function(fig, msg) {\n",
       "    var x0 = msg['x0'] / mpl.ratio;\n",
       "    var y0 = (fig.canvas.height - msg['y0']) / mpl.ratio;\n",
       "    var x1 = msg['x1'] / mpl.ratio;\n",
       "    var y1 = (fig.canvas.height - msg['y1']) / mpl.ratio;\n",
       "    x0 = Math.floor(x0) + 0.5;\n",
       "    y0 = Math.floor(y0) + 0.5;\n",
       "    x1 = Math.floor(x1) + 0.5;\n",
       "    y1 = Math.floor(y1) + 0.5;\n",
       "    var min_x = Math.min(x0, x1);\n",
       "    var min_y = Math.min(y0, y1);\n",
       "    var width = Math.abs(x1 - x0);\n",
       "    var height = Math.abs(y1 - y0);\n",
       "\n",
       "    fig.rubberband_context.clearRect(\n",
       "        0, 0, fig.canvas.width, fig.canvas.height);\n",
       "\n",
       "    fig.rubberband_context.strokeRect(min_x, min_y, width, height);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_figure_label = function(fig, msg) {\n",
       "    // Updates the figure title.\n",
       "    fig.header.textContent = msg['label'];\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_cursor = function(fig, msg) {\n",
       "    var cursor = msg['cursor'];\n",
       "    switch(cursor)\n",
       "    {\n",
       "    case 0:\n",
       "        cursor = 'pointer';\n",
       "        break;\n",
       "    case 1:\n",
       "        cursor = 'default';\n",
       "        break;\n",
       "    case 2:\n",
       "        cursor = 'crosshair';\n",
       "        break;\n",
       "    case 3:\n",
       "        cursor = 'move';\n",
       "        break;\n",
       "    }\n",
       "    fig.rubberband_canvas.style.cursor = cursor;\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_message = function(fig, msg) {\n",
       "    fig.message.textContent = msg['message'];\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_draw = function(fig, msg) {\n",
       "    // Request the server to send over a new figure.\n",
       "    fig.send_draw_message();\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_image_mode = function(fig, msg) {\n",
       "    fig.image_mode = msg['mode'];\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.updated_canvas_event = function() {\n",
       "    // Called whenever the canvas gets updated.\n",
       "    this.send_message(\"ack\", {});\n",
       "}\n",
       "\n",
       "// A function to construct a web socket function for onmessage handling.\n",
       "// Called in the figure constructor.\n",
       "mpl.figure.prototype._make_on_message_function = function(fig) {\n",
       "    return function socket_on_message(evt) {\n",
       "        if (evt.data instanceof Blob) {\n",
       "            /* FIXME: We get \"Resource interpreted as Image but\n",
       "             * transferred with MIME type text/plain:\" errors on\n",
       "             * Chrome.  But how to set the MIME type?  It doesn't seem\n",
       "             * to be part of the websocket stream */\n",
       "            evt.data.type = \"image/png\";\n",
       "\n",
       "            /* Free the memory for the previous frames */\n",
       "            if (fig.imageObj.src) {\n",
       "                (window.URL || window.webkitURL).revokeObjectURL(\n",
       "                    fig.imageObj.src);\n",
       "            }\n",
       "\n",
       "            fig.imageObj.src = (window.URL || window.webkitURL).createObjectURL(\n",
       "                evt.data);\n",
       "            fig.updated_canvas_event();\n",
       "            fig.waiting = false;\n",
       "            return;\n",
       "        }\n",
       "        else if (typeof evt.data === 'string' && evt.data.slice(0, 21) == \"data:image/png;base64\") {\n",
       "            fig.imageObj.src = evt.data;\n",
       "            fig.updated_canvas_event();\n",
       "            fig.waiting = false;\n",
       "            return;\n",
       "        }\n",
       "\n",
       "        var msg = JSON.parse(evt.data);\n",
       "        var msg_type = msg['type'];\n",
       "\n",
       "        // Call the  \"handle_{type}\" callback, which takes\n",
       "        // the figure and JSON message as its only arguments.\n",
       "        try {\n",
       "            var callback = fig[\"handle_\" + msg_type];\n",
       "        } catch (e) {\n",
       "            console.log(\"No handler for the '\" + msg_type + \"' message type: \", msg);\n",
       "            return;\n",
       "        }\n",
       "\n",
       "        if (callback) {\n",
       "            try {\n",
       "                // console.log(\"Handling '\" + msg_type + \"' message: \", msg);\n",
       "                callback(fig, msg);\n",
       "            } catch (e) {\n",
       "                console.log(\"Exception inside the 'handler_\" + msg_type + \"' callback:\", e, e.stack, msg);\n",
       "            }\n",
       "        }\n",
       "    };\n",
       "}\n",
       "\n",
       "// from http://stackoverflow.com/questions/1114465/getting-mouse-location-in-canvas\n",
       "mpl.findpos = function(e) {\n",
       "    //this section is from http://www.quirksmode.org/js/events_properties.html\n",
       "    var targ;\n",
       "    if (!e)\n",
       "        e = window.event;\n",
       "    if (e.target)\n",
       "        targ = e.target;\n",
       "    else if (e.srcElement)\n",
       "        targ = e.srcElement;\n",
       "    if (targ.nodeType == 3) // defeat Safari bug\n",
       "        targ = targ.parentNode;\n",
       "\n",
       "    // jQuery normalizes the pageX and pageY\n",
       "    // pageX,Y are the mouse positions relative to the document\n",
       "    // offset() returns the position of the element relative to the document\n",
       "    var x = e.pageX - $(targ).offset().left;\n",
       "    var y = e.pageY - $(targ).offset().top;\n",
       "\n",
       "    return {\"x\": x, \"y\": y};\n",
       "};\n",
       "\n",
       "/*\n",
       " * return a copy of an object with only non-object keys\n",
       " * we need this to avoid circular references\n",
       " * http://stackoverflow.com/a/24161582/3208463\n",
       " */\n",
       "function simpleKeys (original) {\n",
       "  return Object.keys(original).reduce(function (obj, key) {\n",
       "    if (typeof original[key] !== 'object')\n",
       "        obj[key] = original[key]\n",
       "    return obj;\n",
       "  }, {});\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.mouse_event = function(event, name) {\n",
       "    var canvas_pos = mpl.findpos(event)\n",
       "\n",
       "    if (name === 'button_press')\n",
       "    {\n",
       "        this.canvas.focus();\n",
       "        this.canvas_div.focus();\n",
       "    }\n",
       "\n",
       "    var x = canvas_pos.x * mpl.ratio;\n",
       "    var y = canvas_pos.y * mpl.ratio;\n",
       "\n",
       "    this.send_message(name, {x: x, y: y, button: event.button,\n",
       "                             step: event.step,\n",
       "                             guiEvent: simpleKeys(event)});\n",
       "\n",
       "    /* This prevents the web browser from automatically changing to\n",
       "     * the text insertion cursor when the button is pressed.  We want\n",
       "     * to control all of the cursor setting manually through the\n",
       "     * 'cursor' event from matplotlib */\n",
       "    event.preventDefault();\n",
       "    return false;\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._key_event_extra = function(event, name) {\n",
       "    // Handle any extra behaviour associated with a key event\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.key_event = function(event, name) {\n",
       "\n",
       "    // Prevent repeat events\n",
       "    if (name == 'key_press')\n",
       "    {\n",
       "        if (event.which === this._key)\n",
       "            return;\n",
       "        else\n",
       "            this._key = event.which;\n",
       "    }\n",
       "    if (name == 'key_release')\n",
       "        this._key = null;\n",
       "\n",
       "    var value = '';\n",
       "    if (event.ctrlKey && event.which != 17)\n",
       "        value += \"ctrl+\";\n",
       "    if (event.altKey && event.which != 18)\n",
       "        value += \"alt+\";\n",
       "    if (event.shiftKey && event.which != 16)\n",
       "        value += \"shift+\";\n",
       "\n",
       "    value += 'k';\n",
       "    value += event.which.toString();\n",
       "\n",
       "    this._key_event_extra(event, name);\n",
       "\n",
       "    this.send_message(name, {key: value,\n",
       "                             guiEvent: simpleKeys(event)});\n",
       "    return false;\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.toolbar_button_onclick = function(name) {\n",
       "    if (name == 'download') {\n",
       "        this.handle_save(this, null);\n",
       "    } else {\n",
       "        this.send_message(\"toolbar_button\", {name: name});\n",
       "    }\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.toolbar_button_onmouseover = function(tooltip) {\n",
       "    this.message.textContent = tooltip;\n",
       "};\n",
       "mpl.toolbar_items = [[\"Home\", \"Reset original view\", \"fa fa-home icon-home\", \"home\"], [\"Back\", \"Back to  previous view\", \"fa fa-arrow-left icon-arrow-left\", \"back\"], [\"Forward\", \"Forward to next view\", \"fa fa-arrow-right icon-arrow-right\", \"forward\"], [\"\", \"\", \"\", \"\"], [\"Pan\", \"Pan axes with left mouse, zoom with right\", \"fa fa-arrows icon-move\", \"pan\"], [\"Zoom\", \"Zoom to rectangle\", \"fa fa-square-o icon-check-empty\", \"zoom\"], [\"\", \"\", \"\", \"\"], [\"Download\", \"Download plot\", \"fa fa-floppy-o icon-save\", \"download\"]];\n",
       "\n",
       "mpl.extensions = [\"eps\", \"jpeg\", \"pdf\", \"png\", \"ps\", \"raw\", \"svg\", \"tif\"];\n",
       "\n",
       "mpl.default_extension = \"png\";var comm_websocket_adapter = function(comm) {\n",
       "    // Create a \"websocket\"-like object which calls the given IPython comm\n",
       "    // object with the appropriate methods. Currently this is a non binary\n",
       "    // socket, so there is still some room for performance tuning.\n",
       "    var ws = {};\n",
       "\n",
       "    ws.close = function() {\n",
       "        comm.close()\n",
       "    };\n",
       "    ws.send = function(m) {\n",
       "        //console.log('sending', m);\n",
       "        comm.send(m);\n",
       "    };\n",
       "    // Register the callback with on_msg.\n",
       "    comm.on_msg(function(msg) {\n",
       "        //console.log('receiving', msg['content']['data'], msg);\n",
       "        // Pass the mpl event to the overriden (by mpl) onmessage function.\n",
       "        ws.onmessage(msg['content']['data'])\n",
       "    });\n",
       "    return ws;\n",
       "}\n",
       "\n",
       "mpl.mpl_figure_comm = function(comm, msg) {\n",
       "    // This is the function which gets called when the mpl process\n",
       "    // starts-up an IPython Comm through the \"matplotlib\" channel.\n",
       "\n",
       "    var id = msg.content.data.id;\n",
       "    // Get hold of the div created by the display call when the Comm\n",
       "    // socket was opened in Python.\n",
       "    var element = $(\"#\" + id);\n",
       "    var ws_proxy = comm_websocket_adapter(comm)\n",
       "\n",
       "    function ondownload(figure, format) {\n",
       "        window.open(figure.imageObj.src);\n",
       "    }\n",
       "\n",
       "    var fig = new mpl.figure(id, ws_proxy,\n",
       "                           ondownload,\n",
       "                           element.get(0));\n",
       "\n",
       "    // Call onopen now - mpl needs it, as it is assuming we've passed it a real\n",
       "    // web socket which is closed, not our websocket->open comm proxy.\n",
       "    ws_proxy.onopen();\n",
       "\n",
       "    fig.parent_element = element.get(0);\n",
       "    fig.cell_info = mpl.find_output_cell(\"<div id='\" + id + \"'></div>\");\n",
       "    if (!fig.cell_info) {\n",
       "        console.error(\"Failed to find cell for figure\", id, fig);\n",
       "        return;\n",
       "    }\n",
       "\n",
       "    var output_index = fig.cell_info[2]\n",
       "    var cell = fig.cell_info[0];\n",
       "\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_close = function(fig, msg) {\n",
       "    var width = fig.canvas.width/mpl.ratio\n",
       "    fig.root.unbind('remove')\n",
       "\n",
       "    // Update the output cell to use the data from the current canvas.\n",
       "    fig.push_to_output();\n",
       "    var dataURL = fig.canvas.toDataURL();\n",
       "    // Re-enable the keyboard manager in IPython - without this line, in FF,\n",
       "    // the notebook keyboard shortcuts fail.\n",
       "    IPython.keyboard_manager.enable()\n",
       "    $(fig.parent_element).html('<img src=\"' + dataURL + '\" width=\"' + width + '\">');\n",
       "    fig.close_ws(fig, msg);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.close_ws = function(fig, msg){\n",
       "    fig.send_message('closing', msg);\n",
       "    // fig.ws.close()\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.push_to_output = function(remove_interactive) {\n",
       "    // Turn the data on the canvas into data in the output cell.\n",
       "    var width = this.canvas.width/mpl.ratio\n",
       "    var dataURL = this.canvas.toDataURL();\n",
       "    this.cell_info[1]['text/html'] = '<img src=\"' + dataURL + '\" width=\"' + width + '\">';\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.updated_canvas_event = function() {\n",
       "    // Tell IPython that the notebook contents must change.\n",
       "    IPython.notebook.set_dirty(true);\n",
       "    this.send_message(\"ack\", {});\n",
       "    var fig = this;\n",
       "    // Wait a second, then push the new image to the DOM so\n",
       "    // that it is saved nicely (might be nice to debounce this).\n",
       "    setTimeout(function () { fig.push_to_output() }, 1000);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._init_toolbar = function() {\n",
       "    var fig = this;\n",
       "\n",
       "    var nav_element = $('<div/>')\n",
       "    nav_element.attr('style', 'width: 100%');\n",
       "    this.root.append(nav_element);\n",
       "\n",
       "    // Define a callback function for later on.\n",
       "    function toolbar_event(event) {\n",
       "        return fig.toolbar_button_onclick(event['data']);\n",
       "    }\n",
       "    function toolbar_mouse_event(event) {\n",
       "        return fig.toolbar_button_onmouseover(event['data']);\n",
       "    }\n",
       "\n",
       "    for(var toolbar_ind in mpl.toolbar_items){\n",
       "        var name = mpl.toolbar_items[toolbar_ind][0];\n",
       "        var tooltip = mpl.toolbar_items[toolbar_ind][1];\n",
       "        var image = mpl.toolbar_items[toolbar_ind][2];\n",
       "        var method_name = mpl.toolbar_items[toolbar_ind][3];\n",
       "\n",
       "        if (!name) { continue; };\n",
       "\n",
       "        var button = $('<button class=\"btn btn-default\" href=\"#\" title=\"' + name + '\"><i class=\"fa ' + image + ' fa-lg\"></i></button>');\n",
       "        button.click(method_name, toolbar_event);\n",
       "        button.mouseover(tooltip, toolbar_mouse_event);\n",
       "        nav_element.append(button);\n",
       "    }\n",
       "\n",
       "    // Add the status bar.\n",
       "    var status_bar = $('<span class=\"mpl-message\" style=\"text-align:right; float: right;\"/>');\n",
       "    nav_element.append(status_bar);\n",
       "    this.message = status_bar[0];\n",
       "\n",
       "    // Add the close button to the window.\n",
       "    var buttongrp = $('<div class=\"btn-group inline pull-right\"></div>');\n",
       "    var button = $('<button class=\"btn btn-mini btn-primary\" href=\"#\" title=\"Stop Interaction\"><i class=\"fa fa-power-off icon-remove icon-large\"></i></button>');\n",
       "    button.click(function (evt) { fig.handle_close(fig, {}); } );\n",
       "    button.mouseover('Stop Interaction', toolbar_mouse_event);\n",
       "    buttongrp.append(button);\n",
       "    var titlebar = this.root.find($('.ui-dialog-titlebar'));\n",
       "    titlebar.prepend(buttongrp);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._root_extra_style = function(el){\n",
       "    var fig = this\n",
       "    el.on(\"remove\", function(){\n",
       "\tfig.close_ws(fig, {});\n",
       "    });\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._canvas_extra_style = function(el){\n",
       "    // this is important to make the div 'focusable\n",
       "    el.attr('tabindex', 0)\n",
       "    // reach out to IPython and tell the keyboard manager to turn it's self\n",
       "    // off when our div gets focus\n",
       "\n",
       "    // location in version 3\n",
       "    if (IPython.notebook.keyboard_manager) {\n",
       "        IPython.notebook.keyboard_manager.register_events(el);\n",
       "    }\n",
       "    else {\n",
       "        // location in version 2\n",
       "        IPython.keyboard_manager.register_events(el);\n",
       "    }\n",
       "\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._key_event_extra = function(event, name) {\n",
       "    var manager = IPython.notebook.keyboard_manager;\n",
       "    if (!manager)\n",
       "        manager = IPython.keyboard_manager;\n",
       "\n",
       "    // Check for shift+enter\n",
       "    if (event.shiftKey && event.which == 13) {\n",
       "        this.canvas_div.blur();\n",
       "        // select the cell after this one\n",
       "        var index = IPython.notebook.find_cell_index(this.cell_info[0]);\n",
       "        IPython.notebook.select(index + 1);\n",
       "    }\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_save = function(fig, msg) {\n",
       "    fig.ondownload(fig, null);\n",
       "}\n",
       "\n",
       "\n",
       "mpl.find_output_cell = function(html_output) {\n",
       "    // Return the cell and output element which can be found *uniquely* in the notebook.\n",
       "    // Note - this is a bit hacky, but it is done because the \"notebook_saving.Notebook\"\n",
       "    // IPython event is triggered only after the cells have been serialised, which for\n",
       "    // our purposes (turning an active figure into a static one), is too late.\n",
       "    var cells = IPython.notebook.get_cells();\n",
       "    var ncells = cells.length;\n",
       "    for (var i=0; i<ncells; i++) {\n",
       "        var cell = cells[i];\n",
       "        if (cell.cell_type === 'code'){\n",
       "            for (var j=0; j<cell.output_area.outputs.length; j++) {\n",
       "                var data = cell.output_area.outputs[j];\n",
       "                if (data.data) {\n",
       "                    // IPython >= 3 moved mimebundle to data attribute of output\n",
       "                    data = data.data;\n",
       "                }\n",
       "                if (data['text/html'] == html_output) {\n",
       "                    return [cell, data, j];\n",
       "                }\n",
       "            }\n",
       "        }\n",
       "    }\n",
       "}\n",
       "\n",
       "// Register the function which deals with the matplotlib target/channel.\n",
       "// The kernel may be null if the page has been refreshed.\n",
       "if (IPython.notebook.kernel != null) {\n",
       "    IPython.notebook.kernel.comm_manager.register_target('matplotlib', mpl.mpl_figure_comm);\n",
       "}\n"
      ],
      "text/plain": [
       "<IPython.core.display.Javascript object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<img src=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAoAAAAHgCAYAAAA10dzkAAAgAElEQVR4Xu3c0XFtxRGGUZSG0zBpOA0IgyIMSMNp4DSchlynXLgw6Kq3Wucf7ZlefmWm0azeD19dXfzynf8RIECAAAECBAiMEngZ9VqPJUCAAAECBAgQ+E4A+ggIECBAgAABAsMEBOCwhXsuAQIECBAgQEAA+gYIECBAgAABAsMEBOCwhXsuAQIECBAgQEAA+gYIECBAgAABAsMEBOCwhXsuAQIECBAgQEAA+gYIECBAgAABAsMEBOCwhXsuAQIECBAgQEAA+gYIECBAgAABAsMEBOCwhXsuAQIECBAgQEAA+gYIECBAgAABAsMEBOCwhXsuAQIECBAgQEAA+gYIECBAgAABAsMEBOCwhXsuAQIECBAgQEAA+gYIECBAgAABAsMEBOCwhXsuAQIECBAgQEAA+gYIECBAgAABAsMEBOCwhXsuAQIECBAgQEAA+gYIECBAgAABAsMEBOCwhXsuAQIECBAgQEAA+gYIECBAgAABAsMEBOCwhXsuAQIECBAgQEAA+gYIECBAgAABAsMEBOCwhXsuAQIECBAgQEAA+gYIECBAgAABAsMEBOCwhXsuAQIECBAgQEAA+gYIECBAgAABAsMEBOCwhXsuAQIECBAgQEAA+gYIECBAgAABAsMEBOCwhXsuAQIECBAgQEAA+gYIECBAgAABAsMEBOCwhXsuAQIECBAgQEAA+gYIECBAgAABAsMEBOCwhXsuAQIECBAgQEAA+gYIECBAgAABAsMEBOCwhXsuAQIECBAgQEAA+gYIECBAgAABAsMEBOCwhXsuAQIECBAgQEAA+gYIECBAgAABAsMEBOCwhXsuAQIECBAgQEAA+gYIECBAgAABAsMEBOCwhXsuAQIECBAgQEAA+gYIECBAgAABAsMEBOCwhXsuAQIECBAgQEAA+gYIECBAgAABAsMEBOCwhXsuAQIECBAgQEAA+gYIECBAgAABAsMEBOCwhXsuAQIECBAgQEAA+gYIECBAgAABAsMEBOCwhXsuAQIECBAgQOD4AHx9fX21ZgIECBAgQIDA3QReXl6+rMO+7F+8agkCcJW0fw8BAgQIECDwEQEB+BGtD54VgB8Ec5wAAQIECBBYIiAAg8wCMIhrNAECBAgQINAWEIBtuvqiAKyNnCBAgAABAgTWCwjAoLkADOIaTYAAAQIECLQFBGCbrr4oAGsjJwgQIECAAIH1AgIwaC4Ag7hGEyBAgAABAm0BAdimqy8KwNrICQIECBAgQGC9gAAMmgvAIK7RBAgQIECAQFtAALbp6osCsDZyggABAgQIEFgvIACD5gIwiGs0AQIECBAg0BYQgG26+qIArI2cIECAAAECBNYLCMCguQAM4hpNgAABAgQItAUEYJuuvigAayMnCBAgQIAAgfUCAjBoLgCDuEYTIECAAAECbQEB2KarLwrA2sgJAgQIECBAYL2AAAyaC8AgrtEECBAgQIBAW0AAtunqiwKwNnKCAAECBAgQWC8gAIPmAjCIazQBAgQIECDQFhCAbbr6ogCsjZwgQIAAAQIE1gsIwKC5AAziGk2AAAECBAi0BQRgm66+KABrIycIECBAgACB9QICMGguAIO4RhMgQIAAAQJtAQHYpqsvCsDayAkCBAgQIEBgvYAADJoLwCCu0QQIECBAgEBbQAC26eqLArA2coIAAQIECBBYLyAAg+YCMIhrNAECBAgQINAWEIBtuvqiAKyNnCBAgAABAgTWCwjAoLkADOIaTYAAAQIECLQFBGCbrr4oAGsjJwgQIECAAIH1AgIwaC4Ag7hGEyBAgAABAm0BAdimqy8KwNrICQIECBAgQGC9gAAMmgvAIK7RBAgQIECAQFtAALbp6osCsDZyggABAgQIEFgvIACD5gIwiGs0AQIECBAg0BYQgG26+qIArI2cIECAAAECBNYLCMCguQAM4hpNgAABAgQItAUEYJuuvigAayMnCBAgQIAAgfUCAjBoLgCDuEYTIECAAAECbQEB2KarLwrA2sgJAgQIECBAYL2AAAyaC8AgrtEECBAgQIBAW0AAtunqiwKwNnKCAAECBAgQWC8gAIPmAjCIazQBAgQIECDQFhCAbbr6ogCsjZwgQIAAAQIE1gsIwKC5AAziGk2AAAECBAi0BQRgm66+KABrIycIECBAgACB9QICMGguAIO4RhMgQIAAAQJtAQHYpqsvCsDayAkCBAgQIEBgvYAADJoLwCCu0QQIECBAgEBbQAC26eqLArA2coIAAQIECBBYLyAAg+YCMIhrNAECBAgQINAWEIBtuvqiAKyNnCBAgAABAgTWCwjAoLkADOIaTYAAAQIECLQFBGCbrr4oAGsjJwgQIECAAIH1AgIwaC4Ag7hGEyBAgAABAm0BAdimqy8KwNrICQIECBAgQGC9gAAMmgvAIK7RBAgQIECAQFtAALbp6osCsDZyggABAgQIEFgvIACD5gIwiGs0AQIECBAg0BYQgG26+qIArI2cIECAAAECBNYLCMCguQAM4hpNgAABAgQItAUEYJuuvigAayMnCBAgQIAAgfUCAjBoLgCDuEYTIECAAAECbQEB2KarLwrA2sgJAgQIECBAYL2AAAyaC8AgrtEECBAgQIBAW0AAtunqiwKwNnKCAAECBAgQWC8gAIPmAjCIazQBAgQIECDQFhCAbbr6ogCsjZwgQIAAAQIE1gsIwKC5AAziGk2AAAECBAi0BQRgm66+KABrIycIECBAgACB9QICMGguAIO4RhMgQIAAAQJtAQHYpqsvCsDayAkCBAgQIEBgvYAADJoLwCCu0QQIECBAgEBbQAC26eqLArA2coIAAQIECBBYLyAAg+YCMIhrNAECBAgQINAWEIBtuvqiAKyNnCBAgAABAgTWCwjAoLkADOIaTYAAAQIECLQFBGCbrr4oAGsjJwgQIECAAIH1AgIwaC4Ag7hGEyBAgAABAm0BAdimqy8KwNrICQIECBAgQGC9gAAMmgvAIK7RBAgQIECAQFtAALbp6osCsDZyggABAgQIEFgvIACD5gIwiGs0AQIECBAg0BYQgG26+qIArI2cIECAAAECBNYLCMCguQAM4hpNgAABAgQItAUEYJuuvigAayMnCBAgQIAAgfUCAjBoLgCDuEYTIECAAAECbQEB2KarLwrA2sgJAgQIECBAYL2AAAyaC8AgrtEECBAgQIBAW0AAtunqiwKwNnKCAAECBAgQWC8gAIPmAjCIazQBAgQIECDQFhCAbbr6ogCsjZwgQIAAAQIE1gsIwKC5AAziGk2AAAECBAi0BQRgm66+KABrIycIECBAgACB9QICMGguAIO4RhMgQIAAAQJtAQHYpqsvCsDayAkCBAgQIEBgvYAADJoLwCCu0QQIECBAgEBbQAC26eqLArA2coIAAQIECBBYLyAAg+YCMIhrNAECBAgQINAWEIBtuvqiAKyNnCBAgAABAgTWCwjAoLkADOIaTYAAAQIECLQFBGCbrr4oAGsjJwgQIECAAIH1AgIwaC4Ag7hGEyBAgAABAm0BAdimqy8KwNrICQIECBAgQGC9gAAMmgvAIK7RBAgQIECAQFtAALbp6osCsDZyggABAgQIEFgvIACD5gIwiGs0AQIECBAg0BYQgG26+qIArI2cIECAAAECBNYLCMCguQAM4hpNgAABAgQItAUEYJuuvigAayMnCBAgQIAAgfUCAjBoLgCDuEYTIECAAAECbQEB2KarLwrA2sgJAgQIECBAYL2AAAyaC8AgrtEECBAgQIBAW0AAtunqiwKwNnKCAAECBAgQWC8gAIPmAjCIazQBAgQIECDQFhCAbbr6ogCsjZwgQIAAAQIE1gsIwKC5AAziGk2AAAECBAi0BQRgm66+KABrIycIECBAgACB9QICMGguAIO4RhMgQIAAAQJtAQHYpqsvCsDayAkCBAgQIEBgvYAADJoLwCCu0QQIECBAgEBbQAC26eqLArA2coIAAQIECBBYLyAAg+YCMIhrNAECBAgQINAWEIBtuvqiAKyNnCBAgAABAgTWCwjAoLkADOIaTYAAAQIECLQFBGCbrr4oAGsjJwgQIECAAIH1AgIwaC4Ag7hGEyBAgAABAm0BAdimqy8KwNrICQIECBAgQGC9gAAMmgvAIK7RBAgQIECAQFtAALbp6osCsDZyggABAgQIEFgvIACD5gIwiGs0AQIECBAg0BYQgG26+qIArI2cIECAAAECBNYLCMCguQAM4hpNgAABAgQItAUEYJuuvigAayMnCBAgQIAAgfUCAjBoLgCDuEYTIECAAAECbQEB2KarLwrA2sgJAgQIECBAYL2AAAyaC8AgrtEECBAgQIBAW0AAtunqiwKwNnKCAAECBAgQWC8gAIPmAjCIazQBAgQIECDQFhCAbbr6ogCsjZwgQIAAAQIE1gsIwKC5AAziGk2AAAECBAi0BQRgm66+KABrIycIECBAgACB9QICMGguAIO4RhMgQIAAAQJtAQHYpqsvCsDayAkCBAgQIEBgvYAADJoLwCCu0QQIECBAgEBbQAC26eqLArA2coIAAQIECBBYLyAAg+YCMIhrNAECBAgQINAWEIBtuvqiAKyNnCBAgAABAgTWCwjAoLkADOIaTYAAAQIECLQFBGCbrr4oAGsjJwgQIECAAIH1AgIwaC4Ag7hGEyBAgAABAm0BAdimqy8KwNrICQIECBAgQGC9gAAMmgvAIK7RBAgQIECAQFtAALbp6osCsDZyggABAgQIEFgvIACD5gIwiGs0AQIECBAg0BYQgG26+qIArI2cIECAAAECBNYLCMCguQAM4hpNgAABAgQItAUEYJuuvigAayMnCBAgQIAAgfUCAjBoLgCDuEYTIECAAAECbQEB2KarLwrA2sgJAgR6Av/69cc3L/79h196A90iQGCUgAAMrlsABnGNJjBc4FsB+EcWMTj8I/F8Au8ICMDg5yEAg7hGExguIACHfwCeT+CTAgLwk4DvXReAQVyjCQwXEIDDPwDPJ/BJAQH4SUABGAQ0mgCBNwWuxN/jol8B+4AIEPiWgAAMfhv+BDCIazSBwQJXAlD8Df5APJ3ABQEBeAGpe0QAduXcI0DgPQEB6PsgQOCzAgLws4Lv3BeAQVyjCQwWEICDl+/pBJ4kIACfBPnWGAEYxDWawFCBK/H3oPEr4KEfiGcTuCggAC9CdY4JwI6aOwQIfPbXvwLQN0SAQCUgACuhT/xzAfgJPFcJEHhTwJ8A+jAIEHiGgAB8huI3ZgjAIK7RBIYKCMChi/dsAk8WEIBPBv3jOAEYxDWawECBq/HnV8ADPw5PJvBBAQH4QbCPHBeAH9FylgCBSuBqAPoPQCpJ/5wAAQEY/AYEYBDXaAIDBQTgwKV7MoGQgAAMwT7GCsAgrtEEBgoIwIFL92QCIQEBGIIVgEFYowkMFLgafw8avwIe+IF4MoEPCgjAD4J95Lg/AfyIlrMECLwncDUAxZ/viACBKwIC8IpS84wAbMK5RoDAXwQEoI+CAIFnCgjAZ2r+aZYADOIaTWCYgAActnDPJRAWEIBBYAEYxDWawDABAThs4Z5LICwgAIPAAjCIazSBYQICcNjCPZdAWEAABoEFYBDXaALDBK4EoP8AZNhH4bkEPiEgAD+BV10VgJWQf06AwFUBAXhVyjkCBK4ICMArSs0zArAJ5xoBAv8ncCX+Hhf8CaAPhwCBqwIC8KpU45wAbKC5QoDAXwSuBKD48+EQIPARAQH4Ea0PnhWAHwRznACBNwUEoA+DAIFnCwjAZ4v+YZ4ADOIaTWCQgAActGxPJbBIQAAGoQVgENdoAoMEBOCgZXsqgUUCAjAILQCDuEYTGCJwJf4eFP4O4JAPwjMJPElAAD4J8q0xAjCIazSBIQJXAlD8DfkYPJPAEwUE4BMx/zxKAAZxjSYwREAADlm0ZxJYLCAAg+ACMIhrNIEhAgJwyKI9k8BiAQEYBBeAQVyjCQwQuBJ/Dwa/Ah7wMXgigScLCMAng/5xnAAM4hpNYIDAlQAUfwM+BE8kEBAQgAHU30cKwCCu0QQGCAjAAUv2RAJfJCAAg/ACMIhrNIEBAgJwwJI9kcAXCQjAILwADOIaTeBwgSvx9yDwK+DDPwTPIxASEIAh2MdYARjENZrA4QJXAlD8Hf4ReB6BoIAADOIKwCCu0QQOFxCAhy/Y8wh8sYAADC5AAAZxjSZwuIAAPHzBnkfgiwUEYHABAjCIazSBgwWuxN/j+X4FfPBH4GkEwgICMAgsAIO4RhM4WOBKAIq/gz8ATyOwQEAABpEFYBDXaAIHCwjAg5fraQRuIiAAg4sQgEFcowkcLCAAD16upxG4iYAADC5CAAZxjSZwsIAAPHi5nkbgJgICMLgIARjENZrAoQJX4u/xdH8H8NAPwLMILBIQgEFoARjENZrAoQJXAlD8Hbp8zyKwUEAABrEFYBDXaAKHCgjAQxfrWQRuJiAAgwsRgEFcowkcKHAl/vz698DFexKBLxAQgEF0ARjENZrAgQJXAtCvfw9cvCcR+AIBARhEF4BBXKMJHCggAA9cqicRuKmAAAwuRgAGcY0mcKCAADxwqZ5E4KYCAjC4GAEYxDWawIECAvDApXoSgZsKCMDgYgRgENdoAocJXIm/x5P9HcDDFu85BL5IQAAG4QVgENdoAocJXAlA8XfY0j2HwBcKCMAgvgAM4hpN4DABAXjYQj2HwM0FBGBwQQIwiGs0gcMEBOBhC/UcAjcXEIDBBQnAIK7RBA4SuBJ/j+f6FfBBS/cUAl8sIACDCxCAQVyjCRwkcCUAxd9BC/cUAjcQEIDBJQjAIK7RBA4SEIAHLdNTCGwiIACDixKAQVyjCRwkIAAPWqanENhEQAAGFyUAg7hGEzhE4Er8PZ7qV8CHLNwzCNxEQAAGFyEAg7hGEzhEQAAeskjPILCZgAAMLkwABnGNJnCIgAA8ZJGeQWAzAQEYXJgADOIaTeAQAQF4yCI9g8BmAgIwuDABGMQ1msABAlfj7/FUfwfwgIV7AoEbCQjA4DIEYBDXaAIHCFwNQPF3wLI9gcDNBARgcCECMIhrNIEDBATgAUv0BAKbCgjA4OIEYBDXaAIHCAjAA5boCQQ2FRCAwcUJwCCu0QQOEBCAByzREwhsKiAAg4sTgEFcowkcIHAlAP39vwMW7QkEbiggAINLEYBBXKMJHCAgAA9YoicQ2FRAAAYXJwCDuEYTOEBAAB6wRE8gsKmAAAwuTgAGcY0msLnAlfh7PNGvgDdftB+fwE0FBGBwMQIwiGs0gc0FrgSg+Nt8yX58AjcWEIDB5QjAIK7RBDYXEICbL9CPT2BzAQEYXKAADOIaTWBzAQG4+QL9+AQ2FxCAwQUKwCCu0QQ2FxCAmy/Qj09gcwEBGFygAAziGk1gY4Er8fd4nr8DuPGS/egEbi4gAIMLEoBBXKMJbCxwJQDF38YL9qMT2EBAAAaXJACDuEYT2FhAAG68PD86gUMEBGBwkQIwiGs0gY0FBODGy/OjEzhEQAAGFykAg7hGE9hUQPxtujg/NoHDBARgcKECMIhrNIFNBQTgpovzYxM4TEAABhcqAIO4RhPYVEAAbro4PzaBwwQEYHChAjCIazSBTQUE4KaL82MTOExAAAYXKgCDuEYT2FDgSvw9nuX/AmbD5fqRCWwmIACDCxOAQVyjCWwocCUAxd+Gi/UjE9hQQAAGlyYAg7hGE9hQQABuuDQ/MoFDBQRgcLECMIhrNIENBQTghkvzIxM4VEAABhcrAIO4RhPYTOBK/D2e5FfAmy3Wj0tgUwEBGFycAAziGk1gM4ErASj+NluqH5fAxgICMLg8ARjENZrAZgICcLOF+XEJHC4gAIMLFoBBXKMJbCYgADdbmB+XwOECAjC4YAEYxDWawEYCV+Lv8Ry/At5oqX5UApsLCMDgAgVgENdoAhsJXAlA8bfRQv2oBA4QEIDBJQrAIK7RBDYSEIAbLcuPSmCIgAAMLloABnGNJrCRgADcaFl+VAJDBARgcNECMIhrNIFNBK7E3+MpfgW8yUL9mAQOERCAwUUKwCCu0QQ2EbgSgOJvk2X6MQkcJCAAg8sUgEFcowlsIiAAN1mUH5PAMAEBGFy4AAziGk0gJPDy8vLUyb/98kM57/sffy3PVAdeX1+rI/45AQIE/icgAIMfgwAM4hpNICQgAEOwxhIgcCsBARhchwAM4hpNICTwzAC88qd/j2f4E8DQMo0lQOCbAgIw+HEIwCCu0QRCAs8OwH/+++1fAf/jb//9te8z4u8xx6+AQx+EsQQOFRCAwcUKwCCu0QRCAs8MwJ9++u3dn/IRgQIwtEhjCRB4V0AABj8QARjENZpASOBZAVjF3+PHF4ChJRpLgEApIABLov4BAdi3c5PAVwmsDMCff/7+ac/0K+CnURpEYISAAAyuWQAGcY0mEBIQgCFYYwkQuJWAAAyuQwAGcY0mEBJ4VgA+/gvgb/0HIL//6P4EMLREYwkQKAUEYEnUPyAA+3ZuEvgqAQH4VfL+vQQIrBQQgEFtARjENZpASOCZAfj4Ed/7v4F51n8B/Pj3+DuAoQ/CWAKHCgjA4GIFYBDXaAIhgWcF4Fs/3p//j6EFYGiJxhIgUAoIwJKof0AA9u3cJPBVAskATL7JnwAmdc0mcJ6AAAzuVAAGcY0mEBIQgCFYYwkQuJWAAMyu4zU73nQCBAgQIECAQEvgpXXrCZe+7F/8hJ/90gh/AniJySECtxLwJ4C3WocfhgCBkIA/AQzBPsYKwCCu0QRCAgIwBGssAQK3EhCAwXUIwCCu0QRCAgIwBGssAQK3EhCAwXUIwCCu0QRCAgIwBGssAQK3EhCAwXUIwCCu0QRCAgIwBGssAQK3EhCAwXUIwCCu0QRCAgIwBGssAQK3EhCAwXUIwCCu0QRCAgIwBGssAQK3EhCAwXUIwCCu0QRCAgIwBGssAQK3EhCAwXUIwCCu0QRCAgIwBGssAQK3EhCAwXUIwCCu0QRCAgIwBGssAQK3EhCAwXUIwCCu0QRCAgIwBGssAQK3EhCAwXUIwCCu0QRCAgIwBGssAQK3EhCAwXUIwCCu0QRCAgIwBGssAQK3EhCAwXUIwCCu0QQIECBAgEBbQAC26eqLArA2coIAAQIECBBYLyAAg+YCMIhrNAECBAgQINAWEIBtuvqiAKyNnCBAgAABAgTWCwjAoLkADOIaTYAAAQIECLQFBGCbrr4oAGsjJwgQIECAAIH1AgIwaC4Ag7hGEyBAgAABAm0BAdimqy8KwNrICQIECBAgQGC9gAAMmgvAIK7RBAgQIECAQFtAALbp6osCsDZyggABAgQIEFgvIACD5gIwiGs0AQIECBAg0BYQgG26+qIArI2cIECAAAECBNYLCMCguQAM4hpNgAABAgQItAUEYJuuvigAayMnCBAgQIAAgfUCAjBoLgCDuEYTIECAAAECbQEB2KarLwrA2sgJAgQIECBAYL2AAAyaC8AgrtEECBAgQIBAW0AAtunqiwKwNnKCAAECBAgQWC8gAIPmAjCIazQBAgQIECDQFhCAbbr6ogCsjZwgQIAAAQIE1gsIwKC5AAziGk2AAAECBAi0BQRgm66+KABrIycIECBAgACB9QICMGguAIO4RhMgQIAAAQJtAQHYpqsvCsDayAkCBAgQIEBgvYAADJoLwCCu0QQIECBAgEBbQAC26eqLArA2coIAAQIECBBYLyAAg+YCMIhrNAECBAgQINAWEIBtuvqiAKyNnCBAgAABAgTWCwjAoLkADOIaTYAAAQIECLQFBGCbrr4oAGsjJwgQIECAAIH1AgIwaC4Ag7hGEyBAgAABAm0BAdimqy8KwNrICQIECBAgQGC9gAAMmgvAIK7RBAgQIECAQFtAALbp6osCsDZyggABAgQIEFgvIACD5gIwiGs0AQIECBAg0BYQgG26+qIArI2cIECAAAECBNYLCMCguQAM4hpNgAABAgQItAUEYJuuvigAayMnCBAgQIAAgfUCAjBoLgCDuEYTIECAAAECbQEB2KarLwrA2sgJAgQIECBAYL2AAAyaC8AgrtEECBAgQIBAW0AAtunqiwKwNnKCAAECBAgQWC8gAIPmAjCIazQBAgQIECDQFhCAbbr6ogCsjZwgQIAAAQIE1gsIwKC5AAziGk2AAAECBAi0BQRgm66+KABrIycIECBAgACB9QICMGguAIO4RhMgQIAAAQJtAQHYpqsvCsDayAkCBAgQIEBgvYAADJoLwCCu0QQIECBAgEBbQAC26eqLArA2coIAAQIECBBYLyAAg+YCMIhrNAECBAgQINAWEIBtuvqiAKyNnCBAgAABAgTWCwjAoLkADOIaTYAAAQIECLQFBGCbrr4oAGsjJwgQIECAAIH1AgIwaC4Ag7hGEyBAgAABAm0BAdimqy8KwNrICQIECBAgQGC9gAAMmgvAIK7RBAgQIECAQFtAALbp6osCsDZyggABAgQIEFgvIACD5gIwiGs0AQIECBAg0BYQgG26+qIArI2cIECAAAECBNYLCMCguQAM4hpNgAABAgQItAUEYJuuvigAayMnCBAgQIAAgfUCAjBoLgCDuEYTIECAAAECbQEB2KarLwrA2sgJAgQIECBAYL2AAAyaC8AgrtEECBAgQIBAW0AAtunqiwKwNnKCAAECBAgQWC8gAIPmAjCIazQBAgQIECDQFhCAbbr6ogCsjZwgQIAAAQIE1gsIwKC5AAziGk2AAAECBAi0BQRgm66+KABrIycIECBAgACB9QICMGguAIO4RhMgQIAAAQJtAQHYpqsvCsDayAkCBAgQIEBgvYAADJoLwCCu0QQIECBAgEBbQAC26eqLArA2coIAAQIECBBYLyAAg+YCMIhrNAECBAgQINAWEIBtuvqiAKyNnCBAgAABAgTWCwjAoLkADOIaTYAAAQIECLQFBGCbrr4oAGsjJwgQIECAAIH1AgJwvbl/IwECBAgQIEBgrMDL2Mb2jvEAAANcSURBVJd7OAECBAgQIEBgqIAAHLp4zyZAgAABAgTmCgjAubv3cgIECBAgQGCogAAcunjPJkCAAAECBOYKCMC5u/dyAgQIECBAYKiAABy6eM8mQIAAAQIE5goIwLm793ICBAgQIEBgqIAAHLp4zyZAgAABAgTmCgjAubv3cgIECBAgQGCogAAcunjPJkCAAAECBOYKCMC5u/dyAgQIECBAYKiAABy6eM8mQIAAAQIE5goIwLm793ICBAgQIEBgqIAAHLp4zyZAgAABAgTmCgjAubv3cgIECBAgQGCogAAcunjPJkCAAAECBOYKCMC5u/dyAgQIECBAYKiAABy6eM8mQIAAAQIE5goIwLm793ICBAgQIEBgqIAAHLp4zyZAgAABAgTmCgjAubv3cgIECBAgQGCogAAcunjPJkCAAAECBOYKCMC5u/dyAgQIECBAYKiAABy6eM8mQIAAAQIE5goIwLm793ICBAgQIEBgqIAAHLp4zyZAgAABAgTmCgjAubv3cgIECBAgQGCogAAcunjPJkCAAAECBOYKCMC5u/dyAgQIECBAYKiAABy6eM8mQIAAAQIE5goIwLm793ICBAgQIEBgqIAAHLp4zyZAgAABAgTmCgjAubv3cgIECBAgQGCogAAcunjPJkCAAAECBOYKCMC5u/dyAgQIECBAYKiAABy6eM8mQIAAAQIE5goIwLm793ICBAgQIEBgqIAAHLp4zyZAgAABAgTmCgjAubv3cgIECBAgQGCogAAcunjPJkCAAAECBOYKCMC5u/dyAgQIECBAYKiAABy6eM8mQIAAAQIE5goIwLm793ICBAgQIEBgqIAAHLp4zyZAgAABAgTmCgjAubv3cgIECBAgQGCogAAcunjPJkCAAAECBOYKCMC5u/dyAgQIECBAYKiAABy6eM8mQIAAAQIE5goIwLm793ICBAgQIEBgqIAAHLp4zyZAgAABAgTmCgjAubv3cgIECBAgQGCogAAcunjPJkCAAAECBOYKCMC5u/dyAgQIECBAYKiAABy6eM8mQIAAAQIE5goIwLm793ICBAgQIEBgqIAAHLp4zyZAgAABAgTmCgjAubv3cgIECBAgQGCogAAcunjPJkCAAAECBOYKCMC5u/dyAgQIECBAYKjAfwACtXJKp6oC7gAAAABJRU5ErkJggg==\" width=\"640\">"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Saving figure cart_pole_plot\n"
     ]
    }
   ],
   "source": [
    "plt.close()  # or else nbagg sometimes plots in the previous cell\n",
    "img = render_cart_pole(env, obs)\n",
    "plt.imshow(img)\n",
    "plt.axis(\"off\")\n",
    "save_fig(\"cart_pole_plot\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(400, 600, 3)"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "img.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice that the game is over when the pole tilts too much, not when it actually falls. Now let's reset the environment and push the cart to right instead:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "obs = env.reset()\n",
    "while True:\n",
    "    obs, reward, done, info = env.step(1)\n",
    "    if done:\n",
    "        break"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/javascript": [
       "/* Put everything inside the global mpl namespace */\n",
       "window.mpl = {};\n",
       "\n",
       "\n",
       "mpl.get_websocket_type = function() {\n",
       "    if (typeof(WebSocket) !== 'undefined') {\n",
       "        return WebSocket;\n",
       "    } else if (typeof(MozWebSocket) !== 'undefined') {\n",
       "        return MozWebSocket;\n",
       "    } else {\n",
       "        alert('Your browser does not have WebSocket support.' +\n",
       "              'Please try Chrome, Safari or Firefox ≥ 6. ' +\n",
       "              'Firefox 4 and 5 are also supported but you ' +\n",
       "              'have to enable WebSockets in about:config.');\n",
       "    };\n",
       "}\n",
       "\n",
       "mpl.figure = function(figure_id, websocket, ondownload, parent_element) {\n",
       "    this.id = figure_id;\n",
       "\n",
       "    this.ws = websocket;\n",
       "\n",
       "    this.supports_binary = (this.ws.binaryType != undefined);\n",
       "\n",
       "    if (!this.supports_binary) {\n",
       "        var warnings = document.getElementById(\"mpl-warnings\");\n",
       "        if (warnings) {\n",
       "            warnings.style.display = 'block';\n",
       "            warnings.textContent = (\n",
       "                \"This browser does not support binary websocket messages. \" +\n",
       "                    \"Performance may be slow.\");\n",
       "        }\n",
       "    }\n",
       "\n",
       "    this.imageObj = new Image();\n",
       "\n",
       "    this.context = undefined;\n",
       "    this.message = undefined;\n",
       "    this.canvas = undefined;\n",
       "    this.rubberband_canvas = undefined;\n",
       "    this.rubberband_context = undefined;\n",
       "    this.format_dropdown = undefined;\n",
       "\n",
       "    this.image_mode = 'full';\n",
       "\n",
       "    this.root = $('<div/>');\n",
       "    this._root_extra_style(this.root)\n",
       "    this.root.attr('style', 'display: inline-block');\n",
       "\n",
       "    $(parent_element).append(this.root);\n",
       "\n",
       "    this._init_header(this);\n",
       "    this._init_canvas(this);\n",
       "    this._init_toolbar(this);\n",
       "\n",
       "    var fig = this;\n",
       "\n",
       "    this.waiting = false;\n",
       "\n",
       "    this.ws.onopen =  function () {\n",
       "            fig.send_message(\"supports_binary\", {value: fig.supports_binary});\n",
       "            fig.send_message(\"send_image_mode\", {});\n",
       "            if (mpl.ratio != 1) {\n",
       "                fig.send_message(\"set_dpi_ratio\", {'dpi_ratio': mpl.ratio});\n",
       "            }\n",
       "            fig.send_message(\"refresh\", {});\n",
       "        }\n",
       "\n",
       "    this.imageObj.onload = function() {\n",
       "            if (fig.image_mode == 'full') {\n",
       "                // Full images could contain transparency (where diff images\n",
       "                // almost always do), so we need to clear the canvas so that\n",
       "                // there is no ghosting.\n",
       "                fig.context.clearRect(0, 0, fig.canvas.width, fig.canvas.height);\n",
       "            }\n",
       "            fig.context.drawImage(fig.imageObj, 0, 0);\n",
       "        };\n",
       "\n",
       "    this.imageObj.onunload = function() {\n",
       "        this.ws.close();\n",
       "    }\n",
       "\n",
       "    this.ws.onmessage = this._make_on_message_function(this);\n",
       "\n",
       "    this.ondownload = ondownload;\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._init_header = function() {\n",
       "    var titlebar = $(\n",
       "        '<div class=\"ui-dialog-titlebar ui-widget-header ui-corner-all ' +\n",
       "        'ui-helper-clearfix\"/>');\n",
       "    var titletext = $(\n",
       "        '<div class=\"ui-dialog-title\" style=\"width: 100%; ' +\n",
       "        'text-align: center; padding: 3px;\"/>');\n",
       "    titlebar.append(titletext)\n",
       "    this.root.append(titlebar);\n",
       "    this.header = titletext[0];\n",
       "}\n",
       "\n",
       "\n",
       "\n",
       "mpl.figure.prototype._canvas_extra_style = function(canvas_div) {\n",
       "\n",
       "}\n",
       "\n",
       "\n",
       "mpl.figure.prototype._root_extra_style = function(canvas_div) {\n",
       "\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._init_canvas = function() {\n",
       "    var fig = this;\n",
       "\n",
       "    var canvas_div = $('<div/>');\n",
       "\n",
       "    canvas_div.attr('style', 'position: relative; clear: both; outline: 0');\n",
       "\n",
       "    function canvas_keyboard_event(event) {\n",
       "        return fig.key_event(event, event['data']);\n",
       "    }\n",
       "\n",
       "    canvas_div.keydown('key_press', canvas_keyboard_event);\n",
       "    canvas_div.keyup('key_release', canvas_keyboard_event);\n",
       "    this.canvas_div = canvas_div\n",
       "    this._canvas_extra_style(canvas_div)\n",
       "    this.root.append(canvas_div);\n",
       "\n",
       "    var canvas = $('<canvas/>');\n",
       "    canvas.addClass('mpl-canvas');\n",
       "    canvas.attr('style', \"left: 0; top: 0; z-index: 0; outline: 0\")\n",
       "\n",
       "    this.canvas = canvas[0];\n",
       "    this.context = canvas[0].getContext(\"2d\");\n",
       "\n",
       "    var backingStore = this.context.backingStorePixelRatio ||\n",
       "\tthis.context.webkitBackingStorePixelRatio ||\n",
       "\tthis.context.mozBackingStorePixelRatio ||\n",
       "\tthis.context.msBackingStorePixelRatio ||\n",
       "\tthis.context.oBackingStorePixelRatio ||\n",
       "\tthis.context.backingStorePixelRatio || 1;\n",
       "\n",
       "    mpl.ratio = (window.devicePixelRatio || 1) / backingStore;\n",
       "\n",
       "    var rubberband = $('<canvas/>');\n",
       "    rubberband.attr('style', \"position: absolute; left: 0; top: 0; z-index: 1;\")\n",
       "\n",
       "    var pass_mouse_events = true;\n",
       "\n",
       "    canvas_div.resizable({\n",
       "        start: function(event, ui) {\n",
       "            pass_mouse_events = false;\n",
       "        },\n",
       "        resize: function(event, ui) {\n",
       "            fig.request_resize(ui.size.width, ui.size.height);\n",
       "        },\n",
       "        stop: function(event, ui) {\n",
       "            pass_mouse_events = true;\n",
       "            fig.request_resize(ui.size.width, ui.size.height);\n",
       "        },\n",
       "    });\n",
       "\n",
       "    function mouse_event_fn(event) {\n",
       "        if (pass_mouse_events)\n",
       "            return fig.mouse_event(event, event['data']);\n",
       "    }\n",
       "\n",
       "    rubberband.mousedown('button_press', mouse_event_fn);\n",
       "    rubberband.mouseup('button_release', mouse_event_fn);\n",
       "    // Throttle sequential mouse events to 1 every 20ms.\n",
       "    rubberband.mousemove('motion_notify', mouse_event_fn);\n",
       "\n",
       "    rubberband.mouseenter('figure_enter', mouse_event_fn);\n",
       "    rubberband.mouseleave('figure_leave', mouse_event_fn);\n",
       "\n",
       "    canvas_div.on(\"wheel\", function (event) {\n",
       "        event = event.originalEvent;\n",
       "        event['data'] = 'scroll'\n",
       "        if (event.deltaY < 0) {\n",
       "            event.step = 1;\n",
       "        } else {\n",
       "            event.step = -1;\n",
       "        }\n",
       "        mouse_event_fn(event);\n",
       "    });\n",
       "\n",
       "    canvas_div.append(canvas);\n",
       "    canvas_div.append(rubberband);\n",
       "\n",
       "    this.rubberband = rubberband;\n",
       "    this.rubberband_canvas = rubberband[0];\n",
       "    this.rubberband_context = rubberband[0].getContext(\"2d\");\n",
       "    this.rubberband_context.strokeStyle = \"#000000\";\n",
       "\n",
       "    this._resize_canvas = function(width, height) {\n",
       "        // Keep the size of the canvas, canvas container, and rubber band\n",
       "        // canvas in synch.\n",
       "        canvas_div.css('width', width)\n",
       "        canvas_div.css('height', height)\n",
       "\n",
       "        canvas.attr('width', width * mpl.ratio);\n",
       "        canvas.attr('height', height * mpl.ratio);\n",
       "        canvas.attr('style', 'width: ' + width + 'px; height: ' + height + 'px;');\n",
       "\n",
       "        rubberband.attr('width', width);\n",
       "        rubberband.attr('height', height);\n",
       "    }\n",
       "\n",
       "    // Set the figure to an initial 600x600px, this will subsequently be updated\n",
       "    // upon first draw.\n",
       "    this._resize_canvas(600, 600);\n",
       "\n",
       "    // Disable right mouse context menu.\n",
       "    $(this.rubberband_canvas).bind(\"contextmenu\",function(e){\n",
       "        return false;\n",
       "    });\n",
       "\n",
       "    function set_focus () {\n",
       "        canvas.focus();\n",
       "        canvas_div.focus();\n",
       "    }\n",
       "\n",
       "    window.setTimeout(set_focus, 100);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._init_toolbar = function() {\n",
       "    var fig = this;\n",
       "\n",
       "    var nav_element = $('<div/>')\n",
       "    nav_element.attr('style', 'width: 100%');\n",
       "    this.root.append(nav_element);\n",
       "\n",
       "    // Define a callback function for later on.\n",
       "    function toolbar_event(event) {\n",
       "        return fig.toolbar_button_onclick(event['data']);\n",
       "    }\n",
       "    function toolbar_mouse_event(event) {\n",
       "        return fig.toolbar_button_onmouseover(event['data']);\n",
       "    }\n",
       "\n",
       "    for(var toolbar_ind in mpl.toolbar_items) {\n",
       "        var name = mpl.toolbar_items[toolbar_ind][0];\n",
       "        var tooltip = mpl.toolbar_items[toolbar_ind][1];\n",
       "        var image = mpl.toolbar_items[toolbar_ind][2];\n",
       "        var method_name = mpl.toolbar_items[toolbar_ind][3];\n",
       "\n",
       "        if (!name) {\n",
       "            // put a spacer in here.\n",
       "            continue;\n",
       "        }\n",
       "        var button = $('<button/>');\n",
       "        button.addClass('ui-button ui-widget ui-state-default ui-corner-all ' +\n",
       "                        'ui-button-icon-only');\n",
       "        button.attr('role', 'button');\n",
       "        button.attr('aria-disabled', 'false');\n",
       "        button.click(method_name, toolbar_event);\n",
       "        button.mouseover(tooltip, toolbar_mouse_event);\n",
       "\n",
       "        var icon_img = $('<span/>');\n",
       "        icon_img.addClass('ui-button-icon-primary ui-icon');\n",
       "        icon_img.addClass(image);\n",
       "        icon_img.addClass('ui-corner-all');\n",
       "\n",
       "        var tooltip_span = $('<span/>');\n",
       "        tooltip_span.addClass('ui-button-text');\n",
       "        tooltip_span.html(tooltip);\n",
       "\n",
       "        button.append(icon_img);\n",
       "        button.append(tooltip_span);\n",
       "\n",
       "        nav_element.append(button);\n",
       "    }\n",
       "\n",
       "    var fmt_picker_span = $('<span/>');\n",
       "\n",
       "    var fmt_picker = $('<select/>');\n",
       "    fmt_picker.addClass('mpl-toolbar-option ui-widget ui-widget-content');\n",
       "    fmt_picker_span.append(fmt_picker);\n",
       "    nav_element.append(fmt_picker_span);\n",
       "    this.format_dropdown = fmt_picker[0];\n",
       "\n",
       "    for (var ind in mpl.extensions) {\n",
       "        var fmt = mpl.extensions[ind];\n",
       "        var option = $(\n",
       "            '<option/>', {selected: fmt === mpl.default_extension}).html(fmt);\n",
       "        fmt_picker.append(option)\n",
       "    }\n",
       "\n",
       "    // Add hover states to the ui-buttons\n",
       "    $( \".ui-button\" ).hover(\n",
       "        function() { $(this).addClass(\"ui-state-hover\");},\n",
       "        function() { $(this).removeClass(\"ui-state-hover\");}\n",
       "    );\n",
       "\n",
       "    var status_bar = $('<span class=\"mpl-message\"/>');\n",
       "    nav_element.append(status_bar);\n",
       "    this.message = status_bar[0];\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.request_resize = function(x_pixels, y_pixels) {\n",
       "    // Request matplotlib to resize the figure. Matplotlib will then trigger a resize in the client,\n",
       "    // which will in turn request a refresh of the image.\n",
       "    this.send_message('resize', {'width': x_pixels, 'height': y_pixels});\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.send_message = function(type, properties) {\n",
       "    properties['type'] = type;\n",
       "    properties['figure_id'] = this.id;\n",
       "    this.ws.send(JSON.stringify(properties));\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.send_draw_message = function() {\n",
       "    if (!this.waiting) {\n",
       "        this.waiting = true;\n",
       "        this.ws.send(JSON.stringify({type: \"draw\", figure_id: this.id}));\n",
       "    }\n",
       "}\n",
       "\n",
       "\n",
       "mpl.figure.prototype.handle_save = function(fig, msg) {\n",
       "    var format_dropdown = fig.format_dropdown;\n",
       "    var format = format_dropdown.options[format_dropdown.selectedIndex].value;\n",
       "    fig.ondownload(fig, format);\n",
       "}\n",
       "\n",
       "\n",
       "mpl.figure.prototype.handle_resize = function(fig, msg) {\n",
       "    var size = msg['size'];\n",
       "    if (size[0] != fig.canvas.width || size[1] != fig.canvas.height) {\n",
       "        fig._resize_canvas(size[0], size[1]);\n",
       "        fig.send_message(\"refresh\", {});\n",
       "    };\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_rubberband = function(fig, msg) {\n",
       "    var x0 = msg['x0'] / mpl.ratio;\n",
       "    var y0 = (fig.canvas.height - msg['y0']) / mpl.ratio;\n",
       "    var x1 = msg['x1'] / mpl.ratio;\n",
       "    var y1 = (fig.canvas.height - msg['y1']) / mpl.ratio;\n",
       "    x0 = Math.floor(x0) + 0.5;\n",
       "    y0 = Math.floor(y0) + 0.5;\n",
       "    x1 = Math.floor(x1) + 0.5;\n",
       "    y1 = Math.floor(y1) + 0.5;\n",
       "    var min_x = Math.min(x0, x1);\n",
       "    var min_y = Math.min(y0, y1);\n",
       "    var width = Math.abs(x1 - x0);\n",
       "    var height = Math.abs(y1 - y0);\n",
       "\n",
       "    fig.rubberband_context.clearRect(\n",
       "        0, 0, fig.canvas.width, fig.canvas.height);\n",
       "\n",
       "    fig.rubberband_context.strokeRect(min_x, min_y, width, height);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_figure_label = function(fig, msg) {\n",
       "    // Updates the figure title.\n",
       "    fig.header.textContent = msg['label'];\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_cursor = function(fig, msg) {\n",
       "    var cursor = msg['cursor'];\n",
       "    switch(cursor)\n",
       "    {\n",
       "    case 0:\n",
       "        cursor = 'pointer';\n",
       "        break;\n",
       "    case 1:\n",
       "        cursor = 'default';\n",
       "        break;\n",
       "    case 2:\n",
       "        cursor = 'crosshair';\n",
       "        break;\n",
       "    case 3:\n",
       "        cursor = 'move';\n",
       "        break;\n",
       "    }\n",
       "    fig.rubberband_canvas.style.cursor = cursor;\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_message = function(fig, msg) {\n",
       "    fig.message.textContent = msg['message'];\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_draw = function(fig, msg) {\n",
       "    // Request the server to send over a new figure.\n",
       "    fig.send_draw_message();\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_image_mode = function(fig, msg) {\n",
       "    fig.image_mode = msg['mode'];\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.updated_canvas_event = function() {\n",
       "    // Called whenever the canvas gets updated.\n",
       "    this.send_message(\"ack\", {});\n",
       "}\n",
       "\n",
       "// A function to construct a web socket function for onmessage handling.\n",
       "// Called in the figure constructor.\n",
       "mpl.figure.prototype._make_on_message_function = function(fig) {\n",
       "    return function socket_on_message(evt) {\n",
       "        if (evt.data instanceof Blob) {\n",
       "            /* FIXME: We get \"Resource interpreted as Image but\n",
       "             * transferred with MIME type text/plain:\" errors on\n",
       "             * Chrome.  But how to set the MIME type?  It doesn't seem\n",
       "             * to be part of the websocket stream */\n",
       "            evt.data.type = \"image/png\";\n",
       "\n",
       "            /* Free the memory for the previous frames */\n",
       "            if (fig.imageObj.src) {\n",
       "                (window.URL || window.webkitURL).revokeObjectURL(\n",
       "                    fig.imageObj.src);\n",
       "            }\n",
       "\n",
       "            fig.imageObj.src = (window.URL || window.webkitURL).createObjectURL(\n",
       "                evt.data);\n",
       "            fig.updated_canvas_event();\n",
       "            fig.waiting = false;\n",
       "            return;\n",
       "        }\n",
       "        else if (typeof evt.data === 'string' && evt.data.slice(0, 21) == \"data:image/png;base64\") {\n",
       "            fig.imageObj.src = evt.data;\n",
       "            fig.updated_canvas_event();\n",
       "            fig.waiting = false;\n",
       "            return;\n",
       "        }\n",
       "\n",
       "        var msg = JSON.parse(evt.data);\n",
       "        var msg_type = msg['type'];\n",
       "\n",
       "        // Call the  \"handle_{type}\" callback, which takes\n",
       "        // the figure and JSON message as its only arguments.\n",
       "        try {\n",
       "            var callback = fig[\"handle_\" + msg_type];\n",
       "        } catch (e) {\n",
       "            console.log(\"No handler for the '\" + msg_type + \"' message type: \", msg);\n",
       "            return;\n",
       "        }\n",
       "\n",
       "        if (callback) {\n",
       "            try {\n",
       "                // console.log(\"Handling '\" + msg_type + \"' message: \", msg);\n",
       "                callback(fig, msg);\n",
       "            } catch (e) {\n",
       "                console.log(\"Exception inside the 'handler_\" + msg_type + \"' callback:\", e, e.stack, msg);\n",
       "            }\n",
       "        }\n",
       "    };\n",
       "}\n",
       "\n",
       "// from http://stackoverflow.com/questions/1114465/getting-mouse-location-in-canvas\n",
       "mpl.findpos = function(e) {\n",
       "    //this section is from http://www.quirksmode.org/js/events_properties.html\n",
       "    var targ;\n",
       "    if (!e)\n",
       "        e = window.event;\n",
       "    if (e.target)\n",
       "        targ = e.target;\n",
       "    else if (e.srcElement)\n",
       "        targ = e.srcElement;\n",
       "    if (targ.nodeType == 3) // defeat Safari bug\n",
       "        targ = targ.parentNode;\n",
       "\n",
       "    // jQuery normalizes the pageX and pageY\n",
       "    // pageX,Y are the mouse positions relative to the document\n",
       "    // offset() returns the position of the element relative to the document\n",
       "    var x = e.pageX - $(targ).offset().left;\n",
       "    var y = e.pageY - $(targ).offset().top;\n",
       "\n",
       "    return {\"x\": x, \"y\": y};\n",
       "};\n",
       "\n",
       "/*\n",
       " * return a copy of an object with only non-object keys\n",
       " * we need this to avoid circular references\n",
       " * http://stackoverflow.com/a/24161582/3208463\n",
       " */\n",
       "function simpleKeys (original) {\n",
       "  return Object.keys(original).reduce(function (obj, key) {\n",
       "    if (typeof original[key] !== 'object')\n",
       "        obj[key] = original[key]\n",
       "    return obj;\n",
       "  }, {});\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.mouse_event = function(event, name) {\n",
       "    var canvas_pos = mpl.findpos(event)\n",
       "\n",
       "    if (name === 'button_press')\n",
       "    {\n",
       "        this.canvas.focus();\n",
       "        this.canvas_div.focus();\n",
       "    }\n",
       "\n",
       "    var x = canvas_pos.x * mpl.ratio;\n",
       "    var y = canvas_pos.y * mpl.ratio;\n",
       "\n",
       "    this.send_message(name, {x: x, y: y, button: event.button,\n",
       "                             step: event.step,\n",
       "                             guiEvent: simpleKeys(event)});\n",
       "\n",
       "    /* This prevents the web browser from automatically changing to\n",
       "     * the text insertion cursor when the button is pressed.  We want\n",
       "     * to control all of the cursor setting manually through the\n",
       "     * 'cursor' event from matplotlib */\n",
       "    event.preventDefault();\n",
       "    return false;\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._key_event_extra = function(event, name) {\n",
       "    // Handle any extra behaviour associated with a key event\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.key_event = function(event, name) {\n",
       "\n",
       "    // Prevent repeat events\n",
       "    if (name == 'key_press')\n",
       "    {\n",
       "        if (event.which === this._key)\n",
       "            return;\n",
       "        else\n",
       "            this._key = event.which;\n",
       "    }\n",
       "    if (name == 'key_release')\n",
       "        this._key = null;\n",
       "\n",
       "    var value = '';\n",
       "    if (event.ctrlKey && event.which != 17)\n",
       "        value += \"ctrl+\";\n",
       "    if (event.altKey && event.which != 18)\n",
       "        value += \"alt+\";\n",
       "    if (event.shiftKey && event.which != 16)\n",
       "        value += \"shift+\";\n",
       "\n",
       "    value += 'k';\n",
       "    value += event.which.toString();\n",
       "\n",
       "    this._key_event_extra(event, name);\n",
       "\n",
       "    this.send_message(name, {key: value,\n",
       "                             guiEvent: simpleKeys(event)});\n",
       "    return false;\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.toolbar_button_onclick = function(name) {\n",
       "    if (name == 'download') {\n",
       "        this.handle_save(this, null);\n",
       "    } else {\n",
       "        this.send_message(\"toolbar_button\", {name: name});\n",
       "    }\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.toolbar_button_onmouseover = function(tooltip) {\n",
       "    this.message.textContent = tooltip;\n",
       "};\n",
       "mpl.toolbar_items = [[\"Home\", \"Reset original view\", \"fa fa-home icon-home\", \"home\"], [\"Back\", \"Back to  previous view\", \"fa fa-arrow-left icon-arrow-left\", \"back\"], [\"Forward\", \"Forward to next view\", \"fa fa-arrow-right icon-arrow-right\", \"forward\"], [\"\", \"\", \"\", \"\"], [\"Pan\", \"Pan axes with left mouse, zoom with right\", \"fa fa-arrows icon-move\", \"pan\"], [\"Zoom\", \"Zoom to rectangle\", \"fa fa-square-o icon-check-empty\", \"zoom\"], [\"\", \"\", \"\", \"\"], [\"Download\", \"Download plot\", \"fa fa-floppy-o icon-save\", \"download\"]];\n",
       "\n",
       "mpl.extensions = [\"eps\", \"jpeg\", \"pdf\", \"png\", \"ps\", \"raw\", \"svg\", \"tif\"];\n",
       "\n",
       "mpl.default_extension = \"png\";var comm_websocket_adapter = function(comm) {\n",
       "    // Create a \"websocket\"-like object which calls the given IPython comm\n",
       "    // object with the appropriate methods. Currently this is a non binary\n",
       "    // socket, so there is still some room for performance tuning.\n",
       "    var ws = {};\n",
       "\n",
       "    ws.close = function() {\n",
       "        comm.close()\n",
       "    };\n",
       "    ws.send = function(m) {\n",
       "        //console.log('sending', m);\n",
       "        comm.send(m);\n",
       "    };\n",
       "    // Register the callback with on_msg.\n",
       "    comm.on_msg(function(msg) {\n",
       "        //console.log('receiving', msg['content']['data'], msg);\n",
       "        // Pass the mpl event to the overriden (by mpl) onmessage function.\n",
       "        ws.onmessage(msg['content']['data'])\n",
       "    });\n",
       "    return ws;\n",
       "}\n",
       "\n",
       "mpl.mpl_figure_comm = function(comm, msg) {\n",
       "    // This is the function which gets called when the mpl process\n",
       "    // starts-up an IPython Comm through the \"matplotlib\" channel.\n",
       "\n",
       "    var id = msg.content.data.id;\n",
       "    // Get hold of the div created by the display call when the Comm\n",
       "    // socket was opened in Python.\n",
       "    var element = $(\"#\" + id);\n",
       "    var ws_proxy = comm_websocket_adapter(comm)\n",
       "\n",
       "    function ondownload(figure, format) {\n",
       "        window.open(figure.imageObj.src);\n",
       "    }\n",
       "\n",
       "    var fig = new mpl.figure(id, ws_proxy,\n",
       "                           ondownload,\n",
       "                           element.get(0));\n",
       "\n",
       "    // Call onopen now - mpl needs it, as it is assuming we've passed it a real\n",
       "    // web socket which is closed, not our websocket->open comm proxy.\n",
       "    ws_proxy.onopen();\n",
       "\n",
       "    fig.parent_element = element.get(0);\n",
       "    fig.cell_info = mpl.find_output_cell(\"<div id='\" + id + \"'></div>\");\n",
       "    if (!fig.cell_info) {\n",
       "        console.error(\"Failed to find cell for figure\", id, fig);\n",
       "        return;\n",
       "    }\n",
       "\n",
       "    var output_index = fig.cell_info[2]\n",
       "    var cell = fig.cell_info[0];\n",
       "\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_close = function(fig, msg) {\n",
       "    var width = fig.canvas.width/mpl.ratio\n",
       "    fig.root.unbind('remove')\n",
       "\n",
       "    // Update the output cell to use the data from the current canvas.\n",
       "    fig.push_to_output();\n",
       "    var dataURL = fig.canvas.toDataURL();\n",
       "    // Re-enable the keyboard manager in IPython - without this line, in FF,\n",
       "    // the notebook keyboard shortcuts fail.\n",
       "    IPython.keyboard_manager.enable()\n",
       "    $(fig.parent_element).html('<img src=\"' + dataURL + '\" width=\"' + width + '\">');\n",
       "    fig.close_ws(fig, msg);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.close_ws = function(fig, msg){\n",
       "    fig.send_message('closing', msg);\n",
       "    // fig.ws.close()\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.push_to_output = function(remove_interactive) {\n",
       "    // Turn the data on the canvas into data in the output cell.\n",
       "    var width = this.canvas.width/mpl.ratio\n",
       "    var dataURL = this.canvas.toDataURL();\n",
       "    this.cell_info[1]['text/html'] = '<img src=\"' + dataURL + '\" width=\"' + width + '\">';\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.updated_canvas_event = function() {\n",
       "    // Tell IPython that the notebook contents must change.\n",
       "    IPython.notebook.set_dirty(true);\n",
       "    this.send_message(\"ack\", {});\n",
       "    var fig = this;\n",
       "    // Wait a second, then push the new image to the DOM so\n",
       "    // that it is saved nicely (might be nice to debounce this).\n",
       "    setTimeout(function () { fig.push_to_output() }, 1000);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._init_toolbar = function() {\n",
       "    var fig = this;\n",
       "\n",
       "    var nav_element = $('<div/>')\n",
       "    nav_element.attr('style', 'width: 100%');\n",
       "    this.root.append(nav_element);\n",
       "\n",
       "    // Define a callback function for later on.\n",
       "    function toolbar_event(event) {\n",
       "        return fig.toolbar_button_onclick(event['data']);\n",
       "    }\n",
       "    function toolbar_mouse_event(event) {\n",
       "        return fig.toolbar_button_onmouseover(event['data']);\n",
       "    }\n",
       "\n",
       "    for(var toolbar_ind in mpl.toolbar_items){\n",
       "        var name = mpl.toolbar_items[toolbar_ind][0];\n",
       "        var tooltip = mpl.toolbar_items[toolbar_ind][1];\n",
       "        var image = mpl.toolbar_items[toolbar_ind][2];\n",
       "        var method_name = mpl.toolbar_items[toolbar_ind][3];\n",
       "\n",
       "        if (!name) { continue; };\n",
       "\n",
       "        var button = $('<button class=\"btn btn-default\" href=\"#\" title=\"' + name + '\"><i class=\"fa ' + image + ' fa-lg\"></i></button>');\n",
       "        button.click(method_name, toolbar_event);\n",
       "        button.mouseover(tooltip, toolbar_mouse_event);\n",
       "        nav_element.append(button);\n",
       "    }\n",
       "\n",
       "    // Add the status bar.\n",
       "    var status_bar = $('<span class=\"mpl-message\" style=\"text-align:right; float: right;\"/>');\n",
       "    nav_element.append(status_bar);\n",
       "    this.message = status_bar[0];\n",
       "\n",
       "    // Add the close button to the window.\n",
       "    var buttongrp = $('<div class=\"btn-group inline pull-right\"></div>');\n",
       "    var button = $('<button class=\"btn btn-mini btn-primary\" href=\"#\" title=\"Stop Interaction\"><i class=\"fa fa-power-off icon-remove icon-large\"></i></button>');\n",
       "    button.click(function (evt) { fig.handle_close(fig, {}); } );\n",
       "    button.mouseover('Stop Interaction', toolbar_mouse_event);\n",
       "    buttongrp.append(button);\n",
       "    var titlebar = this.root.find($('.ui-dialog-titlebar'));\n",
       "    titlebar.prepend(buttongrp);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._root_extra_style = function(el){\n",
       "    var fig = this\n",
       "    el.on(\"remove\", function(){\n",
       "\tfig.close_ws(fig, {});\n",
       "    });\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._canvas_extra_style = function(el){\n",
       "    // this is important to make the div 'focusable\n",
       "    el.attr('tabindex', 0)\n",
       "    // reach out to IPython and tell the keyboard manager to turn it's self\n",
       "    // off when our div gets focus\n",
       "\n",
       "    // location in version 3\n",
       "    if (IPython.notebook.keyboard_manager) {\n",
       "        IPython.notebook.keyboard_manager.register_events(el);\n",
       "    }\n",
       "    else {\n",
       "        // location in version 2\n",
       "        IPython.keyboard_manager.register_events(el);\n",
       "    }\n",
       "\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._key_event_extra = function(event, name) {\n",
       "    var manager = IPython.notebook.keyboard_manager;\n",
       "    if (!manager)\n",
       "        manager = IPython.keyboard_manager;\n",
       "\n",
       "    // Check for shift+enter\n",
       "    if (event.shiftKey && event.which == 13) {\n",
       "        this.canvas_div.blur();\n",
       "        // select the cell after this one\n",
       "        var index = IPython.notebook.find_cell_index(this.cell_info[0]);\n",
       "        IPython.notebook.select(index + 1);\n",
       "    }\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_save = function(fig, msg) {\n",
       "    fig.ondownload(fig, null);\n",
       "}\n",
       "\n",
       "\n",
       "mpl.find_output_cell = function(html_output) {\n",
       "    // Return the cell and output element which can be found *uniquely* in the notebook.\n",
       "    // Note - this is a bit hacky, but it is done because the \"notebook_saving.Notebook\"\n",
       "    // IPython event is triggered only after the cells have been serialised, which for\n",
       "    // our purposes (turning an active figure into a static one), is too late.\n",
       "    var cells = IPython.notebook.get_cells();\n",
       "    var ncells = cells.length;\n",
       "    for (var i=0; i<ncells; i++) {\n",
       "        var cell = cells[i];\n",
       "        if (cell.cell_type === 'code'){\n",
       "            for (var j=0; j<cell.output_area.outputs.length; j++) {\n",
       "                var data = cell.output_area.outputs[j];\n",
       "                if (data.data) {\n",
       "                    // IPython >= 3 moved mimebundle to data attribute of output\n",
       "                    data = data.data;\n",
       "                }\n",
       "                if (data['text/html'] == html_output) {\n",
       "                    return [cell, data, j];\n",
       "                }\n",
       "            }\n",
       "        }\n",
       "    }\n",
       "}\n",
       "\n",
       "// Register the function which deals with the matplotlib target/channel.\n",
       "// The kernel may be null if the page has been refreshed.\n",
       "if (IPython.notebook.kernel != null) {\n",
       "    IPython.notebook.kernel.comm_manager.register_target('matplotlib', mpl.mpl_figure_comm);\n",
       "}\n"
      ],
      "text/plain": [
       "<IPython.core.display.Javascript object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<img src=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAoAAAAHgCAYAAAA10dzkAAAgAElEQVR4Xu3d29Edx3WAUSINp2Gl4TSkMFQKQ0zDachpKA24jsusYkEcTOvfM937svTKuXSv3Q+f5oDgt1/8jwABAgQIECBAYJTAt1G7tVkCBAgQIECAAIFfBKBDQIAAAQIECBAYJiAAhw3cdgkQIECAAAECAtAZIECAAAECBAgMExCAwwZuuwQIECBAgAABAegMECBAgAABAgSGCQjAYQO3XQIECBAgQICAAHQGCBAgQIAAAQLDBATgsIHbLgECBAgQIEBAADoDBAgQIECAAIFhAgJw2MBtlwABAgQIECAgAJ0BAgQIECBAgMAwAQE4bOC2S4AAAQIECBAQgM4AAQIECBAgQGCYgAAcNnDbJUCAAAECBAgIQGeAAAECBAgQIDBMQAAOG7jtEiBAgAABAgQEoDNAgAABAgQIEBgmIACHDdx2CRAgQIAAAQIC0BkgQIAAAQIECAwTEIDDBm67BAgQIECAAAEB6AwQIECAAAECBIYJCMBhA7ddAgQIECBAgIAAdAYIECBAgAABAsMEBOCwgdsuAQIECBAgQEAAOgMECBAgQIAAgWECAnDYwG2XAAECBAgQICAAnQECBAgQIECAwDABAThs4LZLgAABAgQIEBCAzgABAgQIECBAYJiAABw2cNslQIAAAQIECAhAZ4AAAQIECBAgMExAAA4buO0SIECAAAECBASgM0CAAAECBAgQGCYgAIcN3HYJECBAgAABAgLQGSBAgAABAgQIDBMQgMMGbrsECBAgQIAAAQHoDBAgQIAAAQIEhgkIwGEDt10CBAgQIECAgAB0BggQIECAAAECwwQE4LCB2y4BAgQIECBAQAA6AwQIECBAgACBYQICcNjAbZcAAQIECBAgIACdAQIECBAgQIDAMAEBOGzgtkuAAAECBAgQEIDOAAECBAgQIEBgmIAAHDZw2yVAgAABAgQICEBngAABAgQIECAwTEAADhu47RIgQIAAAQIEBKAzQIAAAQIECBAYJiAAhw3cdgkQIECAAAECAtAZIECAAAECBAgMExCAwwZuuwQIECBAgAABAegMECBAgAABAgSGCQjAYQO3XQIECBAgQICAAHQGCBAgQIAAAQLDBATgsIHbLgECBAgQIEBAADoDBAgQIECAAIFhAgJw2MBtlwABAgQIECAgAJ0BAgQIECBAgMAwAQE4bOC2S4AAAQIECBAQgM4AAQIECBAgQGCYgAAcNnDbJUCAAAECBAgIQGeAAAECBAgQIDBMQAAOG7jtEiBAgAABAgQEoDNAgAABAgQIEBgmIACHDdx2CRAgQIAAAQIC0BkgQIAAAQIECAwTEIDDBm67BAgQIECAAAEB6AwQIECAAAECBIYJCMBhA7ddAgQIECBAgIAAdAYIECBAgAABAsMEBOCwgdsuAQIECBAgQEAAOgMECBAgQIAAgWECAnDYwG2XAAECBAgQICAAnQECBAgQIECAwDABARgc+Pfv378HH+F2AgQIECBAYLPAt2/fRjfQ6M0/cdYE4BOKnkGAAAECBPYKCMC93u3eJgDbjdSGCBAgQGCAgAAcMOQ3tygA39T1bAIECBAg8I6AAHzHdcxTBeCYUdsoAQIECDQSEICNhnliKwLwhLp3EiBAgACBmIAAjPmNv1sAjj8CAAgQIECgoIAALDi0TEsWgJmmYS0ECBAgQGBNQACuObnqQkAAOhoECBAgQKCegACsN7NUKxaAqcZhMQQIECBAYElAAC4xuehKQAA6GwQIECBAoJ6AAKw3s1QrFoCpxmExBAgQIEBgSUAALjG5yBdAZ4AAAQIECPQREIB9ZnlkJ74AHmH3UgIECBAgEBIQgCE+NwtAZ4AAAQIECNQTEID1ZpZqxQIw1TgshgABAgQILAkIwCUmF10JCEBngwABAgQI1BMQgPVmlmrFAjDVOCyGAAECBAgsCQjAJSYX+QLoDBAgQIAAgT4CArDPLI/sxBfAI+xeSoAAAQIEQgICMMTnZgHoDBAgQIAAgXoCArDezFKtWACmGofFECBAgACBJQEBuMTkoisBAehsECBAgACBegICsN7MUq1YAKYah8UQIECAAIElAQG4xOQiXwCdAQIECBAg0EdAAPaZ5ZGd+AJ4hN1LCRAgQIBASEAAhvjcLACdAQIECBAgUE9AANabWaoVC8BU47AYAgQIECCwJCAAl5hcdCUgAJ0NAgQIECBQT0AA1ptZqhULwFTjsBgCBAgQILAkIACXmFzkC6AzQIAAAQIE+ggIwD6zPLITXwCPsHspAQIECBAICQjAEJ+bBaAzQIAAAQIE6gkIwHozS7ViAZhqHBZDgAABAgSWBATgEpOLrgQEoLNBgAABAgTqCQjAejNLtWIBmGocFkOAAAECBJYEBOASk4t8AXQGCBAgQIBAHwEB2GeWR3biC+ARdi8lQIAAAQIhAQEY4nOzAHQGCBAgQIBAPQEBWG9mqVYsAFONw2IIECBAgMCSgABcYnLRlYAAdDYIECBAgEA9AQFYb2apViwAU43DYggQIECAwJKAAFxicpEvgM4AAQIECBDoIyAA+8zyyE58ATzC7qUECBAgQCAkIABDfG4WgM4AAQIECBCoJyAA680s1YoFYKpxWAwBAgQIEFgSEIBLTC66EhCAzgYBAgQIEKgnIADrzSzVigVgqnFYDAECBAgQWBIQgEtMLvIF0BkgQIAAAQJ9BARgn1ke2YkvgEfYvZQAAQIECIQEBGCIz80C0BkgQIAAAQL1BARgvZmlWrEATDUOiyFAgAABAksCAnCJyUVXAgLQ2SBAgAABAvUEBGC9maVasQBMNQ6LIUCAAAECSwICcInJRb4AOgMECBAgQKCPgADsM8sjO/EF8Ai7lxIgQIAAgZCAAAzxuVkAOgMECBAgQKCegACsN7NUKxaAqcZhMQQIECBAYElAAC4xuehKQAA6GwQIECBAoJ6AAKw3s1QrFoCpxmExBAgQIEBgSUAALjG5yBdAZ4AAAQIECPQREIB9ZnlkJ74AHmH3UgIECBAgEBIQgCE+NwtAZ4AAAQIECNQTEID1ZpZqxQIw1TgshgABAgQILAkIwCUmF10JCEBngwABAgQI1BMQgPVmlmrFAjDVOCyGAAECBAgsCQjAJSYX+QLoDBAgQIAAgT4CArDPLI/sxBfAI+xeSoAAAQIEQgICMMTnZgHoDBAgQIAAgXoCArDezFKtWACmGofFECBAgACBJQEBuMTkoisBAehsECBAgACBegICsN7MUq1YAKYah8UQIECAAIElAQG4xOQiXwCdAQIECBAg0EdAAPaZ5ZGd+AJ4hN1LCRAgQIBASEAAhvjcLACdAQIECBAgUE9AANabWaoVC8BU47AYAgQIECCwJCAAl5hcdCUgAJ0NAgQIECBQT0AA1ptZqhULwFTjsBgCBAgQILAkIACXmFzkC6AzQIAAAQIE+ggIwD6zPLITXwCPsHspAQIECBAICQjAEJ+bBaAzQIAAAQIE6gkIwHozS7ViAZhqHBZDgAABAgSWBATgEpOLrgQEoLNBgAABAgTqCQjAejNLtWIBmGocFkOAAAECBJYEBOASk4t8AXQGCBAgQIBAHwEB2GeWR3biC+ARdi8lQIAAAQIhAQEY4nOzAHQGCBAgQIBAPQEBWG9mqVYsAFONw2IIECBAgMCSgABcYnLRlYAAdDYIECBAgEA9AQFYb2apViwAU43DYggQIECAwJKAAFxicpEvgM4AAQIECBDoIyAA+8zyyE58ATzC7qUECBAgQCAkIABDfG4WgM4AAQIECBCoJyAA680s1YoFYKpxWAwBAgQIEFgSEIBLTC66EhCAzgYBAgQIEKgnIADrzSzVigVgqnFYDAECBAgQWBIQgEtMLvIF0BkgQIAAAQJ9BARgn1ke2YkvgEfYvZQAAQIECIQEBGCIz80C0BkgQIAAAQL1BARgvZmlWrEATDUOiyFAgAABAksCAnCJyUVXAgLQ2SBAgAABAvUEBGC9maVasQBMNQ6LIUCAAAECSwICcInJRb4AOgMECBAgQKCPgADsM8sjO/EF8Ai7lxIgQIAAgZCAAAzxuVkAOgMECBAgQKCegACsN7NUKxaAqcZhMQQIECBAYElAAC4xuehKQAA6GwQIECBAoJ6AAKw3s1QrFoCpxmExBAgQIEBgSUAALjG5yBdAZ4AAAQIECPQREIB9ZnlkJ74AHmH3UgIECBAgEBIQgCE+NwtAZ4AAAQIECNQTEID1ZpZqxQIw1TgshgABAgQILAkIwCUmF10JCEBngwABAgQI1BMQgPVmlmrFAjDVOCyGAAECBAgsCQjAJSYX+QLoDBAgQIAAgT4CArDPLI/sxBfAI+xeSoAAAQIEQgICMMTnZgHoDBAgQIAAgXoCArDezFKtWACmGofFECBAgACBJQEBuMTkoisBAehsECBAgACBegICsN7MUq1YAKYah8UQIECAAIElAQG4xOQiXwCdAQIECBAg0EdAAPaZ5ZGd+AJ4hN1LCRAgQIBASEAAhvjcLACdAQIECBAgUE9AANabWaoVC8BU47AYAgQIECCwJCAAl5hcdCUgAJ0NAgQIECBQT0AA1ptZqhULwFTjsBgCBAgQILAkIACXmFzkC6AzQIAAAQIE+ggIwD6zPLITXwCPsHspAQIECBAICQjAEJ+bBaAzQIAAAQIE6gkIwHozS7ViAZhqHBZDgAABAgSWBATgEpOLrgQEoLNBgAABAgTqCQjAejNLtWIBmGocFkOAAAECBJYEBOASk4t8AXQGCBAgQIBAHwEB2GeWR3biC+ARdi8lQIAAAQIhAQEY4nOzAHQGCBAgQIBAPQEBWG9mqVYsAFONw2IIECBAgMCSgABcYnLRlYAAdDYIECBAgEA9AQFYb2apViwAU43DYggQIECAwJKAAFxicpEvgM4AAQIECBDoIyAA+8zyyE58ATzC7qUECBAgQCAkIABDfG4WgM4AAQIECBCoJyAA680s1YoFYKpxWAwBAgQIEFgSEIBLTC66EhCAzgYBAgQIEKgnIADrzSzVigVgqnFYDAECBAgQWBIQgEtMLvIF0BkgQIAAAQJ9BARgn1ke2YkvgEfYvZTAlwT+59e//N99//nnv3/pfjcRINBHQAD2meWRnQjAI+xeSuBW4LfYu7pQBN4SuoBAawEB2Hq8729OAL5v7A0EviIgAL+i5h4CcwQE4JxZv7JTAfgKq4cSCAsIwDChBxBoLSAAW4/3/c0JwPeNvYHAVwTuAvDzTD8Df0XWPQR6CAjAHnM8tgsBeIzeiwn8VEAAOiAECPxMQAA6HyEBARjiczOB1wQE4Gu0HkyghYAAbDHGc5sQgOfsvZnAzwRWAtDPwM4QgbkCAnDu7B/ZuQB8hNFDCLwisBKB/hzgK/QeSiC9gABMP6LcCxSAuedjdbMFBODs+ds9gZ8JCEDnIyQgAEN8bibwqoAAfJXXwwmUFhCApcd3fvEC8PwMrIDAzwTuItBPwM4PgZkCAnDm3B/btQB8jNKDCLwiIABfYfVQAuUFBGD5EZ7dgAA86+/tBO4EBOCdkH9OYKaAAJw598d2LQAfo/QgAq8I3AXg56V+Bn6F3kMJpBYQgKnHk39xAjD/jKyQwF0ECkBnhMA8AQE4b+aP7lgAPsrpYQReERCAr7B6KIHSAgKw9PjOL14Anp+BFRC4E7gLQD8D3wn65wT6CQjAfjPduiMBuJXbywh8WeAuAv0M/GVaNxIoKSAAS44tz6IFYJ5ZWAmBnwkIQOeDAIHfCwhA5yEkIABDfG4msE3gLgD9DLxtFF5EIIWAAEwxhrqLEIB1Z2fl8wTuItDPwPPOhB3PFRCAc2f/yM4F4COMHkJgi4AA3MLsJQRKCAjAEmPKu0gBmHc2VkbgRwEB6EwQIPCbgAB0FkICAjDE52YCWwXuAvCzGD8Dbx2JlxE4JiAAj9H3eLEA7DFHu5ghIABnzNkuCawICMAVJddcCghAh4NAHQEBWGdWVkrgbQEB+LZw8+cLwOYDtr1WAisB6GfgViO3GQKXAgLQ4QgJCMAQn5sJbBUQgFu5vYxAagEBmHo8+RcnAPPPyAoJ/F5gJQL9iyDODIH+AgKw/4xf3aEAfJXXwwk8LiAAHyf1QAIlBQRgybHlWbQAzDMLKyGwIiAAV5RcQ6C/gADsP+NXdygAX+X1cAKvCNxFoJ+AX2H3UAKpBARgqnHUW4wArDczKyYgAJ0BAgQEoDMQEhCAIT43EzgicBeAn0X5CnhkNF5KYJuAANxG3fNFArDnXO2qv8BdBArA/mfADmcLCMDZ8w/vXgCGCT2AwBEBAXiE3UsJpBEQgGlGUXMhArDm3KyagAB0BgjMFhCAs+cf3r0ADBN6AIEjAncB+FmUn4GPjMZLCWwREIBbmPu+RAD2na2d9Re4i0AB2P8M2OFcAQE4d/aP7FwAPsLoIQSOCAjAI+xeSiCFgABMMYa6ixCAdWdn5QTuAtDPwM4Igb4CArDvbLfsTABuYfYSAq8J3EWgn4Ffo/dgAkcFBOBR/vovF4D1Z2gHswUE4Oz52/1cAQE4d/aP7FwAPsLoIQSOCdwFoJ+Bj43Giwm8KiAAX+Xt/3AB2H/GdthbQAD2nq/dEbgSEIDORkhAAIb43EzguIAAPD4CCyBwREAAHmHv81IB2GeWdjJTQADOnLtdExCAzkBIQACG+NxM4LiAADw+AgsgcERAAB5h7/NSAdhnlnYyV+AuAv1VMHPPhp33FRCAfWe7ZWcCcAuzlxB4VUAAvsrr4QRSCgjAlGOpsygBWGdWVkrgSuAuAD/3+Qro/BDoJSAAe81z+24E4HZyLyTwisBdBArAV9g9lMAxAQF4jL7HiwVgjznaBQEB6AwQmCUgAGfN+/HdCsDHST2QwBGBuwD0M/CRsXgpgdcEBOBrtDMeLABnzNkuZwjcRaCfgWecA7ucISAAZ8z5tV0KwNdoPZjAdgEBuJ3cCwkcExCAx+h7vFgA9pijXRD4CAhA54DAHAEBOGfWr+xUAL7C6qEEjgjcBeBnUX4GPjIaLyXwuIAAfJx01gMF4Kx5221/gbsIFID9z4AdzhAQgDPm/NouBeBrtB5M4IiAADzC7qUEtgsIwO3kvV4oAHvN024I3AWgn4GdEQI9BARgjzke24UAPEbvxQReE7iLQD8Dv0bvwQS2CQjAbdQ9XyQAe87VrmYLCMDZ87f7GQICcMacX9ulAHyN1oMJHBO4C0A/Ax8bjRcTeExAAD5GOfNBAnDm3O26t4AA7D1fuyPwERCAzkFIQACG+NxMIKWAAEw5Fosi8KiAAHyUc97DBOC8mdtxfwEB2H/GdkhAADoDIQEBGOJzM4GUAgIw5VgsisCjAgLwUc55DxOA82ZuxzME7iLQXwUz4xzYZV8BAdh3tlt2JgC3MHsJge0CAnA7uRcS2CogALdy93uZAOw3Uzsi8BEQgM4Bgd4CArD3fF/fnQB8ndgLCBwRuAvAz6L8DHxkNF5K4BEBAfgI49yHCMC5s7fz/gJ3ESgA+58BO+wrIAD7znbLzgTgFmYvIXBEQAAeYfdSAlsEBOAW5r4vEYB9Z2tnBO4C0M/AzgiBugICsO7sUqxcAKYYg0UQeE3gLgL9DPwavQcTeFVAAL7K2//hArD/jO1wtoAAnD1/u+8rIAD7znbLzgTgFmYvIXBM4C4A/Qx8bDReTCAkIABDfG4WgM4Agf4CdxHoZ+D+Z8AO+wkIwH4z3bojAbiV28sIHBEQgEfYvZTAqwIC8FXe/g8XgP1nbIcEBKAzQKCfgADsN9OtOxKAW7m9jMARgbsA/CzKz8BHRuOlBL4sIAC/TOfGj4AAdA4I9BcQgP1nbIfzBATgvJk/umMB+CinhxFIKSAAU47FogiEBARgiM/NAtAZINBfYCUA/Qzc/xzYYS8BAdhrntt3IwC3k3shgSMCKxHozwEeGY2XEviSgAD8EpubfhMQgM4CgRkCAnDGnO1yjoAAnDPrV3YqAF9h9VAC6QQEYLqRWBCBkIAADPG5WQA6AwTmCNxFoJ+A55wFO60vIADrz/DoDgTgUX4vJ7BVQABu5fYyAq8KCMBXefs/XAD2n7Ed9hL49u3blzf0j7//+af3/ukvv3752T/e+P3798ee5UEECPyrgAB0KkICAjDE52YC2wUiAfhZ7K4IFIDbj4YXDhMQgMMG/vR2BeDTop5H4F2BpwLwv//5r18D/+s/fv3lqa+AAvDdc+DpBASgMxASEIAhPjcT2C7wZgB+NvO3v/3pkT0JwEcYPYTApYAAdDhCAgIwxOdmAtsFngjAP/r699tGnvoKKAC3Hw0vHCYgAIcN/OntCsCnRT2PwLsC0QD8rO6vf/3HTxf5xFdAAfjuOfB0AgLQGQgJCMAQn5sJbBcQgNvJvZBASgEBmHIsdRYlAOvMykoJfATeDsDPT8Cf/0X/ZRBfAJ1XAu8KCMB3fds/XQC2H7ENNhN4IgA/fxXM1Z8DFIDNDozttBUQgG1Hu2djAnCPs7cQeErgiQD8rOWP/hzgU/H3eb4vgE9N3HMI/LGAAHQyQgICMMTnZgLbBZ4KwB//QujoT74/QgjA7UfDC4cJCMBhA396uwLwaVHPI/CuwFMB+O4qfQF829fzCQhAZyAq4D/YGRV0PwECBAgQ2C/w9f8w+P61Pv7G0Zt/QtMXwCcUPYPAPgFfAPdZexOBzAK+AGaeToG1CcACQ7JEAr8TEICOAwECHwEB6ByEBARgiM/NBLYLCMDt5F5IIKWAAEw5ljqLEoB1ZmWlBP7///WXgPBvAZcYk0UWFhCAhYeXYekCMMMUrIHAuoAvgOtWriTQWUAAdp7uhr0JwA3IXkHgQQEB+CCmRxEoLCAACw8vw9IFYIYpWAOBdQEBuG7lSgKdBQRg5+lu2JsA3IDsFQQeFBCAD2J6FIHCAgKw8PAyLF0AZpiCNRBYFxCA61auJNBZQAB2nu6GvQnADcheQeBBAQH4IKZHESgsIAALDy/D0gVghilYA4F1AQG4buVKAp0FBGDn6W7YmwDcgOwVBB4UEIAPYnoUgcICArDw8DIsXQBmmII1ECBAgACBf09AAP57Xq7+QUAAOhIECBAgQKCegACsN7NUKxaAqcZhMQQIECBAYElAAC4xuehKQAA6GwQIECBAoJ6AAKw3s1QrFoCpxmExBAgQIEBgSUAALjG5yBdAZ4AAAQIECPQREIB9ZnlkJ74AHmH3UgIECBAgEBIQgCE+NwtAZ4AAAQIECNQTEID1ZpZqxQIw1TgshgABAgQILAkIwCUmF10JCEBngwABAgQI1BMQgPVmlmrFAjDVOCyGAAECBAgsCQjAJSYX+QLoDBAgQIAAgT4CArDPLI/sxBfAI+xeSoAAAQIEQgICMMTnZgHoDBAgQIAAgXoCArDezFKtWACmGofFECBAgACBJQEBuMTkoisBAehsECBAgACBegICsN7MUq1YAKYah8UQIECAAIElAQG4xOQiXwCdAQIECBAg0EdAAPaZ5ZGd+AJ4hN1LCRAgQIBASEAAhvjcLACdAQIECBAgUE9AANabWaoVC8BU47AYAgQIECCwJCAAl5hcdCUgAJ0NAgQIECBQT0AA1ptZqhULwFTjsBgCBAgQILAkIACXmFzkC6AzQIAAAQIE+ggIwD6zPLITXwCPsHspAQIECBAICQjAEJ+bBaAzQIAAAQIE6gkIwHozS7ViAZhqHBZDgAABAgSWBATgEpOLrgQEoLNBgAABAgTqCQjAejNLtWIBmGocFkOAAAECBJYEBOASk4t8AXQGCBAgQIBAHwEB2GeWR3biC+ARdi8lQIAAAQIhAQEY4nOzAHQGCBAgQIBAPQEBWG9mqVYsAFONw2IIECBAgMCSgABcYnLRlYAAdDYIECBAgEA9AQFYb2apViwAU43DYggQIECAwJKAAFxicpEvgM4AAQIECBDoIyAA+8zyyE58ATzC7qUECBAgQCAkIABDfG4WgM4AAQIECBCoJyAA680s1YoFYKpxWAwBAgQIEFgSEIBLTC66EhCAzgYBAgQIEKgnIADrzSzVigVgqnFYDAECBAgQWBIQgEtMLvIF0BkgQIAAAQJ9BARgn1ke2YkvgEfYvZQAAQIECIQEBGCIz80C0BkgQIAAAQL1BARgvZmlWrEATDUOiyFAgAABAksCAnCJyUVXAgLQ2SBAgAABAvUEBGC9maVasQBMNQ6LIUCAAAECSwICcInJRb4AOgMECBAgQKCPgADsM8sjO/EF8Ai7lxIgQIAAgZCAAAzxuVkAOgMECBAgQKCegACsN7NUKxaAqcZhMQQIECBAYElAAC4xuehKQAA6GwQIECBAoJ6AAKw3s1QrFoCpxmExBAgQIEBgSUAALjG5yBdAZ4AAAQIECPQREIB9ZnlkJ74AHmH3UgIECBAgEBIQgCE+NwtAZ4AAAQIECNQTEID1ZpZqxQIw1TgshgABAgQILAkIwCUmF10JCEBngwABAgQI1BMQgPVmlmrFAjDVOCyGAAECBAgsCQjAJSYX+QLoDBAgQIAAgT4CArDPLI/sxBfAI+xeSoAAAQIEQgICMMTnZgHoDBAgQIAAgXoCArDezFKtWACmGofFECBAgACBJQEBuMTkoisBAehsECBAgACBegICsN7MUq1YAKYah8UQIECAAIElAQG4xOQiXwCdAQIECBAg0EdAAPaZ5ZGd+AJ4hN1LCRAgQIBASEAAhvjcLACdAQIECBAgUE9AANabWaoVC8BU47AYAgQIECCwJCAAl5hcdCUgAJ0NAgQIECBQT0AA1puZFRMgQIAAAQIECAQEvgXudSsBAgQIECBAgEBBAQFYcGiWTIAAAQIECBCICAjAiJ57CRAgQIAAAQIFBQRgwaFZMgECBAgQIEAgIiAAI3ruJUCAAAECBAgUFBCABYdmyQQIECBAgACBiIAAjOi5lwABAgQIECBQUEAAFhyaJRMgQIAAAQIEIgICMKLnXgIECBAgQIBAQQEBWJZqphAAAAOuSURBVHBolkyAAAECBAgQiAgIwIieewkQIECAAAECBQUEYMGhWTIBAgQIECBAICIgACN67iVAgAABAgQIFBQQgAWHZskECBAgQIAAgYiAAIzouZcAAQIECBAgUFBAABYcmiUTIECAAAECBCICAjCi514CBAgQIECAQEEBAVhwaJZMgAABAgQIEIgICMCInnsJECBAgAABAgUFBGDBoVkyAQIECBAgQCAiIAAjeu4lQIAAAQIECBQUEIAFh2bJBAgQIECAAIGIgACM6LmXAAECBAgQIFBQQAAWHJolEyBAgAABAgQiAgIwoudeAgQIECBAgEBBAQFYcGiWTIAAAQIECBCICAjAiJ57CRAgQIAAAQIFBQRgwaFZMgECBAgQIEAgIiAAI3ruJUCAAAECBAgUFBCABYdmyQQIECBAgACBiIAAjOi5lwABAgQIECBQUEAAFhyaJRMgQIAAAQIEIgICMKLnXgIECBAgQIBAQQEBWHBolkyAAAECBAgQiAgIwIieewkQIECAAAECBQUEYMGhWTIBAgQIECBAICIgACN67iVAgAABAgQIFBQQgAWHZskECBAgQIAAgYiAAIzouZcAAQIECBAgUFBAABYcmiUTIECAAAECBCICAjCi514CBAgQIECAQEEBAVhwaJZMgAABAgQIEIgICMCInnsJECBAgAABAgUFBGDBoVkyAQIECBAgQCAiIAAjeu4lQIAAAQIECBQUEIAFh2bJBAgQIECAAIGIgACM6LmXAAECBAgQIFBQQAAWHJolEyBAgAABAgQiAgIwoudeAgQIECBAgEBBAQFYcGiWTIAAAQIECBCICAjAiJ57CRAgQIAAAQIFBQRgwaFZMgECBAgQIEAgIiAAI3ruJUCAAAECBAgUFBCABYdmyQQIECBAgACBiIAAjOi5lwABAgQIECBQUEAAFhyaJRMgQIAAAQIEIgICMKLnXgIECBAgQIBAQQEBWHBolkyAAAECBAgQiAgIwIieewkQIECAAAECBQUEYMGhWTIBAgQIECBAICIgACN67iVAgAABAgQIFBQQgAWHZskECBAgQIAAgYiAAIzouZcAAQIECBAgUFBAABYcmiUTIECAAAECBCICAjCi514CBAgQIECAQEEBAVhwaJZMgAABAgQIEIgICMCInnsJECBAgAABAgUFBGDBoVkyAQIECBAgQCAiIAAjeu4lQIAAAQIECBQUEIAFh2bJBAgQIECAAIGIgACM6LmXAAECBAgQIFBQ4H8Bb7SyO0jQSXkAAAAASUVORK5CYII=\" width=\"640\">"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_cart_pole(env, obs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Looks like it's doing what we're telling it to do. Now how can we make the poll remain upright? We will need to define a _policy_ for that. This is the strategy that the agent will use to select an action at each step. It can use all the past actions and observations to decide what to do."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# A simple hard-coded policy"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's hard code a simple strategy: if the pole is tilting to the left, then push the cart to the left, and _vice versa_. Let's see if that works:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "frames = []\n",
    "\n",
    "n_max_steps = 1000\n",
    "n_change_steps = 10\n",
    "\n",
    "obs = env.reset()\n",
    "for step in range(n_max_steps):\n",
    "    img = render_cart_pole(env, obs)\n",
    "    frames.append(img)\n",
    "\n",
    "    # hard-coded policy\n",
    "    position, velocity, angle, angular_velocity = obs\n",
    "    if angle < 0:\n",
    "        action = 0\n",
    "    else:\n",
    "        action = 1\n",
    "\n",
    "    obs, reward, done, info = env.step(action)\n",
    "    if done:\n",
    "        break"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/javascript": [
       "/* Put everything inside the global mpl namespace */\n",
       "window.mpl = {};\n",
       "\n",
       "\n",
       "mpl.get_websocket_type = function() {\n",
       "    if (typeof(WebSocket) !== 'undefined') {\n",
       "        return WebSocket;\n",
       "    } else if (typeof(MozWebSocket) !== 'undefined') {\n",
       "        return MozWebSocket;\n",
       "    } else {\n",
       "        alert('Your browser does not have WebSocket support.' +\n",
       "              'Please try Chrome, Safari or Firefox ≥ 6. ' +\n",
       "              'Firefox 4 and 5 are also supported but you ' +\n",
       "              'have to enable WebSockets in about:config.');\n",
       "    };\n",
       "}\n",
       "\n",
       "mpl.figure = function(figure_id, websocket, ondownload, parent_element) {\n",
       "    this.id = figure_id;\n",
       "\n",
       "    this.ws = websocket;\n",
       "\n",
       "    this.supports_binary = (this.ws.binaryType != undefined);\n",
       "\n",
       "    if (!this.supports_binary) {\n",
       "        var warnings = document.getElementById(\"mpl-warnings\");\n",
       "        if (warnings) {\n",
       "            warnings.style.display = 'block';\n",
       "            warnings.textContent = (\n",
       "                \"This browser does not support binary websocket messages. \" +\n",
       "                    \"Performance may be slow.\");\n",
       "        }\n",
       "    }\n",
       "\n",
       "    this.imageObj = new Image();\n",
       "\n",
       "    this.context = undefined;\n",
       "    this.message = undefined;\n",
       "    this.canvas = undefined;\n",
       "    this.rubberband_canvas = undefined;\n",
       "    this.rubberband_context = undefined;\n",
       "    this.format_dropdown = undefined;\n",
       "\n",
       "    this.image_mode = 'full';\n",
       "\n",
       "    this.root = $('<div/>');\n",
       "    this._root_extra_style(this.root)\n",
       "    this.root.attr('style', 'display: inline-block');\n",
       "\n",
       "    $(parent_element).append(this.root);\n",
       "\n",
       "    this._init_header(this);\n",
       "    this._init_canvas(this);\n",
       "    this._init_toolbar(this);\n",
       "\n",
       "    var fig = this;\n",
       "\n",
       "    this.waiting = false;\n",
       "\n",
       "    this.ws.onopen =  function () {\n",
       "            fig.send_message(\"supports_binary\", {value: fig.supports_binary});\n",
       "            fig.send_message(\"send_image_mode\", {});\n",
       "            if (mpl.ratio != 1) {\n",
       "                fig.send_message(\"set_dpi_ratio\", {'dpi_ratio': mpl.ratio});\n",
       "            }\n",
       "            fig.send_message(\"refresh\", {});\n",
       "        }\n",
       "\n",
       "    this.imageObj.onload = function() {\n",
       "            if (fig.image_mode == 'full') {\n",
       "                // Full images could contain transparency (where diff images\n",
       "                // almost always do), so we need to clear the canvas so that\n",
       "                // there is no ghosting.\n",
       "                fig.context.clearRect(0, 0, fig.canvas.width, fig.canvas.height);\n",
       "            }\n",
       "            fig.context.drawImage(fig.imageObj, 0, 0);\n",
       "        };\n",
       "\n",
       "    this.imageObj.onunload = function() {\n",
       "        this.ws.close();\n",
       "    }\n",
       "\n",
       "    this.ws.onmessage = this._make_on_message_function(this);\n",
       "\n",
       "    this.ondownload = ondownload;\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._init_header = function() {\n",
       "    var titlebar = $(\n",
       "        '<div class=\"ui-dialog-titlebar ui-widget-header ui-corner-all ' +\n",
       "        'ui-helper-clearfix\"/>');\n",
       "    var titletext = $(\n",
       "        '<div class=\"ui-dialog-title\" style=\"width: 100%; ' +\n",
       "        'text-align: center; padding: 3px;\"/>');\n",
       "    titlebar.append(titletext)\n",
       "    this.root.append(titlebar);\n",
       "    this.header = titletext[0];\n",
       "}\n",
       "\n",
       "\n",
       "\n",
       "mpl.figure.prototype._canvas_extra_style = function(canvas_div) {\n",
       "\n",
       "}\n",
       "\n",
       "\n",
       "mpl.figure.prototype._root_extra_style = function(canvas_div) {\n",
       "\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._init_canvas = function() {\n",
       "    var fig = this;\n",
       "\n",
       "    var canvas_div = $('<div/>');\n",
       "\n",
       "    canvas_div.attr('style', 'position: relative; clear: both; outline: 0');\n",
       "\n",
       "    function canvas_keyboard_event(event) {\n",
       "        return fig.key_event(event, event['data']);\n",
       "    }\n",
       "\n",
       "    canvas_div.keydown('key_press', canvas_keyboard_event);\n",
       "    canvas_div.keyup('key_release', canvas_keyboard_event);\n",
       "    this.canvas_div = canvas_div\n",
       "    this._canvas_extra_style(canvas_div)\n",
       "    this.root.append(canvas_div);\n",
       "\n",
       "    var canvas = $('<canvas/>');\n",
       "    canvas.addClass('mpl-canvas');\n",
       "    canvas.attr('style', \"left: 0; top: 0; z-index: 0; outline: 0\")\n",
       "\n",
       "    this.canvas = canvas[0];\n",
       "    this.context = canvas[0].getContext(\"2d\");\n",
       "\n",
       "    var backingStore = this.context.backingStorePixelRatio ||\n",
       "\tthis.context.webkitBackingStorePixelRatio ||\n",
       "\tthis.context.mozBackingStorePixelRatio ||\n",
       "\tthis.context.msBackingStorePixelRatio ||\n",
       "\tthis.context.oBackingStorePixelRatio ||\n",
       "\tthis.context.backingStorePixelRatio || 1;\n",
       "\n",
       "    mpl.ratio = (window.devicePixelRatio || 1) / backingStore;\n",
       "\n",
       "    var rubberband = $('<canvas/>');\n",
       "    rubberband.attr('style', \"position: absolute; left: 0; top: 0; z-index: 1;\")\n",
       "\n",
       "    var pass_mouse_events = true;\n",
       "\n",
       "    canvas_div.resizable({\n",
       "        start: function(event, ui) {\n",
       "            pass_mouse_events = false;\n",
       "        },\n",
       "        resize: function(event, ui) {\n",
       "            fig.request_resize(ui.size.width, ui.size.height);\n",
       "        },\n",
       "        stop: function(event, ui) {\n",
       "            pass_mouse_events = true;\n",
       "            fig.request_resize(ui.size.width, ui.size.height);\n",
       "        },\n",
       "    });\n",
       "\n",
       "    function mouse_event_fn(event) {\n",
       "        if (pass_mouse_events)\n",
       "            return fig.mouse_event(event, event['data']);\n",
       "    }\n",
       "\n",
       "    rubberband.mousedown('button_press', mouse_event_fn);\n",
       "    rubberband.mouseup('button_release', mouse_event_fn);\n",
       "    // Throttle sequential mouse events to 1 every 20ms.\n",
       "    rubberband.mousemove('motion_notify', mouse_event_fn);\n",
       "\n",
       "    rubberband.mouseenter('figure_enter', mouse_event_fn);\n",
       "    rubberband.mouseleave('figure_leave', mouse_event_fn);\n",
       "\n",
       "    canvas_div.on(\"wheel\", function (event) {\n",
       "        event = event.originalEvent;\n",
       "        event['data'] = 'scroll'\n",
       "        if (event.deltaY < 0) {\n",
       "            event.step = 1;\n",
       "        } else {\n",
       "            event.step = -1;\n",
       "        }\n",
       "        mouse_event_fn(event);\n",
       "    });\n",
       "\n",
       "    canvas_div.append(canvas);\n",
       "    canvas_div.append(rubberband);\n",
       "\n",
       "    this.rubberband = rubberband;\n",
       "    this.rubberband_canvas = rubberband[0];\n",
       "    this.rubberband_context = rubberband[0].getContext(\"2d\");\n",
       "    this.rubberband_context.strokeStyle = \"#000000\";\n",
       "\n",
       "    this._resize_canvas = function(width, height) {\n",
       "        // Keep the size of the canvas, canvas container, and rubber band\n",
       "        // canvas in synch.\n",
       "        canvas_div.css('width', width)\n",
       "        canvas_div.css('height', height)\n",
       "\n",
       "        canvas.attr('width', width * mpl.ratio);\n",
       "        canvas.attr('height', height * mpl.ratio);\n",
       "        canvas.attr('style', 'width: ' + width + 'px; height: ' + height + 'px;');\n",
       "\n",
       "        rubberband.attr('width', width);\n",
       "        rubberband.attr('height', height);\n",
       "    }\n",
       "\n",
       "    // Set the figure to an initial 600x600px, this will subsequently be updated\n",
       "    // upon first draw.\n",
       "    this._resize_canvas(600, 600);\n",
       "\n",
       "    // Disable right mouse context menu.\n",
       "    $(this.rubberband_canvas).bind(\"contextmenu\",function(e){\n",
       "        return false;\n",
       "    });\n",
       "\n",
       "    function set_focus () {\n",
       "        canvas.focus();\n",
       "        canvas_div.focus();\n",
       "    }\n",
       "\n",
       "    window.setTimeout(set_focus, 100);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._init_toolbar = function() {\n",
       "    var fig = this;\n",
       "\n",
       "    var nav_element = $('<div/>')\n",
       "    nav_element.attr('style', 'width: 100%');\n",
       "    this.root.append(nav_element);\n",
       "\n",
       "    // Define a callback function for later on.\n",
       "    function toolbar_event(event) {\n",
       "        return fig.toolbar_button_onclick(event['data']);\n",
       "    }\n",
       "    function toolbar_mouse_event(event) {\n",
       "        return fig.toolbar_button_onmouseover(event['data']);\n",
       "    }\n",
       "\n",
       "    for(var toolbar_ind in mpl.toolbar_items) {\n",
       "        var name = mpl.toolbar_items[toolbar_ind][0];\n",
       "        var tooltip = mpl.toolbar_items[toolbar_ind][1];\n",
       "        var image = mpl.toolbar_items[toolbar_ind][2];\n",
       "        var method_name = mpl.toolbar_items[toolbar_ind][3];\n",
       "\n",
       "        if (!name) {\n",
       "            // put a spacer in here.\n",
       "            continue;\n",
       "        }\n",
       "        var button = $('<button/>');\n",
       "        button.addClass('ui-button ui-widget ui-state-default ui-corner-all ' +\n",
       "                        'ui-button-icon-only');\n",
       "        button.attr('role', 'button');\n",
       "        button.attr('aria-disabled', 'false');\n",
       "        button.click(method_name, toolbar_event);\n",
       "        button.mouseover(tooltip, toolbar_mouse_event);\n",
       "\n",
       "        var icon_img = $('<span/>');\n",
       "        icon_img.addClass('ui-button-icon-primary ui-icon');\n",
       "        icon_img.addClass(image);\n",
       "        icon_img.addClass('ui-corner-all');\n",
       "\n",
       "        var tooltip_span = $('<span/>');\n",
       "        tooltip_span.addClass('ui-button-text');\n",
       "        tooltip_span.html(tooltip);\n",
       "\n",
       "        button.append(icon_img);\n",
       "        button.append(tooltip_span);\n",
       "\n",
       "        nav_element.append(button);\n",
       "    }\n",
       "\n",
       "    var fmt_picker_span = $('<span/>');\n",
       "\n",
       "    var fmt_picker = $('<select/>');\n",
       "    fmt_picker.addClass('mpl-toolbar-option ui-widget ui-widget-content');\n",
       "    fmt_picker_span.append(fmt_picker);\n",
       "    nav_element.append(fmt_picker_span);\n",
       "    this.format_dropdown = fmt_picker[0];\n",
       "\n",
       "    for (var ind in mpl.extensions) {\n",
       "        var fmt = mpl.extensions[ind];\n",
       "        var option = $(\n",
       "            '<option/>', {selected: fmt === mpl.default_extension}).html(fmt);\n",
       "        fmt_picker.append(option)\n",
       "    }\n",
       "\n",
       "    // Add hover states to the ui-buttons\n",
       "    $( \".ui-button\" ).hover(\n",
       "        function() { $(this).addClass(\"ui-state-hover\");},\n",
       "        function() { $(this).removeClass(\"ui-state-hover\");}\n",
       "    );\n",
       "\n",
       "    var status_bar = $('<span class=\"mpl-message\"/>');\n",
       "    nav_element.append(status_bar);\n",
       "    this.message = status_bar[0];\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.request_resize = function(x_pixels, y_pixels) {\n",
       "    // Request matplotlib to resize the figure. Matplotlib will then trigger a resize in the client,\n",
       "    // which will in turn request a refresh of the image.\n",
       "    this.send_message('resize', {'width': x_pixels, 'height': y_pixels});\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.send_message = function(type, properties) {\n",
       "    properties['type'] = type;\n",
       "    properties['figure_id'] = this.id;\n",
       "    this.ws.send(JSON.stringify(properties));\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.send_draw_message = function() {\n",
       "    if (!this.waiting) {\n",
       "        this.waiting = true;\n",
       "        this.ws.send(JSON.stringify({type: \"draw\", figure_id: this.id}));\n",
       "    }\n",
       "}\n",
       "\n",
       "\n",
       "mpl.figure.prototype.handle_save = function(fig, msg) {\n",
       "    var format_dropdown = fig.format_dropdown;\n",
       "    var format = format_dropdown.options[format_dropdown.selectedIndex].value;\n",
       "    fig.ondownload(fig, format);\n",
       "}\n",
       "\n",
       "\n",
       "mpl.figure.prototype.handle_resize = function(fig, msg) {\n",
       "    var size = msg['size'];\n",
       "    if (size[0] != fig.canvas.width || size[1] != fig.canvas.height) {\n",
       "        fig._resize_canvas(size[0], size[1]);\n",
       "        fig.send_message(\"refresh\", {});\n",
       "    };\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_rubberband = function(fig, msg) {\n",
       "    var x0 = msg['x0'] / mpl.ratio;\n",
       "    var y0 = (fig.canvas.height - msg['y0']) / mpl.ratio;\n",
       "    var x1 = msg['x1'] / mpl.ratio;\n",
       "    var y1 = (fig.canvas.height - msg['y1']) / mpl.ratio;\n",
       "    x0 = Math.floor(x0) + 0.5;\n",
       "    y0 = Math.floor(y0) + 0.5;\n",
       "    x1 = Math.floor(x1) + 0.5;\n",
       "    y1 = Math.floor(y1) + 0.5;\n",
       "    var min_x = Math.min(x0, x1);\n",
       "    var min_y = Math.min(y0, y1);\n",
       "    var width = Math.abs(x1 - x0);\n",
       "    var height = Math.abs(y1 - y0);\n",
       "\n",
       "    fig.rubberband_context.clearRect(\n",
       "        0, 0, fig.canvas.width, fig.canvas.height);\n",
       "\n",
       "    fig.rubberband_context.strokeRect(min_x, min_y, width, height);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_figure_label = function(fig, msg) {\n",
       "    // Updates the figure title.\n",
       "    fig.header.textContent = msg['label'];\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_cursor = function(fig, msg) {\n",
       "    var cursor = msg['cursor'];\n",
       "    switch(cursor)\n",
       "    {\n",
       "    case 0:\n",
       "        cursor = 'pointer';\n",
       "        break;\n",
       "    case 1:\n",
       "        cursor = 'default';\n",
       "        break;\n",
       "    case 2:\n",
       "        cursor = 'crosshair';\n",
       "        break;\n",
       "    case 3:\n",
       "        cursor = 'move';\n",
       "        break;\n",
       "    }\n",
       "    fig.rubberband_canvas.style.cursor = cursor;\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_message = function(fig, msg) {\n",
       "    fig.message.textContent = msg['message'];\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_draw = function(fig, msg) {\n",
       "    // Request the server to send over a new figure.\n",
       "    fig.send_draw_message();\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_image_mode = function(fig, msg) {\n",
       "    fig.image_mode = msg['mode'];\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.updated_canvas_event = function() {\n",
       "    // Called whenever the canvas gets updated.\n",
       "    this.send_message(\"ack\", {});\n",
       "}\n",
       "\n",
       "// A function to construct a web socket function for onmessage handling.\n",
       "// Called in the figure constructor.\n",
       "mpl.figure.prototype._make_on_message_function = function(fig) {\n",
       "    return function socket_on_message(evt) {\n",
       "        if (evt.data instanceof Blob) {\n",
       "            /* FIXME: We get \"Resource interpreted as Image but\n",
       "             * transferred with MIME type text/plain:\" errors on\n",
       "             * Chrome.  But how to set the MIME type?  It doesn't seem\n",
       "             * to be part of the websocket stream */\n",
       "            evt.data.type = \"image/png\";\n",
       "\n",
       "            /* Free the memory for the previous frames */\n",
       "            if (fig.imageObj.src) {\n",
       "                (window.URL || window.webkitURL).revokeObjectURL(\n",
       "                    fig.imageObj.src);\n",
       "            }\n",
       "\n",
       "            fig.imageObj.src = (window.URL || window.webkitURL).createObjectURL(\n",
       "                evt.data);\n",
       "            fig.updated_canvas_event();\n",
       "            fig.waiting = false;\n",
       "            return;\n",
       "        }\n",
       "        else if (typeof evt.data === 'string' && evt.data.slice(0, 21) == \"data:image/png;base64\") {\n",
       "            fig.imageObj.src = evt.data;\n",
       "            fig.updated_canvas_event();\n",
       "            fig.waiting = false;\n",
       "            return;\n",
       "        }\n",
       "\n",
       "        var msg = JSON.parse(evt.data);\n",
       "        var msg_type = msg['type'];\n",
       "\n",
       "        // Call the  \"handle_{type}\" callback, which takes\n",
       "        // the figure and JSON message as its only arguments.\n",
       "        try {\n",
       "            var callback = fig[\"handle_\" + msg_type];\n",
       "        } catch (e) {\n",
       "            console.log(\"No handler for the '\" + msg_type + \"' message type: \", msg);\n",
       "            return;\n",
       "        }\n",
       "\n",
       "        if (callback) {\n",
       "            try {\n",
       "                // console.log(\"Handling '\" + msg_type + \"' message: \", msg);\n",
       "                callback(fig, msg);\n",
       "            } catch (e) {\n",
       "                console.log(\"Exception inside the 'handler_\" + msg_type + \"' callback:\", e, e.stack, msg);\n",
       "            }\n",
       "        }\n",
       "    };\n",
       "}\n",
       "\n",
       "// from http://stackoverflow.com/questions/1114465/getting-mouse-location-in-canvas\n",
       "mpl.findpos = function(e) {\n",
       "    //this section is from http://www.quirksmode.org/js/events_properties.html\n",
       "    var targ;\n",
       "    if (!e)\n",
       "        e = window.event;\n",
       "    if (e.target)\n",
       "        targ = e.target;\n",
       "    else if (e.srcElement)\n",
       "        targ = e.srcElement;\n",
       "    if (targ.nodeType == 3) // defeat Safari bug\n",
       "        targ = targ.parentNode;\n",
       "\n",
       "    // jQuery normalizes the pageX and pageY\n",
       "    // pageX,Y are the mouse positions relative to the document\n",
       "    // offset() returns the position of the element relative to the document\n",
       "    var x = e.pageX - $(targ).offset().left;\n",
       "    var y = e.pageY - $(targ).offset().top;\n",
       "\n",
       "    return {\"x\": x, \"y\": y};\n",
       "};\n",
       "\n",
       "/*\n",
       " * return a copy of an object with only non-object keys\n",
       " * we need this to avoid circular references\n",
       " * http://stackoverflow.com/a/24161582/3208463\n",
       " */\n",
       "function simpleKeys (original) {\n",
       "  return Object.keys(original).reduce(function (obj, key) {\n",
       "    if (typeof original[key] !== 'object')\n",
       "        obj[key] = original[key]\n",
       "    return obj;\n",
       "  }, {});\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.mouse_event = function(event, name) {\n",
       "    var canvas_pos = mpl.findpos(event)\n",
       "\n",
       "    if (name === 'button_press')\n",
       "    {\n",
       "        this.canvas.focus();\n",
       "        this.canvas_div.focus();\n",
       "    }\n",
       "\n",
       "    var x = canvas_pos.x * mpl.ratio;\n",
       "    var y = canvas_pos.y * mpl.ratio;\n",
       "\n",
       "    this.send_message(name, {x: x, y: y, button: event.button,\n",
       "                             step: event.step,\n",
       "                             guiEvent: simpleKeys(event)});\n",
       "\n",
       "    /* This prevents the web browser from automatically changing to\n",
       "     * the text insertion cursor when the button is pressed.  We want\n",
       "     * to control all of the cursor setting manually through the\n",
       "     * 'cursor' event from matplotlib */\n",
       "    event.preventDefault();\n",
       "    return false;\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._key_event_extra = function(event, name) {\n",
       "    // Handle any extra behaviour associated with a key event\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.key_event = function(event, name) {\n",
       "\n",
       "    // Prevent repeat events\n",
       "    if (name == 'key_press')\n",
       "    {\n",
       "        if (event.which === this._key)\n",
       "            return;\n",
       "        else\n",
       "            this._key = event.which;\n",
       "    }\n",
       "    if (name == 'key_release')\n",
       "        this._key = null;\n",
       "\n",
       "    var value = '';\n",
       "    if (event.ctrlKey && event.which != 17)\n",
       "        value += \"ctrl+\";\n",
       "    if (event.altKey && event.which != 18)\n",
       "        value += \"alt+\";\n",
       "    if (event.shiftKey && event.which != 16)\n",
       "        value += \"shift+\";\n",
       "\n",
       "    value += 'k';\n",
       "    value += event.which.toString();\n",
       "\n",
       "    this._key_event_extra(event, name);\n",
       "\n",
       "    this.send_message(name, {key: value,\n",
       "                             guiEvent: simpleKeys(event)});\n",
       "    return false;\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.toolbar_button_onclick = function(name) {\n",
       "    if (name == 'download') {\n",
       "        this.handle_save(this, null);\n",
       "    } else {\n",
       "        this.send_message(\"toolbar_button\", {name: name});\n",
       "    }\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.toolbar_button_onmouseover = function(tooltip) {\n",
       "    this.message.textContent = tooltip;\n",
       "};\n",
       "mpl.toolbar_items = [[\"Home\", \"Reset original view\", \"fa fa-home icon-home\", \"home\"], [\"Back\", \"Back to  previous view\", \"fa fa-arrow-left icon-arrow-left\", \"back\"], [\"Forward\", \"Forward to next view\", \"fa fa-arrow-right icon-arrow-right\", \"forward\"], [\"\", \"\", \"\", \"\"], [\"Pan\", \"Pan axes with left mouse, zoom with right\", \"fa fa-arrows icon-move\", \"pan\"], [\"Zoom\", \"Zoom to rectangle\", \"fa fa-square-o icon-check-empty\", \"zoom\"], [\"\", \"\", \"\", \"\"], [\"Download\", \"Download plot\", \"fa fa-floppy-o icon-save\", \"download\"]];\n",
       "\n",
       "mpl.extensions = [\"eps\", \"jpeg\", \"pdf\", \"png\", \"ps\", \"raw\", \"svg\", \"tif\"];\n",
       "\n",
       "mpl.default_extension = \"png\";var comm_websocket_adapter = function(comm) {\n",
       "    // Create a \"websocket\"-like object which calls the given IPython comm\n",
       "    // object with the appropriate methods. Currently this is a non binary\n",
       "    // socket, so there is still some room for performance tuning.\n",
       "    var ws = {};\n",
       "\n",
       "    ws.close = function() {\n",
       "        comm.close()\n",
       "    };\n",
       "    ws.send = function(m) {\n",
       "        //console.log('sending', m);\n",
       "        comm.send(m);\n",
       "    };\n",
       "    // Register the callback with on_msg.\n",
       "    comm.on_msg(function(msg) {\n",
       "        //console.log('receiving', msg['content']['data'], msg);\n",
       "        // Pass the mpl event to the overriden (by mpl) onmessage function.\n",
       "        ws.onmessage(msg['content']['data'])\n",
       "    });\n",
       "    return ws;\n",
       "}\n",
       "\n",
       "mpl.mpl_figure_comm = function(comm, msg) {\n",
       "    // This is the function which gets called when the mpl process\n",
       "    // starts-up an IPython Comm through the \"matplotlib\" channel.\n",
       "\n",
       "    var id = msg.content.data.id;\n",
       "    // Get hold of the div created by the display call when the Comm\n",
       "    // socket was opened in Python.\n",
       "    var element = $(\"#\" + id);\n",
       "    var ws_proxy = comm_websocket_adapter(comm)\n",
       "\n",
       "    function ondownload(figure, format) {\n",
       "        window.open(figure.imageObj.src);\n",
       "    }\n",
       "\n",
       "    var fig = new mpl.figure(id, ws_proxy,\n",
       "                           ondownload,\n",
       "                           element.get(0));\n",
       "\n",
       "    // Call onopen now - mpl needs it, as it is assuming we've passed it a real\n",
       "    // web socket which is closed, not our websocket->open comm proxy.\n",
       "    ws_proxy.onopen();\n",
       "\n",
       "    fig.parent_element = element.get(0);\n",
       "    fig.cell_info = mpl.find_output_cell(\"<div id='\" + id + \"'></div>\");\n",
       "    if (!fig.cell_info) {\n",
       "        console.error(\"Failed to find cell for figure\", id, fig);\n",
       "        return;\n",
       "    }\n",
       "\n",
       "    var output_index = fig.cell_info[2]\n",
       "    var cell = fig.cell_info[0];\n",
       "\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_close = function(fig, msg) {\n",
       "    var width = fig.canvas.width/mpl.ratio\n",
       "    fig.root.unbind('remove')\n",
       "\n",
       "    // Update the output cell to use the data from the current canvas.\n",
       "    fig.push_to_output();\n",
       "    var dataURL = fig.canvas.toDataURL();\n",
       "    // Re-enable the keyboard manager in IPython - without this line, in FF,\n",
       "    // the notebook keyboard shortcuts fail.\n",
       "    IPython.keyboard_manager.enable()\n",
       "    $(fig.parent_element).html('<img src=\"' + dataURL + '\" width=\"' + width + '\">');\n",
       "    fig.close_ws(fig, msg);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.close_ws = function(fig, msg){\n",
       "    fig.send_message('closing', msg);\n",
       "    // fig.ws.close()\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.push_to_output = function(remove_interactive) {\n",
       "    // Turn the data on the canvas into data in the output cell.\n",
       "    var width = this.canvas.width/mpl.ratio\n",
       "    var dataURL = this.canvas.toDataURL();\n",
       "    this.cell_info[1]['text/html'] = '<img src=\"' + dataURL + '\" width=\"' + width + '\">';\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.updated_canvas_event = function() {\n",
       "    // Tell IPython that the notebook contents must change.\n",
       "    IPython.notebook.set_dirty(true);\n",
       "    this.send_message(\"ack\", {});\n",
       "    var fig = this;\n",
       "    // Wait a second, then push the new image to the DOM so\n",
       "    // that it is saved nicely (might be nice to debounce this).\n",
       "    setTimeout(function () { fig.push_to_output() }, 1000);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._init_toolbar = function() {\n",
       "    var fig = this;\n",
       "\n",
       "    var nav_element = $('<div/>')\n",
       "    nav_element.attr('style', 'width: 100%');\n",
       "    this.root.append(nav_element);\n",
       "\n",
       "    // Define a callback function for later on.\n",
       "    function toolbar_event(event) {\n",
       "        return fig.toolbar_button_onclick(event['data']);\n",
       "    }\n",
       "    function toolbar_mouse_event(event) {\n",
       "        return fig.toolbar_button_onmouseover(event['data']);\n",
       "    }\n",
       "\n",
       "    for(var toolbar_ind in mpl.toolbar_items){\n",
       "        var name = mpl.toolbar_items[toolbar_ind][0];\n",
       "        var tooltip = mpl.toolbar_items[toolbar_ind][1];\n",
       "        var image = mpl.toolbar_items[toolbar_ind][2];\n",
       "        var method_name = mpl.toolbar_items[toolbar_ind][3];\n",
       "\n",
       "        if (!name) { continue; };\n",
       "\n",
       "        var button = $('<button class=\"btn btn-default\" href=\"#\" title=\"' + name + '\"><i class=\"fa ' + image + ' fa-lg\"></i></button>');\n",
       "        button.click(method_name, toolbar_event);\n",
       "        button.mouseover(tooltip, toolbar_mouse_event);\n",
       "        nav_element.append(button);\n",
       "    }\n",
       "\n",
       "    // Add the status bar.\n",
       "    var status_bar = $('<span class=\"mpl-message\" style=\"text-align:right; float: right;\"/>');\n",
       "    nav_element.append(status_bar);\n",
       "    this.message = status_bar[0];\n",
       "\n",
       "    // Add the close button to the window.\n",
       "    var buttongrp = $('<div class=\"btn-group inline pull-right\"></div>');\n",
       "    var button = $('<button class=\"btn btn-mini btn-primary\" href=\"#\" title=\"Stop Interaction\"><i class=\"fa fa-power-off icon-remove icon-large\"></i></button>');\n",
       "    button.click(function (evt) { fig.handle_close(fig, {}); } );\n",
       "    button.mouseover('Stop Interaction', toolbar_mouse_event);\n",
       "    buttongrp.append(button);\n",
       "    var titlebar = this.root.find($('.ui-dialog-titlebar'));\n",
       "    titlebar.prepend(buttongrp);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._root_extra_style = function(el){\n",
       "    var fig = this\n",
       "    el.on(\"remove\", function(){\n",
       "\tfig.close_ws(fig, {});\n",
       "    });\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._canvas_extra_style = function(el){\n",
       "    // this is important to make the div 'focusable\n",
       "    el.attr('tabindex', 0)\n",
       "    // reach out to IPython and tell the keyboard manager to turn it's self\n",
       "    // off when our div gets focus\n",
       "\n",
       "    // location in version 3\n",
       "    if (IPython.notebook.keyboard_manager) {\n",
       "        IPython.notebook.keyboard_manager.register_events(el);\n",
       "    }\n",
       "    else {\n",
       "        // location in version 2\n",
       "        IPython.keyboard_manager.register_events(el);\n",
       "    }\n",
       "\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._key_event_extra = function(event, name) {\n",
       "    var manager = IPython.notebook.keyboard_manager;\n",
       "    if (!manager)\n",
       "        manager = IPython.keyboard_manager;\n",
       "\n",
       "    // Check for shift+enter\n",
       "    if (event.shiftKey && event.which == 13) {\n",
       "        this.canvas_div.blur();\n",
       "        // select the cell after this one\n",
       "        var index = IPython.notebook.find_cell_index(this.cell_info[0]);\n",
       "        IPython.notebook.select(index + 1);\n",
       "    }\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_save = function(fig, msg) {\n",
       "    fig.ondownload(fig, null);\n",
       "}\n",
       "\n",
       "\n",
       "mpl.find_output_cell = function(html_output) {\n",
       "    // Return the cell and output element which can be found *uniquely* in the notebook.\n",
       "    // Note - this is a bit hacky, but it is done because the \"notebook_saving.Notebook\"\n",
       "    // IPython event is triggered only after the cells have been serialised, which for\n",
       "    // our purposes (turning an active figure into a static one), is too late.\n",
       "    var cells = IPython.notebook.get_cells();\n",
       "    var ncells = cells.length;\n",
       "    for (var i=0; i<ncells; i++) {\n",
       "        var cell = cells[i];\n",
       "        if (cell.cell_type === 'code'){\n",
       "            for (var j=0; j<cell.output_area.outputs.length; j++) {\n",
       "                var data = cell.output_area.outputs[j];\n",
       "                if (data.data) {\n",
       "                    // IPython >= 3 moved mimebundle to data attribute of output\n",
       "                    data = data.data;\n",
       "                }\n",
       "                if (data['text/html'] == html_output) {\n",
       "                    return [cell, data, j];\n",
       "                }\n",
       "            }\n",
       "        }\n",
       "    }\n",
       "}\n",
       "\n",
       "// Register the function which deals with the matplotlib target/channel.\n",
       "// The kernel may be null if the page has been refreshed.\n",
       "if (IPython.notebook.kernel != null) {\n",
       "    IPython.notebook.kernel.comm_manager.register_target('matplotlib', mpl.mpl_figure_comm);\n",
       "}\n"
      ],
      "text/plain": [
       "<IPython.core.display.Javascript object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<img src=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAoAAAAHgCAYAAAA10dzkAAAgAElEQVR4Xu3d660dyXWAUd40nIaVhtOQwhAUhiYNpyGn4TRoHAsEhBGbp+bs6q79WPP39qNq7RrgQ18O5+ubfwgQIECAAAECBEYJfI3arc0SIECAAAECBAh8E4AOAQECBAgQIEBgmIAAHDZw2yVAgAABAgQICEBngAABAgQIECAwTEAADhu47RIgQIAAAQIEBKAzQIAAAQIECBAYJiAAhw3cdgkQIECAAAECAtAZIECAAAECBAgMExCAwwZuuwQIECBAgAABAegMECBAgAABAgSGCQjAYQO3XQIECBAgQICAAHQGCBAgQIAAAQLDBATgsIHbLgECBAgQIEBAADoDBAgQIECAAIFhAgJw2MBtlwABAgQIECAgAJ0BAgQIECBAgMAwAQE4bOC2S4AAAQIECBAQgM4AAQIECBAgQGCYgAAcNnDbJUCAAAECBAgIQGeAAAECBAgQIDBMQAAOG7jtEiBAgAABAgQEoDNAgAABAgQIEBgmIACHDdx2CRAgQIAAAQIC0BkgQIAAAQIECAwTEIDDBm67BAgQIECAAAEB6AwQIECAAAECBIYJCMBhA7ddAgQIECBAgIAAdAYIECBAgAABAsMEBOCwgdsuAQIECBAgQEAAOgMECBAgQIAAgWECAnDYwG2XAAECBAgQICAAnQECBAgQIECAwDABAThs4LZLgAABAgQIEBCAzgABAgQIECBAYJiAABw2cNslQIAAAQIECAhAZ4AAAQIECBAgMExAAA4buO0SIECAAAECBASgM0CAAAECBAgQGCYgAIcN3HYJECBAgAABAgLQGSBAgAABAgQIDBMQgMMGbrsECBAgQIAAAQHoDBAgQIAAAQIEhgkIwGEDt10CBAgQIECAgAB0BggQIECAAAECwwQE4LCB2y4BAgQIECBAQAA6AwQIECBAgACBYQICcNjAbZcAAQIECBAgIACdAQIECBAgQIDAMAEBOGzgtkuAAAECBAgQEIDOAAECBAgQIEBgmIAAHDZw2yVAgAABAgQICEBngAABAgQIECAwTEAADhu47RIgQIAAAQIEBKAzQIAAAQIECBAYJiAAhw3cdgkQIECAAAECAtAZIECAAAECBAgMExCAwwZuuwQIECBAgAABAegMECBAgAABAgSGCQjAYQO3XQIECBAgQICAAHQGCBAgQIAAAQLDBATgsIHbLgECBAgQIEBAADoDBAgQIECAAIFhAgJw2MBtlwABAgQIECAgAJ0BAgQIECBAgMAwAQE4bOC2S4AAAQIECBAQgM4AAQIECBAgQGCYgAAcNnDbJUCAAAECBAgIQGeAAAECBAgQIDBMQAAOG7jtEiBAgAABAgQEoDNAgAABAgQIEBgmIACHDdx2CRAgQIAAAQIC0BkgQIAAAQIECAwTEIDDBm67BAgQIECAAAEB6AwQIECAAAECBIYJCMBhA7ddAgQIECBAgIAAdAYIECBAgAABAsMEBGBw4N+/f/8efITbCRAgQIAAgYcFvr6+RjfQ6M3vOGsCcIeiZxAgQIAAgWcFBOCz3u3eJgDbjdSGCBAgQGCAgAAcMOQ7tygA79T1bAIECBAgcI+AALzHdcxTBeCYUdsoAQIECDQSEICNhnliKwLwhLp3EiBAgACBmIAAjPmNv1sAjj8CAAgQIECgoIAALDi0TEsWgJmmYS0ECBAgQGBNQACuObnqQkAAOhoECBAgQKCegACsN7NUKxaAqcZhMQQIECBAYElAAC4xuehKQAA6GwQIECBAoJ6AAKw3s1QrFoCpxmExBAgQIEBgSUAALjG5yBdAZ4AAAQIECPQREIB9ZnlkJ74AHmH3UgIECBAgEBIQgCE+NwtAZ4AAAQIECNQTEID1ZpZqxQIw1TgshgABAgQILAkIwCUmF10JCEBngwABAgQI1BMQgPVmlmrFAjDVOCyGAAECBAgsCQjAJSYX+QLoDBAgQIAAgT4CArDPLI/sxBfAI+xeSoAAAQIEQgICMMTnZgHoDBAgQIAAgXoCArDezFKtWACmGofFECBAgACBJQEBuMTkoisBAehsECBAgACBegICsN7MUq1YAKYah8UQIECAAIElAQG4xOQiXwCdAQIECBAg0EdAAPaZ5ZGd+AJ4hN1LCRAgQIBASEAAhvjcLACdAQIECBAgUE9AANabWaoVC8BU47AYAgQIECCwJCAAl5hcdCUgAJ0NAgQIECBQT0AA1ptZqhULwFTjsBgCBAgQILAkIACXmFzkC6AzQIAAAQIE+ggIwD6zPLITXwCPsHspAQIECBAICQjAEJ+bBaAzQIAAAQIE6gkIwHozS7ViAZhqHBZDgAABAgSWBATgEpOLrgQEoLNBgAABAgTqCQjAejNLtWIBmGocFkOAAAECBJYEBOASk4t8AXQGCBAgQIBAHwEB2GeWR3biC+ARdi8lQIAAAQIhAQEY4nOzAHQGCBAgQIBAPQEBWG9mqVYsAFONw2IIECBAgMCSgABcYnLRlYAAdDYIECBAgEA9AQFYb2apViwAU43DYggQIECAwJKAAFxicpEvgM4AAQIECBDoIyAA+8zyyE58ATzC7qUECBAgQCAkIABDfG4WgM4AAQIECBCoJyAA680s1YoFYKpxWAwBAgQIEFgSEIBLTC66EhCAzgYBAgQIEKgnIADrzSzVigVgqnFYDAECBAgQWBIQgEtMLvIF0BkgQIAAAQJ9BARgn1ke2YkvgEfYvZQAAQIECIQEBGCIz80C0BkgQIAAAQL1BARgvZmlWrEATDUOiyFAgAABAksCAnCJyUVXAgLQ2SBAgAABAvUEBGC9maVasQBMNQ6LIUCAAAECSwICcInJRb4AOgMECBAgQKCPgADsM8sjO/EF8Ai7lxIgQIAAgZCAAAzxuVkAOgMECBAgQKCegACsN7NUKxaAqcZhMQQIECBAYElAAC4xuehKQAA6GwQIECBAoJ6AAKw3s1QrFoCpxmExBAgQIEBgSUAALjG5yBdAZ4AAAQIECPQREIB9ZnlkJ74AHmH3UgIECBAgEBIQgCE+NwtAZ4AAAQIECNQTEID1ZpZqxQIw1TgshgABAgQILAkIwCUmF10JCEBngwABAgQI1BMQgPVmlmrFAjDVOCyGAAECBAgsCQjAJSYX+QLoDBAgQIAAgT4CArDPLI/sxBfAI+xeSoAAAQIEQgICMMTnZgHoDBAgQIAAgXoCArDezFKtWACmGofFECBAgACBJQEBuMTkoisBAehsECBAgACBegICsN7MUq1YAKYah8UQIECAAIElAQG4xOQiXwCdAQIECBAg0EdAAPaZ5ZGd+AJ4hN1LCRAgQIBASEAAhvjcLACdAQIECBAgUE9AANabWaoVC8BU47AYAgQIECCwJCAAl5hcdCUgAJ0NAgQIECBQT0AA1ptZqhULwFTjsBgCBAgQILAkIACXmFzkC6AzQIAAAQIE+ggIwD6zPLITXwCPsHspAQIECBAICQjAEJ+bBaAzQIAAAQIE6gkIwHozS7ViAZhqHBZDgAABAgSWBATgEpOLrgQEoLNBgAABAgTqCQjAejNLtWIBmGocFkOAAAECBJYEBOASk4t8AXQGCBAgQIBAHwEB2GeWR3biC+ARdi8lQIAAAQIhAQEY4nOzAHQGCBAgQIBAPQEBWG9mqVYsAFONw2IIECBAgMCSgABcYnLRlYAAdDYIECBAgEA9AQFYb2apViwAU43DYggQIECAwJKAAFxicpEvgM4AAQIECBDoIyAA+8zyyE58ATzC7qUECBAgQCAkIABDfG4WgM4AAQIECBCoJyAA680s1YoFYKpxWAwBAgQIEFgSEIBLTC66EhCAzgYBAgQIEKgnIADrzSzVigVgqnFYDAECBAgQWBIQgEtMLvIF0BkgQIAAAQJ9BARgn1ke2YkvgEfYvZQAAQIECIQEBGCIz80C0BkgQIAAAQL1BARgvZmlWrEATDUOiyFAgAABAksCAnCJyUVXAgLQ2SBAgAABAvUEBGC9maVasQBMNQ6LIUCAAAECSwICcInJRb4AOgMECBAgQKCPgADsM8sjO/EF8Ai7lxIgQIAAgZCAAAzxuVkAOgMECBAgQKCegACsN7NUKxaAqcZhMQQIECBAYElAAC4xuehKQAA6GwQIECBAoJ6AAKw3s1QrFoCpxmExBAgQIEBgSUAALjG5yBdAZ4AAAQIECPQREIB9ZnlkJ74AHmH3UgIECBAgEBIQgCE+NwtAZ4AAAQIECNQTEID1ZpZqxQIw1TgshgABAgQILAkIwCUmF10JCEBngwABAgQI1BMQgPVmlmrFAjDVOCyGAAECBAgsCQjAJSYX+QLoDBAgQIAAgT4CArDPLI/sxBfAI+xeSoAAAQIEQgICMMTnZgHoDBAgQIAAgXoCArDezFKtWACmGofFECBAgACBJQEBuMTkoisBAehsECBAgACBegICsN7MUq1YAKYah8UQIECAAIElAQG4xOQiXwCdAQIECBAg0EdAAPaZ5ZGd+AJ4hN1LCRAgQIBASEAAhvjcLACdAQIECBAgUE9AANabWaoVC8BU47AYAgQIECCwJCAAl5hcdCUgAJ0NAgQIECBQT0AA1ptZqhULwFTjsBgCBAgQILAkIACXmFzkC6AzQIAAAQIE+ggIwD6zPLITXwCPsHspAQIECBAICQjAEJ+bBaAzQIAAAQIE6gkIwHozS7ViAZhqHBZDgAABAgSWBATgEpOLrgQEoLNBgAABAgTqCQjAejNLtWIBmGocFkOAAAECBJYEBOASk4t8AXQGCBAgQIBAHwEB2GeWR3biC+ARdi8lQIAAAQIhAQEY4nOzAHQGCBAgQIBAPQEBWG9mqVYsAFONw2IIECBAgMCSgABcYnLRlYAAdDYIECBAgEA9AQFYb2apViwAU43DYggQIECAwJKAAFxicpEvgM4AAQIECBDoIyAA+8zyyE58ATzC7qUECBAgQCAkIABDfG4WgM4AAQIECBCoJyAA680s1YoFYKpxWAwBAgQIEFgSEIBLTC66EhCAzgYBAgQIEKgnIADrzSzVigVgqnFYDAECBAgQWBIQgEtMLvIF0BkgMFfgf377y7f//PPf5wLYOYGGAgKw4VCf3JIvgE9qexeBZwRewfezf0TgM/7eQuAJAQH4hHLjdwjAxsO1tXECV+H3A0IAjjsSNtxYQAA2Hu4TWxOATyh7B4FnBATgM87eQiCDgADMMIXCaxCAhYdn6QR+JyAAHQkCcwQE4JxZ37JTAXgLq4cSOCLwLgBfi/Jr4COj8VIC2wUE4HbSWQ8UgLPmbbf9Bd5FoADsfwbscIaAAJwx59t2KQBvo/VgAkcEBOARdi8l8LiAAHycvNcLBWCvedoNgXcB6NfAzgiBHgICsMccj+1CAB6j92ICtwm8i0C/Br6N3oMJPCYgAB+j7vkiAdhzrnY1W0AAzp6/3c8QEIAz5nzbLgXgbbQeTOCYgAA8Ru/FBB4TEICPUfd8kQDsOVe7mi3wLgBfOn4NPPuM2H19AQFYf4ZHdyAAj/J7OYFbBATgLaweSiCVgABMNY56ixGA9WZmxQTeCQjAd0J+TqC+gACsP8OjOxCAR/m9nMAtAgLwFlYPJZBKQACmGke9xQjAejOzYgLvBATgOyE/J1BfQADWn+HRHQjAo/xeTuA2gXcR6D8CuY3egwk8IiAAH2Hu+xIB2He2djZbQADOnr/d9xcQgP1nfOsOBeCtvB5O4JiAADxG78UEHhEQgI8w932JAOw7WzubLSAAZ8/f7vsLCMD+M751hwLwVl4PJ3BUQAQe5fdyArcKCMBbefs/XAD2n7EdzhUQgHNnb+f9BQRg/xnfukMBeCuvhxM4KiAAj/J7OYFbBQTgrbz9Hy4A+8/YDucKCMC5s7fz/gICsP+Mb92hALyV18MJHBV4F4Cvxfn7AI+OyMsJfCwgAD+mc+NLQAA6BwR6C7yLQAHYe/5211dAAPad7SM7E4CPMHsJgWMCAvAYvRcTuFVAAN7K2//hArD/jO1wtoAAnD1/u+8rIAD7zvaRnQnAR5i9hMAxgXcB+FqYXwMfG48XE/hYQAB+TOfGl4AAdA4I9Bd4F4ECsP8ZsMN+AgKw30wf3ZEAfJTbywgcERCAR9i9lMCtAgLwVt7+DxeA/WdshwQEoDNAoJ+AAOw300d3JAAf5fYyAkcE3gXga1F+DXxkNF5K4GMBAfgxnRtfAgLQOSDQX0AA9p+xHc4TEIDzZr51xwJwK6eHEUgpIABTjsWiCIQEBGCIz80C0Bkg0F9gJQD9Grj/ObDDXgICsNc8H9+NAHyc3AsJHBFYiUB/DvDIaLyUwEcCAvAjNjf9EBCAzgKBGQICcMac7XKOgACcM+tbdioAb2H1UALpBARgupFYEIGQgAAM8blZADoDBGYICMAZc7bLOQICcM6sb9mpALyF1UMJpBR4F4H+DGDKsVkUgZ8KCEAHIyQgAEN8biZQSkAAlhqXxRL4pYAAdEBCAgIwxOdmAqUEBGCpcVksAQH4C4Ev5yMmIABjfu4mUElAAFaalrUS+LWAL4BOSEhAAIb43EygnIAILDcyCybwUwEB6GCEBARgiM/NBMoJCMByI7NgAgLwJwJ+BRz8F0MABgHdTqCYgAAsNjDLJXAh4AugoxESEIAhPjcTKCcgAMuNzIIJ+ALoC+D+fwsE4H5TTySQWeBdAL7W7u8DzDxBayPwTwFfAJ2EkIAADPG5mUBJgXcRKABLjtWihwkIwGED371dAbhb1PMI5BcQgPlnZIUE3gkIwHdCfv5LAQHogBCYJyAA583cjvsJCMB+M310RwLwUW4vI5BC4F0Avhbp18ApRmURBC4FBKDDERIQgCE+NxMoK/AuAgVg2dFa+BABAThk0HdtUwDeJeu5BHILCMDc87E6Au8EBOA7IT//pYAAdEAIzBQQgDPnbtd9BARgn1ke2YkAPMLupQSOC7wLwNcC/Rr4+JgsgMClgAB0OEICAjDE52YCZQUEYNnRWTiB/xcQgA5CSEAAhvjcTKCsgAAsOzoLJyAAXwHsHMQEBGDMz90EqgqsBKBfA1edrnVPEPAFcMKUb9yjALwR16MJJBdYiUB/DjD5EC1vrIAAHDv6PRsXgHscPYVARQEBWHFq1kzgnwIC0EkICQjAEJ+bCZQWEIClx2fxwwUE4PADEN2+AIwKup9AXQEBWHd2Vk5AADoDIQEBGOJzM4HyAu8i0J8BLD9iG2gqIACbDvapbQnAp6S9h0BOAQGYcy5WReCdgAB8J+TnvxQQgA4IgdkCAnD2/O2+roAArDu7FCsXgCnGYBEEjgkIwGP0XkwgJCAAQ3xuFoDOAAECItAZIFBPQADWm1mqFQvAVOOwGAJHBATgEXYvJRASEIAhPjcLQGeAAAEB6AwQqCcgAOvNLNWKBWCqcVgMgSMCAvAIu5cSCAkIwBCfmwWgM0CAwLsAfAn5+wCdEwK5BARgrnmUW40ALDcyCyZwi8C7CBSAt7B7KIGPBQTgx3RufAkIQOeAAIGXgAB0DgjUEhCAteaVbrUCMN1ILIjAEQEBeITdSwl8LCAAP6Zzoy+AzgABAj8E3gXg6zq/BnZeCOQREIB5ZlFyJb4AlhybRRO4ReBdBArAW9g9lMBHAgLwIzY3/RAQgM4CAQKrXwEFoLNCII+AAMwzi5IrEYAlx2bRBG4R8AXwFlYPJXCLgAC8hXXOQwXgnFnbKYF3Au8C8HW/r4DvFP2cwDMCAvAZ57ZvEYBtR2tjBP6wgAD8w2RuIHBMQAAeo+/xYgHYY452QWCHgADcoegZBJ4REIDPOLd9iwBsO1obI/CHBQTgHyZzA4FjAgLwGH2PFwvAHnO0CwI7BATgDkXPIPCMgAB8xrntWwRg29HaGIGPBN5FoP8I5CNWNxHYLiAAt5POeqAAnDVvuyXwTkAAvhPycwI5BARgjjmUXYUALDs6Cydwi4AAvIXVQwlsFxCA20lnPVAAzpq33RJYERCBK0quIXBWQACe9S//dgFYfoQ2QGC7gADcTuqBBLYLCMDtpLMeKABnzdtuCawICMAVJdcQOCsgAM/6l3+7ACw/Qhsg8FOBr6+vj2X+8fc///LeP/3lt4+f/fsbv3//vu1ZHkRgkoAAnDTtG/YqAG9A9UgCCQQiAfha/lMRKAATHBZLKCkgAEuOLc+iBWCeWVgJgZ0CuwLwv//3378G/td//PZt11dAAbhz6p41SUAATpr2DXsVgDegeiSBBAJ3BuBre3/725+27FIAbmH0kIECAnDg0HduWQDu1PQsAnkEdgTgz77+/dihAMwzayuZKSAAZ859264F4DZKDyKQSuDuANz1FdAXwFTHxmIKCQjAQsPKuFQBmHEq1kQgLhANwNcK/vrXf/xyITu+AgrA+Kw9YaaAAJw59227FoDbKD2IQCqBuwNw138IIgBTHRuLKSQgAAsNK+NSBWDGqVgTgbjAjgB8/VUwV38OUADGZ+QJBCICAjCi595vAtAhINBT4M4AfMXf658dfxWML4A9z59d3S8gAO83bv0GAdh6vDY3WGBHAL74/vUvhN4RfL8fiQAcfEhtPSQgAEN8bhaAzgCBngK7AvBuHQF4t7DndxUQgF0n+9y+/I84n7P2JgIECBAgsEvg8//h964VHHzO6M3vcPcFcIeiZxDIJ+ALYL6ZWBGBnQK+AO7UHPgsAThw6LY8QkAAjhizTQ4WEICDh79j6wJwh6JnEMgnIADzzcSKCOwUEIA7NQc+SwAOHLotjxAQgCPGbJODBQTg4OHv2LoA3KHoGQTyCQjAfDOxIgI7BQTgTs2BzxKAA4duyyMEBOCIMdvkYAEBOHj4O7YuAHcoegaBfAICMN9MrIjATgEBuFNz4LME4MCh2/IIAQE4Ysw2OVhAAA4e/o6tC8Adip5BIJ+AAMw3EysisFNAAO7UHPgsAThw6LY8QkAAjhizTQ4WEICDh79j6wJwh6JnEMgnIADzzcSKCOwUEIA7NQc+SwAOHLotjxAQgCPGbJODBQTg4OHv2LoA3KHoGQQIECBA4FkBAfisd7u3CcB2I7UhAgQIEBggIAAHDPnOLQrAO3U9mwABAgQI3CMgAO9xHfNUAThm1DZKgAABAo0EBGCjYZ7YigA8oe6dBAgQIEAgJiAAY37j7xaA448AAAIECBAoKCAACw4t05IFYKZpWAsBAgQIEFgTEIBrTq66EBCAjgYBAgQIEKgnIADrzSzVigVgqnFYDAECBAgQWBIQgEtMLroSEIDOBgECBAgQqCcgAOvNLNWKBWCqcVgMAQIECBBYEhCAS0wu8gXQGSBAgAABAn0EBGCfWR7ZiS+AR9i9lAABAgQIhAQEYIjPzQLQGSBAgAABAvUEBGC9maVasQBMNQ6LIUCAAAECSwICcInJRVcCAtDZIECAAAEC9QQEYL2ZpVqxAEw1DoshQIAAAQJLAgJwiclFvgA6AwQIECBAoI+AAOwzyyM78QXwCLuXEiBAgACBkIAADPG5WQA6AwQIECBAoJ6AAKw3s1QrFoCpxmExBAgQIEBgSUAALjG56EpAADobBAgQIECgnoAArDezVCsWgKnGYTEECBAgQGBJQAAuMbnIF0BngAABAgQI9BEQgH1meWQnvgAeYfdSAgQIECAQEhCAIT43C0BngAABAgQI1BMQgPVmlmrFAjDVOCyGAAECBAgsCQjAJSYXXQkIQGeDAAECBAjUExCA9WaWasUCMNU4LIYAAQIECCwJCMAlJhf5AugMECBAgACBPgICsM8sj+zEF8Aj7F5KgAABAgRCAgIwxOdmAegMECBAgACBegICsN7MUq1YAKYah8UQIECAAIElAQG4xOSiKwEB6GwQIECAAIF6AgKw3sxSrVgAphqHxRAgQIAAgSUBAbjE5CJfAJ0BAgQIECDQR0AA9pnlkZ34AniE3UsJECBAgEBIQACG+NwsAJ0BAgQIECBQT0AA1ptZqhULwFTjsBgCBAgQILAkIACXmFx0JSAAnQ0CBAgQIFBPQADWm1mqFQvAVOOwGAIECBAgsCQgAJeYXOQLoDNAgAABAgT6CAjAPrM8shNfAI+weykBAgQIEAgJCMAQn5sFoDNAgAABAgTqCQjAejNLtWIBmGocFkOAAAECBJYEBOASk4uuBASgs0GAAAECBOoJCMB6M0u1YgGYahwWQ4AAAQIElgQE4BKTi3wBdAYIECBAgEAfAQHYZ5ZHduIL4BF2LyVAgAABAiEBARjic7MAdAYIECBAgEA9AQFYb2apViwAU43DYggQIECAwJKAAFxictGVgAB0NggQIECAQD0BAVhvZqlWLABTjcNiCBAgQIDAkoAAXGJykS+AzgABAgQIEOgjIAD7zPLITnwBPMLupQQIECBAICQgAEN8bhaAzgABAgQIEKgnIADrzSzVigVgqnFYDAECBAgQWBIQgEtMLroSEIDOBgECBAgQqCcgAOvNLNWKBWCqcVgMAQIECBBYEhCAS0wu8gXQGSBAgAABAn0EBGCfWR7ZiS+AR9i9lAABAgQIhAQEYIjPzQLQGSBAgAABAvUEBGC9maVasQBMNQ6LIUCAAAECSwICcInJRVcCAtDZIECAAAEC9QQEYL2ZpVqxAEw1DoshQIAAAQJLAgJwiclFvgA6AwQIECBAoI+AAOwzyyM78QXwCLuXEiBAgACBkIAADPG5WQA6AwQIECBAoJ6AAKw3s1QrFoCpxmExBAgQIEBgSUAALjG56EpAADobBAgQIECgnoAArDczKyZAgAABAgQIEAgIfAXudSsBAgQIECBAgEBBAQFYcGiWTIAAAQIECBCICAjAiJ57CRAgQIAAAQIFBQRgwaFZMgECBAgQIEAgIiAAI3ruJUCAAAECBAgUFBCABYdmyQQIECBAgACBiIAAjOi5lwABAgQIECBQUEAAFhyaJRMgQIAAAQIEIgICMKLnXgIECBAgQIBAQQEBWHBolkyAAAECBAgQiAgIwIieewkQIECAAAECBQUEYNC4WUgAAAOQSURBVMGhWTIBAgQIECBAICIgACN67iVAgAABAgQIFBQQgAWHZskECBAgQIAAgYiAAIzouZcAAQIECBAgUFBAABYcmiUTIECAAAECBCICAjCi514CBAgQIECAQEEBAVhwaJZMgAABAgQIEIgICMCInnsJECBAgAABAgUFBGDBoVkyAQIECBAgQCAiIAAjeu4lQIAAAQIECBQUEIAFh2bJBAgQIECAAIGIgACM6LmXAAECBAgQIFBQQAAWHJolEyBAgAABAgQiAgIwoudeAgQIECBAgEBBAQFYcGiWTIAAAQIECBCICAjAiJ57CRAgQIAAAQIFBQRgwaFZMgECBAgQIEAgIiAAI3ruJUCAAAECBAgUFBCABYdmyQQIECBAgACBiIAAjOi5lwABAgQIECBQUEAAFhyaJRMgQIAAAQIEIgICMKLnXgIECBAgQIBAQQEBWHBolkyAAAECBAgQiAgIwIieewkQIECAAAECBQUEYMGhWTIBAgQIECBAICIgACN67iVAgAABAgQIFBQQgAWHZskECBAgQIAAgYiAAIzouZcAAQIECBAgUFBAABYcmiUTIECAAAECBCICAjCi514CBAgQIECAQEEBAVhwaJZMgAABAgQIEIgICMCInnsJECBAgAABAgUFBGDBoVkyAQIECBAgQCAiIAAjeu4lQIAAAQIECBQUEIAFh2bJBAgQIECAAIGIgACM6LmXAAECBAgQIFBQQAAWHJolEyBAgAABAgQiAgIwoudeAgQIECBAgEBBAQFYcGiWTIAAAQIECBCICAjAiJ57CRAgQIAAAQIFBQRgwaFZMgECBAgQIEAgIiAAI3ruJUCAAAECBAgUFBCABYdmyQQIECBAgACBiIAAjOi5lwABAgQIECBQUEAAFhyaJRMgQIAAAQIEIgICMKLnXgIECBAgQIBAQQEBWHBolkyAAAECBAgQiAgIwIieewkQIECAAAECBQUEYMGhWTIBAgQIECBAICIgACN67iVAgAABAgQIFBQQgAWHZskECBAgQIAAgYiAAIzouZcAAQIECBAgUFBAABYcmiUTIECAAAECBCICAjCi514CBAgQIECAQEEBAVhwaJZMgAABAgQIEIgICMCInnsJECBAgAABAgUFBGDBoVkyAQIECBAgQCAiIAAjeu4lQIAAAQIECBQUEIAFh2bJBAgQIECAAIGIgACM6LmXAAECBAgQIFBQ4P8Agfy4OwKNRNQAAAAASUVORK5CYII=\" width=\"640\">"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "video = plot_animation(frames)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Nope, the system is unstable and after just a few wobbles, the pole ends up too tilted: game over. We will need to be smarter than that!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Neural Network Policies"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's create a neural network that will take observations as inputs, and output the action to take for each observation. To choose an action, the network will first estimate a probability for each action, then select an action randomly according to the estimated probabilities. In the case of the Cart-Pole environment, there are just two possible actions (left or right), so we only need one output neuron: it will output the probability `p` of the action 0 (left), and of course the probability of action 1 (right) will be `1 - p`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note: instead of using the `fully_connected()` function from the `tensorflow.contrib.layers` module (as in the book), we now use the `dense()` function from the `tf.layers` module, which did not exist when this chapter was written. This is preferable because anything in contrib may change or be deleted without notice, while `tf.layers` is part of the official API. As you will see, the code is mostly the same.\n",
    "\n",
    "The main differences relevant to this chapter are:\n",
    "* the `_fn` suffix was removed in all the parameters that had it (for example the `activation_fn` parameter was renamed to `activation`).\n",
    "* the `weights` parameter was renamed to `kernel`,\n",
    "* the default activation is `None` instead of `tf.nn.relu`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import tensorflow as tf\n",
    "\n",
    "# 1. Specify the network architecture\n",
    "n_inputs = 4  # == env.observation_space.shape[0]\n",
    "n_hidden = 4  # it's a simple task, we don't need more than this\n",
    "n_outputs = 1 # only outputs the probability of accelerating left\n",
    "initializer = tf.contrib.layers.variance_scaling_initializer()\n",
    "\n",
    "# 2. Build the neural network\n",
    "X = tf.placeholder(tf.float32, shape=[None, n_inputs])\n",
    "hidden = tf.layers.dense(X, n_hidden, activation=tf.nn.elu,\n",
    "                         kernel_initializer=initializer)\n",
    "outputs = tf.layers.dense(hidden, n_outputs, activation=tf.nn.sigmoid,\n",
    "                          kernel_initializer=initializer)\n",
    "\n",
    "# 3. Select a random action based on the estimated probabilities\n",
    "p_left_and_right = tf.concat(axis=1, values=[outputs, 1 - outputs])\n",
    "action = tf.multinomial(tf.log(p_left_and_right), num_samples=1)\n",
    "\n",
    "init = tf.global_variables_initializer()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this particular environment, the past actions and observations can safely be ignored, since each observation contains the environment's full state. If there were some hidden state then you may need to consider past actions and observations in order to try to infer the hidden state of the environment. For example, if the environment only revealed the position of the cart but not its velocity, you would have to consider not only the current observation but also the previous observation in order to estimate the current velocity. Another example is if the observations are noisy: you may want to use the past few observations to estimate the most likely current state. Our problem is thus as simple as can be: the current observation is noise-free and contains the environment's full state."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You may wonder why we are picking a random action based on the probability given by the policy network, rather than just picking the action with the highest probability. This approach lets the agent find the right balance between _exploring_ new actions and _exploiting_ the actions that are known to work well. Here's an analogy: suppose you go to a restaurant for the first time, and all the dishes look equally appealing so you randomly pick one. If it turns out to be good, you can increase the probability to order it next time, but you shouldn't increase that probability to 100%, or else you will never try out the other dishes, some of which may be even better than the one you tried."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's randomly initialize this policy neural network and use it to play one game:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "n_max_steps = 1000\n",
    "frames = []\n",
    "\n",
    "with tf.Session() as sess:\n",
    "    init.run()\n",
    "    obs = env.reset()\n",
    "    for step in range(n_max_steps):\n",
    "        img = render_cart_pole(env, obs)\n",
    "        frames.append(img)\n",
    "        action_val = action.eval(feed_dict={X: obs.reshape(1, n_inputs)})\n",
    "        obs, reward, done, info = env.step(action_val[0][0])\n",
    "        if done:\n",
    "            break\n",
    "\n",
    "env.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's look at how well this randomly initialized policy network performed:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/javascript": [
       "/* Put everything inside the global mpl namespace */\n",
       "window.mpl = {};\n",
       "\n",
       "\n",
       "mpl.get_websocket_type = function() {\n",
       "    if (typeof(WebSocket) !== 'undefined') {\n",
       "        return WebSocket;\n",
       "    } else if (typeof(MozWebSocket) !== 'undefined') {\n",
       "        return MozWebSocket;\n",
       "    } else {\n",
       "        alert('Your browser does not have WebSocket support.' +\n",
       "              'Please try Chrome, Safari or Firefox ≥ 6. ' +\n",
       "              'Firefox 4 and 5 are also supported but you ' +\n",
       "              'have to enable WebSockets in about:config.');\n",
       "    };\n",
       "}\n",
       "\n",
       "mpl.figure = function(figure_id, websocket, ondownload, parent_element) {\n",
       "    this.id = figure_id;\n",
       "\n",
       "    this.ws = websocket;\n",
       "\n",
       "    this.supports_binary = (this.ws.binaryType != undefined);\n",
       "\n",
       "    if (!this.supports_binary) {\n",
       "        var warnings = document.getElementById(\"mpl-warnings\");\n",
       "        if (warnings) {\n",
       "            warnings.style.display = 'block';\n",
       "            warnings.textContent = (\n",
       "                \"This browser does not support binary websocket messages. \" +\n",
       "                    \"Performance may be slow.\");\n",
       "        }\n",
       "    }\n",
       "\n",
       "    this.imageObj = new Image();\n",
       "\n",
       "    this.context = undefined;\n",
       "    this.message = undefined;\n",
       "    this.canvas = undefined;\n",
       "    this.rubberband_canvas = undefined;\n",
       "    this.rubberband_context = undefined;\n",
       "    this.format_dropdown = undefined;\n",
       "\n",
       "    this.image_mode = 'full';\n",
       "\n",
       "    this.root = $('<div/>');\n",
       "    this._root_extra_style(this.root)\n",
       "    this.root.attr('style', 'display: inline-block');\n",
       "\n",
       "    $(parent_element).append(this.root);\n",
       "\n",
       "    this._init_header(this);\n",
       "    this._init_canvas(this);\n",
       "    this._init_toolbar(this);\n",
       "\n",
       "    var fig = this;\n",
       "\n",
       "    this.waiting = false;\n",
       "\n",
       "    this.ws.onopen =  function () {\n",
       "            fig.send_message(\"supports_binary\", {value: fig.supports_binary});\n",
       "            fig.send_message(\"send_image_mode\", {});\n",
       "            if (mpl.ratio != 1) {\n",
       "                fig.send_message(\"set_dpi_ratio\", {'dpi_ratio': mpl.ratio});\n",
       "            }\n",
       "            fig.send_message(\"refresh\", {});\n",
       "        }\n",
       "\n",
       "    this.imageObj.onload = function() {\n",
       "            if (fig.image_mode == 'full') {\n",
       "                // Full images could contain transparency (where diff images\n",
       "                // almost always do), so we need to clear the canvas so that\n",
       "                // there is no ghosting.\n",
       "                fig.context.clearRect(0, 0, fig.canvas.width, fig.canvas.height);\n",
       "            }\n",
       "            fig.context.drawImage(fig.imageObj, 0, 0);\n",
       "        };\n",
       "\n",
       "    this.imageObj.onunload = function() {\n",
       "        this.ws.close();\n",
       "    }\n",
       "\n",
       "    this.ws.onmessage = this._make_on_message_function(this);\n",
       "\n",
       "    this.ondownload = ondownload;\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._init_header = function() {\n",
       "    var titlebar = $(\n",
       "        '<div class=\"ui-dialog-titlebar ui-widget-header ui-corner-all ' +\n",
       "        'ui-helper-clearfix\"/>');\n",
       "    var titletext = $(\n",
       "        '<div class=\"ui-dialog-title\" style=\"width: 100%; ' +\n",
       "        'text-align: center; padding: 3px;\"/>');\n",
       "    titlebar.append(titletext)\n",
       "    this.root.append(titlebar);\n",
       "    this.header = titletext[0];\n",
       "}\n",
       "\n",
       "\n",
       "\n",
       "mpl.figure.prototype._canvas_extra_style = function(canvas_div) {\n",
       "\n",
       "}\n",
       "\n",
       "\n",
       "mpl.figure.prototype._root_extra_style = function(canvas_div) {\n",
       "\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._init_canvas = function() {\n",
       "    var fig = this;\n",
       "\n",
       "    var canvas_div = $('<div/>');\n",
       "\n",
       "    canvas_div.attr('style', 'position: relative; clear: both; outline: 0');\n",
       "\n",
       "    function canvas_keyboard_event(event) {\n",
       "        return fig.key_event(event, event['data']);\n",
       "    }\n",
       "\n",
       "    canvas_div.keydown('key_press', canvas_keyboard_event);\n",
       "    canvas_div.keyup('key_release', canvas_keyboard_event);\n",
       "    this.canvas_div = canvas_div\n",
       "    this._canvas_extra_style(canvas_div)\n",
       "    this.root.append(canvas_div);\n",
       "\n",
       "    var canvas = $('<canvas/>');\n",
       "    canvas.addClass('mpl-canvas');\n",
       "    canvas.attr('style', \"left: 0; top: 0; z-index: 0; outline: 0\")\n",
       "\n",
       "    this.canvas = canvas[0];\n",
       "    this.context = canvas[0].getContext(\"2d\");\n",
       "\n",
       "    var backingStore = this.context.backingStorePixelRatio ||\n",
       "\tthis.context.webkitBackingStorePixelRatio ||\n",
       "\tthis.context.mozBackingStorePixelRatio ||\n",
       "\tthis.context.msBackingStorePixelRatio ||\n",
       "\tthis.context.oBackingStorePixelRatio ||\n",
       "\tthis.context.backingStorePixelRatio || 1;\n",
       "\n",
       "    mpl.ratio = (window.devicePixelRatio || 1) / backingStore;\n",
       "\n",
       "    var rubberband = $('<canvas/>');\n",
       "    rubberband.attr('style', \"position: absolute; left: 0; top: 0; z-index: 1;\")\n",
       "\n",
       "    var pass_mouse_events = true;\n",
       "\n",
       "    canvas_div.resizable({\n",
       "        start: function(event, ui) {\n",
       "            pass_mouse_events = false;\n",
       "        },\n",
       "        resize: function(event, ui) {\n",
       "            fig.request_resize(ui.size.width, ui.size.height);\n",
       "        },\n",
       "        stop: function(event, ui) {\n",
       "            pass_mouse_events = true;\n",
       "            fig.request_resize(ui.size.width, ui.size.height);\n",
       "        },\n",
       "    });\n",
       "\n",
       "    function mouse_event_fn(event) {\n",
       "        if (pass_mouse_events)\n",
       "            return fig.mouse_event(event, event['data']);\n",
       "    }\n",
       "\n",
       "    rubberband.mousedown('button_press', mouse_event_fn);\n",
       "    rubberband.mouseup('button_release', mouse_event_fn);\n",
       "    // Throttle sequential mouse events to 1 every 20ms.\n",
       "    rubberband.mousemove('motion_notify', mouse_event_fn);\n",
       "\n",
       "    rubberband.mouseenter('figure_enter', mouse_event_fn);\n",
       "    rubberband.mouseleave('figure_leave', mouse_event_fn);\n",
       "\n",
       "    canvas_div.on(\"wheel\", function (event) {\n",
       "        event = event.originalEvent;\n",
       "        event['data'] = 'scroll'\n",
       "        if (event.deltaY < 0) {\n",
       "            event.step = 1;\n",
       "        } else {\n",
       "            event.step = -1;\n",
       "        }\n",
       "        mouse_event_fn(event);\n",
       "    });\n",
       "\n",
       "    canvas_div.append(canvas);\n",
       "    canvas_div.append(rubberband);\n",
       "\n",
       "    this.rubberband = rubberband;\n",
       "    this.rubberband_canvas = rubberband[0];\n",
       "    this.rubberband_context = rubberband[0].getContext(\"2d\");\n",
       "    this.rubberband_context.strokeStyle = \"#000000\";\n",
       "\n",
       "    this._resize_canvas = function(width, height) {\n",
       "        // Keep the size of the canvas, canvas container, and rubber band\n",
       "        // canvas in synch.\n",
       "        canvas_div.css('width', width)\n",
       "        canvas_div.css('height', height)\n",
       "\n",
       "        canvas.attr('width', width * mpl.ratio);\n",
       "        canvas.attr('height', height * mpl.ratio);\n",
       "        canvas.attr('style', 'width: ' + width + 'px; height: ' + height + 'px;');\n",
       "\n",
       "        rubberband.attr('width', width);\n",
       "        rubberband.attr('height', height);\n",
       "    }\n",
       "\n",
       "    // Set the figure to an initial 600x600px, this will subsequently be updated\n",
       "    // upon first draw.\n",
       "    this._resize_canvas(600, 600);\n",
       "\n",
       "    // Disable right mouse context menu.\n",
       "    $(this.rubberband_canvas).bind(\"contextmenu\",function(e){\n",
       "        return false;\n",
       "    });\n",
       "\n",
       "    function set_focus () {\n",
       "        canvas.focus();\n",
       "        canvas_div.focus();\n",
       "    }\n",
       "\n",
       "    window.setTimeout(set_focus, 100);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._init_toolbar = function() {\n",
       "    var fig = this;\n",
       "\n",
       "    var nav_element = $('<div/>')\n",
       "    nav_element.attr('style', 'width: 100%');\n",
       "    this.root.append(nav_element);\n",
       "\n",
       "    // Define a callback function for later on.\n",
       "    function toolbar_event(event) {\n",
       "        return fig.toolbar_button_onclick(event['data']);\n",
       "    }\n",
       "    function toolbar_mouse_event(event) {\n",
       "        return fig.toolbar_button_onmouseover(event['data']);\n",
       "    }\n",
       "\n",
       "    for(var toolbar_ind in mpl.toolbar_items) {\n",
       "        var name = mpl.toolbar_items[toolbar_ind][0];\n",
       "        var tooltip = mpl.toolbar_items[toolbar_ind][1];\n",
       "        var image = mpl.toolbar_items[toolbar_ind][2];\n",
       "        var method_name = mpl.toolbar_items[toolbar_ind][3];\n",
       "\n",
       "        if (!name) {\n",
       "            // put a spacer in here.\n",
       "            continue;\n",
       "        }\n",
       "        var button = $('<button/>');\n",
       "        button.addClass('ui-button ui-widget ui-state-default ui-corner-all ' +\n",
       "                        'ui-button-icon-only');\n",
       "        button.attr('role', 'button');\n",
       "        button.attr('aria-disabled', 'false');\n",
       "        button.click(method_name, toolbar_event);\n",
       "        button.mouseover(tooltip, toolbar_mouse_event);\n",
       "\n",
       "        var icon_img = $('<span/>');\n",
       "        icon_img.addClass('ui-button-icon-primary ui-icon');\n",
       "        icon_img.addClass(image);\n",
       "        icon_img.addClass('ui-corner-all');\n",
       "\n",
       "        var tooltip_span = $('<span/>');\n",
       "        tooltip_span.addClass('ui-button-text');\n",
       "        tooltip_span.html(tooltip);\n",
       "\n",
       "        button.append(icon_img);\n",
       "        button.append(tooltip_span);\n",
       "\n",
       "        nav_element.append(button);\n",
       "    }\n",
       "\n",
       "    var fmt_picker_span = $('<span/>');\n",
       "\n",
       "    var fmt_picker = $('<select/>');\n",
       "    fmt_picker.addClass('mpl-toolbar-option ui-widget ui-widget-content');\n",
       "    fmt_picker_span.append(fmt_picker);\n",
       "    nav_element.append(fmt_picker_span);\n",
       "    this.format_dropdown = fmt_picker[0];\n",
       "\n",
       "    for (var ind in mpl.extensions) {\n",
       "        var fmt = mpl.extensions[ind];\n",
       "        var option = $(\n",
       "            '<option/>', {selected: fmt === mpl.default_extension}).html(fmt);\n",
       "        fmt_picker.append(option)\n",
       "    }\n",
       "\n",
       "    // Add hover states to the ui-buttons\n",
       "    $( \".ui-button\" ).hover(\n",
       "        function() { $(this).addClass(\"ui-state-hover\");},\n",
       "        function() { $(this).removeClass(\"ui-state-hover\");}\n",
       "    );\n",
       "\n",
       "    var status_bar = $('<span class=\"mpl-message\"/>');\n",
       "    nav_element.append(status_bar);\n",
       "    this.message = status_bar[0];\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.request_resize = function(x_pixels, y_pixels) {\n",
       "    // Request matplotlib to resize the figure. Matplotlib will then trigger a resize in the client,\n",
       "    // which will in turn request a refresh of the image.\n",
       "    this.send_message('resize', {'width': x_pixels, 'height': y_pixels});\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.send_message = function(type, properties) {\n",
       "    properties['type'] = type;\n",
       "    properties['figure_id'] = this.id;\n",
       "    this.ws.send(JSON.stringify(properties));\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.send_draw_message = function() {\n",
       "    if (!this.waiting) {\n",
       "        this.waiting = true;\n",
       "        this.ws.send(JSON.stringify({type: \"draw\", figure_id: this.id}));\n",
       "    }\n",
       "}\n",
       "\n",
       "\n",
       "mpl.figure.prototype.handle_save = function(fig, msg) {\n",
       "    var format_dropdown = fig.format_dropdown;\n",
       "    var format = format_dropdown.options[format_dropdown.selectedIndex].value;\n",
       "    fig.ondownload(fig, format);\n",
       "}\n",
       "\n",
       "\n",
       "mpl.figure.prototype.handle_resize = function(fig, msg) {\n",
       "    var size = msg['size'];\n",
       "    if (size[0] != fig.canvas.width || size[1] != fig.canvas.height) {\n",
       "        fig._resize_canvas(size[0], size[1]);\n",
       "        fig.send_message(\"refresh\", {});\n",
       "    };\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_rubberband = function(fig, msg) {\n",
       "    var x0 = msg['x0'] / mpl.ratio;\n",
       "    var y0 = (fig.canvas.height - msg['y0']) / mpl.ratio;\n",
       "    var x1 = msg['x1'] / mpl.ratio;\n",
       "    var y1 = (fig.canvas.height - msg['y1']) / mpl.ratio;\n",
       "    x0 = Math.floor(x0) + 0.5;\n",
       "    y0 = Math.floor(y0) + 0.5;\n",
       "    x1 = Math.floor(x1) + 0.5;\n",
       "    y1 = Math.floor(y1) + 0.5;\n",
       "    var min_x = Math.min(x0, x1);\n",
       "    var min_y = Math.min(y0, y1);\n",
       "    var width = Math.abs(x1 - x0);\n",
       "    var height = Math.abs(y1 - y0);\n",
       "\n",
       "    fig.rubberband_context.clearRect(\n",
       "        0, 0, fig.canvas.width, fig.canvas.height);\n",
       "\n",
       "    fig.rubberband_context.strokeRect(min_x, min_y, width, height);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_figure_label = function(fig, msg) {\n",
       "    // Updates the figure title.\n",
       "    fig.header.textContent = msg['label'];\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_cursor = function(fig, msg) {\n",
       "    var cursor = msg['cursor'];\n",
       "    switch(cursor)\n",
       "    {\n",
       "    case 0:\n",
       "        cursor = 'pointer';\n",
       "        break;\n",
       "    case 1:\n",
       "        cursor = 'default';\n",
       "        break;\n",
       "    case 2:\n",
       "        cursor = 'crosshair';\n",
       "        break;\n",
       "    case 3:\n",
       "        cursor = 'move';\n",
       "        break;\n",
       "    }\n",
       "    fig.rubberband_canvas.style.cursor = cursor;\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_message = function(fig, msg) {\n",
       "    fig.message.textContent = msg['message'];\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_draw = function(fig, msg) {\n",
       "    // Request the server to send over a new figure.\n",
       "    fig.send_draw_message();\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_image_mode = function(fig, msg) {\n",
       "    fig.image_mode = msg['mode'];\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.updated_canvas_event = function() {\n",
       "    // Called whenever the canvas gets updated.\n",
       "    this.send_message(\"ack\", {});\n",
       "}\n",
       "\n",
       "// A function to construct a web socket function for onmessage handling.\n",
       "// Called in the figure constructor.\n",
       "mpl.figure.prototype._make_on_message_function = function(fig) {\n",
       "    return function socket_on_message(evt) {\n",
       "        if (evt.data instanceof Blob) {\n",
       "            /* FIXME: We get \"Resource interpreted as Image but\n",
       "             * transferred with MIME type text/plain:\" errors on\n",
       "             * Chrome.  But how to set the MIME type?  It doesn't seem\n",
       "             * to be part of the websocket stream */\n",
       "            evt.data.type = \"image/png\";\n",
       "\n",
       "            /* Free the memory for the previous frames */\n",
       "            if (fig.imageObj.src) {\n",
       "                (window.URL || window.webkitURL).revokeObjectURL(\n",
       "                    fig.imageObj.src);\n",
       "            }\n",
       "\n",
       "            fig.imageObj.src = (window.URL || window.webkitURL).createObjectURL(\n",
       "                evt.data);\n",
       "            fig.updated_canvas_event();\n",
       "            fig.waiting = false;\n",
       "            return;\n",
       "        }\n",
       "        else if (typeof evt.data === 'string' && evt.data.slice(0, 21) == \"data:image/png;base64\") {\n",
       "            fig.imageObj.src = evt.data;\n",
       "            fig.updated_canvas_event();\n",
       "            fig.waiting = false;\n",
       "            return;\n",
       "        }\n",
       "\n",
       "        var msg = JSON.parse(evt.data);\n",
       "        var msg_type = msg['type'];\n",
       "\n",
       "        // Call the  \"handle_{type}\" callback, which takes\n",
       "        // the figure and JSON message as its only arguments.\n",
       "        try {\n",
       "            var callback = fig[\"handle_\" + msg_type];\n",
       "        } catch (e) {\n",
       "            console.log(\"No handler for the '\" + msg_type + \"' message type: \", msg);\n",
       "            return;\n",
       "        }\n",
       "\n",
       "        if (callback) {\n",
       "            try {\n",
       "                // console.log(\"Handling '\" + msg_type + \"' message: \", msg);\n",
       "                callback(fig, msg);\n",
       "            } catch (e) {\n",
       "                console.log(\"Exception inside the 'handler_\" + msg_type + \"' callback:\", e, e.stack, msg);\n",
       "            }\n",
       "        }\n",
       "    };\n",
       "}\n",
       "\n",
       "// from http://stackoverflow.com/questions/1114465/getting-mouse-location-in-canvas\n",
       "mpl.findpos = function(e) {\n",
       "    //this section is from http://www.quirksmode.org/js/events_properties.html\n",
       "    var targ;\n",
       "    if (!e)\n",
       "        e = window.event;\n",
       "    if (e.target)\n",
       "        targ = e.target;\n",
       "    else if (e.srcElement)\n",
       "        targ = e.srcElement;\n",
       "    if (targ.nodeType == 3) // defeat Safari bug\n",
       "        targ = targ.parentNode;\n",
       "\n",
       "    // jQuery normalizes the pageX and pageY\n",
       "    // pageX,Y are the mouse positions relative to the document\n",
       "    // offset() returns the position of the element relative to the document\n",
       "    var x = e.pageX - $(targ).offset().left;\n",
       "    var y = e.pageY - $(targ).offset().top;\n",
       "\n",
       "    return {\"x\": x, \"y\": y};\n",
       "};\n",
       "\n",
       "/*\n",
       " * return a copy of an object with only non-object keys\n",
       " * we need this to avoid circular references\n",
       " * http://stackoverflow.com/a/24161582/3208463\n",
       " */\n",
       "function simpleKeys (original) {\n",
       "  return Object.keys(original).reduce(function (obj, key) {\n",
       "    if (typeof original[key] !== 'object')\n",
       "        obj[key] = original[key]\n",
       "    return obj;\n",
       "  }, {});\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.mouse_event = function(event, name) {\n",
       "    var canvas_pos = mpl.findpos(event)\n",
       "\n",
       "    if (name === 'button_press')\n",
       "    {\n",
       "        this.canvas.focus();\n",
       "        this.canvas_div.focus();\n",
       "    }\n",
       "\n",
       "    var x = canvas_pos.x * mpl.ratio;\n",
       "    var y = canvas_pos.y * mpl.ratio;\n",
       "\n",
       "    this.send_message(name, {x: x, y: y, button: event.button,\n",
       "                             step: event.step,\n",
       "                             guiEvent: simpleKeys(event)});\n",
       "\n",
       "    /* This prevents the web browser from automatically changing to\n",
       "     * the text insertion cursor when the button is pressed.  We want\n",
       "     * to control all of the cursor setting manually through the\n",
       "     * 'cursor' event from matplotlib */\n",
       "    event.preventDefault();\n",
       "    return false;\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._key_event_extra = function(event, name) {\n",
       "    // Handle any extra behaviour associated with a key event\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.key_event = function(event, name) {\n",
       "\n",
       "    // Prevent repeat events\n",
       "    if (name == 'key_press')\n",
       "    {\n",
       "        if (event.which === this._key)\n",
       "            return;\n",
       "        else\n",
       "            this._key = event.which;\n",
       "    }\n",
       "    if (name == 'key_release')\n",
       "        this._key = null;\n",
       "\n",
       "    var value = '';\n",
       "    if (event.ctrlKey && event.which != 17)\n",
       "        value += \"ctrl+\";\n",
       "    if (event.altKey && event.which != 18)\n",
       "        value += \"alt+\";\n",
       "    if (event.shiftKey && event.which != 16)\n",
       "        value += \"shift+\";\n",
       "\n",
       "    value += 'k';\n",
       "    value += event.which.toString();\n",
       "\n",
       "    this._key_event_extra(event, name);\n",
       "\n",
       "    this.send_message(name, {key: value,\n",
       "                             guiEvent: simpleKeys(event)});\n",
       "    return false;\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.toolbar_button_onclick = function(name) {\n",
       "    if (name == 'download') {\n",
       "        this.handle_save(this, null);\n",
       "    } else {\n",
       "        this.send_message(\"toolbar_button\", {name: name});\n",
       "    }\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.toolbar_button_onmouseover = function(tooltip) {\n",
       "    this.message.textContent = tooltip;\n",
       "};\n",
       "mpl.toolbar_items = [[\"Home\", \"Reset original view\", \"fa fa-home icon-home\", \"home\"], [\"Back\", \"Back to  previous view\", \"fa fa-arrow-left icon-arrow-left\", \"back\"], [\"Forward\", \"Forward to next view\", \"fa fa-arrow-right icon-arrow-right\", \"forward\"], [\"\", \"\", \"\", \"\"], [\"Pan\", \"Pan axes with left mouse, zoom with right\", \"fa fa-arrows icon-move\", \"pan\"], [\"Zoom\", \"Zoom to rectangle\", \"fa fa-square-o icon-check-empty\", \"zoom\"], [\"\", \"\", \"\", \"\"], [\"Download\", \"Download plot\", \"fa fa-floppy-o icon-save\", \"download\"]];\n",
       "\n",
       "mpl.extensions = [\"eps\", \"jpeg\", \"pdf\", \"png\", \"ps\", \"raw\", \"svg\", \"tif\"];\n",
       "\n",
       "mpl.default_extension = \"png\";var comm_websocket_adapter = function(comm) {\n",
       "    // Create a \"websocket\"-like object which calls the given IPython comm\n",
       "    // object with the appropriate methods. Currently this is a non binary\n",
       "    // socket, so there is still some room for performance tuning.\n",
       "    var ws = {};\n",
       "\n",
       "    ws.close = function() {\n",
       "        comm.close()\n",
       "    };\n",
       "    ws.send = function(m) {\n",
       "        //console.log('sending', m);\n",
       "        comm.send(m);\n",
       "    };\n",
       "    // Register the callback with on_msg.\n",
       "    comm.on_msg(function(msg) {\n",
       "        //console.log('receiving', msg['content']['data'], msg);\n",
       "        // Pass the mpl event to the overriden (by mpl) onmessage function.\n",
       "        ws.onmessage(msg['content']['data'])\n",
       "    });\n",
       "    return ws;\n",
       "}\n",
       "\n",
       "mpl.mpl_figure_comm = function(comm, msg) {\n",
       "    // This is the function which gets called when the mpl process\n",
       "    // starts-up an IPython Comm through the \"matplotlib\" channel.\n",
       "\n",
       "    var id = msg.content.data.id;\n",
       "    // Get hold of the div created by the display call when the Comm\n",
       "    // socket was opened in Python.\n",
       "    var element = $(\"#\" + id);\n",
       "    var ws_proxy = comm_websocket_adapter(comm)\n",
       "\n",
       "    function ondownload(figure, format) {\n",
       "        window.open(figure.imageObj.src);\n",
       "    }\n",
       "\n",
       "    var fig = new mpl.figure(id, ws_proxy,\n",
       "                           ondownload,\n",
       "                           element.get(0));\n",
       "\n",
       "    // Call onopen now - mpl needs it, as it is assuming we've passed it a real\n",
       "    // web socket which is closed, not our websocket->open comm proxy.\n",
       "    ws_proxy.onopen();\n",
       "\n",
       "    fig.parent_element = element.get(0);\n",
       "    fig.cell_info = mpl.find_output_cell(\"<div id='\" + id + \"'></div>\");\n",
       "    if (!fig.cell_info) {\n",
       "        console.error(\"Failed to find cell for figure\", id, fig);\n",
       "        return;\n",
       "    }\n",
       "\n",
       "    var output_index = fig.cell_info[2]\n",
       "    var cell = fig.cell_info[0];\n",
       "\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_close = function(fig, msg) {\n",
       "    var width = fig.canvas.width/mpl.ratio\n",
       "    fig.root.unbind('remove')\n",
       "\n",
       "    // Update the output cell to use the data from the current canvas.\n",
       "    fig.push_to_output();\n",
       "    var dataURL = fig.canvas.toDataURL();\n",
       "    // Re-enable the keyboard manager in IPython - without this line, in FF,\n",
       "    // the notebook keyboard shortcuts fail.\n",
       "    IPython.keyboard_manager.enable()\n",
       "    $(fig.parent_element).html('<img src=\"' + dataURL + '\" width=\"' + width + '\">');\n",
       "    fig.close_ws(fig, msg);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.close_ws = function(fig, msg){\n",
       "    fig.send_message('closing', msg);\n",
       "    // fig.ws.close()\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.push_to_output = function(remove_interactive) {\n",
       "    // Turn the data on the canvas into data in the output cell.\n",
       "    var width = this.canvas.width/mpl.ratio\n",
       "    var dataURL = this.canvas.toDataURL();\n",
       "    this.cell_info[1]['text/html'] = '<img src=\"' + dataURL + '\" width=\"' + width + '\">';\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.updated_canvas_event = function() {\n",
       "    // Tell IPython that the notebook contents must change.\n",
       "    IPython.notebook.set_dirty(true);\n",
       "    this.send_message(\"ack\", {});\n",
       "    var fig = this;\n",
       "    // Wait a second, then push the new image to the DOM so\n",
       "    // that it is saved nicely (might be nice to debounce this).\n",
       "    setTimeout(function () { fig.push_to_output() }, 1000);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._init_toolbar = function() {\n",
       "    var fig = this;\n",
       "\n",
       "    var nav_element = $('<div/>')\n",
       "    nav_element.attr('style', 'width: 100%');\n",
       "    this.root.append(nav_element);\n",
       "\n",
       "    // Define a callback function for later on.\n",
       "    function toolbar_event(event) {\n",
       "        return fig.toolbar_button_onclick(event['data']);\n",
       "    }\n",
       "    function toolbar_mouse_event(event) {\n",
       "        return fig.toolbar_button_onmouseover(event['data']);\n",
       "    }\n",
       "\n",
       "    for(var toolbar_ind in mpl.toolbar_items){\n",
       "        var name = mpl.toolbar_items[toolbar_ind][0];\n",
       "        var tooltip = mpl.toolbar_items[toolbar_ind][1];\n",
       "        var image = mpl.toolbar_items[toolbar_ind][2];\n",
       "        var method_name = mpl.toolbar_items[toolbar_ind][3];\n",
       "\n",
       "        if (!name) { continue; };\n",
       "\n",
       "        var button = $('<button class=\"btn btn-default\" href=\"#\" title=\"' + name + '\"><i class=\"fa ' + image + ' fa-lg\"></i></button>');\n",
       "        button.click(method_name, toolbar_event);\n",
       "        button.mouseover(tooltip, toolbar_mouse_event);\n",
       "        nav_element.append(button);\n",
       "    }\n",
       "\n",
       "    // Add the status bar.\n",
       "    var status_bar = $('<span class=\"mpl-message\" style=\"text-align:right; float: right;\"/>');\n",
       "    nav_element.append(status_bar);\n",
       "    this.message = status_bar[0];\n",
       "\n",
       "    // Add the close button to the window.\n",
       "    var buttongrp = $('<div class=\"btn-group inline pull-right\"></div>');\n",
       "    var button = $('<button class=\"btn btn-mini btn-primary\" href=\"#\" title=\"Stop Interaction\"><i class=\"fa fa-power-off icon-remove icon-large\"></i></button>');\n",
       "    button.click(function (evt) { fig.handle_close(fig, {}); } );\n",
       "    button.mouseover('Stop Interaction', toolbar_mouse_event);\n",
       "    buttongrp.append(button);\n",
       "    var titlebar = this.root.find($('.ui-dialog-titlebar'));\n",
       "    titlebar.prepend(buttongrp);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._root_extra_style = function(el){\n",
       "    var fig = this\n",
       "    el.on(\"remove\", function(){\n",
       "\tfig.close_ws(fig, {});\n",
       "    });\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._canvas_extra_style = function(el){\n",
       "    // this is important to make the div 'focusable\n",
       "    el.attr('tabindex', 0)\n",
       "    // reach out to IPython and tell the keyboard manager to turn it's self\n",
       "    // off when our div gets focus\n",
       "\n",
       "    // location in version 3\n",
       "    if (IPython.notebook.keyboard_manager) {\n",
       "        IPython.notebook.keyboard_manager.register_events(el);\n",
       "    }\n",
       "    else {\n",
       "        // location in version 2\n",
       "        IPython.keyboard_manager.register_events(el);\n",
       "    }\n",
       "\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._key_event_extra = function(event, name) {\n",
       "    var manager = IPython.notebook.keyboard_manager;\n",
       "    if (!manager)\n",
       "        manager = IPython.keyboard_manager;\n",
       "\n",
       "    // Check for shift+enter\n",
       "    if (event.shiftKey && event.which == 13) {\n",
       "        this.canvas_div.blur();\n",
       "        // select the cell after this one\n",
       "        var index = IPython.notebook.find_cell_index(this.cell_info[0]);\n",
       "        IPython.notebook.select(index + 1);\n",
       "    }\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_save = function(fig, msg) {\n",
       "    fig.ondownload(fig, null);\n",
       "}\n",
       "\n",
       "\n",
       "mpl.find_output_cell = function(html_output) {\n",
       "    // Return the cell and output element which can be found *uniquely* in the notebook.\n",
       "    // Note - this is a bit hacky, but it is done because the \"notebook_saving.Notebook\"\n",
       "    // IPython event is triggered only after the cells have been serialised, which for\n",
       "    // our purposes (turning an active figure into a static one), is too late.\n",
       "    var cells = IPython.notebook.get_cells();\n",
       "    var ncells = cells.length;\n",
       "    for (var i=0; i<ncells; i++) {\n",
       "        var cell = cells[i];\n",
       "        if (cell.cell_type === 'code'){\n",
       "            for (var j=0; j<cell.output_area.outputs.length; j++) {\n",
       "                var data = cell.output_area.outputs[j];\n",
       "                if (data.data) {\n",
       "                    // IPython >= 3 moved mimebundle to data attribute of output\n",
       "                    data = data.data;\n",
       "                }\n",
       "                if (data['text/html'] == html_output) {\n",
       "                    return [cell, data, j];\n",
       "                }\n",
       "            }\n",
       "        }\n",
       "    }\n",
       "}\n",
       "\n",
       "// Register the function which deals with the matplotlib target/channel.\n",
       "// The kernel may be null if the page has been refreshed.\n",
       "if (IPython.notebook.kernel != null) {\n",
       "    IPython.notebook.kernel.comm_manager.register_target('matplotlib', mpl.mpl_figure_comm);\n",
       "}\n"
      ],
      "text/plain": [
       "<IPython.core.display.Javascript object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<img src=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAoAAAAHgCAYAAAA10dzkAAAgAElEQVR4Xu3d4dEcS5EF0Cc3cGNxAzfADAIzwA3cADdwQxtDBBCrfaMpvuzuypt19q+6p7LOzR83WnrLt1/8HwECBAgQIECAwFEC3466rcsSIECAAAECBAj8ogBaAgIECBAgQIDAYQIK4GGBuy4BAgQIECBAQAG0AwQIECBAgACBwwQUwMMCd10CBAgQIECAgAJoBwgQIECAAAEChwkogIcF7roECBAgQIAAAQXQDhAgQIAAAQIEDhNQAA8L3HUJECBAgAABAgqgHSBAgAABAgQIHCagAB4WuOsSIECAAAECBBRAO0CAAAECBAgQOExAATwscNclQIAAAQIECCiAdoAAAQIECBAgcJiAAnhY4K5LgAABAgQIEFAA7QABAgQIECBA4DABBfCwwF2XAAECBAgQIKAA2gECBAgQIECAwGECCuBhgbsuAQIECBAgQEABtAMECBAgQIAAgcMEFMDDAnddAgQIECBAgIACaAcIECBAgAABAocJKICHBe66BAgQIECAAAEF0A4QIECAAAECBA4TUAAPC9x1CRAgQIAAAQIKoB0gQIAAAQIECBwmoAAeFrjrEiBAgAABAgQUQDtAgAABAgQIEDhMQAE8LHDXJUCAAAECBAgogHaAAAECBAgQIHCYgAJ4WOCuS4AAAQIECBBQAO0AAQIECBAgQOAwAQXwsMBdlwABAgQIECCgANoBAgQIECBAgMBhAgrgYYG7LgECBAgQIEBAAbQDBAgQIECAAIHDBBTAwwJ3XQIECBAgQICAAmgHCBAgQIAAAQKHCSiAhwXuugQIECBAgAABBdAOECBAgAABAgQOE1AADwvcdQkQIECAAAECCqAdIECAAAECBAgcJqAAHha46xIgQIAAAQIEFEA7QIAAAQIECBA4TEABPCxw1yVAgAABAgQIKIB2gAABAgQIECBwmIACeFjgrkuAAAECBAgQUADtAAECBAgQIEDgMAEF8LDAXZcAAQIECBAgoADaAQIECBAgQIDAYQIK4GGBuy4BAgQIECBAQAG0AwQIECBAgACBwwQUwMMCd10CBAgQIECAgAJoBwgQIECAAAEChwkogIcF7roECBAgQIAAAQXQDhAgQIAAAQIEDhNQAA8L3HUJECBAgAABAgqgHSBAgAABAgQIHCagAB4WuOsSIECAAAECBBRAO0CAAAECBAgQOExAATwscNclQIAAAQIECCiAdoAAAQIECBAgcJiAAnhY4K5LgAABAgQIEFAA7QABAgQIECBA4DABBfCwwF2XAAECBAgQIKAA2gECBAgQIECAwGECCuBhgbsuAQIECBAgQEABtAMECBAgQIAAgcMEFMDDAnddAgQIECBAgIACaAcIECBAgAABAocJKICHBe66BAgQIECAAAEF0A4QIECAAAECBA4TUAAPC9x1CRAgQIAAAQIKoB0gQIAAAQIECBwmoAAeFrjrEiBAgAABAgQUQDtAgAABAgQIEDhMQAEsBv79+/fvxZ/wOgECBAgQIPCwwLdv347uQEdf/opdUwCvUPQbBAgQIEDgWQEF8FnvcacpgOMidSECBAgQOEBAATwg5DuvqADeqeu3CRAgQIDAPQIK4D2ux/yqAnhM1C5KgAABAoMEFMBBYe64igK4Q92ZBAgQIECgJqAA1vyOf1sBPH4FABAgQIBAoIACGBhap5EVwE5pmIUAAQIECKwJKIBrTp56I6AAWg0CBAgQIJAnoADmZdZqYgWwVRyGIUCAAAECSwIK4BKTh94JKIB2gwABAgQI5AkogHmZtZpYAWwVh2EIECBAgMCSgAK4xOQhXwDtAAECBAgQmCOgAM7JcstNfAHcwu5QAgQIECBQElAAS3xeVgDtAAECBAgQyBNQAPMyazWxAtgqDsMQIECAAIElAQVwiclD7wQUQLtBgAABAgTyBBTAvMxaTawAtorDMAQIECBAYElAAVxi8pAvgHaAAAECBAjMEVAA52S55Sa+AG5hdygBAgQIECgJKIAlPi8rgHaAAAECBAjkCSiAeZm1mlgBbBWHYQgQIECAwJKAArjE5KF3Agqg3SBAgAABAnkCCmBeZq0mVgBbxWEYAgQIECCwJKAALjF5yBdAO0CAAAECBOYIKIBzstxyE18At7A7lAABAgQIlAQUwBKflxVAO0CAAAECBPIEFMC8zFpNrAC2isMwBAgQIEBgSUABXGLy0DsBBdBuECBAgACBPAEFMC+zVhMrgK3iMAwBAgQIEFgSUACXmDzkC6AdIECAAAECcwQUwDlZbrmJL4Bb2B1KgAABAgRKAgpgic/LCqAdIECAAAECeQIKYF5mrSZWAFvFYRgCBAgQILAkoAAuMXnonYACaDcIECBAgECegAKYl1mriRXAVnEYhgABAgQILAkogEtMHvIF0A4QIECAAIE5AgrgnCy33MQXwC3sDiVAgAABAiUBBbDE52UF0A4QIECAAIE8AQUwL7NWEyuAreIwDAECBAgQWBJQAJeYPPROQAG0GwQIECBAIE9AAczLrNXECmCrOAxDgAABAgSWBBTAJSYP+QJoBwgQIECAwBwBBXBOlltu4gvgFnaHEiBAgACBkoACWOLzsgJoBwgQIECAQJ6AApiXWauJFcBWcRiGAAECBAgsCSiAS0weeiegANoNAgQIECCQJ6AA5mXWamIFsFUchiFAgAABAksCCuASk4d8AbQDBAgQIEBgjoACOCfLLTfxBXALu0MJECBAgEBJQAEs8XlZAbQDBAgQIEAgT0ABzMus1cQKYKs4DEOAAAECBJYEFMAlJg+9E1AA7QYBAgQIEMgTUADzMms1sQLYKg7DECBAgACBJQEFcInJQ74A2gECBAgQIDBHQAGck+WWm/gCuIXdoQQIECBAoCSgAJb4vKwA2gECBAgQIJAnoADmZdZqYgWwVRyGIUCAAAECSwIK4BKTh94JKIB2gwABAgQI5AkogHmZtZpYAWwVh2EIECBAgMCSgAK4xOQhXwDtAAECBAgQmCOgAM7JcstNfAHcwu5QAgQIECBQElAAS3xeVgDtAAECBAgQyBNQAPMyazWxAtgqDsMQIECAAIElAQVwiclD7wQUQLtBgAABAgTyBBTAvMxaTawAtorDMAQIECBAYElAAVxi8pAvgHaAAAECBAjMEVAA52S55Sa+AG5hdygBAgQIECgJKIAlPi8rgHaAAAECBAjkCSiAeZm1mlgBbBWHYQgQIECAwJKAArjE5KF3Agqg3SBAgAABAnkCCmBeZq0mVgBbxWEYAgQIECCwJKAALjF5yBdAO0CAAAECBOYIKIBzstxyE18At7A7lAABAgQIlAQUwBKflxVAO0CAAAECBPIEFMC8zFpNrAC2isMwBAgQIEBgSUABXGLy0DsBBdBuECBAgACBPAEFMC+zVhMrgK3iMAwBAgQIEFgSUACXmDzkC6AdIECAAAECcwQUwDlZbrmJL4Bb2B1KgAABAgRKAgpgic/LCqAdIECAAAECeQIKYF5mrSZWAFvFYRgCBAgQILAkoAAuMXnonYACaDcIECBAgECegAKYl1mriRXAVnEYhgABAgQILAkogEtMHvIF0A4QIECAAIE5AgrgnCy33MQXwC3sDiVAgAABAiUBBbDE52UF0A4QIECAAIE8AQUwL7NWEyuAreIwDAECBAgQWBJQAJeYPPROQAG0GwQIECBAIE9AAczLrNXECmCrOAxDgAABAgSWBBTAJSYP+QJoBwgQIECAwBwBBXBOlltu4gvgFnaHEiBAgACBkoACWOLzsgJoBwgQIECAQJ6AApiXWauJFcBWcRiGAAECBAgsCSiAS0weeiegANoNAgQIECCQJ6AA5mXWamIFsFUchiFAgAABAksCCuASk4d8AbQDBAgQIEBgjoACOCfLLTfxBXALu0MJECBAgEBJQAEs8XlZAbQDBAgQIEAgT0ABzMus1cQKYKs4DEOAAAECBJYEFMAlJg+9E1AA7QYBAgQIEMgTUADzMms1sQLYKg7DECBAgACBJQEFcInJQ74A2gECBAgQIDBHQAGck+WWm/gCuIXdoQQIECBAoCSgAJb4vKwA2gECBAgQIJAnoADmZdZqYgWwVRyGIUCAAAECSwIK4BKTh94JKIB2gwABAgQI5AkogHmZtZpYAWwVh2EIECBAgMCSgAK4xOQhXwDtAAECBAgQmCOgAM7JcstNfAHcwu5QAgQIECBQElAAS3xeVgDtAAECBAgQyBNQAPMyazWxAtgqDsMQIECAAIElAQVwiclD7wQUQLtBgAABAgTyBBTAvMxaTawAtorDMAQIECBAYElAAVxi8pAvgHaAAAECBAjMEVAA52S55Sa+AG5hdygBAgQIECgJKIAlPi8rgHaAAAECBAjkCSiAeZm1mlgBbBWHYQgQIECAwJKAArjE5KF3Agqg3SBAgAABAnkCCmBeZq0mVgBbxWEYAgQIECCwJKAALjF5yBdAO0CAAAECBOYIKIBzstxyE18At7A7lAABAgQIlAQUwBKflxVAO0CAAAECBPIEFMC8zFpNrAC2isMwBAgQIEBgSUABXGLy0DsBBdBuECBAgACBPAEFMC+zVhMrgK3iMAwBAgQIEFgSUACXmDzkC6AdIECAAAECcwQUwDlZbrmJL4Bb2B1KgAABAgRKAgpgic/LCqAdIECAAAECeQIKYF5mrSZWAFvFYRgCBAgQILAkoAAuMXnonYACaDcIECBAgECegAKYl1mriRXAVnEYhgABAgQILAkogEtMHvIF0A4QIECAAIE5AgrgnCy33MQXwC3sDiVAgAABAiUBBbDE52UF0A4QIECAAIE8AQUwL7NWEyuAreIwDAECBAgQWBJQAJeYPPROQAG0GwQIECBAIE9AAczLrNXECmCrOAxDgAABAgSWBBTAJSYP+QJoBwgQIECAwBwBBXBOlltu4gvgFnaHEiBAgACBkoACWOLzsgJoBwgQIECAQJ6AApiXWauJFcBWcRiGAAECBAgsCSiAS0weeiegANoNAgQIECCQJ6AA5mXWamIFsFUchiFAgAABAksCCuASk4d8AbQDBAi8BP7+lz/8H4j/+f2fwRAgECigAAaG1mlkXwA7pWEWAvcL/FgAfzxRIbw/AycQuEJAAbxC8eDfUAAPDt/VjxNQ/o6L3IUHCyiAg8N94moK4BPKziDQQ0AB7JGDKQhcIaAAXqF48G8ogAeH7+rHCSiAx0XuwoMFFMDB4T5xNQXwCWVnEOghoAD2yMEUBK4QUACvUDz4NxTAg8N39aMEPpW/F4b/AOSolXDZcAEFMDzA3eMrgLsTcD6BZwQUwGecnULgKQEF8CnpoecogEODdS0CPwgogFaCwCwBBXBWno/fRgF8nNyBBLYIKIBb2B1K4DYBBfA22jN+WAE8I2e3JKAA2gECswQUwFl5Pn4bBfBxcgcSeFxgpfy9hvIfgTwejQMJfFlAAfwynRdfAgqgPSAwX2ClACp/8/fADWcJKICz8nz8Ngrg4+QOJPC4gAL4OLkDCdwuoADeTjz7AAVwdr5uR+AloADaAwLzBBTAeZk+eiMF8FFuhxHYIqAAbmF3KIFbBRTAW3nn/7gCOD9jNyTwqQD69392hECegAKYl1mriRXAVnEYhsAtAgrgLax+lMBWAQVwK3/+4QpgfoZuQOCTgAL4ScifE8gTUADzMms1sQLYKg7DELhc4FP5ex3or4AvZ/eDBG4XUABvJ559gAI4O1+3I/CpACp/doRApoACmJlbm6kVwDZRGITALQIK4C2sfpTAdgEFcHsE2QMogNn5mZ7AJwEF8JOQPyeQKaAAZubWZmoFsE0UBiFwi4ACeAurHyWwXUAB3B5B9gAKYHZ+pifwM4FP5e/1rn8DaIcIZAoogJm5tZlaAWwThUEIXC7wqQAqf5eT+0ECjwkogI9RzzxIAZyZq1sReAkogPaAwFwBBXButo/cTAF8hNkhBLYIKIBb2B1K4BEBBfAR5rmHKIBzs3WzswU+lb+Xjr8CPntH3D5bQAHMzm/79Arg9ggMQOAWgU8FUPm7hd2PEnhMQAF8jHrmQQrgzFzdioACaAcIzBZQAGfne/vtFMDbiR1AYIuAAriF3aEEHhNQAB+jnnmQAjgzV7c6W+BT+Xvp+Cvgs3fE7fMFFMD8DLfeQAHcyu9wArcIKIC3sPpRAq0EFMBWceQNowDmZWZiAp8EFMBPQv6cQL6AApif4dYbKIBb+R1O4BYBBfAWVj9KoJWAAtgqjrxhFMC8zExM4JOAAvhJyJ8TyBdQAPMz3HoDBXArv8MJ3CKgAN7C6kcJtBJQAFvFkTeMApiXmYkJfBL4VAD9F8CfBP05gf4CCmD/jFpPqAC2jsdwBL4koAB+ic1LBKIEFMCouPoNqwD2y8REBKoCCmBV0PsE+gsogP0zaj2hAtg6HsMR+K8FPpW/1w/6K+D/mtULBNoJKIDtIskaSAHMysu0BD4JfCqAyt8nQX9OIENAAczIqe2UCmDbaAxG4EsCCuCX2LxEIE5AAYyLrNfACmCvPExDoCqgAFYFvU8gQ0ABzMip7ZQKYNtoDEbgSwIK4JfYvEQgTkABjIus18AKYK88TEOgIqD8VfS8SyBLQAHMyqvdtApgu0gMRODLAgrgl+m8SCBOQAGMi6zXwApgrzxMQ6AioABW9LxLIEtAAczKq920CmC7SAxE4MsCCuCX6bxIIE5AAYyLrNfACmCvPExD4KsCn8rf63f9/wD8qq73CPQTUAD7ZRI1kQIYFZdhCbwV+FQAlT/LQ2CWgAI4K8/Hb6MAPk7uQAK3CCiAt7D6UQJtBRTAttFkDKYAZuRkSgKfBBTAT0L+nMAsAQVwVp6P30YBfJzcgQRuEVAAb2H1owTaCiiAbaPJGEwBzMjJlAR+JqD82Q8C5wkogOdlfumNFcBLOf0YgS0CCuAWdocS2CqgAG7lzz9cAczP0A0IKIB2gMB5AgrgeZlfemMF8FJOP0Zgi4ACuIXdoQS2CiiAW/nzD1cA8zN0AwIKoB0gcJ6AAnhe5pfeWAG8lNOPEXhc4FP5ew3k/wn047E4kMDtAgrg7cSzD1AAZ+frdvMFFMD5GbshgV8TUADtRUlAASzxeZnAdgEFcHsEBiCwRUAB3MI+51AFcE6WbnKmgAJ4Zu5uTUABtAMlAQWwxOdlAtsFFMDtERiAwBYBBXAL+5xDFcA5WbrJmQKfCqD/AOTMvXDr+QIK4PyMb72hAngrrx8ncLuAAng7sQMItBRQAFvGkjOUApiTlUkJ/JqAAmgvCJwpoACemftlt1YAL6P0QwQeF/hU/l4D+Svgx2NxIIFHBBTAR5jnHqIAzs3WzeYLfCqAyt/8HXDDcwUUwHOzv+TmCuAljH6EwBYBBXALu0MJtBBQAFvEkDuEApibnckJKIB2gMC5AgrgudlfcnMF8BJGP0Jgi4ACuIXdoQRaCCiALWLIHUIBzM3O5GcLfCp/Lx3/BvDsHXH72QIK4Ox8b7+dAng7sQMI3CLwqQAqf7ew+1ECbQQUwDZRZA6iAGbmZmoCCqAdIHC2gAJ4dv7l2yuAZUI/QGCLgAK4hd2hBNoIKIBtosgcRAHMzM3UZwt8Kn8vHX8FfPaOuP18AQVwfsa33lABvJXXjxO4ReBTAVT+bmH3owRaCSiAreLIG0YBzMvMxAQUQDtAgIACaAdKAgpgic/LBLYIKIBb2B1KoJWAAtgqjrxhFMC8zExMQAG0AwQIKIB2oCSgAJb4vEzgcYFP5e81kH8D+HgsDiTwuIAC+Dj5rAMVwFl5us18AQVwfsZuSGBFQAFcUfLMWwEF0HIQyBJQALPyMi2BuwQUwLtkD/ldBfCQoF1zjIACOCZKFyFQElAAS3xeVgDtAIEcgZXy97qNfwOYk6lJCXxVQAH8qpz3/imgAFoEAjkCKwVQ+cvJ06QEKgIKYEXPuwqgHSAQJKAABoVlVAI3CyiANwNP/3lfAKcn7H6TBBTASWm6C4GagAJY8zv+bQXw+BUAECSgAAaFZVQCNwsogDcDT/95BXB6wu43SeBTAfTv/yal7S4Efi6gANqQkoACWOLzMoElgW/fvi099+mhv/359z995Ld/+Munn/jpn3///r30vpcJEHhOQAF8znrkSQrgyFhdqpmAAtgsEOMQGCCgAA4IcecVFMCd+s4+ReCqAvjHP/7t/5H97jf/+ernC+ApG+WeBH75RQG0BSUBBbDE52UCSwJXFMDXX//+9R+//lfA/yqBCuBSHB4iMEJAARwR475LKID77J18jsAVBfDXvv79S/BVAKvl7/Vb/g3gOTvppvkCCmB+hltvoABu5Xf4IQJ3F8AX45/+9NuypgJYJvQDBB4TUAAfo555kAI4M1e36iWgAPbKwzQEJggogBNS3HgHBXAjvqOPEagWwJ/9+78Xor8CPmaVXJTAvwUUQMtQElAAS3xeJrAkcEUBfB307j8CueKvf1+/76+Al+L0EIEWAgpgixhyh1AAc7MzeY7AnQXwqq9/CmDOPpmUwEtAAbQHJQEFsMTnZQJLAtUC+OMhP/4vglzxXwArgEtReohAGwEFsE0UmYMogJm5mTpL4OoCeNft/RXwXbJ+l8D1Agrg9aan/aL/8c/TEndfAgQIEJggcM3/yHioxNGXvyIzXwCvUPQbBH4u4AugDSFA4GoBXwCvFj3s9xTAwwJ33S0CCuAWdocSGC2gAI6O9/7LKYD3GzuBgAJoBwgQuFpAAbxa9LDfUwAPC9x1twgogFvYHUpgtIACODre+y+nAN5v7AQCCqAdIEDgagEF8GrRw35PATwscNfdIqAAbmF3KIHRAgrg6Hjvv5wCeL+xEwgogHaAAIGrBRTAq0UP+z0F8LDAXXeLgAK4hd2hBEYLKICj473/cgrg/cZOIKAA2gECBK4WUACvFj3s9xTAwwJ33S0CCuAWdocSGC2gAI6O9/7LKYD3GzuBgAJoBwgQuFpAAbxa9LDfUwAPC9x1twgogFvYHUpgtIACODre+y+nAN5v7AQCBAgQIHC1gAJ4tehhv6cAHha46xIgQIDACAEFcESM+y6hAO6zdzIBAgQIEPiqgAL4VTnv/VNAAbQIBAgQIEAgT0ABzMus1cQKYKs4DEOAAAECBJYEFMAlJg+9E1AA7QYBAgQIEMgTUADzMms1sQLYKg7DECBAgACBJQEFcInJQ74A2gECBAgQIDBHQAGck+WWm/gCuIXdoQQIECBAoCSgAJb4vKwA2gECBAgQIJAnoADmZdZqYgWwVRyGIUCAAAECSwIK4BKTh94JKIB2gwABAgQI5AkogHmZtZpYAWwVh2EIECBAgMCSgAK4xOQhXwDtAAECBAgQmCOgAM7JcstNfAHcwu5QAgQIECBQElAAS3xeVgDtAAECBAgQyBNQAPMyazWxAtgqDsMQIECAAIElAQVwiclD7wQUQLtBgAABAgTyBBTAvMxaTawAtorDMAQIECBAYElAAVxi8pAvgHaAAAECBAjMEVAA52S55Sa+AG5hdygBAgQIECgJKIAlPi8rgHaAAAECBAjkCSiAeZm1mlgBbBWHYQgQIECAwJKAArjE5KF3Agqg3SBAgAABAnkCCmBeZq0mVgBbxWEYAgQIECCwJKAALjF5yBdAO0CAAAECBOYIKIBzstxyE18At7A7lAABAgQIlAQUwBKflxVAO0CAAAECBPIEFMC8zFpNrAC2isMwBAgQIEBgSUABXGLy0DsBBdBuECBAgACBPAEFMC+zVhMrgK3iMAwBAgQIEFgSUACXmDzkC6AdIECAAAECcwQUwDlZbrmJL4Bb2B1KgAABAgRKAgpgic/LCqAdIECAAAECeQIKYF5mrSZWAFvFYRgCBAgQILAkoAAuMXnonYACaDcIECBAgECegAKYl1mriRXAVnEYhgABAgQILAkogEtMHvIF0A4QIECAAIE5AgrgnCy33MQXwC3sDiVAgAABAiUBBbDE52UF0A4QIECAAIE8AQUwL7NWEyuAreIwDAECBAgQWBJQAJeYPPROQAG0GwQIECBAIE9AAczLrNXECmCrOAxDgAABAgSWBBTAJSYP+QJoBwgQIECAwBwBBXBOlltu4gvgFnaHEiBAgACBkoACWOLzsgJoBwgQIECAQJ6AApiXWauJFcBWcRiGAAECBAgsCSiAS0weeiegANoNAgQIECCQJ6AA5mXWamIFsFUchiFAgAABAksCCuASk4d8AbQDBAgQIEBgjoACOCfLLTfxBXALu0MJECBAgEBJQAEs8XlZAbQDBAgQIEAgT0ABzMus1cQKYKs4DEOAAAECBJYEFMAlJg+9E1AA7QYBAgQIEMgTUADzMms1sQLYKg7DECBAgACBJQEFcInJQ74A2gECBAgQIDBHQAGck+WWm/gCuIXdoQQIECBAoCSgAJb4vKwA2gECBAgQIJAnoADmZdZqYgWwVRyGIUCAAAECSwIK4BKTh94JKIB2gwABAgQI5AkogHmZtZpYAWwVh2EIECBAgMCSgAK4xOQhXwDtAAECBAgQmCOgAM7JcstNfAHcwu5QAgQIECBQElAAS3xeVgDtAAECBAgQyBNQAPMyazWxAtgqDsMQIECAAIElAQVwiclD7wQUQLtBgAABAgTyBBTAvMxaTawAtorDMAQIECBAYElAAVxi8pAvgHaAAAECBAjMEVAA52S55Sa+AG5hdygBAgQIECgJKIAlPi8rgHaAAAECBAjkCSiAeZmZmAABAgQIECBAoCDwrfCuVwkQIECAAAECBAIFFMDA0IxMgAABAgQIEKgIKIAVPe8SIECAAAECBAIFFMDA0IxMgAABAgQIEKgIKIAVPe8SIECAAAECBAIFFMDA0IxMgAABAgQIEKgIKIAVPe8SIECAAAECBAIFFMDA0IxMgAABAgQIEKgIKIAVPe8SIECAAAECBAIFFMDA0IxMgAABAgQIEKgIKIAVPe8SIECAAAECBAIFFMDA0IxMgAABAgQIEKgIKIAVPSwqvOAAAAN3SURBVO8SIECAAAECBAIFFMDA0IxMgAABAgQIEKgIKIAVPe8SIECAAAECBAIFFMDA0IxMgAABAgQIEKgIKIAVPe8SIECAAAECBAIFFMDA0IxMgAABAgQIEKgIKIAVPe8SIECAAAECBAIFFMDA0IxMgAABAgQIEKgIKIAVPe8SIECAAAECBAIFFMDA0IxMgAABAgQIEKgIKIAVPe8SIECAAAECBAIFFMDA0IxMgAABAgQIEKgIKIAVPe8SIECAAAECBAIFFMDA0IxMgAABAgQIEKgIKIAVPe8SIECAAAECBAIFFMDA0IxMgAABAgQIEKgIKIAVPe8SIECAAAECBAIFFMDA0IxMgAABAgQIEKgIKIAVPe8SIECAAAECBAIFFMDA0IxMgAABAgQIEKgIKIAVPe8SIECAAAECBAIFFMDA0IxMgAABAgQIEKgIKIAVPe8SIECAAAECBAIFFMDA0IxMgAABAgQIEKgIKIAVPe8SIECAAAECBAIFFMDA0IxMgAABAgQIEKgIKIAVPe8SIECAAAECBAIFFMDA0IxMgAABAgQIEKgIKIAVPe8SIECAAAECBAIFFMDA0IxMgAABAgQIEKgIKIAVPe8SIECAAAECBAIFFMDA0IxMgAABAgQIEKgIKIAVPe8SIECAAAECBAIFFMDA0IxMgAABAgQIEKgIKIAVPe8SIECAAAECBAIFFMDA0IxMgAABAgQIEKgIKIAVPe8SIECAAAECBAIFFMDA0IxMgAABAgQIEKgIKIAVPe8SIECAAAECBAIFFMDA0IxMgAABAgQIEKgIKIAVPe8SIECAAAECBAIFFMDA0IxMgAABAgQIEKgIKIAVPe8SIECAAAECBAIFFMDA0IxMgAABAgQIEKgIKIAVPe8SIECAAAECBAIFFMDA0IxMgAABAgQIEKgIKIAVPe8SIECAAAECBAIFFMDA0IxMgAABAgQIEKgIKIAVPe8SIECAAAECBAIFFMDA0IxMgAABAgQIEKgIKIAVPe8SIECAAAECBAIFFMDA0IxMgAABAgQIEKgIKIAVPe8SIECAAAECBAIFFMDA0IxMgAABAgQIEKgIKIAVPe8SIECAAAECBAIFFMDA0IxMgAABAgQIEKgIKIAVPe8SIECAAAECBAIFFMDA0IxMgAABAgQIEKgIKIAVPe8SIECAAAECBAIF/hcKDbU7M03aSQAAAABJRU5ErkJggg==\" width=\"640\">"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "video = plot_animation(frames)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Yeah... pretty bad. The neural network will have to learn to do better. First let's see if it is capable of learning the basic policy we used earlier: go left if the pole is tilting left, and go right if it is tilting right. The following code defines the same neural network but we add the target probabilities `y`, and the training operations (`cross_entropy`,  `optimizer` and `training_op`):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import tensorflow as tf\n",
    "\n",
    "reset_graph()\n",
    "\n",
    "n_inputs = 4\n",
    "n_hidden = 4\n",
    "n_outputs = 1\n",
    "\n",
    "learning_rate = 0.01\n",
    "\n",
    "initializer = tf.contrib.layers.variance_scaling_initializer()\n",
    "\n",
    "X = tf.placeholder(tf.float32, shape=[None, n_inputs])\n",
    "y = tf.placeholder(tf.float32, shape=[None, n_outputs])\n",
    "\n",
    "hidden = tf.layers.dense(X, n_hidden, activation=tf.nn.elu, kernel_initializer=initializer)\n",
    "logits = tf.layers.dense(hidden, n_outputs)\n",
    "outputs = tf.nn.sigmoid(logits) # probability of action 0 (left)\n",
    "p_left_and_right = tf.concat(axis=1, values=[outputs, 1 - outputs])\n",
    "action = tf.multinomial(tf.log(p_left_and_right), num_samples=1)\n",
    "\n",
    "cross_entropy = tf.nn.sigmoid_cross_entropy_with_logits(labels=y, logits=logits)\n",
    "optimizer = tf.train.AdamOptimizer(learning_rate)\n",
    "training_op = optimizer.minimize(cross_entropy)\n",
    "\n",
    "init = tf.global_variables_initializer()\n",
    "saver = tf.train.Saver()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can make the same net play in 10 different environments in parallel, and train for 1000 iterations. We also reset environments when they are done."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[2017-09-25 11:37:10,300] Making new env: CartPole-v0\n",
      "[2017-09-25 11:37:10,305] Making new env: CartPole-v0\n",
      "[2017-09-25 11:37:10,308] Making new env: CartPole-v0\n",
      "[2017-09-25 11:37:10,312] Making new env: CartPole-v0\n",
      "[2017-09-25 11:37:10,316] Making new env: CartPole-v0\n",
      "[2017-09-25 11:37:10,319] Making new env: CartPole-v0\n",
      "[2017-09-25 11:37:10,322] Making new env: CartPole-v0\n",
      "[2017-09-25 11:37:10,325] Making new env: CartPole-v0\n",
      "[2017-09-25 11:37:10,328] Making new env: CartPole-v0\n",
      "[2017-09-25 11:37:10,331] Making new env: CartPole-v0\n"
     ]
    }
   ],
   "source": [
    "n_environments = 10\n",
    "n_iterations = 1000\n",
    "\n",
    "envs = [gym.make(\"CartPole-v0\") for _ in range(n_environments)]\n",
    "observations = [env.reset() for env in envs]\n",
    "\n",
    "with tf.Session() as sess:\n",
    "    init.run()\n",
    "    for iteration in range(n_iterations):\n",
    "        target_probas = np.array([([1.] if obs[2] < 0 else [0.]) for obs in observations]) # if angle<0 we want proba(left)=1., or else proba(left)=0.\n",
    "        action_val, _ = sess.run([action, training_op], feed_dict={X: np.array(observations), y: target_probas})\n",
    "        for env_index, env in enumerate(envs):\n",
    "            obs, reward, done, info = env.step(action_val[env_index][0])\n",
    "            observations[env_index] = obs if not done else env.reset()\n",
    "    saver.save(sess, \"./my_policy_net_basic.ckpt\")\n",
    "\n",
    "for env in envs:\n",
    "    env.close()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def render_policy_net(model_path, action, X, n_max_steps = 1000):\n",
    "    frames = []\n",
    "    env = gym.make(\"CartPole-v0\")\n",
    "    obs = env.reset()\n",
    "    with tf.Session() as sess:\n",
    "        saver.restore(sess, model_path)\n",
    "        for step in range(n_max_steps):\n",
    "            img = render_cart_pole(env, obs)\n",
    "            frames.append(img)\n",
    "            action_val = action.eval(feed_dict={X: obs.reshape(1, n_inputs)})\n",
    "            obs, reward, done, info = env.step(action_val[0][0])\n",
    "            if done:\n",
    "                break\n",
    "    env.close()\n",
    "    return frames        "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[2017-09-25 11:37:30,225] Making new env: CartPole-v0\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Restoring parameters from ./my_policy_net_basic.ckpt\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[2017-09-25 11:37:30,231] Restoring parameters from ./my_policy_net_basic.ckpt\n"
     ]
    },
    {
     "data": {
      "application/javascript": [
       "/* Put everything inside the global mpl namespace */\n",
       "window.mpl = {};\n",
       "\n",
       "\n",
       "mpl.get_websocket_type = function() {\n",
       "    if (typeof(WebSocket) !== 'undefined') {\n",
       "        return WebSocket;\n",
       "    } else if (typeof(MozWebSocket) !== 'undefined') {\n",
       "        return MozWebSocket;\n",
       "    } else {\n",
       "        alert('Your browser does not have WebSocket support.' +\n",
       "              'Please try Chrome, Safari or Firefox ≥ 6. ' +\n",
       "              'Firefox 4 and 5 are also supported but you ' +\n",
       "              'have to enable WebSockets in about:config.');\n",
       "    };\n",
       "}\n",
       "\n",
       "mpl.figure = function(figure_id, websocket, ondownload, parent_element) {\n",
       "    this.id = figure_id;\n",
       "\n",
       "    this.ws = websocket;\n",
       "\n",
       "    this.supports_binary = (this.ws.binaryType != undefined);\n",
       "\n",
       "    if (!this.supports_binary) {\n",
       "        var warnings = document.getElementById(\"mpl-warnings\");\n",
       "        if (warnings) {\n",
       "            warnings.style.display = 'block';\n",
       "            warnings.textContent = (\n",
       "                \"This browser does not support binary websocket messages. \" +\n",
       "                    \"Performance may be slow.\");\n",
       "        }\n",
       "    }\n",
       "\n",
       "    this.imageObj = new Image();\n",
       "\n",
       "    this.context = undefined;\n",
       "    this.message = undefined;\n",
       "    this.canvas = undefined;\n",
       "    this.rubberband_canvas = undefined;\n",
       "    this.rubberband_context = undefined;\n",
       "    this.format_dropdown = undefined;\n",
       "\n",
       "    this.image_mode = 'full';\n",
       "\n",
       "    this.root = $('<div/>');\n",
       "    this._root_extra_style(this.root)\n",
       "    this.root.attr('style', 'display: inline-block');\n",
       "\n",
       "    $(parent_element).append(this.root);\n",
       "\n",
       "    this._init_header(this);\n",
       "    this._init_canvas(this);\n",
       "    this._init_toolbar(this);\n",
       "\n",
       "    var fig = this;\n",
       "\n",
       "    this.waiting = false;\n",
       "\n",
       "    this.ws.onopen =  function () {\n",
       "            fig.send_message(\"supports_binary\", {value: fig.supports_binary});\n",
       "            fig.send_message(\"send_image_mode\", {});\n",
       "            if (mpl.ratio != 1) {\n",
       "                fig.send_message(\"set_dpi_ratio\", {'dpi_ratio': mpl.ratio});\n",
       "            }\n",
       "            fig.send_message(\"refresh\", {});\n",
       "        }\n",
       "\n",
       "    this.imageObj.onload = function() {\n",
       "            if (fig.image_mode == 'full') {\n",
       "                // Full images could contain transparency (where diff images\n",
       "                // almost always do), so we need to clear the canvas so that\n",
       "                // there is no ghosting.\n",
       "                fig.context.clearRect(0, 0, fig.canvas.width, fig.canvas.height);\n",
       "            }\n",
       "            fig.context.drawImage(fig.imageObj, 0, 0);\n",
       "        };\n",
       "\n",
       "    this.imageObj.onunload = function() {\n",
       "        this.ws.close();\n",
       "    }\n",
       "\n",
       "    this.ws.onmessage = this._make_on_message_function(this);\n",
       "\n",
       "    this.ondownload = ondownload;\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._init_header = function() {\n",
       "    var titlebar = $(\n",
       "        '<div class=\"ui-dialog-titlebar ui-widget-header ui-corner-all ' +\n",
       "        'ui-helper-clearfix\"/>');\n",
       "    var titletext = $(\n",
       "        '<div class=\"ui-dialog-title\" style=\"width: 100%; ' +\n",
       "        'text-align: center; padding: 3px;\"/>');\n",
       "    titlebar.append(titletext)\n",
       "    this.root.append(titlebar);\n",
       "    this.header = titletext[0];\n",
       "}\n",
       "\n",
       "\n",
       "\n",
       "mpl.figure.prototype._canvas_extra_style = function(canvas_div) {\n",
       "\n",
       "}\n",
       "\n",
       "\n",
       "mpl.figure.prototype._root_extra_style = function(canvas_div) {\n",
       "\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._init_canvas = function() {\n",
       "    var fig = this;\n",
       "\n",
       "    var canvas_div = $('<div/>');\n",
       "\n",
       "    canvas_div.attr('style', 'position: relative; clear: both; outline: 0');\n",
       "\n",
       "    function canvas_keyboard_event(event) {\n",
       "        return fig.key_event(event, event['data']);\n",
       "    }\n",
       "\n",
       "    canvas_div.keydown('key_press', canvas_keyboard_event);\n",
       "    canvas_div.keyup('key_release', canvas_keyboard_event);\n",
       "    this.canvas_div = canvas_div\n",
       "    this._canvas_extra_style(canvas_div)\n",
       "    this.root.append(canvas_div);\n",
       "\n",
       "    var canvas = $('<canvas/>');\n",
       "    canvas.addClass('mpl-canvas');\n",
       "    canvas.attr('style', \"left: 0; top: 0; z-index: 0; outline: 0\")\n",
       "\n",
       "    this.canvas = canvas[0];\n",
       "    this.context = canvas[0].getContext(\"2d\");\n",
       "\n",
       "    var backingStore = this.context.backingStorePixelRatio ||\n",
       "\tthis.context.webkitBackingStorePixelRatio ||\n",
       "\tthis.context.mozBackingStorePixelRatio ||\n",
       "\tthis.context.msBackingStorePixelRatio ||\n",
       "\tthis.context.oBackingStorePixelRatio ||\n",
       "\tthis.context.backingStorePixelRatio || 1;\n",
       "\n",
       "    mpl.ratio = (window.devicePixelRatio || 1) / backingStore;\n",
       "\n",
       "    var rubberband = $('<canvas/>');\n",
       "    rubberband.attr('style', \"position: absolute; left: 0; top: 0; z-index: 1;\")\n",
       "\n",
       "    var pass_mouse_events = true;\n",
       "\n",
       "    canvas_div.resizable({\n",
       "        start: function(event, ui) {\n",
       "            pass_mouse_events = false;\n",
       "        },\n",
       "        resize: function(event, ui) {\n",
       "            fig.request_resize(ui.size.width, ui.size.height);\n",
       "        },\n",
       "        stop: function(event, ui) {\n",
       "            pass_mouse_events = true;\n",
       "            fig.request_resize(ui.size.width, ui.size.height);\n",
       "        },\n",
       "    });\n",
       "\n",
       "    function mouse_event_fn(event) {\n",
       "        if (pass_mouse_events)\n",
       "            return fig.mouse_event(event, event['data']);\n",
       "    }\n",
       "\n",
       "    rubberband.mousedown('button_press', mouse_event_fn);\n",
       "    rubberband.mouseup('button_release', mouse_event_fn);\n",
       "    // Throttle sequential mouse events to 1 every 20ms.\n",
       "    rubberband.mousemove('motion_notify', mouse_event_fn);\n",
       "\n",
       "    rubberband.mouseenter('figure_enter', mouse_event_fn);\n",
       "    rubberband.mouseleave('figure_leave', mouse_event_fn);\n",
       "\n",
       "    canvas_div.on(\"wheel\", function (event) {\n",
       "        event = event.originalEvent;\n",
       "        event['data'] = 'scroll'\n",
       "        if (event.deltaY < 0) {\n",
       "            event.step = 1;\n",
       "        } else {\n",
       "            event.step = -1;\n",
       "        }\n",
       "        mouse_event_fn(event);\n",
       "    });\n",
       "\n",
       "    canvas_div.append(canvas);\n",
       "    canvas_div.append(rubberband);\n",
       "\n",
       "    this.rubberband = rubberband;\n",
       "    this.rubberband_canvas = rubberband[0];\n",
       "    this.rubberband_context = rubberband[0].getContext(\"2d\");\n",
       "    this.rubberband_context.strokeStyle = \"#000000\";\n",
       "\n",
       "    this._resize_canvas = function(width, height) {\n",
       "        // Keep the size of the canvas, canvas container, and rubber band\n",
       "        // canvas in synch.\n",
       "        canvas_div.css('width', width)\n",
       "        canvas_div.css('height', height)\n",
       "\n",
       "        canvas.attr('width', width * mpl.ratio);\n",
       "        canvas.attr('height', height * mpl.ratio);\n",
       "        canvas.attr('style', 'width: ' + width + 'px; height: ' + height + 'px;');\n",
       "\n",
       "        rubberband.attr('width', width);\n",
       "        rubberband.attr('height', height);\n",
       "    }\n",
       "\n",
       "    // Set the figure to an initial 600x600px, this will subsequently be updated\n",
       "    // upon first draw.\n",
       "    this._resize_canvas(600, 600);\n",
       "\n",
       "    // Disable right mouse context menu.\n",
       "    $(this.rubberband_canvas).bind(\"contextmenu\",function(e){\n",
       "        return false;\n",
       "    });\n",
       "\n",
       "    function set_focus () {\n",
       "        canvas.focus();\n",
       "        canvas_div.focus();\n",
       "    }\n",
       "\n",
       "    window.setTimeout(set_focus, 100);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._init_toolbar = function() {\n",
       "    var fig = this;\n",
       "\n",
       "    var nav_element = $('<div/>')\n",
       "    nav_element.attr('style', 'width: 100%');\n",
       "    this.root.append(nav_element);\n",
       "\n",
       "    // Define a callback function for later on.\n",
       "    function toolbar_event(event) {\n",
       "        return fig.toolbar_button_onclick(event['data']);\n",
       "    }\n",
       "    function toolbar_mouse_event(event) {\n",
       "        return fig.toolbar_button_onmouseover(event['data']);\n",
       "    }\n",
       "\n",
       "    for(var toolbar_ind in mpl.toolbar_items) {\n",
       "        var name = mpl.toolbar_items[toolbar_ind][0];\n",
       "        var tooltip = mpl.toolbar_items[toolbar_ind][1];\n",
       "        var image = mpl.toolbar_items[toolbar_ind][2];\n",
       "        var method_name = mpl.toolbar_items[toolbar_ind][3];\n",
       "\n",
       "        if (!name) {\n",
       "            // put a spacer in here.\n",
       "            continue;\n",
       "        }\n",
       "        var button = $('<button/>');\n",
       "        button.addClass('ui-button ui-widget ui-state-default ui-corner-all ' +\n",
       "                        'ui-button-icon-only');\n",
       "        button.attr('role', 'button');\n",
       "        button.attr('aria-disabled', 'false');\n",
       "        button.click(method_name, toolbar_event);\n",
       "        button.mouseover(tooltip, toolbar_mouse_event);\n",
       "\n",
       "        var icon_img = $('<span/>');\n",
       "        icon_img.addClass('ui-button-icon-primary ui-icon');\n",
       "        icon_img.addClass(image);\n",
       "        icon_img.addClass('ui-corner-all');\n",
       "\n",
       "        var tooltip_span = $('<span/>');\n",
       "        tooltip_span.addClass('ui-button-text');\n",
       "        tooltip_span.html(tooltip);\n",
       "\n",
       "        button.append(icon_img);\n",
       "        button.append(tooltip_span);\n",
       "\n",
       "        nav_element.append(button);\n",
       "    }\n",
       "\n",
       "    var fmt_picker_span = $('<span/>');\n",
       "\n",
       "    var fmt_picker = $('<select/>');\n",
       "    fmt_picker.addClass('mpl-toolbar-option ui-widget ui-widget-content');\n",
       "    fmt_picker_span.append(fmt_picker);\n",
       "    nav_element.append(fmt_picker_span);\n",
       "    this.format_dropdown = fmt_picker[0];\n",
       "\n",
       "    for (var ind in mpl.extensions) {\n",
       "        var fmt = mpl.extensions[ind];\n",
       "        var option = $(\n",
       "            '<option/>', {selected: fmt === mpl.default_extension}).html(fmt);\n",
       "        fmt_picker.append(option)\n",
       "    }\n",
       "\n",
       "    // Add hover states to the ui-buttons\n",
       "    $( \".ui-button\" ).hover(\n",
       "        function() { $(this).addClass(\"ui-state-hover\");},\n",
       "        function() { $(this).removeClass(\"ui-state-hover\");}\n",
       "    );\n",
       "\n",
       "    var status_bar = $('<span class=\"mpl-message\"/>');\n",
       "    nav_element.append(status_bar);\n",
       "    this.message = status_bar[0];\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.request_resize = function(x_pixels, y_pixels) {\n",
       "    // Request matplotlib to resize the figure. Matplotlib will then trigger a resize in the client,\n",
       "    // which will in turn request a refresh of the image.\n",
       "    this.send_message('resize', {'width': x_pixels, 'height': y_pixels});\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.send_message = function(type, properties) {\n",
       "    properties['type'] = type;\n",
       "    properties['figure_id'] = this.id;\n",
       "    this.ws.send(JSON.stringify(properties));\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.send_draw_message = function() {\n",
       "    if (!this.waiting) {\n",
       "        this.waiting = true;\n",
       "        this.ws.send(JSON.stringify({type: \"draw\", figure_id: this.id}));\n",
       "    }\n",
       "}\n",
       "\n",
       "\n",
       "mpl.figure.prototype.handle_save = function(fig, msg) {\n",
       "    var format_dropdown = fig.format_dropdown;\n",
       "    var format = format_dropdown.options[format_dropdown.selectedIndex].value;\n",
       "    fig.ondownload(fig, format);\n",
       "}\n",
       "\n",
       "\n",
       "mpl.figure.prototype.handle_resize = function(fig, msg) {\n",
       "    var size = msg['size'];\n",
       "    if (size[0] != fig.canvas.width || size[1] != fig.canvas.height) {\n",
       "        fig._resize_canvas(size[0], size[1]);\n",
       "        fig.send_message(\"refresh\", {});\n",
       "    };\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_rubberband = function(fig, msg) {\n",
       "    var x0 = msg['x0'] / mpl.ratio;\n",
       "    var y0 = (fig.canvas.height - msg['y0']) / mpl.ratio;\n",
       "    var x1 = msg['x1'] / mpl.ratio;\n",
       "    var y1 = (fig.canvas.height - msg['y1']) / mpl.ratio;\n",
       "    x0 = Math.floor(x0) + 0.5;\n",
       "    y0 = Math.floor(y0) + 0.5;\n",
       "    x1 = Math.floor(x1) + 0.5;\n",
       "    y1 = Math.floor(y1) + 0.5;\n",
       "    var min_x = Math.min(x0, x1);\n",
       "    var min_y = Math.min(y0, y1);\n",
       "    var width = Math.abs(x1 - x0);\n",
       "    var height = Math.abs(y1 - y0);\n",
       "\n",
       "    fig.rubberband_context.clearRect(\n",
       "        0, 0, fig.canvas.width, fig.canvas.height);\n",
       "\n",
       "    fig.rubberband_context.strokeRect(min_x, min_y, width, height);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_figure_label = function(fig, msg) {\n",
       "    // Updates the figure title.\n",
       "    fig.header.textContent = msg['label'];\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_cursor = function(fig, msg) {\n",
       "    var cursor = msg['cursor'];\n",
       "    switch(cursor)\n",
       "    {\n",
       "    case 0:\n",
       "        cursor = 'pointer';\n",
       "        break;\n",
       "    case 1:\n",
       "        cursor = 'default';\n",
       "        break;\n",
       "    case 2:\n",
       "        cursor = 'crosshair';\n",
       "        break;\n",
       "    case 3:\n",
       "        cursor = 'move';\n",
       "        break;\n",
       "    }\n",
       "    fig.rubberband_canvas.style.cursor = cursor;\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_message = function(fig, msg) {\n",
       "    fig.message.textContent = msg['message'];\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_draw = function(fig, msg) {\n",
       "    // Request the server to send over a new figure.\n",
       "    fig.send_draw_message();\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_image_mode = function(fig, msg) {\n",
       "    fig.image_mode = msg['mode'];\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.updated_canvas_event = function() {\n",
       "    // Called whenever the canvas gets updated.\n",
       "    this.send_message(\"ack\", {});\n",
       "}\n",
       "\n",
       "// A function to construct a web socket function for onmessage handling.\n",
       "// Called in the figure constructor.\n",
       "mpl.figure.prototype._make_on_message_function = function(fig) {\n",
       "    return function socket_on_message(evt) {\n",
       "        if (evt.data instanceof Blob) {\n",
       "            /* FIXME: We get \"Resource interpreted as Image but\n",
       "             * transferred with MIME type text/plain:\" errors on\n",
       "             * Chrome.  But how to set the MIME type?  It doesn't seem\n",
       "             * to be part of the websocket stream */\n",
       "            evt.data.type = \"image/png\";\n",
       "\n",
       "            /* Free the memory for the previous frames */\n",
       "            if (fig.imageObj.src) {\n",
       "                (window.URL || window.webkitURL).revokeObjectURL(\n",
       "                    fig.imageObj.src);\n",
       "            }\n",
       "\n",
       "            fig.imageObj.src = (window.URL || window.webkitURL).createObjectURL(\n",
       "                evt.data);\n",
       "            fig.updated_canvas_event();\n",
       "            fig.waiting = false;\n",
       "            return;\n",
       "        }\n",
       "        else if (typeof evt.data === 'string' && evt.data.slice(0, 21) == \"data:image/png;base64\") {\n",
       "            fig.imageObj.src = evt.data;\n",
       "            fig.updated_canvas_event();\n",
       "            fig.waiting = false;\n",
       "            return;\n",
       "        }\n",
       "\n",
       "        var msg = JSON.parse(evt.data);\n",
       "        var msg_type = msg['type'];\n",
       "\n",
       "        // Call the  \"handle_{type}\" callback, which takes\n",
       "        // the figure and JSON message as its only arguments.\n",
       "        try {\n",
       "            var callback = fig[\"handle_\" + msg_type];\n",
       "        } catch (e) {\n",
       "            console.log(\"No handler for the '\" + msg_type + \"' message type: \", msg);\n",
       "            return;\n",
       "        }\n",
       "\n",
       "        if (callback) {\n",
       "            try {\n",
       "                // console.log(\"Handling '\" + msg_type + \"' message: \", msg);\n",
       "                callback(fig, msg);\n",
       "            } catch (e) {\n",
       "                console.log(\"Exception inside the 'handler_\" + msg_type + \"' callback:\", e, e.stack, msg);\n",
       "            }\n",
       "        }\n",
       "    };\n",
       "}\n",
       "\n",
       "// from http://stackoverflow.com/questions/1114465/getting-mouse-location-in-canvas\n",
       "mpl.findpos = function(e) {\n",
       "    //this section is from http://www.quirksmode.org/js/events_properties.html\n",
       "    var targ;\n",
       "    if (!e)\n",
       "        e = window.event;\n",
       "    if (e.target)\n",
       "        targ = e.target;\n",
       "    else if (e.srcElement)\n",
       "        targ = e.srcElement;\n",
       "    if (targ.nodeType == 3) // defeat Safari bug\n",
       "        targ = targ.parentNode;\n",
       "\n",
       "    // jQuery normalizes the pageX and pageY\n",
       "    // pageX,Y are the mouse positions relative to the document\n",
       "    // offset() returns the position of the element relative to the document\n",
       "    var x = e.pageX - $(targ).offset().left;\n",
       "    var y = e.pageY - $(targ).offset().top;\n",
       "\n",
       "    return {\"x\": x, \"y\": y};\n",
       "};\n",
       "\n",
       "/*\n",
       " * return a copy of an object with only non-object keys\n",
       " * we need this to avoid circular references\n",
       " * http://stackoverflow.com/a/24161582/3208463\n",
       " */\n",
       "function simpleKeys (original) {\n",
       "  return Object.keys(original).reduce(function (obj, key) {\n",
       "    if (typeof original[key] !== 'object')\n",
       "        obj[key] = original[key]\n",
       "    return obj;\n",
       "  }, {});\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.mouse_event = function(event, name) {\n",
       "    var canvas_pos = mpl.findpos(event)\n",
       "\n",
       "    if (name === 'button_press')\n",
       "    {\n",
       "        this.canvas.focus();\n",
       "        this.canvas_div.focus();\n",
       "    }\n",
       "\n",
       "    var x = canvas_pos.x * mpl.ratio;\n",
       "    var y = canvas_pos.y * mpl.ratio;\n",
       "\n",
       "    this.send_message(name, {x: x, y: y, button: event.button,\n",
       "                             step: event.step,\n",
       "                             guiEvent: simpleKeys(event)});\n",
       "\n",
       "    /* This prevents the web browser from automatically changing to\n",
       "     * the text insertion cursor when the button is pressed.  We want\n",
       "     * to control all of the cursor setting manually through the\n",
       "     * 'cursor' event from matplotlib */\n",
       "    event.preventDefault();\n",
       "    return false;\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._key_event_extra = function(event, name) {\n",
       "    // Handle any extra behaviour associated with a key event\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.key_event = function(event, name) {\n",
       "\n",
       "    // Prevent repeat events\n",
       "    if (name == 'key_press')\n",
       "    {\n",
       "        if (event.which === this._key)\n",
       "            return;\n",
       "        else\n",
       "            this._key = event.which;\n",
       "    }\n",
       "    if (name == 'key_release')\n",
       "        this._key = null;\n",
       "\n",
       "    var value = '';\n",
       "    if (event.ctrlKey && event.which != 17)\n",
       "        value += \"ctrl+\";\n",
       "    if (event.altKey && event.which != 18)\n",
       "        value += \"alt+\";\n",
       "    if (event.shiftKey && event.which != 16)\n",
       "        value += \"shift+\";\n",
       "\n",
       "    value += 'k';\n",
       "    value += event.which.toString();\n",
       "\n",
       "    this._key_event_extra(event, name);\n",
       "\n",
       "    this.send_message(name, {key: value,\n",
       "                             guiEvent: simpleKeys(event)});\n",
       "    return false;\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.toolbar_button_onclick = function(name) {\n",
       "    if (name == 'download') {\n",
       "        this.handle_save(this, null);\n",
       "    } else {\n",
       "        this.send_message(\"toolbar_button\", {name: name});\n",
       "    }\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.toolbar_button_onmouseover = function(tooltip) {\n",
       "    this.message.textContent = tooltip;\n",
       "};\n",
       "mpl.toolbar_items = [[\"Home\", \"Reset original view\", \"fa fa-home icon-home\", \"home\"], [\"Back\", \"Back to  previous view\", \"fa fa-arrow-left icon-arrow-left\", \"back\"], [\"Forward\", \"Forward to next view\", \"fa fa-arrow-right icon-arrow-right\", \"forward\"], [\"\", \"\", \"\", \"\"], [\"Pan\", \"Pan axes with left mouse, zoom with right\", \"fa fa-arrows icon-move\", \"pan\"], [\"Zoom\", \"Zoom to rectangle\", \"fa fa-square-o icon-check-empty\", \"zoom\"], [\"\", \"\", \"\", \"\"], [\"Download\", \"Download plot\", \"fa fa-floppy-o icon-save\", \"download\"]];\n",
       "\n",
       "mpl.extensions = [\"eps\", \"jpeg\", \"pdf\", \"png\", \"ps\", \"raw\", \"svg\", \"tif\"];\n",
       "\n",
       "mpl.default_extension = \"png\";var comm_websocket_adapter = function(comm) {\n",
       "    // Create a \"websocket\"-like object which calls the given IPython comm\n",
       "    // object with the appropriate methods. Currently this is a non binary\n",
       "    // socket, so there is still some room for performance tuning.\n",
       "    var ws = {};\n",
       "\n",
       "    ws.close = function() {\n",
       "        comm.close()\n",
       "    };\n",
       "    ws.send = function(m) {\n",
       "        //console.log('sending', m);\n",
       "        comm.send(m);\n",
       "    };\n",
       "    // Register the callback with on_msg.\n",
       "    comm.on_msg(function(msg) {\n",
       "        //console.log('receiving', msg['content']['data'], msg);\n",
       "        // Pass the mpl event to the overriden (by mpl) onmessage function.\n",
       "        ws.onmessage(msg['content']['data'])\n",
       "    });\n",
       "    return ws;\n",
       "}\n",
       "\n",
       "mpl.mpl_figure_comm = function(comm, msg) {\n",
       "    // This is the function which gets called when the mpl process\n",
       "    // starts-up an IPython Comm through the \"matplotlib\" channel.\n",
       "\n",
       "    var id = msg.content.data.id;\n",
       "    // Get hold of the div created by the display call when the Comm\n",
       "    // socket was opened in Python.\n",
       "    var element = $(\"#\" + id);\n",
       "    var ws_proxy = comm_websocket_adapter(comm)\n",
       "\n",
       "    function ondownload(figure, format) {\n",
       "        window.open(figure.imageObj.src);\n",
       "    }\n",
       "\n",
       "    var fig = new mpl.figure(id, ws_proxy,\n",
       "                           ondownload,\n",
       "                           element.get(0));\n",
       "\n",
       "    // Call onopen now - mpl needs it, as it is assuming we've passed it a real\n",
       "    // web socket which is closed, not our websocket->open comm proxy.\n",
       "    ws_proxy.onopen();\n",
       "\n",
       "    fig.parent_element = element.get(0);\n",
       "    fig.cell_info = mpl.find_output_cell(\"<div id='\" + id + \"'></div>\");\n",
       "    if (!fig.cell_info) {\n",
       "        console.error(\"Failed to find cell for figure\", id, fig);\n",
       "        return;\n",
       "    }\n",
       "\n",
       "    var output_index = fig.cell_info[2]\n",
       "    var cell = fig.cell_info[0];\n",
       "\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_close = function(fig, msg) {\n",
       "    var width = fig.canvas.width/mpl.ratio\n",
       "    fig.root.unbind('remove')\n",
       "\n",
       "    // Update the output cell to use the data from the current canvas.\n",
       "    fig.push_to_output();\n",
       "    var dataURL = fig.canvas.toDataURL();\n",
       "    // Re-enable the keyboard manager in IPython - without this line, in FF,\n",
       "    // the notebook keyboard shortcuts fail.\n",
       "    IPython.keyboard_manager.enable()\n",
       "    $(fig.parent_element).html('<img src=\"' + dataURL + '\" width=\"' + width + '\">');\n",
       "    fig.close_ws(fig, msg);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.close_ws = function(fig, msg){\n",
       "    fig.send_message('closing', msg);\n",
       "    // fig.ws.close()\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.push_to_output = function(remove_interactive) {\n",
       "    // Turn the data on the canvas into data in the output cell.\n",
       "    var width = this.canvas.width/mpl.ratio\n",
       "    var dataURL = this.canvas.toDataURL();\n",
       "    this.cell_info[1]['text/html'] = '<img src=\"' + dataURL + '\" width=\"' + width + '\">';\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.updated_canvas_event = function() {\n",
       "    // Tell IPython that the notebook contents must change.\n",
       "    IPython.notebook.set_dirty(true);\n",
       "    this.send_message(\"ack\", {});\n",
       "    var fig = this;\n",
       "    // Wait a second, then push the new image to the DOM so\n",
       "    // that it is saved nicely (might be nice to debounce this).\n",
       "    setTimeout(function () { fig.push_to_output() }, 1000);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._init_toolbar = function() {\n",
       "    var fig = this;\n",
       "\n",
       "    var nav_element = $('<div/>')\n",
       "    nav_element.attr('style', 'width: 100%');\n",
       "    this.root.append(nav_element);\n",
       "\n",
       "    // Define a callback function for later on.\n",
       "    function toolbar_event(event) {\n",
       "        return fig.toolbar_button_onclick(event['data']);\n",
       "    }\n",
       "    function toolbar_mouse_event(event) {\n",
       "        return fig.toolbar_button_onmouseover(event['data']);\n",
       "    }\n",
       "\n",
       "    for(var toolbar_ind in mpl.toolbar_items){\n",
       "        var name = mpl.toolbar_items[toolbar_ind][0];\n",
       "        var tooltip = mpl.toolbar_items[toolbar_ind][1];\n",
       "        var image = mpl.toolbar_items[toolbar_ind][2];\n",
       "        var method_name = mpl.toolbar_items[toolbar_ind][3];\n",
       "\n",
       "        if (!name) { continue; };\n",
       "\n",
       "        var button = $('<button class=\"btn btn-default\" href=\"#\" title=\"' + name + '\"><i class=\"fa ' + image + ' fa-lg\"></i></button>');\n",
       "        button.click(method_name, toolbar_event);\n",
       "        button.mouseover(tooltip, toolbar_mouse_event);\n",
       "        nav_element.append(button);\n",
       "    }\n",
       "\n",
       "    // Add the status bar.\n",
       "    var status_bar = $('<span class=\"mpl-message\" style=\"text-align:right; float: right;\"/>');\n",
       "    nav_element.append(status_bar);\n",
       "    this.message = status_bar[0];\n",
       "\n",
       "    // Add the close button to the window.\n",
       "    var buttongrp = $('<div class=\"btn-group inline pull-right\"></div>');\n",
       "    var button = $('<button class=\"btn btn-mini btn-primary\" href=\"#\" title=\"Stop Interaction\"><i class=\"fa fa-power-off icon-remove icon-large\"></i></button>');\n",
       "    button.click(function (evt) { fig.handle_close(fig, {}); } );\n",
       "    button.mouseover('Stop Interaction', toolbar_mouse_event);\n",
       "    buttongrp.append(button);\n",
       "    var titlebar = this.root.find($('.ui-dialog-titlebar'));\n",
       "    titlebar.prepend(buttongrp);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._root_extra_style = function(el){\n",
       "    var fig = this\n",
       "    el.on(\"remove\", function(){\n",
       "\tfig.close_ws(fig, {});\n",
       "    });\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._canvas_extra_style = function(el){\n",
       "    // this is important to make the div 'focusable\n",
       "    el.attr('tabindex', 0)\n",
       "    // reach out to IPython and tell the keyboard manager to turn it's self\n",
       "    // off when our div gets focus\n",
       "\n",
       "    // location in version 3\n",
       "    if (IPython.notebook.keyboard_manager) {\n",
       "        IPython.notebook.keyboard_manager.register_events(el);\n",
       "    }\n",
       "    else {\n",
       "        // location in version 2\n",
       "        IPython.keyboard_manager.register_events(el);\n",
       "    }\n",
       "\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._key_event_extra = function(event, name) {\n",
       "    var manager = IPython.notebook.keyboard_manager;\n",
       "    if (!manager)\n",
       "        manager = IPython.keyboard_manager;\n",
       "\n",
       "    // Check for shift+enter\n",
       "    if (event.shiftKey && event.which == 13) {\n",
       "        this.canvas_div.blur();\n",
       "        // select the cell after this one\n",
       "        var index = IPython.notebook.find_cell_index(this.cell_info[0]);\n",
       "        IPython.notebook.select(index + 1);\n",
       "    }\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_save = function(fig, msg) {\n",
       "    fig.ondownload(fig, null);\n",
       "}\n",
       "\n",
       "\n",
       "mpl.find_output_cell = function(html_output) {\n",
       "    // Return the cell and output element which can be found *uniquely* in the notebook.\n",
       "    // Note - this is a bit hacky, but it is done because the \"notebook_saving.Notebook\"\n",
       "    // IPython event is triggered only after the cells have been serialised, which for\n",
       "    // our purposes (turning an active figure into a static one), is too late.\n",
       "    var cells = IPython.notebook.get_cells();\n",
       "    var ncells = cells.length;\n",
       "    for (var i=0; i<ncells; i++) {\n",
       "        var cell = cells[i];\n",
       "        if (cell.cell_type === 'code'){\n",
       "            for (var j=0; j<cell.output_area.outputs.length; j++) {\n",
       "                var data = cell.output_area.outputs[j];\n",
       "                if (data.data) {\n",
       "                    // IPython >= 3 moved mimebundle to data attribute of output\n",
       "                    data = data.data;\n",
       "                }\n",
       "                if (data['text/html'] == html_output) {\n",
       "                    return [cell, data, j];\n",
       "                }\n",
       "            }\n",
       "        }\n",
       "    }\n",
       "}\n",
       "\n",
       "// Register the function which deals with the matplotlib target/channel.\n",
       "// The kernel may be null if the page has been refreshed.\n",
       "if (IPython.notebook.kernel != null) {\n",
       "    IPython.notebook.kernel.comm_manager.register_target('matplotlib', mpl.mpl_figure_comm);\n",
       "}\n"
      ],
      "text/plain": [
       "<IPython.core.display.Javascript object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<img src=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAoAAAAHgCAYAAAA10dzkAAAgAElEQVR4Xu3d3bElR5WA0b5u4MbgBm6AGQRmgBu4AW6MGz1xUHSMRqOiNsqqzP2zeFXVycy18+GLalr6+uZ/BAgQIECAAAECowS+Rp3WYQkQIECAAAECBL4JQJeAAAECBAgQIDBMQAAOG7jjEiBAgAABAgQEoDtAgAABAgQIEBgmIACHDdxxCRAgQIAAAQIC0B0gQIAAAQIECAwTEIDDBu64BAgQIECAAAEB6A4QIECAAAECBIYJCMBhA3dcAgQIECBAgIAAdAcIECBAgAABAsMEBOCwgTsuAQIECBAgQEAAugMECBAgQIAAgWECAnDYwB2XAAECBAgQICAA3QECBAgQIECAwDABAThs4I5LgAABAgQIEBCA7gABAgQIECBAYJiAABw2cMclQIAAAQIECAhAd4AAAQIECBAgMExAAA4buOMSIECAAAECBASgO0CAAAECBAgQGCYgAIcN3HEJECBAgAABAgLQHSBAgAABAgQIDBMQgMMG7rgECBAgQIAAAQHoDhAgQIAAAQIEhgkIwGEDd1wCBAgQIECAgAB0BwgQIECAAAECwwQE4LCBOy4BAgQIECBAQAC6AwQIECBAgACBYQICcNjAHZcAAQIECBAgIADdAQIECBAgQIDAMAEBOGzgjkuAAAECBAgQEIDuAAECBAgQIEBgmIAAHDZwxyVAgAABAgQICEB3gAABAgQIECAwTEAADhu44xIgQIAAAQIEBKA7QIAAAQIECBAYJiAAhw3ccQkQIECAAAECAtAdIECAAAECBAgMExCAwwbuuAQIECBAgAABAegOECBAgAABAgSGCQjAYQN3XAIECBAgQICAAHQHCBAgQIAAAQLDBATgsIE7LgECBAgQIEBAALoDBAgQIECAAIFhAgJw2MAdlwABAgQIECAgAN0BAgQIECBAgMAwAQE4bOCOS4AAAQIECBAQgO4AAQIECBAgQGCYgAAcNnDHJUCAAAECBAgIQHeAAAECBAgQIDBMQAAOG7jjEiBAgAABAgQEoDtAgAABAgQIEBgmIACHDdxxCRAgQIAAAQIC0B0gQIAAAQIECAwTEIDDBu64BAgQIECAAAEB6A4QIECAAAECBIYJCMBhA3dcAgQIECBAgIAAdAcIECBAgAABAsMEBOCwgTsuAQIECBAgQEAAugMECBAgQIAAgWECAnDYwB2XAAECBAgQICAA3QECBAgQIECAwDABAThs4I5LgAABAgQIEBCA7gABAgQIECBAYJiAABw2cMclQIAAAQIECAhAd4AAAQIECBAgMExAAA4buOMSIECAAAECBASgO0CAAAECBAgQGCYgAIcN3HEJECBAgAABAgLQHSBAgAABAgQIDBMQgMMG7rgECBAgQIAAAQHoDhAgQIAAAQIEhgkIwGEDd1wCBAgQIECAgAB0BwgQIECAAAECwwQE4LCBOy4BAgQIECBAQAC6AwQIECBAgACBYQICcHHg379//774E14nQIAAAQIENgt8fX2NbqDRh3/irgnAJxT9BgECBAgQ2CsgAPd6t1tNALYbqQMRIECAwAABAThgyG8eUQC+qeu3CRAgQIDAOwIC8B3XMb8qAMeM2kEJECBAoJGAAGw0zBNHEYAn1K1JgAABAgTWBATgmt/4twXg+CsAgAABAgQKCgjAgkPLtGUBmGka9kKAAAECBGICAjDm5KkLAQHoahAgQIAAgXoCArDezFLtWACmGofNECBAgACBkIAADDF56EpAALobBAgQIECgnoAArDezVDsWgKnGYTMECBAgQCAkIABDTB7yBdAdIECAAAECfQQEYJ9ZHjmJL4BH2C1KgAABAgSWBATgEp+XBaA7QIAAAQIE6gkIwHozS7VjAZhqHDZDgAABAgRCAgIwxOShKwEB6G4QIECAAIF6AgKw3sxS7VgAphqHzRAgQIAAgZCAAAwxecgXQHeAAAECBAj0ERCAfWZ55CS+AB5htygBAgQIEFgSEIBLfF4WgO4AAQIECBCoJyAA680s1Y4FYKpx2AwBAgQIEAgJCMAQk4euBASgu0GAAAECBOoJCMB6M0u1YwGYahw2Q4AAAQIEQgICMMTkIV8A3QECBAgQINBHQAD2meWRk/gCeITdogQIECBAYElAAC7xeVkAugMECBAgQKCegACsN7NUOxaAqcZhMwQIECBAICQgAENMHroSEIDuBgECBAgQqCcgAOvNLNWOBWCqcdgMAQIECBAICQjAEJOHfAF0BwgQIECAQB8BAdhnlkdO4gvgEXaLEiBAgACBJQEBuMTnZQHoDhAgQIAAgXoCArDezFLtWACmGofNECBAgACBkIAADDF56EpAALobBAgQIECgnoAArDezVDsWgKnGYTMECBAgQCAkIABDTB7yBdAdIECAAAECfQQEYJ9ZHjmJL4BH2C1KgAABAgSWBATgEp+XBaA7QIAAAQIE6gkIwHozS7VjAZhqHDZDgAABAgRCAgIwxOShKwEB6G4QIECAAIF6AgKw3sxS7VgAphqHzRAgQIAAgZCAAAwxecgXQHeAAAECBAj0ERCAfWZ55CS+AB5htygBAgQIEFgSEIBLfF4WgO4AAQIECBCoJyAA680s1Y4FYKpx2AwBAgQIEAgJCMAQk4euBASgu0GAAAECBOoJCMB6M0u1YwGYahw2Q4AAAQIEQgICMMTkIV8A3QECBAgQINBHQAD2meWRk/gCeITdogQIECBAYElAAC7xeVkAugMECBAgQKCegACsN7NUOxaAqcZhMwQIECBAICQgAENMHroSEIDuBgECBAgQqCcgAOvNLNWOBWCqcdgMAQIECBAICQjAEJOHfAF0BwgQIECAQB8BAdhnlkdO4gvgEXaLEiBAgACBJQEBuMTnZQHoDhAgQIAAgXoCArDezFLtWACmGofNECBAgACBkIAADDF56EpAALobBAgQIECgnoAArDezVDsWgKnGYTMECBAgQCAkIABDTB7yBdAdIECAAAECfQQEYJ9ZHjmJL4BH2C1KgAABAgSWBATgEp+XBaA7QIAAAQIE6gkIwHozS7VjAZhqHDZDgAABAgRCAgIwxOShKwEB6G4QIECAAIF6AgKw3sxS7VgAphqHzRAgQIAAgZCAAAwxecgXQHeAAAECBAj0ERCAfWZ55CS+AB5htygBAgQIEFgSEIBLfF4WgO4AAQIECBCoJyAA680s1Y4FYKpx2AwBAgQIEAgJCMAQk4euBASgu0GAAAECBOoJCMB6M0u1YwGYahw2Q4AAAQIEQgICMMTkIV8A3QECBAgQINBHQAD2meWRk/gCeITdogQIECBAYElAAC7xeVkAugMECBAgQKCegACsN7NUOxaAqcZhMwQIECBAICQgAENMHroSEIDuBgECBAgQqCcgAOvNLNWOBWCqcdgMAQIECBAICQjAEJOHfAF0BwgQIECAQB8BAdhnlkdO4gvgEXaLEiBAgACBJQEBuMTnZQHoDhAgQIAAgXoCArDezFLtWACmGofNECBAgACBkIAADDF56EpAALobBAgQIECgnoAArDezVDsWgKnGYTMECBAgQCAkIABDTB7yBdAdIECAAAECfQQEYJ9ZHjmJL4BH2C1KgAABAgSWBATgEp+XBaA7QIAAAQIE6gkIwHozS7VjAZhqHDZDgAABAgRCAgIwxOShKwEB6G4QIECAAIF6AgKw3sxS7VgAphqHzRAgQIAAgZCAAAwxecgXQHeAAAECBAj0ERCAfWZ55CS+AB5htygBAgQIEFgSEIBLfF4WgO4AAQIECBCoJyAA680s1Y4FYKpx2AwBAgQIEAgJCMAQk4euBASgu0GAAAECBOoJCMB6M0u1YwGYahw2Q4AAAQIEQgICMMTkIV8A3QECBAgQINBHQAD2meWRk/gCeITdogQIECBAYElAAC7xeVkAugMECBAgQKCegACsN7NUOxaAqcZhMwQIECBAICQgAENMHroSEIDuBgECBAgQqCcgAOvNLNWOBWCqcdgMAQIECBAICQjAEJOHfAF0BwgQIECAQB8BAdhnlkdO4gvgEXaLEiBAgACBJQEBuMTnZQHoDhAgQIAAgXoCArDezFLtWACmGofNECBAgACBkIAADDF56EpAALobBAgQIECgnoAArDezVDsWgKnGYTMECBAgQCAkIABDTB7yBdAdIECAAAECfQQEYJ9ZHjmJL4BH2C1KgAABAgSWBATgEp+XBaA7QIAAAQIE6gkIwHozS7VjAZhqHDZDgAABAgRCAgIwxOShKwEB6G4QIECAAIF6AgKw3sxS7VgAphqHzRAgQIAAgZCAAAwxecgXQHeAAAECBAj0ERCAfWZ55CS+AB5htygBAgQIEFgSEIBLfF4WgO4AAQIECBCoJyAA680s1Y4FYKpx2AwBAgQIEAgJCMAQk4euBASgu0GAAAECBOoJCMB6M0u1YwGYahw2Q4AAAQIEQgICMMTkIV8A3QECBAgQINBHQAD2meWRk/gCeITdogQIECBAYElAAC7xeVkAugMECBAgQKCegACsN7NUOxaAqcZhMwQIECBAICQgAENMHroSEIDuBgECBAgQqCcgAOvNLNWOBWCqcdgMAQIECBAICQjAEJOHfAF0BwgQIECAQB8BAdhnlkdO4gvgEXaLEiBAgACBJQEBuMTnZQHoDhAgQIAAgXoCArDezFLtWACmGofNECBAgACBkIAADDF56EpAALobBAgQIECgnoAArDezVDsWgKnGYTMECBAgQCAkIABDTB7yBdAdIECAAAECfQQEYJ9ZHjmJL4BH2C1KgAABAgSWBATgEp+XBaA7QIAAAQIE6gkIwHozS7VjAZhqHDZDgAABAgRCAgIwxOShKwEB6G4QIECAAIF6AgKw3sxS7VgAphqHzRAgQIAAgZCAAAwxecgXQHeAAAECBAj0ERCAfWZ55CS+AB5htygBAgQIEFgSEIBLfF4WgO4AAQIECBCoJyAA680s1Y4FYKpx2AwBAgQIEAgJCMAQk4euBASgu0GAAAECBOoJCMB6M0u1YwGYahw2Q4AAAQIEQgICMMTkIV8A3QECMwX++bc//evg//XHv84EcGoCTQUEYNPB7jqWL4C7pK1D4IzAjwD85eqC8Mw8rErgKQEB+JTk0N8RgEMH79hjBK4C8AeAEBxzFRy0mYAAbDbQ3ccRgLvFrUdgr4AA3OttNQK7BATgLumm6wjApoN1LALfvn27i78Pki+ArgqBmgICsObc0uxaAKYZhY0QeFzgLgDF3+PkfpDANgEBuI2650ICsOdcnYrAR0AAugcE+goIwL6z3XIyAbiF2SIEjggIwCPsFiWwRUAAbmHuu4gA7DtbJ5stcBd/Hx1/BDz7jjh9bQEBWHt+x3cvAI+PwAYIvCJwF4Di7xV2P0pgm4AA3EbdcyEB2HOuTkVAALoDBHoLCMDe8339dALwdWILEDgiIACPsFuUwDYBAbiNuudCArDnXJ1qtsBd/H10/BHw7Dvi9PUFBGD9GR49gQA8ym9xAq8ICMBXWP0ogVQCAjDVOOptRgDWm5kdE7gTEIB3Qv45gfoCArD+DI+eQAAe5bc4gVcEBOArrH6UQCoBAZhqHPU2IwDrzcyOCdwJCMA7If+cQH0BAVh/hkdPIACP8lucwCsCdwHoL4C8wu5HCWwVEIBbufstJgD7zdSJCAhAd4BAfwEB2H/Gr55QAL7K68cJHBEQgEfYLUpgq4AA3MrdbzEB2G+mTkRAALoDBPoLCMD+M371hALwVV4/TmC7gPjbTm5BAkcEBOAR9j6LCsA+s3QSAh8BAegeEJghIABnzPm1UwrA12j9MIEjAgLwCLtFCWwXEIDbyXstKAB7zdNpZgvcxd9Hx78CZvYdcfo+AgKwzyyPnEQAHmG3KIFXBO4CUPy9wu5HCRwREIBH2PssKgD7zNJJCAhAd4DAHAEBOGfWr5xUAL7C6kcJHBEQgEfYLUrgiIAAPMLeZ1EB2GeWTjJb4C7+Pjr+CHj2HXH6XgICsNc8t59GAG4ntyCBVwTuAlD8vcLuRwkcExCAx+h7LCwAe8zRKQgIQHeAwCwBAThr3o+fVgA+TuoHCRwREIBH2C1K4JiAADxG32NhAdhjjk4xW+Au/j46/gh49h1x+n4CArDfTLeeSABu5bYYgVcE7gJQ/L3C7kcJHBUQgEf56y8uAOvP0AkICEB3gMA8AQE4b+aPnlgAPsrpxwgcERCAR9gtSuCogAA8yl9/cQFYf4ZOMFvgLv4+Ov4IePYdcfqeAgKw51y3nUoAbqO2EIFXBATgK6x+lEB6AQGYfkS5NygAc8/H7gjcCQjAOyH/nEBPAQHYc67bTiUAt1FbiMArAgLwFVY/SiC9gABMP6LcGxSAuedjdwTuBATgnZB/TqCngADsOddtpxKA26gtROAVgbsA9BdAXmH3owSOCwjA4yOovQEBWHt+dk9AALoDBGYKCMCZc3/s1ALwMUo/ROCIgAA8wm5RAscFBODxEdTegACsPT+7JyAA3QECMwUE4My5P3ZqAfgYpR8isF1A/G0ntyCBNAICMM0oam5EANacm10T+AgIQPeAwFwBATh39o+cXAA+wuhHCBwREIBH2C1KIIWAAEwxhrqbEIB1Z2fnBASgO0BgroAAnDv7R04uAB9h9CMEtgvcxd9nQ/4dgNvHYkEC2wQE4DbqngsJwJ5zdar+AncBKP763wEnnC0gAGfPf/n0AnCZ0A8QOCIgAI+wW5RAGgEBmGYUNTciAGvOza4JCEB3gMBsAQE4e/7LpxeAy4R+gMB2gbv4+2zIHwFvH4sFCWwVEIBbufstJgD7zdSJ+gvcBaD4638HnJCAAHQHlgQE4BKflwkcERCAR9gtSiCVgABMNY56mxGA9WZmx7MF7uLPH//Ovh9OP0dAAM6Z9SsnFYCvsPpRAq8JCMDXaP0wgVICArDUuPJtVgDmm4kdEfh3AgLQ/SBA4CMgAN2DJQEBuMTnZQLbBQTgdnILEkgpIABTjqXOpgRgnVnZKYFI/H2U/C1gd4VAfwEB2H/Gr55QAL7K68cJPCoQCUDx9yi5HyOQVkAAph1NjY0JwBpzsksCHwEB6B4QIPBDQAC6C0sCAnCJz8sEtgoIwK3cFiOQWkAAph5P/s0JwPwzskMCPwQEoLtAgIAvgD8JfLkKawICcM3P2wR2CtwFoP//385pWIvAWQFfAM/6l19dAJYfoQMMEhCAg4btqARuBASgK7IkIACX+LxMYKuAANzKbTECqQUEYOrx5N+cAMw/Izsk8ENAALoLBAj8EBCA7sKSgABc4vMygW0C4m8btYUIlBAQgCXGlHeTAjDvbOyMwM8FBKD7QIDAzwUEoPuwJCAAl/i8TGCbgADcRm0hAiUEBGCJMeXdpADMOxs7I+ALoDtAgMCVgAB0N5YEBOASn5cJbBG4+/r32YR/B+CWUViEQBoBAZhmFDU3IgBrzs2uZwncBaD4m3UfnJbAR0AAugdLAgJwic/LBLYICMAtzBYhUEpAAJYaV77NCsB8M7EjAr8UEIDuBAECvxQQgO7EkoAAXOLzMoHXBe7i77MBfwT8+hgsQCCdgABMN5JaGxKAteZlt/ME7gJQ/M27E05M4CMgAN2DJQEBuMTnZQKvCwjA14ktQKCkgAAsObY8mxaAeWZhJwR+TUAAuhcECPyagAB0L5YEBOASn5cJvCpwF3+fxf0R8Ksj8OME0goIwLSjqbExAVhjTnY5U0AAzpy7UxOICAjAiJJnLgUEoMtBIK+AAMw7GzsjcFpAAJ6eQPH1BWDxAdp+awEB2Hq8DkdgSUAALvF5WQC6AwRyCkTi77Nz/x/AnPOzKwJvCwjAt4Wb/74AbD5gxysrEAlA8Vd2vDZOYFlAAC4Tzv4BATh7/k6fV0AA5p2NnRHIICAAM0yh8B4EYOHh2XprAQHYerwOR2BZQAAuE87+AQE4e/5On1fgLgD98W/e2dkZgR0CAnCHcuM1BGDj4TpaaQEBWHp8Nk/gdQEB+Dpx7wUEYO/5Ol1dAQFYd3Z2TmCHgADcodx4DQHYeLiOtl3g6+vrsTX/8dc//tvf+v2f/vab1/r+/ftvfteLBAjkEBCAOeZQdhcCsOzobDyhwFMB+Gb8fdgEYMLLY0sE/kMBAfgfgnn8/woIQDeCwHMCTwbg3//7/38B/MPvfvrqt/L1TwA+N2+/ROCkgAA8qd9gbQHYYIiOkEbgqQD885//8atnEoBpRm0jBI4LCMDjI6i9AQFYe352n0vg7QD8nPYTgb4A5pq73RA4ISAAT6g3WlMANhqmoxwXeCIAr77+/TicADw+ZhsgkEJAAKYYQ91NCMC6s7PzfAI7AvAvf/n98sH9JZBlQj9A4LiAADw+gtobEIC152f3uQSeCMDP3wD+tb8A8uOkAjDXzO2GwCkBAXhKvsm6ArDJIB0jhcDbAfjEH/9+oHwBTHFdbILAkoAAXOLzsgB0Bwg8J/BUAH52dPWvgVn9CyAC8Ll5+yUCJwUE4En9BmsLwAZDdIQ0Ak8E4C8P8/N/KfQT8ScA01wXGyGwJCAAl/i8LADdAQLPCbwRgM/t7n9/yR8Bv6HqNwnsFRCAe707ruY/Ctpxqs5EgAABAt0FnvuPjxeUGn34J+blC+ATin6DwE8CvgC6CQQI7BLwBXCXdNN1BGDTwTrWEQEBeITdogRGCgjAkWN/7tAC8DlLv0RAALoDBAjsEhCAu6SbriMAmw7WsY4ICMAj7BYlMFJAAI4c+3OHFoDPWfolAgLQHSBAYJeAANwl3XQdAdh0sI51REAAHmG3KIGRAgJw5NifO7QAfM7SLxEQgO4AAQK7BATgLumm6wjApoN1rCMCAvAIu0UJjBQQgCPH/tyhBeBzln6JgAB0BwgQ2CUgAHdJN11HADYdrGMdERCAR9gtSmCkgAAcOfbnDi0An7P0SwQEoDtAgMAuAQG4S7rpOgKw6WAd64iAADzCblECIwUE4MixP3doAficpV8iQIAAAQK7BATgLumm6wjApoN1LAIECBBoLSAAW4/3/cMJwPeNrUCAAAECBJ4WEIBPiw77PQE4bOCOS4AAAQItBARgizGeO4QAPGdvZQIECBAg8FsFBOBvlfPevwQEoItAgAABAgTqCQjAejNLtWMBmGocNkOAAAECBEICAjDE5KErAQHobhAgQIAAgXoCArDezFLtWACmGofNECBAgACBkIAADDF5yBdAd4AAAQIECPQREIB9ZnnkJL4AHmG3KAECBAgQWBIQgEt8XhaA7gABAgQIEKgnIADrzSzVjgVgqnHYDAECBAgQCAkIwBCTh64EBKC7QYAAAQIE6gkIwHozS7VjAZhqHDZDgAABAgRCAgIwxOQhXwDdAQIECBAg0EdAAPaZ5ZGT+AJ4hN2iBAgQIEBgSUAALvF5WQC6AwQIECBAoJ6AAKw3s1Q7FoCpxmEzBAgQIEAgJCAAQ0weuhIQgO4GAQIECBCoJyAA680s1Y4FYKpx2AwBAgQIEAgJCMAQk4d8AXQHCBAgQIBAHwEB2GeWR07iC+ARdosSIECAAIElAQG4xOdlAegOECBAgACBegICsN7MUu1YAKYah80QIECAAIGQgAAMMXnoSkAAuhsECBAgQKCegACsN7NUOxaAqcZhMwQIECBAICQgAENMHvIF0B0gQIAAAQJ9BARgn1keOYkvgEfYLUqAAAECBJYEBOASn5cFoDtAgAABAgTqCQjAejNLtWMBmGocNkOAAAECBEICAjDE5KErAQHobhAgQIAAgXoCArDezFLtWACmGofNECBAgACBkIAADDF5yBdAd4AAAQIECPQREIB9ZnnkJL4AHmG3KAECBAgQWBIQgEt8XhaA7gABAgQIEKgnIADrzSzVjgVgqnHYDAECBAgQCAkIwBCTh64EBKC7QYAAAQIE6gkIwHozS7VjAZhqHDZDgAABAgRCAgIwxOQhXwDdAQIECBAg0EdAAPaZ5ZGT+AJ4hN2iBAgQIEBgSUAALvF5WQC6AwQIECBAoJ6AAKw3s1Q7FoCpxmEzBAgQIEAgJCAAQ0weuhIQgO4GAQIECBCoJyAA680s1Y4FYKpx2AwBAgQIEAgJCMAQk4d8AXQHCBAgQIBAHwEB2GeWR07iC+ARdosSIECAAIElAQG4xOdlAegOECBAgACBegICsN7MUu1YAKYah80QIECAAIGQgAAMMXnoSkAAuhsECBAgQKCegACsN7NUOxaAqcZhMwQIECBAICQgAENMHvIF0B0gQIAAAQJ9BARgn1keOYkvgEfYLUqAAAECBJYEBOASn5cFoDtAgAABAgTqCQjAejNLtWMBmGocNkOAAAECBEICAjDE5KErAQHobhAgQIAAgXoCArDezFLtWACmGofNECBAgACBkIAADDF5yBdAd4AAAQIECPQREIB9ZnnkJL4AHmG3KAECBAgQWBIQgEt8XhaA7gABAgQIEKgnIADrzSzVjgVgqnHYDAECBAgQCAkIwBCTh64EBKC7QYAAAQIE6gkIwHozS7VjAZhqHDZDgAABAgRCAgIwxOQhXwDdAQIECBAg0EdAAPaZ5ZGT+AJ4hN2iBAgQIEBgSUAALvF5WQC6AwQIECBAoJ6AAKw3s1Q7FoCpxmEzBAgQIEAgJCAAQ0weuhIQgO4GAQIECBCoJyAA680s1Y4FYKpx2AwBAgQIEAgJCMAQk4d8AXQHCBAgQIBAHwEB2GeWTkKAAAECBAgQIBAQ+Ao84xECBAgQIECAAIFGAgKw0TAdhQABAgQIECAQERCAESXPECBAgAABAgQaCQjARsN0FAIECBAgQIBAREAARpQ8Q4AAAQIECBBoJCAAGw3TUQgQIECAAAECEQEBGFHyDAECBAgQIECgkYAAbDRMRyFAgAABAgQIRAQEYETJMwQIECBAgACBRgICsNEwHYUAAQIECBAgEBEQgBElzxAgQIAAAQIEGgkIwEbDdBQCBAgQIECAQERAAEaUPEOAAAECBAgQaCQgAKZkCbEAAANySURBVBsN01EIECBAgAABAhEBARhR8gwBAgQIECBAoJGAAGw0TEchQIAAAQIECEQEBGBEyTMECBAgQIAAgUYCArDRMB2FAAECBAgQIBAREIARJc8QIECAAAECBBoJCMBGw3QUAgQIECBAgEBEQABGlDxDgAABAgQIEGgkIAAbDdNRCBAgQIAAAQIRAQEYUfIMAQIECBAgQKCRgABsNExHIUCAAAECBAhEBARgRMkzBAgQIECAAIFGAgKw0TAdhQABAgQIECAQERCAESXPECBAgAABAgQaCQjARsN0FAIECBAgQIBAREAARpQ8Q4AAAQIECBBoJCAAGw3TUQgQIECAAAECEQEBGFHyDAECBAgQIECgkYAAbDRMRyFAgAABAgQIRAQEYETJMwQIECBAgACBRgICsNEwHYUAAQIECBAgEBEQgBElzxAgQIAAAQIEGgkIwEbDdBQCBAgQIECAQERAAEaUPEOAAAECBAgQaCQgABsN01EIECBAgAABAhEBARhR8gwBAgQIECBAoJGAAGw0TEchQIAAAQIECEQEBGBEyTMECBAgQIAAgUYCArDRMB2FAAECBAgQIBAREIARJc8QIECAAAECBBoJCMBGw3QUAgQIECBAgEBEQABGlDxDgAABAgQIEGgkIAAbDdNRCBAgQIAAAQIRAQEYUfIMAQIECBAgQKCRgABsNExHIUCAAAECBAhEBARgRMkzBAgQIECAAIFGAgKw0TAdhQABAgQIECAQERCAESXPECBAgAABAgQaCQjARsN0FAIECBAgQIBAREAARpQ8Q4AAAQIECBBoJCAAGw3TUQgQIECAAAECEQEBGFHyDAECBAgQIECgkYAAbDRMRyFAgAABAgQIRAQEYETJMwQIECBAgACBRgICsNEwHYUAAQIECBAgEBEQgBElzxAgQIAAAQIEGgkIwEbDdBQCBAgQIECAQERAAEaUPEOAAAECBAgQaCQgABsN01EIECBAgAABAhEBARhR8gwBAgQIECBAoJGAAGw0TEchQIAAAQIECEQEBGBEyTMECBAgQIAAgUYCArDRMB2FAAECBAgQIBAREIARJc8QIECAAAECBBoJCMBGw3QUAgQIECBAgEBEQABGlDxDgAABAgQIEGgkIAAbDdNRCBAgQIAAAQIRAQEYUfIMAQIECBAgQKCRwP8A6V62OzC1lKQAAAAASUVORK5CYII=\" width=\"640\">"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "frames = render_policy_net(\"./my_policy_net_basic.ckpt\", action, X)\n",
    "video = plot_animation(frames)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Looks like it learned the policy correctly. Now let's see if it can learn a better policy on its own."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Policy Gradients"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To train this neural network we will need to define the target probabilities `y`. If an action is good we should increase its probability, and conversely if it is bad we should reduce it. But how do we know whether an action is good or bad? The problem is that most actions have delayed effects, so when you win or lose points in a game, it is not clear which actions contributed to this result: was it just the last action? Or the last 10? Or just one action 50 steps earlier? This is called the _credit assignment problem_.\n",
    "\n",
    "The _Policy Gradients_ algorithm tackles this problem by first playing multiple games, then making the actions in good games slightly more likely, while actions in bad games are made slightly less likely. First we play, then we go back and think about what we did."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import tensorflow as tf\n",
    "\n",
    "reset_graph()\n",
    "\n",
    "n_inputs = 4\n",
    "n_hidden = 4\n",
    "n_outputs = 1\n",
    "\n",
    "learning_rate = 0.01\n",
    "\n",
    "initializer = tf.contrib.layers.variance_scaling_initializer()\n",
    "\n",
    "X = tf.placeholder(tf.float32, shape=[None, n_inputs])\n",
    "\n",
    "hidden = tf.layers.dense(X, n_hidden, activation=tf.nn.elu, kernel_initializer=initializer)\n",
    "logits = tf.layers.dense(hidden, n_outputs)\n",
    "outputs = tf.nn.sigmoid(logits)  # probability of action 0 (left)\n",
    "p_left_and_right = tf.concat(axis=1, values=[outputs, 1 - outputs])\n",
    "action = tf.multinomial(tf.log(p_left_and_right), num_samples=1)\n",
    "\n",
    "y = 1. - tf.to_float(action)\n",
    "cross_entropy = tf.nn.sigmoid_cross_entropy_with_logits(labels=y, logits=logits)\n",
    "optimizer = tf.train.AdamOptimizer(learning_rate)\n",
    "grads_and_vars = optimizer.compute_gradients(cross_entropy)\n",
    "gradients = [grad for grad, variable in grads_and_vars]\n",
    "gradient_placeholders = []\n",
    "grads_and_vars_feed = []\n",
    "for grad, variable in grads_and_vars:\n",
    "    gradient_placeholder = tf.placeholder(tf.float32, shape=grad.get_shape())\n",
    "    gradient_placeholders.append(gradient_placeholder)\n",
    "    grads_and_vars_feed.append((gradient_placeholder, variable))\n",
    "training_op = optimizer.apply_gradients(grads_and_vars_feed)\n",
    "\n",
    "init = tf.global_variables_initializer()\n",
    "saver = tf.train.Saver()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def discount_rewards(rewards, discount_rate):\n",
    "    discounted_rewards = np.zeros(len(rewards))\n",
    "    cumulative_rewards = 0\n",
    "    for step in reversed(range(len(rewards))):\n",
    "        cumulative_rewards = rewards[step] + cumulative_rewards * discount_rate\n",
    "        discounted_rewards[step] = cumulative_rewards\n",
    "    return discounted_rewards\n",
    "\n",
    "def discount_and_normalize_rewards(all_rewards, discount_rate):\n",
    "    all_discounted_rewards = [discount_rewards(rewards, discount_rate) for rewards in all_rewards]\n",
    "    flat_rewards = np.concatenate(all_discounted_rewards)\n",
    "    reward_mean = flat_rewards.mean()\n",
    "    reward_std = flat_rewards.std()\n",
    "    return [(discounted_rewards - reward_mean)/reward_std for discounted_rewards in all_discounted_rewards]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([-22., -40., -50.])"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "discount_rewards([10, 0, -50], discount_rate=0.8)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[array([-0.28435071, -0.86597718, -1.18910299]),\n",
       " array([ 1.26665318,  1.0727777 ])]"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "discount_and_normalize_rewards([[10, 0, -50], [10, 20]], discount_rate=0.8)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[2017-09-25 11:37:41,946] Making new env: CartPole-v0\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Iteration: 249"
     ]
    }
   ],
   "source": [
    "env = gym.make(\"CartPole-v0\")\n",
    "\n",
    "n_games_per_update = 10\n",
    "n_max_steps = 1000\n",
    "n_iterations = 250\n",
    "save_iterations = 10\n",
    "discount_rate = 0.95\n",
    "\n",
    "with tf.Session() as sess:\n",
    "    init.run()\n",
    "    for iteration in range(n_iterations):\n",
    "        print(\"\\rIteration: {}\".format(iteration), end=\"\")\n",
    "        all_rewards = []\n",
    "        all_gradients = []\n",
    "        for game in range(n_games_per_update):\n",
    "            current_rewards = []\n",
    "            current_gradients = []\n",
    "            obs = env.reset()\n",
    "            for step in range(n_max_steps):\n",
    "                action_val, gradients_val = sess.run([action, gradients], feed_dict={X: obs.reshape(1, n_inputs)})\n",
    "                obs, reward, done, info = env.step(action_val[0][0])\n",
    "                current_rewards.append(reward)\n",
    "                current_gradients.append(gradients_val)\n",
    "                if done:\n",
    "                    break\n",
    "            all_rewards.append(current_rewards)\n",
    "            all_gradients.append(current_gradients)\n",
    "\n",
    "        all_rewards = discount_and_normalize_rewards(all_rewards, discount_rate=discount_rate)\n",
    "        feed_dict = {}\n",
    "        for var_index, gradient_placeholder in enumerate(gradient_placeholders):\n",
    "            mean_gradients = np.mean([reward * all_gradients[game_index][step][var_index]\n",
    "                                      for game_index, rewards in enumerate(all_rewards)\n",
    "                                          for step, reward in enumerate(rewards)], axis=0)\n",
    "            feed_dict[gradient_placeholder] = mean_gradients\n",
    "        sess.run(training_op, feed_dict=feed_dict)\n",
    "        if iteration % save_iterations == 0:\n",
    "            saver.save(sess, \"./my_policy_net_pg.ckpt\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "env.close()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[2017-09-25 11:51:05,425] Making new env: CartPole-v0\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Restoring parameters from ./my_policy_net_pg.ckpt\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[2017-09-25 11:51:05,430] Restoring parameters from ./my_policy_net_pg.ckpt\n"
     ]
    },
    {
     "data": {
      "application/javascript": [
       "/* Put everything inside the global mpl namespace */\n",
       "window.mpl = {};\n",
       "\n",
       "\n",
       "mpl.get_websocket_type = function() {\n",
       "    if (typeof(WebSocket) !== 'undefined') {\n",
       "        return WebSocket;\n",
       "    } else if (typeof(MozWebSocket) !== 'undefined') {\n",
       "        return MozWebSocket;\n",
       "    } else {\n",
       "        alert('Your browser does not have WebSocket support.' +\n",
       "              'Please try Chrome, Safari or Firefox ≥ 6. ' +\n",
       "              'Firefox 4 and 5 are also supported but you ' +\n",
       "              'have to enable WebSockets in about:config.');\n",
       "    };\n",
       "}\n",
       "\n",
       "mpl.figure = function(figure_id, websocket, ondownload, parent_element) {\n",
       "    this.id = figure_id;\n",
       "\n",
       "    this.ws = websocket;\n",
       "\n",
       "    this.supports_binary = (this.ws.binaryType != undefined);\n",
       "\n",
       "    if (!this.supports_binary) {\n",
       "        var warnings = document.getElementById(\"mpl-warnings\");\n",
       "        if (warnings) {\n",
       "            warnings.style.display = 'block';\n",
       "            warnings.textContent = (\n",
       "                \"This browser does not support binary websocket messages. \" +\n",
       "                    \"Performance may be slow.\");\n",
       "        }\n",
       "    }\n",
       "\n",
       "    this.imageObj = new Image();\n",
       "\n",
       "    this.context = undefined;\n",
       "    this.message = undefined;\n",
       "    this.canvas = undefined;\n",
       "    this.rubberband_canvas = undefined;\n",
       "    this.rubberband_context = undefined;\n",
       "    this.format_dropdown = undefined;\n",
       "\n",
       "    this.image_mode = 'full';\n",
       "\n",
       "    this.root = $('<div/>');\n",
       "    this._root_extra_style(this.root)\n",
       "    this.root.attr('style', 'display: inline-block');\n",
       "\n",
       "    $(parent_element).append(this.root);\n",
       "\n",
       "    this._init_header(this);\n",
       "    this._init_canvas(this);\n",
       "    this._init_toolbar(this);\n",
       "\n",
       "    var fig = this;\n",
       "\n",
       "    this.waiting = false;\n",
       "\n",
       "    this.ws.onopen =  function () {\n",
       "            fig.send_message(\"supports_binary\", {value: fig.supports_binary});\n",
       "            fig.send_message(\"send_image_mode\", {});\n",
       "            if (mpl.ratio != 1) {\n",
       "                fig.send_message(\"set_dpi_ratio\", {'dpi_ratio': mpl.ratio});\n",
       "            }\n",
       "            fig.send_message(\"refresh\", {});\n",
       "        }\n",
       "\n",
       "    this.imageObj.onload = function() {\n",
       "            if (fig.image_mode == 'full') {\n",
       "                // Full images could contain transparency (where diff images\n",
       "                // almost always do), so we need to clear the canvas so that\n",
       "                // there is no ghosting.\n",
       "                fig.context.clearRect(0, 0, fig.canvas.width, fig.canvas.height);\n",
       "            }\n",
       "            fig.context.drawImage(fig.imageObj, 0, 0);\n",
       "        };\n",
       "\n",
       "    this.imageObj.onunload = function() {\n",
       "        this.ws.close();\n",
       "    }\n",
       "\n",
       "    this.ws.onmessage = this._make_on_message_function(this);\n",
       "\n",
       "    this.ondownload = ondownload;\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._init_header = function() {\n",
       "    var titlebar = $(\n",
       "        '<div class=\"ui-dialog-titlebar ui-widget-header ui-corner-all ' +\n",
       "        'ui-helper-clearfix\"/>');\n",
       "    var titletext = $(\n",
       "        '<div class=\"ui-dialog-title\" style=\"width: 100%; ' +\n",
       "        'text-align: center; padding: 3px;\"/>');\n",
       "    titlebar.append(titletext)\n",
       "    this.root.append(titlebar);\n",
       "    this.header = titletext[0];\n",
       "}\n",
       "\n",
       "\n",
       "\n",
       "mpl.figure.prototype._canvas_extra_style = function(canvas_div) {\n",
       "\n",
       "}\n",
       "\n",
       "\n",
       "mpl.figure.prototype._root_extra_style = function(canvas_div) {\n",
       "\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._init_canvas = function() {\n",
       "    var fig = this;\n",
       "\n",
       "    var canvas_div = $('<div/>');\n",
       "\n",
       "    canvas_div.attr('style', 'position: relative; clear: both; outline: 0');\n",
       "\n",
       "    function canvas_keyboard_event(event) {\n",
       "        return fig.key_event(event, event['data']);\n",
       "    }\n",
       "\n",
       "    canvas_div.keydown('key_press', canvas_keyboard_event);\n",
       "    canvas_div.keyup('key_release', canvas_keyboard_event);\n",
       "    this.canvas_div = canvas_div\n",
       "    this._canvas_extra_style(canvas_div)\n",
       "    this.root.append(canvas_div);\n",
       "\n",
       "    var canvas = $('<canvas/>');\n",
       "    canvas.addClass('mpl-canvas');\n",
       "    canvas.attr('style', \"left: 0; top: 0; z-index: 0; outline: 0\")\n",
       "\n",
       "    this.canvas = canvas[0];\n",
       "    this.context = canvas[0].getContext(\"2d\");\n",
       "\n",
       "    var backingStore = this.context.backingStorePixelRatio ||\n",
       "\tthis.context.webkitBackingStorePixelRatio ||\n",
       "\tthis.context.mozBackingStorePixelRatio ||\n",
       "\tthis.context.msBackingStorePixelRatio ||\n",
       "\tthis.context.oBackingStorePixelRatio ||\n",
       "\tthis.context.backingStorePixelRatio || 1;\n",
       "\n",
       "    mpl.ratio = (window.devicePixelRatio || 1) / backingStore;\n",
       "\n",
       "    var rubberband = $('<canvas/>');\n",
       "    rubberband.attr('style', \"position: absolute; left: 0; top: 0; z-index: 1;\")\n",
       "\n",
       "    var pass_mouse_events = true;\n",
       "\n",
       "    canvas_div.resizable({\n",
       "        start: function(event, ui) {\n",
       "            pass_mouse_events = false;\n",
       "        },\n",
       "        resize: function(event, ui) {\n",
       "            fig.request_resize(ui.size.width, ui.size.height);\n",
       "        },\n",
       "        stop: function(event, ui) {\n",
       "            pass_mouse_events = true;\n",
       "            fig.request_resize(ui.size.width, ui.size.height);\n",
       "        },\n",
       "    });\n",
       "\n",
       "    function mouse_event_fn(event) {\n",
       "        if (pass_mouse_events)\n",
       "            return fig.mouse_event(event, event['data']);\n",
       "    }\n",
       "\n",
       "    rubberband.mousedown('button_press', mouse_event_fn);\n",
       "    rubberband.mouseup('button_release', mouse_event_fn);\n",
       "    // Throttle sequential mouse events to 1 every 20ms.\n",
       "    rubberband.mousemove('motion_notify', mouse_event_fn);\n",
       "\n",
       "    rubberband.mouseenter('figure_enter', mouse_event_fn);\n",
       "    rubberband.mouseleave('figure_leave', mouse_event_fn);\n",
       "\n",
       "    canvas_div.on(\"wheel\", function (event) {\n",
       "        event = event.originalEvent;\n",
       "        event['data'] = 'scroll'\n",
       "        if (event.deltaY < 0) {\n",
       "            event.step = 1;\n",
       "        } else {\n",
       "            event.step = -1;\n",
       "        }\n",
       "        mouse_event_fn(event);\n",
       "    });\n",
       "\n",
       "    canvas_div.append(canvas);\n",
       "    canvas_div.append(rubberband);\n",
       "\n",
       "    this.rubberband = rubberband;\n",
       "    this.rubberband_canvas = rubberband[0];\n",
       "    this.rubberband_context = rubberband[0].getContext(\"2d\");\n",
       "    this.rubberband_context.strokeStyle = \"#000000\";\n",
       "\n",
       "    this._resize_canvas = function(width, height) {\n",
       "        // Keep the size of the canvas, canvas container, and rubber band\n",
       "        // canvas in synch.\n",
       "        canvas_div.css('width', width)\n",
       "        canvas_div.css('height', height)\n",
       "\n",
       "        canvas.attr('width', width * mpl.ratio);\n",
       "        canvas.attr('height', height * mpl.ratio);\n",
       "        canvas.attr('style', 'width: ' + width + 'px; height: ' + height + 'px;');\n",
       "\n",
       "        rubberband.attr('width', width);\n",
       "        rubberband.attr('height', height);\n",
       "    }\n",
       "\n",
       "    // Set the figure to an initial 600x600px, this will subsequently be updated\n",
       "    // upon first draw.\n",
       "    this._resize_canvas(600, 600);\n",
       "\n",
       "    // Disable right mouse context menu.\n",
       "    $(this.rubberband_canvas).bind(\"contextmenu\",function(e){\n",
       "        return false;\n",
       "    });\n",
       "\n",
       "    function set_focus () {\n",
       "        canvas.focus();\n",
       "        canvas_div.focus();\n",
       "    }\n",
       "\n",
       "    window.setTimeout(set_focus, 100);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._init_toolbar = function() {\n",
       "    var fig = this;\n",
       "\n",
       "    var nav_element = $('<div/>')\n",
       "    nav_element.attr('style', 'width: 100%');\n",
       "    this.root.append(nav_element);\n",
       "\n",
       "    // Define a callback function for later on.\n",
       "    function toolbar_event(event) {\n",
       "        return fig.toolbar_button_onclick(event['data']);\n",
       "    }\n",
       "    function toolbar_mouse_event(event) {\n",
       "        return fig.toolbar_button_onmouseover(event['data']);\n",
       "    }\n",
       "\n",
       "    for(var toolbar_ind in mpl.toolbar_items) {\n",
       "        var name = mpl.toolbar_items[toolbar_ind][0];\n",
       "        var tooltip = mpl.toolbar_items[toolbar_ind][1];\n",
       "        var image = mpl.toolbar_items[toolbar_ind][2];\n",
       "        var method_name = mpl.toolbar_items[toolbar_ind][3];\n",
       "\n",
       "        if (!name) {\n",
       "            // put a spacer in here.\n",
       "            continue;\n",
       "        }\n",
       "        var button = $('<button/>');\n",
       "        button.addClass('ui-button ui-widget ui-state-default ui-corner-all ' +\n",
       "                        'ui-button-icon-only');\n",
       "        button.attr('role', 'button');\n",
       "        button.attr('aria-disabled', 'false');\n",
       "        button.click(method_name, toolbar_event);\n",
       "        button.mouseover(tooltip, toolbar_mouse_event);\n",
       "\n",
       "        var icon_img = $('<span/>');\n",
       "        icon_img.addClass('ui-button-icon-primary ui-icon');\n",
       "        icon_img.addClass(image);\n",
       "        icon_img.addClass('ui-corner-all');\n",
       "\n",
       "        var tooltip_span = $('<span/>');\n",
       "        tooltip_span.addClass('ui-button-text');\n",
       "        tooltip_span.html(tooltip);\n",
       "\n",
       "        button.append(icon_img);\n",
       "        button.append(tooltip_span);\n",
       "\n",
       "        nav_element.append(button);\n",
       "    }\n",
       "\n",
       "    var fmt_picker_span = $('<span/>');\n",
       "\n",
       "    var fmt_picker = $('<select/>');\n",
       "    fmt_picker.addClass('mpl-toolbar-option ui-widget ui-widget-content');\n",
       "    fmt_picker_span.append(fmt_picker);\n",
       "    nav_element.append(fmt_picker_span);\n",
       "    this.format_dropdown = fmt_picker[0];\n",
       "\n",
       "    for (var ind in mpl.extensions) {\n",
       "        var fmt = mpl.extensions[ind];\n",
       "        var option = $(\n",
       "            '<option/>', {selected: fmt === mpl.default_extension}).html(fmt);\n",
       "        fmt_picker.append(option)\n",
       "    }\n",
       "\n",
       "    // Add hover states to the ui-buttons\n",
       "    $( \".ui-button\" ).hover(\n",
       "        function() { $(this).addClass(\"ui-state-hover\");},\n",
       "        function() { $(this).removeClass(\"ui-state-hover\");}\n",
       "    );\n",
       "\n",
       "    var status_bar = $('<span class=\"mpl-message\"/>');\n",
       "    nav_element.append(status_bar);\n",
       "    this.message = status_bar[0];\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.request_resize = function(x_pixels, y_pixels) {\n",
       "    // Request matplotlib to resize the figure. Matplotlib will then trigger a resize in the client,\n",
       "    // which will in turn request a refresh of the image.\n",
       "    this.send_message('resize', {'width': x_pixels, 'height': y_pixels});\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.send_message = function(type, properties) {\n",
       "    properties['type'] = type;\n",
       "    properties['figure_id'] = this.id;\n",
       "    this.ws.send(JSON.stringify(properties));\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.send_draw_message = function() {\n",
       "    if (!this.waiting) {\n",
       "        this.waiting = true;\n",
       "        this.ws.send(JSON.stringify({type: \"draw\", figure_id: this.id}));\n",
       "    }\n",
       "}\n",
       "\n",
       "\n",
       "mpl.figure.prototype.handle_save = function(fig, msg) {\n",
       "    var format_dropdown = fig.format_dropdown;\n",
       "    var format = format_dropdown.options[format_dropdown.selectedIndex].value;\n",
       "    fig.ondownload(fig, format);\n",
       "}\n",
       "\n",
       "\n",
       "mpl.figure.prototype.handle_resize = function(fig, msg) {\n",
       "    var size = msg['size'];\n",
       "    if (size[0] != fig.canvas.width || size[1] != fig.canvas.height) {\n",
       "        fig._resize_canvas(size[0], size[1]);\n",
       "        fig.send_message(\"refresh\", {});\n",
       "    };\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_rubberband = function(fig, msg) {\n",
       "    var x0 = msg['x0'] / mpl.ratio;\n",
       "    var y0 = (fig.canvas.height - msg['y0']) / mpl.ratio;\n",
       "    var x1 = msg['x1'] / mpl.ratio;\n",
       "    var y1 = (fig.canvas.height - msg['y1']) / mpl.ratio;\n",
       "    x0 = Math.floor(x0) + 0.5;\n",
       "    y0 = Math.floor(y0) + 0.5;\n",
       "    x1 = Math.floor(x1) + 0.5;\n",
       "    y1 = Math.floor(y1) + 0.5;\n",
       "    var min_x = Math.min(x0, x1);\n",
       "    var min_y = Math.min(y0, y1);\n",
       "    var width = Math.abs(x1 - x0);\n",
       "    var height = Math.abs(y1 - y0);\n",
       "\n",
       "    fig.rubberband_context.clearRect(\n",
       "        0, 0, fig.canvas.width, fig.canvas.height);\n",
       "\n",
       "    fig.rubberband_context.strokeRect(min_x, min_y, width, height);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_figure_label = function(fig, msg) {\n",
       "    // Updates the figure title.\n",
       "    fig.header.textContent = msg['label'];\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_cursor = function(fig, msg) {\n",
       "    var cursor = msg['cursor'];\n",
       "    switch(cursor)\n",
       "    {\n",
       "    case 0:\n",
       "        cursor = 'pointer';\n",
       "        break;\n",
       "    case 1:\n",
       "        cursor = 'default';\n",
       "        break;\n",
       "    case 2:\n",
       "        cursor = 'crosshair';\n",
       "        break;\n",
       "    case 3:\n",
       "        cursor = 'move';\n",
       "        break;\n",
       "    }\n",
       "    fig.rubberband_canvas.style.cursor = cursor;\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_message = function(fig, msg) {\n",
       "    fig.message.textContent = msg['message'];\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_draw = function(fig, msg) {\n",
       "    // Request the server to send over a new figure.\n",
       "    fig.send_draw_message();\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_image_mode = function(fig, msg) {\n",
       "    fig.image_mode = msg['mode'];\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.updated_canvas_event = function() {\n",
       "    // Called whenever the canvas gets updated.\n",
       "    this.send_message(\"ack\", {});\n",
       "}\n",
       "\n",
       "// A function to construct a web socket function for onmessage handling.\n",
       "// Called in the figure constructor.\n",
       "mpl.figure.prototype._make_on_message_function = function(fig) {\n",
       "    return function socket_on_message(evt) {\n",
       "        if (evt.data instanceof Blob) {\n",
       "            /* FIXME: We get \"Resource interpreted as Image but\n",
       "             * transferred with MIME type text/plain:\" errors on\n",
       "             * Chrome.  But how to set the MIME type?  It doesn't seem\n",
       "             * to be part of the websocket stream */\n",
       "            evt.data.type = \"image/png\";\n",
       "\n",
       "            /* Free the memory for the previous frames */\n",
       "            if (fig.imageObj.src) {\n",
       "                (window.URL || window.webkitURL).revokeObjectURL(\n",
       "                    fig.imageObj.src);\n",
       "            }\n",
       "\n",
       "            fig.imageObj.src = (window.URL || window.webkitURL).createObjectURL(\n",
       "                evt.data);\n",
       "            fig.updated_canvas_event();\n",
       "            fig.waiting = false;\n",
       "            return;\n",
       "        }\n",
       "        else if (typeof evt.data === 'string' && evt.data.slice(0, 21) == \"data:image/png;base64\") {\n",
       "            fig.imageObj.src = evt.data;\n",
       "            fig.updated_canvas_event();\n",
       "            fig.waiting = false;\n",
       "            return;\n",
       "        }\n",
       "\n",
       "        var msg = JSON.parse(evt.data);\n",
       "        var msg_type = msg['type'];\n",
       "\n",
       "        // Call the  \"handle_{type}\" callback, which takes\n",
       "        // the figure and JSON message as its only arguments.\n",
       "        try {\n",
       "            var callback = fig[\"handle_\" + msg_type];\n",
       "        } catch (e) {\n",
       "            console.log(\"No handler for the '\" + msg_type + \"' message type: \", msg);\n",
       "            return;\n",
       "        }\n",
       "\n",
       "        if (callback) {\n",
       "            try {\n",
       "                // console.log(\"Handling '\" + msg_type + \"' message: \", msg);\n",
       "                callback(fig, msg);\n",
       "            } catch (e) {\n",
       "                console.log(\"Exception inside the 'handler_\" + msg_type + \"' callback:\", e, e.stack, msg);\n",
       "            }\n",
       "        }\n",
       "    };\n",
       "}\n",
       "\n",
       "// from http://stackoverflow.com/questions/1114465/getting-mouse-location-in-canvas\n",
       "mpl.findpos = function(e) {\n",
       "    //this section is from http://www.quirksmode.org/js/events_properties.html\n",
       "    var targ;\n",
       "    if (!e)\n",
       "        e = window.event;\n",
       "    if (e.target)\n",
       "        targ = e.target;\n",
       "    else if (e.srcElement)\n",
       "        targ = e.srcElement;\n",
       "    if (targ.nodeType == 3) // defeat Safari bug\n",
       "        targ = targ.parentNode;\n",
       "\n",
       "    // jQuery normalizes the pageX and pageY\n",
       "    // pageX,Y are the mouse positions relative to the document\n",
       "    // offset() returns the position of the element relative to the document\n",
       "    var x = e.pageX - $(targ).offset().left;\n",
       "    var y = e.pageY - $(targ).offset().top;\n",
       "\n",
       "    return {\"x\": x, \"y\": y};\n",
       "};\n",
       "\n",
       "/*\n",
       " * return a copy of an object with only non-object keys\n",
       " * we need this to avoid circular references\n",
       " * http://stackoverflow.com/a/24161582/3208463\n",
       " */\n",
       "function simpleKeys (original) {\n",
       "  return Object.keys(original).reduce(function (obj, key) {\n",
       "    if (typeof original[key] !== 'object')\n",
       "        obj[key] = original[key]\n",
       "    return obj;\n",
       "  }, {});\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.mouse_event = function(event, name) {\n",
       "    var canvas_pos = mpl.findpos(event)\n",
       "\n",
       "    if (name === 'button_press')\n",
       "    {\n",
       "        this.canvas.focus();\n",
       "        this.canvas_div.focus();\n",
       "    }\n",
       "\n",
       "    var x = canvas_pos.x * mpl.ratio;\n",
       "    var y = canvas_pos.y * mpl.ratio;\n",
       "\n",
       "    this.send_message(name, {x: x, y: y, button: event.button,\n",
       "                             step: event.step,\n",
       "                             guiEvent: simpleKeys(event)});\n",
       "\n",
       "    /* This prevents the web browser from automatically changing to\n",
       "     * the text insertion cursor when the button is pressed.  We want\n",
       "     * to control all of the cursor setting manually through the\n",
       "     * 'cursor' event from matplotlib */\n",
       "    event.preventDefault();\n",
       "    return false;\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._key_event_extra = function(event, name) {\n",
       "    // Handle any extra behaviour associated with a key event\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.key_event = function(event, name) {\n",
       "\n",
       "    // Prevent repeat events\n",
       "    if (name == 'key_press')\n",
       "    {\n",
       "        if (event.which === this._key)\n",
       "            return;\n",
       "        else\n",
       "            this._key = event.which;\n",
       "    }\n",
       "    if (name == 'key_release')\n",
       "        this._key = null;\n",
       "\n",
       "    var value = '';\n",
       "    if (event.ctrlKey && event.which != 17)\n",
       "        value += \"ctrl+\";\n",
       "    if (event.altKey && event.which != 18)\n",
       "        value += \"alt+\";\n",
       "    if (event.shiftKey && event.which != 16)\n",
       "        value += \"shift+\";\n",
       "\n",
       "    value += 'k';\n",
       "    value += event.which.toString();\n",
       "\n",
       "    this._key_event_extra(event, name);\n",
       "\n",
       "    this.send_message(name, {key: value,\n",
       "                             guiEvent: simpleKeys(event)});\n",
       "    return false;\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.toolbar_button_onclick = function(name) {\n",
       "    if (name == 'download') {\n",
       "        this.handle_save(this, null);\n",
       "    } else {\n",
       "        this.send_message(\"toolbar_button\", {name: name});\n",
       "    }\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.toolbar_button_onmouseover = function(tooltip) {\n",
       "    this.message.textContent = tooltip;\n",
       "};\n",
       "mpl.toolbar_items = [[\"Home\", \"Reset original view\", \"fa fa-home icon-home\", \"home\"], [\"Back\", \"Back to  previous view\", \"fa fa-arrow-left icon-arrow-left\", \"back\"], [\"Forward\", \"Forward to next view\", \"fa fa-arrow-right icon-arrow-right\", \"forward\"], [\"\", \"\", \"\", \"\"], [\"Pan\", \"Pan axes with left mouse, zoom with right\", \"fa fa-arrows icon-move\", \"pan\"], [\"Zoom\", \"Zoom to rectangle\", \"fa fa-square-o icon-check-empty\", \"zoom\"], [\"\", \"\", \"\", \"\"], [\"Download\", \"Download plot\", \"fa fa-floppy-o icon-save\", \"download\"]];\n",
       "\n",
       "mpl.extensions = [\"eps\", \"jpeg\", \"pdf\", \"png\", \"ps\", \"raw\", \"svg\", \"tif\"];\n",
       "\n",
       "mpl.default_extension = \"png\";var comm_websocket_adapter = function(comm) {\n",
       "    // Create a \"websocket\"-like object which calls the given IPython comm\n",
       "    // object with the appropriate methods. Currently this is a non binary\n",
       "    // socket, so there is still some room for performance tuning.\n",
       "    var ws = {};\n",
       "\n",
       "    ws.close = function() {\n",
       "        comm.close()\n",
       "    };\n",
       "    ws.send = function(m) {\n",
       "        //console.log('sending', m);\n",
       "        comm.send(m);\n",
       "    };\n",
       "    // Register the callback with on_msg.\n",
       "    comm.on_msg(function(msg) {\n",
       "        //console.log('receiving', msg['content']['data'], msg);\n",
       "        // Pass the mpl event to the overriden (by mpl) onmessage function.\n",
       "        ws.onmessage(msg['content']['data'])\n",
       "    });\n",
       "    return ws;\n",
       "}\n",
       "\n",
       "mpl.mpl_figure_comm = function(comm, msg) {\n",
       "    // This is the function which gets called when the mpl process\n",
       "    // starts-up an IPython Comm through the \"matplotlib\" channel.\n",
       "\n",
       "    var id = msg.content.data.id;\n",
       "    // Get hold of the div created by the display call when the Comm\n",
       "    // socket was opened in Python.\n",
       "    var element = $(\"#\" + id);\n",
       "    var ws_proxy = comm_websocket_adapter(comm)\n",
       "\n",
       "    function ondownload(figure, format) {\n",
       "        window.open(figure.imageObj.src);\n",
       "    }\n",
       "\n",
       "    var fig = new mpl.figure(id, ws_proxy,\n",
       "                           ondownload,\n",
       "                           element.get(0));\n",
       "\n",
       "    // Call onopen now - mpl needs it, as it is assuming we've passed it a real\n",
       "    // web socket which is closed, not our websocket->open comm proxy.\n",
       "    ws_proxy.onopen();\n",
       "\n",
       "    fig.parent_element = element.get(0);\n",
       "    fig.cell_info = mpl.find_output_cell(\"<div id='\" + id + \"'></div>\");\n",
       "    if (!fig.cell_info) {\n",
       "        console.error(\"Failed to find cell for figure\", id, fig);\n",
       "        return;\n",
       "    }\n",
       "\n",
       "    var output_index = fig.cell_info[2]\n",
       "    var cell = fig.cell_info[0];\n",
       "\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_close = function(fig, msg) {\n",
       "    var width = fig.canvas.width/mpl.ratio\n",
       "    fig.root.unbind('remove')\n",
       "\n",
       "    // Update the output cell to use the data from the current canvas.\n",
       "    fig.push_to_output();\n",
       "    var dataURL = fig.canvas.toDataURL();\n",
       "    // Re-enable the keyboard manager in IPython - without this line, in FF,\n",
       "    // the notebook keyboard shortcuts fail.\n",
       "    IPython.keyboard_manager.enable()\n",
       "    $(fig.parent_element).html('<img src=\"' + dataURL + '\" width=\"' + width + '\">');\n",
       "    fig.close_ws(fig, msg);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.close_ws = function(fig, msg){\n",
       "    fig.send_message('closing', msg);\n",
       "    // fig.ws.close()\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.push_to_output = function(remove_interactive) {\n",
       "    // Turn the data on the canvas into data in the output cell.\n",
       "    var width = this.canvas.width/mpl.ratio\n",
       "    var dataURL = this.canvas.toDataURL();\n",
       "    this.cell_info[1]['text/html'] = '<img src=\"' + dataURL + '\" width=\"' + width + '\">';\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.updated_canvas_event = function() {\n",
       "    // Tell IPython that the notebook contents must change.\n",
       "    IPython.notebook.set_dirty(true);\n",
       "    this.send_message(\"ack\", {});\n",
       "    var fig = this;\n",
       "    // Wait a second, then push the new image to the DOM so\n",
       "    // that it is saved nicely (might be nice to debounce this).\n",
       "    setTimeout(function () { fig.push_to_output() }, 1000);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._init_toolbar = function() {\n",
       "    var fig = this;\n",
       "\n",
       "    var nav_element = $('<div/>')\n",
       "    nav_element.attr('style', 'width: 100%');\n",
       "    this.root.append(nav_element);\n",
       "\n",
       "    // Define a callback function for later on.\n",
       "    function toolbar_event(event) {\n",
       "        return fig.toolbar_button_onclick(event['data']);\n",
       "    }\n",
       "    function toolbar_mouse_event(event) {\n",
       "        return fig.toolbar_button_onmouseover(event['data']);\n",
       "    }\n",
       "\n",
       "    for(var toolbar_ind in mpl.toolbar_items){\n",
       "        var name = mpl.toolbar_items[toolbar_ind][0];\n",
       "        var tooltip = mpl.toolbar_items[toolbar_ind][1];\n",
       "        var image = mpl.toolbar_items[toolbar_ind][2];\n",
       "        var method_name = mpl.toolbar_items[toolbar_ind][3];\n",
       "\n",
       "        if (!name) { continue; };\n",
       "\n",
       "        var button = $('<button class=\"btn btn-default\" href=\"#\" title=\"' + name + '\"><i class=\"fa ' + image + ' fa-lg\"></i></button>');\n",
       "        button.click(method_name, toolbar_event);\n",
       "        button.mouseover(tooltip, toolbar_mouse_event);\n",
       "        nav_element.append(button);\n",
       "    }\n",
       "\n",
       "    // Add the status bar.\n",
       "    var status_bar = $('<span class=\"mpl-message\" style=\"text-align:right; float: right;\"/>');\n",
       "    nav_element.append(status_bar);\n",
       "    this.message = status_bar[0];\n",
       "\n",
       "    // Add the close button to the window.\n",
       "    var buttongrp = $('<div class=\"btn-group inline pull-right\"></div>');\n",
       "    var button = $('<button class=\"btn btn-mini btn-primary\" href=\"#\" title=\"Stop Interaction\"><i class=\"fa fa-power-off icon-remove icon-large\"></i></button>');\n",
       "    button.click(function (evt) { fig.handle_close(fig, {}); } );\n",
       "    button.mouseover('Stop Interaction', toolbar_mouse_event);\n",
       "    buttongrp.append(button);\n",
       "    var titlebar = this.root.find($('.ui-dialog-titlebar'));\n",
       "    titlebar.prepend(buttongrp);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._root_extra_style = function(el){\n",
       "    var fig = this\n",
       "    el.on(\"remove\", function(){\n",
       "\tfig.close_ws(fig, {});\n",
       "    });\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._canvas_extra_style = function(el){\n",
       "    // this is important to make the div 'focusable\n",
       "    el.attr('tabindex', 0)\n",
       "    // reach out to IPython and tell the keyboard manager to turn it's self\n",
       "    // off when our div gets focus\n",
       "\n",
       "    // location in version 3\n",
       "    if (IPython.notebook.keyboard_manager) {\n",
       "        IPython.notebook.keyboard_manager.register_events(el);\n",
       "    }\n",
       "    else {\n",
       "        // location in version 2\n",
       "        IPython.keyboard_manager.register_events(el);\n",
       "    }\n",
       "\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._key_event_extra = function(event, name) {\n",
       "    var manager = IPython.notebook.keyboard_manager;\n",
       "    if (!manager)\n",
       "        manager = IPython.keyboard_manager;\n",
       "\n",
       "    // Check for shift+enter\n",
       "    if (event.shiftKey && event.which == 13) {\n",
       "        this.canvas_div.blur();\n",
       "        // select the cell after this one\n",
       "        var index = IPython.notebook.find_cell_index(this.cell_info[0]);\n",
       "        IPython.notebook.select(index + 1);\n",
       "    }\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_save = function(fig, msg) {\n",
       "    fig.ondownload(fig, null);\n",
       "}\n",
       "\n",
       "\n",
       "mpl.find_output_cell = function(html_output) {\n",
       "    // Return the cell and output element which can be found *uniquely* in the notebook.\n",
       "    // Note - this is a bit hacky, but it is done because the \"notebook_saving.Notebook\"\n",
       "    // IPython event is triggered only after the cells have been serialised, which for\n",
       "    // our purposes (turning an active figure into a static one), is too late.\n",
       "    var cells = IPython.notebook.get_cells();\n",
       "    var ncells = cells.length;\n",
       "    for (var i=0; i<ncells; i++) {\n",
       "        var cell = cells[i];\n",
       "        if (cell.cell_type === 'code'){\n",
       "            for (var j=0; j<cell.output_area.outputs.length; j++) {\n",
       "                var data = cell.output_area.outputs[j];\n",
       "                if (data.data) {\n",
       "                    // IPython >= 3 moved mimebundle to data attribute of output\n",
       "                    data = data.data;\n",
       "                }\n",
       "                if (data['text/html'] == html_output) {\n",
       "                    return [cell, data, j];\n",
       "                }\n",
       "            }\n",
       "        }\n",
       "    }\n",
       "}\n",
       "\n",
       "// Register the function which deals with the matplotlib target/channel.\n",
       "// The kernel may be null if the page has been refreshed.\n",
       "if (IPython.notebook.kernel != null) {\n",
       "    IPython.notebook.kernel.comm_manager.register_target('matplotlib', mpl.mpl_figure_comm);\n",
       "}\n"
      ],
      "text/plain": [
       "<IPython.core.display.Javascript object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<img src=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAoAAAAHgCAYAAAA10dzkAAAgAElEQVR4Xu3d3a1tSXWAUU4aTsOk4TQgDEQYkIbTwGk4jWu1LCwZetPVPevU/Bs8r7Wrasx6+LTUnPv1O/8jQIAAAQIECBBYJfC16rQOS4AAAQIECBAg8DsB6BIQIECAAAECBJYJCMBlA3dcAgQIECBAgIAAdAcIECBAgAABAssEBOCygTsuAQIECBAgQEAAugMECBAgQIAAgWUCAnDZwB2XAAECBAgQICAA3QECBAgQIECAwDIBAbhs4I5LgAABAgQIEBCA7gABAgQIECBAYJmAAFw2cMclQIAAAQIECAhAd4AAAQIECBAgsExAAC4buOMSIECAAAECBASgO0CAAAECBAgQWCYgAJcN3HEJECBAgAABAgLQHSBAgAABAgQILBMQgMsG7rgECBAgQIAAAQHoDhAgQIAAAQIElgkIwGUDd1wCBAgQIECAgAB0BwgQIECAAAECywQE4LKBOy4BAgQIECBAQAC6AwQIECBAgACBZQICcNnAHZcAAQIECBAgIADdAQIECBAgQIDAMgEBuGzgjkuAAAECBAgQEIDuAAECBAgQIEBgmYAAXDZwxyVAgAABAgQICEB3gAABAgQIECCwTEAALhu44xIgQIAAAQIEBKA7QIAAAQIECBBYJiAAlw3ccQkQIECAAAECAtAdIECAAAECBAgsExCAywbuuAQIECBAgAABAegOECBAgAABAgSWCQjAZQN3XAIECBAgQICAAHQHCBAgQIAAAQLLBATgsoE7LgECBAgQIEBAALoDBAgQIECAAIFlAgJw2cAdlwABAgQIECAgAN0BAgQIECBAgMAyAQG4bOCOS4AAAQIECBAQgO4AAQIECBAgQGCZgABcNnDHJUCAAAECBAgIQHeAAAECBAgQILBMQAAuG7jjEiBAgAABAgQEoDtAgAABAgQIEFgmIACXDdxxCRAgQIAAAQIC0B0gQIAAAQIECCwTEIDLBu64BAgQIECAAAEB6A4QIECAAAECBJYJCMBlA3dcAgQIECBAgIAAdAcIECBAgAABAssEBOCygTsuAQIECBAgQEAAugMECBAgQIAAgWUCAnDZwB2XAAECBAgQICAA3QECBAgQIECAwDIBAbhs4I5LgAABAgQIEBCA7gABAgQIECBAYJmAAFw2cMclQIAAAQIECAhAd4AAAQIECBAgsExAAC4buOMSIECAAAECBASgO0CAAAECBAgQWCYgAJcN3HEJECBAgAABAgLQHSBAgAABAgQILBMQgMsG7rgECBAgQIAAAQHoDhAgQIAAAQIElgkIwGUDd1wCBAgQIECAgAB0BwgQIECAAAECywQE4LKBOy4BAgQIECBAQAC6AwQIECBAgACBZQICcNnAHZcAAQIECBAgIADdAQIECBAgQIDAMgEBuGzgjkuAAAECBAgQEIDuAAECBAgQIEBgmYAAXDZwxyVAgAABAgQICEB3gAABAgQIECCwTEAABgf+48ePH8Gf8DoBAgQIECDwWODr62t1A60+/I27JgBvKPoNAgQIECDwVkAAvvUet5oAHDdSByJAgACBBQICcMGQv/OIAvA7df02AQIECBD4HgEB+D2ua35VAK4ZtYMSIECAwCABAThomBlHEYAZ6tYkQIAAAQIxAQEY81v/tgBcfwUAECBAgEBDAQHYcGiVtiwAK03DXggQIECAwJmAADxz8tQHAQHoahAgQIAAgX4CArDfzErtWACWGofNECBAgACBIwEBeMTkoU8CAtDdIECAAAEC/QQEYL+ZldqxACw1DpshQIAAAQJHAgLwiMlDvgC6AwQIECBAYI6AAJwzy5ST+AKYwm5RAgQIECAQEhCAIT4vC0B3gAABAgQI9BMQgP1mVmrHArDUOGyGAAECBAgcCQjAIyYPfRIQgO4GAQIECBDoJyAA+82s1I4FYKlx2AwBAgQIEDgSEIBHTB7yBdAdIECAAAECcwQE4JxZppzEF8AUdosSIECAAIGQgAAM8XlZALoDBAgQIECgn4AA7DezUjsWgKXGYTMECBAgQOBIQAAeMXnok4AAdDcIECBAgEA/AQHYb2aldiwAS43DZggQIECAwJGAADxi8pAvgO4AAQIECBCYIyAA58wy5SS+AKawW5QAAQIECIQEBGCIz8sC0B0gQIAAAQL9BARgv5mV2rEALDUOmyFAgAABAkcCAvCIyUOfBASgu0GAAAECBPoJCMB+Myu1YwFYahw2Q4AAAQIEjgQE4BGTh3wBdAcIECBAgMAcAQE4Z5YpJ/EFMIXdogQIECBAICQgAEN8XhaA7gABAgQIEOgnIAD7zazUjgVgqXHYDAECBAgQOBIQgEdMHvokIADdDQIECBAg0E9AAPabWakdC8BS47AZAgQIECBwJCAAj5g85AugO0CAAAECBOYICMA5s0w5iS+AKewWJUCAAAECIQEBGOLzsgB0BwgQIECAQD8BAdhvZqV2LABLjcNmCBAgQIDAkYAAPGLy0CcBAehuECBAgACBfgICsN/MSu1YAJYah80QIECAAIEjAQF4xOQhXwDdAQIECBAgMEdAAM6ZZcpJfAFMYbcoAQIECBAICQjAEJ+XBaA7QIAAAQIE+gkIwH4zK7VjAVhqHDZDgAABAgSOBATgEZOHPgkIQHeDAAECBAj0ExCA/WZWascCsNQ4bIYAAQIECBwJCMAjJg/5AugOECBAgACBOQICcM4sU07iC2AKu0UJECBAgEBIQACG+LwsAN0BAgQIECDQT0AA9ptZqR0LwFLjsBkCBAgQIHAkIACPmDz0SUAAuhsECBAgQKCfgADsN7NSOxaApcZhMwQIECBA4EhAAB4xecgXQHeAAAECBAjMERCAc2aZchJfAFPYLUqAAAECBEICAjDE52UB6A4QIECAAIF+AgKw38xK7VgAlhqHzRAgQIAAgSMBAXjE5KFPAgLQ3SBAgAABAv0EBGC/mZXasQAsNQ6bIUCAAAECRwIC8IjJQ74AugMECBAgQGCOgACcM8uUk/gCmMJuUQIECBAgEBIQgCE+LwtAd4AAAQIECPQTEID9ZlZqxwKw1DhshgABAgQIHAkIwCMmD30SEIDuBgECBAgQ6CcgAPvNrNSOBWCpcdgMAQIECBA4EhCAR0we8gXQHSBAgAABAnMEBOCcWaacxBfAFHaLEiBAgACBkIAADPF5WQC6AwQIECBAoJ+AAOw3s1I7FoClxmEzBAgQIEDgSEAAHjF56JOAAHQ3CBAgQIBAPwEB2G9mpXYsAEuNw2YIECBAgMCRgAA8YvKQL4DuAAECBAgQmCMgAOfMMuUkvgCmsFuUAAECBAiEBARgiM/LAtAdIECAAAEC/QQEYL+ZldqxACw1DpshQIAAAQJHAgLwiMlDnwQEoLtBgAABAgT6CQjAfjMrtWMBWGocNkOAAAECBI4EBOARk4d8AXQHCBAgQIDAHAEBOGeWKSfxBTCF3aIECBAgQCAkIABDfF4WgO4AAQIECBDoJyAA+82s1I4FYKlx2AwBAgQIEDgSEIBHTB76JCAA3Q0CBAgQINBPQAD2m1mpHQvAUuOwGQIECBAgcCQgAI+YPOQLoDtAgAABAgTmCAjAObNMOYkvgCnsFiVAgAABAiEBARji87IAdAcIECBAgEA/AQHYb2aldiwAS43DZggQIECAwJGAADxi8tAnAQHobhAgQIAAgX4CArDfzErtWACWGofNECBAgACBIwEBeMTkIV8A3QECBAgQIDBHQADOmWXKSXwBTGG3KAECBAgQCAkIwBCflwWgO0CAAAECBPoJCMB+Myu1YwFYahw2Q4AAAQIEjgQE4BGThz4JCEB3gwABAgQI9BMQgP1mVmrHArDUOGyGAAECBAgcCQjAIyYP+QLoDhAgQIAAgTkCAnDOLFNO4gtgCrtFCRAgQIBASEAAhvi8LADdAQIECBAg0E9AAPabWakdC8BS47AZAgQIECBwJCAAj5g89ElAALobBAgQIECgn4AA7DezUjsWgKXGYTMECBAgQOBIQAAeMXnIF0B3gAABAgQIzBEQgHNmmXISXwBT2C1KgAABAgRCAgIwxOdlAegOECBAgACBfgICsN/MSu1YAJYah80QIECAAIEjAQF4xOShTwIC0N0gQIAAAQL9BARgv5mV2rEALDUOmyFAgAABAkcCAvCIyUO+ALoDBAgQIEBgjoAAnDPLlJP4ApjCblECBAgQIBASEIAhPi8LQHeAAAECBAj0ExCA/WZWascCsNQ4bIYAAQIECBwJCMAjJg99EhCA7gYBAgQIEOgnIAD7zazUjgVgqXHYDAECBAgQOBIQgEdMHvIF0B0gQIAAAQJzBATgnFmmnMQXwBR2ixIgQIAAgZCAAAzxeVkAugMECBAgQKCfgADsN7NSOxaApcZhMwQIECBA4EhAAB4xeeiTgAB0NwgQIECAQD8BAdhvZqV2LABLjcNmCBAgQIDAkYAAPGLykC+A7gABAgQIEJgjIADnzDLlJL4AprBblAABAgQIhAQEYIjPywLQHSBAgAABAv0EBGC/mZXasQAsNQ6bIUCAAAECRwIC8IjJQ58EBKC7QYAAAQIE+gkIwH4zK7VjAVhqHDZDgAABAgSOBATgEZOHfAF0BwgQIECAwBwBAThnlikn8QUwhd2iBAgQIEAgJCAAQ3xeFoDuAAECBAgQ6CcgAPvNrNSOBWCpcdgMAQIECBA4EhCAR0we+iQgAN0NAgQIECDQT0AA9ptZqR0LwFLjsBkCBAgQIHAkIACPmDzkC6A7QIAAAQIE5ggIwDmzTDmJL4Ap7BYlQIAAAQIhAQEY4vOyAHQHCBAgQIBAPwEB2G9mpXYsAEuNw2YIECBAgMCRgAA8YvLQJwEB6G4QIECAAIF+AgKw38xK7VgAlhqHzRAgQIAAgSMBAXjE5CFfAN0BAgQIECAwR0AAzpllykl8AUxhtygBAgQIEAgJCMAQn5cFoDtAgAABAgT6CQjAfjMrtWMBWGocNkOAAAECBI4EBOARk4c+CQhAd+OlwH/99Y//b7l//8NfXi5vLQIECIwREIBjRplzEAGY47511X8MwJ9zEIVbb4dzEyDwawQE4K/R8uw/CQhAl+KlgAB8qW0tAgQmCwjAydN9cDYB+ADZEv8nIABdBgIECNwREIB3HNf+igBcO/qUgwvAFHaLEiAwUEAADhzqyyMJwJfa1hKA7gABAgTuCAjAO45rf0UArh19ysEFYAq7RQkQGCggAAcO9eWRBOBLbWsJQHeAAAECdwQE4B3Htb8iANeOPuXgAjCF3aIECAwUEIADh/rySALwpba1BKA7QIAAgTsCAvCO49pfEYBrR59ycAGYwm5RAgQGCgjAgUN9eSQB+FLbWgLQHSBAgMAdAQF4x3HtrwjAtaNPObgATGG3KAECAwUE4MChvjySAHypbS0B6A4QIEDgjoAAvOO49lcE4NrRpxxcAKawW5QAgYECAnDgUF8eSQC+1LaWAHQHCBAgcEdAAN5xXPsrAnDt6FMOLgBT2C1KgMBAAQE4cKgvjyQAX2pbSwC6AwQIELgjIADvOK79FQG4dvQpBxeAKewWJUBgoIAAHDjUl0cSgC+1rSUA3QECBAjcERCAdxzX/ooAXDv6lIMLwBR2ixIgMFBAAA4c6ssjCcCX2tYSgO4AAQIE7ggIwDuOa39FAK4dfcrBBWAKu0UJEBgoIAAHDvXlkQTgS21rCUB3gAABAncEBOAdx7W/IgDXjj7l4AIwhd2iBAgMFBCAA4f68kgC8KW2tQSgO0CAAIE7AgLwjuPaXxGAa0efcnABmMJuUQIEBgoIwIFDfXkkAfhS21oC0B0gQIDAHQEBeMdx7a8IwLWjTzm4AExhtygBAgMFBODAob48kgB8qW0tAegOECBA4I6AALzjuPZXBODa0accXACmsFuUAIGBAgJw4FBfHkkAvtS2lgB0BwgQIHBHQADecVz7KwJw7ehTDi4AU9gtSoDAQAEBOHCoL48kAF9qW0sAugMECBC4IyAA7ziu/RUBuHb0KQcXgCnsFiVAYKCAABw41JdHEoAvta0lAN0BAgQI3BEQgHcc1/6KAFw7+pSDC8AUdosSIDBQQAAOHOrLIwnAl9rWEoDuAAECBO4ICMA7jmt/RQCuHX3KwQVgCrtFCRAYKCAABw715ZEE4EttawlAd4AAAQJ3BATgHce1vyIA144+5eACMIXdogQIDBQQgAOH+vJIAvCltrUEoDtAgACBOwIC8I7j2l8RgGtHn3JwAZjCblECBAYKCMCBQ315JAH4UttaAtAdIECAwB0BAXjHce2vCMC1o085uABMYbcoAQIDBQTgwKG+PJIAfKltLQHoDhAgQOCOgAC847j2VwTg2tGnHFwAprBblACBgQICcOBQXx5JAL7UtpYAdAcIECBwR0AA3nFc+ysCcO3oUw4uAFPYLUqAwEABAThwqC+PJABfaltLALoDBAgQuCMgAO84rv0VAbh29CkHF4Ap7BYlQGCggAAcONSXRxKAL7WtJQDdAQIECNwREIB3HNf+igBcO/qUgwvAFHaLEiAwUEAADhzqyyMJwJfa1hKA7gABAgTuCAjAO45rf0UArh19ysEFYAq7RQkQGCggAAcO9eWRBOBLbWsJQHeAAAECdwQE4B3Htb8iANeOPuXgAjCF3aIECAwUEIADh/rySALwpfbutcTf7vk7PQECdwUE4F3Pdb8mANeNPO3AAjCN3sIECAwUEIADh/rySALwpfbutQTg7vk7PQECdwUE4F3Pdb8mANeNPO3AAjCN3sIECAwUEIADh/rySALwpfbutQTg7vk7PQECdwUE4F3Pdb8mANeNPO3AAjCN3sIECAwUEIADh/rySALwpfbutQTg7vk7PQECdwUE4F3Pdb8mANeNPO3AAjCN3sIECAwUEIADh/rySALwpfbutQTg7vk7PQECdwUE4F3Pdb8mANeNPO3AAjCN3sIECAwUEIADh/rySALwpfbutQTg7vk7PQECdwUE4F3Pdb8mANeNPO3AAjCN3sIECAwUEIADh/rySALwpfbutQTg7vk7PQECdwUE4F3Pdb8mANeNPO3AAjCN3sIECAwUEIADh/rySALwpfbutQTg7vk7PQECdwUE4F3Pdb8mANeNPO3AAjCN3sIECAwUEIADh/rySALwpfbutQTg7vk7PQECdwUE4F3Pdb8mANeNPO3AAjCN3sIECAwUEIADh/rySALwpfbutQTg7vk7PQECdwUE4F3Pdb8mANeNPO3AAjCN3sIECAwUEIADh/rySALwpfbutQTg7vk7PQECdwUE4F3Pdb8mANeNPO3AAjCN3sIECAwUEIADh/rySALwpfbutQTg7vk7PQECdwUE4F3Pdb8mANeNPO3AAjCN3sIECAwUEIADh/rySALwpfbutQTg7vk7PQECdwUE4F3Pdb8mANeNPO3AAjCN3sIECAwUEIADh/rySALwpfbutQTg7vk7PQECdwUE4F3Pdb8mANeNPO3AAjCN3sIECAwUEIADh/rySALwpfbutQTg7vk7PQECdwUE4F3Pdb8mANeNPO3AAjCN3sIECAwUEIADh/rySALwpfbutQTg7vk7PQECdwUE4F3Pdb8mANeNPO3AAjCN3sIECAwUEIADh/rySALwpfbutQTg7vk7PQECdwUE4F3Pdb8mANeNPO3AAjCN3sIECAwUEIADh/rySALwpfbutQTg7vk7PQECdwUE4F3Pdb8mANeNPO3AAjCN3sIECAwUEIADh/rySALwpfbutQTg7vk7PQECdwUE4F3Pdb8mANeN/Dcf+Ovr6ze/+9OLf/vLH37x/d//8a+/+My/euDHjx+h971MgACBLgICsMukiu5TABYdTMFtCcCCQ7ElAgTWCgjAtaO/c3ABeMdxw6/cCsD//O9//hL4H//2v1/+fAHccJOckQCBGwIC8Ibi4t8QgIuH/yuPfiMAfy7+/r6NnyJQAP7KoXicAIG1AgJw7ejvHFwA3nHc8CvRAPzTn/72i0x//vPvf/GZf/WA/wYwxOdlAgQaCQjARsOquFUBWHEqNfckAGvOxa4IENgpIAB3zv3aqQXgNcrxPyQAx4/YAQkQaCQgABsNq+JWBWDFqdTckwCsORe7IkBgp4AA3Dn3a6cWgNcox/9QNAB/+juA/k8g46+JAxIg8EhAAD6CnrqMAJw62fvnuhGAP+3Kn4G5Pxu/SIDAPgEBuG/mV08sAK9yjv6xaAD+HM4//usg/gzM6CvkcAQIXBQQgBcxN/6UANw49d925u8IwN+2k89v+TMwt0X9HgECVQUEYNXJ9NmXfzy1z6zslAABAgQI/F0g9g+0N3dcffgbs/MF8Ibijt/wBXDHnJ2SAIEeAr4A9phT2V0KwLKjKbcxAVhuJDZEgMBiAQG4ePg3ji4Abyju+A0BuGPOTkmAQA8BAdhjTmV3KQDLjqbcxgRguZHYEAECiwUE4OLh3zi6ALyhuOM3BOCOOTslAQI9BARgjzmV3aUALDuachsTgOVGYkMECCwWEICLh3/j6ALwhuKO3xCAO+bslAQI9BAQgD3mVHaXArDsaMptTACWG4kNESCwWEAALh7+jaMLwBuKO35DAO6Ys1MSINBDQAD2mFPZXQrAsqMptzEBWG4kNkSAwGIBAbh4+DeOLgBvKO74DQG4Y85OSYBADwEB2GNOZXcpAMuOptzGBGC5kdgQAQKLBQTg4uHfOLoAvKHoNwgQIECAwFsBAfjWe9xqAnDcSB2IAAECBBYICMAFQ/7OIwrA79T12wQIECBA4HsEBOD3uK75VQG4ZtQOSoAAAQKDBATgoGFmHEUAZqhbkwABAgQIxAQEYMxv/dsCcP0VAECAAAECDQUEYMOhVdqyAKw0DXshQIAAAQJnAgLwzMlTHwQEoKtBgAABAgT6CQjAfjMrtWMBWGocNkOAAAECBI4EBOARk4c+CQhAd4MAAQIECPQTEID9ZlZqxwKw1DhshgABAgQIHAkIwCMmD/kC6A4QIECAAIE5AgJwzixTTuILYAq7RQkQIECAQEhAAIb4vCwA3QECBAgQINBPQAD2m1mpHQvAUuOwGQIECBAgcCQgAI+YPPRJQAC6GwQIECBAoJ+AAOw3s1I7FoClxmEzBAgQIEDgSEAAHjF5yBdAd4AAAQIECMwREIBzZplyEl8AU9gtSoAAAQIEQgICMMTnZQHoDhAgQIAAgX4CArDfzErtWACWGofNECBAgACBIwEBeMTkoU8CAtDdIECAAAEC/QQEYL+ZldqxACw1DpshQIAAAQJHAgLwiMlDvgC6AwQIECBAYI6AAJwzy5ST+AKYwm5RAgQIECAQEhCAIT4vC0B3gAABAgQI9BMQgP1mVmrHArDUOGyGAAECBAgcCQjAIyYPfRIQgO4GAQIECBDoJyAA+82s1I4FYKlx2AwBAgQIEDgSEIBHTB7yBdAdIECAAAECcwQE4JxZppzEF8AUdosSIECAAIGQgAAM8XlZALoDBAgQIECgn4AA7DezUjsWgKXGYTMECBAgQOBIQAAeMXnok4AAdDcIECBAgEA/AQHYb2aldiwAS43DZggQIECAwJGAADxi8pAvgO4AAQIECBCYIyAA58wy5SS+AKawW5QAAQIECIQEBGCIz8sC0B0gQIAAAQL9BARgv5mV2rEALDUOmyFAgAABAkcCAvCIyUOfBASgu0GAAAECBPoJCMB+Myu1YwFYahw2Q4AAAQIEjgQE4BGTh3wBdAcIECBAgMAcAQE4Z5YpJ/EFMIXdogQIECBAICQgAEN8XhaA7gABAgQIEOgnIAD7zazUjgVgqXHYDAECBAgQOBIQgEdMHvokIADdDQIECBAg0E9AAPabWakdC8BS47AZAgQIECBwJCAAj5g85AugO0CAAAECBOYICMA5s0w5iS+AKewWJUCAAAECIQEBGOLzsgB0BwgQIECAQD8BAdhvZqV2LABLjcNmCBAgQIDAkYAAPGLy0CcBAehuECBAgACBfgICsN/MSu1YAJYah80QIECAAIEjAQF4xOQhXwDdAQIECBAgMEdAAM6ZZcpJfAFMYbcoAQIECBAICQjAEJ+XBaA7QIAAAQIE+gkIwH4zK7VjAVhqHDZDgAABAgSOBATgEZOHPgkIQHeDAAECBAj0ExCA/WZWascCsNQ4bIYAAQIECBwJCMAjJg/5AugOECBAgACBOQICcM4sU07iC2AKu0UJECBAgEBIQACG+LwsAN0BAgQIECDQT0AA9ptZqR0LwFLjsBkCBAgQIHAkIACPmDz0SUAAuhsECBAgQKCfgADsN7NSOxaApcZhMwQIECBA4EhAAB4xecgXQHeAAAECBAjMERCAc2aZchJfAFPYLUqAAAECBEICAjDE52UB6A4QIECAAIF+AgKw38xK7VgAlhqHzRAgQIAAgSMBAXjE5KFPAgLQ3SBAgAABAv0EBGC/mdkxAQIECBAgQIBAQOAr8K5XCRAgQIAAAQIEGgoIwIZDs2UCBAgQIECAQERAAEb0vEuAAAECBAgQaCggABsOzZYJECBAgAABAhEBARjR8y4BAgQIECBAoKGAAGw4NFsmQIAAAQIECEQEBGBEz7sECBAgQIAAgYYCArDh0GyZAAECBAgQIBAREIARPe8SIECAAAECBBoKCMCGQ7NlAgQIECBAgEBEQABG9LxLgAABAgQIEGgoIAAbDs2WCRAgQIAAAQIRAQEY0fMuAQIECBAgQKChgABsODRbJkCAAAECBAhEBARgRM+7BAgQIECAAIGGAgKw4dBsmQABAgQIECAQERCAET3vEiBAgAABAgQaCgjAhkOzZQIECBAgQIBAREAARvS8S4AAAQIECBBoKCAAGw7NlgkQIECAAAECEQEBGNHzLgECBAgQIECgoYAAbDg0WyZAgAABAgQIRAQEYETPuwQIECBAgACBhgICsOHQbJkAAQIECBAgEBEQgBE97xIgQIAAAQIEGgoIwIZDs2UCBAgQIECAQClBqQgAAAKwSURBVERAAEb0vEuAAAECBAgQaCggABsOzZYJECBAgAABAhEBARjR8y4BAgQIECBAoKGAAGw4NFsmQIAAAQIECEQEBGBEz7sECBAgQIAAgYYCArDh0GyZAAECBAgQIBAREIARPe8SIECAAAECBBoKCMCGQ7NlAgQIECBAgEBEQABG9LxLgAABAgQIEGgoIAAbDs2WCRAgQIAAAQIRAQEY0fMuAQIECBAgQKChgABsODRbJkCAAAECBAhEBARgRM+7BAgQIECAAIGGAgKw4dBsmQABAgQIECAQERCAET3vEiBAgAABAgQaCgjAhkOzZQIECBAgQIBAREAARvS8S4AAAQIECBBoKCAAGw7NlgkQIECAAAECEQEBGNHzLgECBAgQIECgoYAAbDg0WyZAgAABAgQIRAQEYETPuwQIECBAgACBhgICsOHQbJkAAQIECBAgEBEQgBE97xIgQIAAAQIEGgoIwIZDs2UCBAgQIECAQERAAEb0vEuAAAECBAgQaCggABsOzZYJECBAgAABAhEBARjR8y4BAgQIECBAoKGAAGw4NFsmQIAAAQIECEQEBGBEz7sECBAgQIAAgYYCArDh0GyZAAECBAgQIBAREIARPe8SIECAAAECBBoKCMCGQ7NlAgQIECBAgEBEQABG9LxLgAABAgQIEGgoIAAbDs2WCRAgQIAAAQIRAQEY0fMuAQIECBAgQKChgABsODRbJkCAAAECBAhEBARgRM+7BAgQIECAAIGGAgKw4dBsmQABAgQIECAQERCAET3vEiBAgAABAgQaCgjAhkOzZQIECBAgQIBAREAARvS8S4AAAQIECBBoKCAAGw7NlgkQIECAAAECEQEBGNHzLgECBAgQIECgoYAAbDg0WyZAgAABAgQIRAQEYETPuwQIECBAgACBhgL/A6w1ujvLdzIpAAAAAElFTkSuQmCC\" width=\"640\">"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "frames = render_policy_net(\"./my_policy_net_pg.ckpt\", action, X, n_max_steps=1000)\n",
    "video = plot_animation(frames)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Markov Chains"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "States: 0 0 3 \n",
      "States: 0 1 2 1 2 1 2 1 2 1 3 \n",
      "States: 0 1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 3 \n",
      "States: 0 3 \n",
      "States: 0 1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 3 \n",
      "States: 0 1 3 \n",
      "States: 0 1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 ...\n",
      "States: 0 0 3 \n",
      "States: 0 0 0 1 2 1 2 1 3 \n",
      "States: 0 1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 3 \n"
     ]
    }
   ],
   "source": [
    "transition_probabilities = [\n",
    "        [0.7, 0.2, 0.0, 0.1],  # from s0 to s0, s1, s2, s3\n",
    "        [0.0, 0.0, 0.9, 0.1],  # from s1 to ...\n",
    "        [0.0, 1.0, 0.0, 0.0],  # from s2 to ...\n",
    "        [0.0, 0.0, 0.0, 1.0],  # from s3 to ...\n",
    "    ]\n",
    "\n",
    "n_max_steps = 50\n",
    "\n",
    "def print_sequence(start_state=0):\n",
    "    current_state = start_state\n",
    "    print(\"States:\", end=\" \")\n",
    "    for step in range(n_max_steps):\n",
    "        print(current_state, end=\" \")\n",
    "        if current_state == 3:\n",
    "            break\n",
    "        current_state = np.random.choice(range(4), p=transition_probabilities[current_state])\n",
    "    else:\n",
    "        print(\"...\", end=\"\")\n",
    "    print()\n",
    "\n",
    "for _ in range(10):\n",
    "    print_sequence()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Markov Decision Process"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "policy_fire\n",
      "States (+rewards): 0 (10) 0 (10) 0 1 (-50) 2 2 2 (40) 0 (10) 0 (10) 0 (10) ... Total rewards = 210\n",
      "States (+rewards): 0 1 (-50) 2 (40) 0 (10) 0 (10) 0 1 (-50) 2 2 (40) 0 (10) ... Total rewards = 70\n",
      "States (+rewards): 0 (10) 0 1 (-50) 2 (40) 0 (10) 0 (10) 0 (10) 0 (10) 0 (10) 0 (10) ... Total rewards = 70\n",
      "States (+rewards): 0 1 (-50) 2 1 (-50) 2 (40) 0 (10) 0 1 (-50) 2 (40) 0 ... Total rewards = -10\n",
      "States (+rewards): 0 1 (-50) 2 (40) 0 (10) 0 (10) 0 1 (-50) 2 (40) 0 (10) 0 (10) ... Total rewards = 290\n",
      "Summary: mean=121.1, std=129.333766, min=-330, max=470\n",
      "\n",
      "policy_random\n",
      "States (+rewards): 0 1 (-50) 2 1 (-50) 2 (40) 0 1 (-50) 2 2 (40) 0 ... Total rewards = -60\n",
      "States (+rewards): 0 (10) 0 0 0 0 0 (10) 0 0 0 (10) 0 ... Total rewards = -30\n",
      "States (+rewards): 0 1 1 (-50) 2 (40) 0 0 1 1 1 1 ... Total rewards = 10\n",
      "States (+rewards): 0 (10) 0 (10) 0 0 0 0 1 (-50) 2 (40) 0 0 ... Total rewards = 0\n",
      "States (+rewards): 0 0 (10) 0 1 (-50) 2 (40) 0 0 0 0 (10) 0 (10) ... Total rewards = 40\n",
      "Summary: mean=-22.1, std=88.152740, min=-380, max=200\n",
      "\n",
      "policy_safe\n",
      "States (+rewards): 0 1 1 1 1 1 1 1 1 1 ... Total rewards = 0\n",
      "States (+rewards): 0 1 1 1 1 1 1 1 1 1 ... Total rewards = 0\n",
      "States (+rewards): 0 (10) 0 (10) 0 (10) 0 1 1 1 1 1 1 ... Total rewards = 30\n",
      "States (+rewards): 0 (10) 0 1 1 1 1 1 1 1 1 ... Total rewards = 10\n",
      "States (+rewards): 0 1 1 1 1 1 1 1 1 1 ... Total rewards = 0\n",
      "Summary: mean=22.3, std=26.244312, min=0, max=170\n",
      "\n"
     ]
    }
   ],
   "source": [
    "transition_probabilities = [\n",
    "        [[0.7, 0.3, 0.0], [1.0, 0.0, 0.0], [0.8, 0.2, 0.0]], # in s0, if action a0 then proba 0.7 to state s0 and 0.3 to state s1, etc.\n",
    "        [[0.0, 1.0, 0.0], None, [0.0, 0.0, 1.0]],\n",
    "        [None, [0.8, 0.1, 0.1], None],\n",
    "    ]\n",
    "\n",
    "rewards = [\n",
    "        [[+10, 0, 0], [0, 0, 0], [0, 0, 0]],\n",
    "        [[0, 0, 0], [0, 0, 0], [0, 0, -50]],\n",
    "        [[0, 0, 0], [+40, 0, 0], [0, 0, 0]],\n",
    "    ]\n",
    "\n",
    "possible_actions = [[0, 1, 2], [0, 2], [1]]\n",
    "\n",
    "def policy_fire(state):\n",
    "    return [0, 2, 1][state]\n",
    "\n",
    "def policy_random(state):\n",
    "    return np.random.choice(possible_actions[state])\n",
    "\n",
    "def policy_safe(state):\n",
    "    return [0, 0, 1][state]\n",
    "\n",
    "class MDPEnvironment(object):\n",
    "    def __init__(self, start_state=0):\n",
    "        self.start_state=start_state\n",
    "        self.reset()\n",
    "    def reset(self):\n",
    "        self.total_rewards = 0\n",
    "        self.state = self.start_state\n",
    "    def step(self, action):\n",
    "        next_state = np.random.choice(range(3), p=transition_probabilities[self.state][action])\n",
    "        reward = rewards[self.state][action][next_state]\n",
    "        self.state = next_state\n",
    "        self.total_rewards += reward\n",
    "        return self.state, reward\n",
    "\n",
    "def run_episode(policy, n_steps, start_state=0, display=True):\n",
    "    env = MDPEnvironment()\n",
    "    if display:\n",
    "        print(\"States (+rewards):\", end=\" \")\n",
    "    for step in range(n_steps):\n",
    "        if display:\n",
    "            if step == 10:\n",
    "                print(\"...\", end=\" \")\n",
    "            elif step < 10:\n",
    "                print(env.state, end=\" \")\n",
    "        action = policy(env.state)\n",
    "        state, reward = env.step(action)\n",
    "        if display and step < 10:\n",
    "            if reward:\n",
    "                print(\"({})\".format(reward), end=\" \")\n",
    "    if display:\n",
    "        print(\"Total rewards =\", env.total_rewards)\n",
    "    return env.total_rewards\n",
    "\n",
    "for policy in (policy_fire, policy_random, policy_safe):\n",
    "    all_totals = []\n",
    "    print(policy.__name__)\n",
    "    for episode in range(1000):\n",
    "        all_totals.append(run_episode(policy, n_steps=100, display=(episode<5)))\n",
    "    print(\"Summary: mean={:.1f}, std={:1f}, min={}, max={}\".format(np.mean(all_totals), np.std(all_totals), np.min(all_totals), np.max(all_totals)))\n",
    "    print()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Q-Learning"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Q-Learning works by watching an agent play (e.g., randomly) and gradually improving its estimates of the Q-Values. Once it has accurate Q-Value estimates (or close enough), then the optimal policy consists in choosing the action that has the highest Q-Value (i.e., the greedy policy)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "n_states = 3\n",
    "n_actions = 3\n",
    "n_steps = 20000\n",
    "alpha = 0.01\n",
    "gamma = 0.99\n",
    "exploration_policy = policy_random\n",
    "q_values = np.full((n_states, n_actions), -np.inf)\n",
    "for state, actions in enumerate(possible_actions):\n",
    "    q_values[state][actions]=0\n",
    "\n",
    "env = MDPEnvironment()\n",
    "for step in range(n_steps):\n",
    "    action = exploration_policy(env.state)\n",
    "    state = env.state\n",
    "    next_state, reward = env.step(action)\n",
    "    next_value = np.max(q_values[next_state]) # greedy policy\n",
    "    q_values[state, action] = (1-alpha)*q_values[state, action] + alpha*(reward + gamma * next_value)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def optimal_policy(state):\n",
    "    return np.argmax(q_values[state])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 39.13508139,  38.88079412,  35.23025716],\n",
       "       [ 18.9117071 ,         -inf,  20.54567816],\n",
       "       [        -inf,  72.53192111,         -inf]])"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "q_values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "States (+rewards): 0 (10) 0 (10) 0 1 (-50) 2 (40) 0 (10) 0 1 (-50) 2 (40) 0 (10) ... Total rewards = 230\n",
      "States (+rewards): 0 (10) 0 (10) 0 (10) 0 1 (-50) 2 2 1 (-50) 2 (40) 0 (10) ... Total rewards = 90\n",
      "States (+rewards): 0 1 (-50) 2 (40) 0 (10) 0 (10) 0 (10) 0 (10) 0 (10) 0 (10) 0 (10) ... Total rewards = 170\n",
      "States (+rewards): 0 1 (-50) 2 (40) 0 (10) 0 (10) 0 (10) 0 (10) 0 (10) 0 (10) 0 (10) ... Total rewards = 220\n",
      "States (+rewards): 0 1 (-50) 2 (40) 0 (10) 0 1 (-50) 2 (40) 0 (10) 0 (10) 0 (10) ... Total rewards = -50\n",
      "Summary: mean=125.6, std=127.363464, min=-290, max=500\n",
      "\n"
     ]
    }
   ],
   "source": [
    "all_totals = []\n",
    "for episode in range(1000):\n",
    "    all_totals.append(run_episode(optimal_policy, n_steps=100, display=(episode<5)))\n",
    "print(\"Summary: mean={:.1f}, std={:1f}, min={}, max={}\".format(np.mean(all_totals), np.std(all_totals), np.min(all_totals), np.max(all_totals)))\n",
    "print()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Learning to Play MsPacman Using the DQN Algorithm"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Warning**: Unfortunately, the first version of the book contained two important errors in this section.\n",
    "\n",
    "1. The actor DQN and critic DQN should have been named _online DQN_ and _target DQN_ respectively. Actor-critic algorithms are a distinct class of algorithms.\n",
    "2. The online DQN is the one that learns and is copied to the target DQN at regular intervals. The target DQN's only role is to estimate the next state's Q-Values for each possible action. This is needed to compute the target Q-Values for training the online DQN, as shown in this equation:\n",
    "\n",
    "$y(s,a) = \\text{r} + \\gamma . \\underset{a'}{\\max} \\, Q_\\text{target}(s', a')$\n",
    "\n",
    "* $y(s,a)$ is the target Q-Value to train the online DQN for the state-action pair $(s, a)$.\n",
    "* $r$ is the reward actually collected after playing action $a$ in state $s$.\n",
    "* $\\gamma$ is the discount rate.\n",
    "* $s'$ is the state actually reached after played action $a$ in state $s$.\n",
    "* $a'$ is one of the possible actions in state $s'$.\n",
    "* $Q_\\text{target}(s', a')$ is the target DQN's estimate of the Q-Value of playing action $a'$ while in state $s'$.\n",
    "\n",
    "I hope these errors did not affect you, and if they did, I sincerely apologize."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Creating the MsPacman environment"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[2017-09-25 11:52:24,838] Making new env: MsPacman-v0\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "(210, 160, 3)"
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "env = gym.make(\"MsPacman-v0\")\n",
    "obs = env.reset()\n",
    "obs.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Discrete(9)"
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "env.action_space"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Preprocessing"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Preprocessing the images is optional but greatly speeds up training."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "mspacman_color = np.array([210, 164, 74]).mean()\n",
    "\n",
    "def preprocess_observation(obs):\n",
    "    img = obs[1:176:2, ::2] # crop and downsize\n",
    "    img = img.mean(axis=2) # to greyscale\n",
    "    img[img==mspacman_color] = 0 # Improve contrast\n",
    "    img = (img - 128) / 128 - 1 # normalize from -1. to 1.\n",
    "    return img.reshape(88, 80, 1)\n",
    "\n",
    "img = preprocess_observation(obs)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/javascript": [
       "/* Put everything inside the global mpl namespace */\n",
       "window.mpl = {};\n",
       "\n",
       "\n",
       "mpl.get_websocket_type = function() {\n",
       "    if (typeof(WebSocket) !== 'undefined') {\n",
       "        return WebSocket;\n",
       "    } else if (typeof(MozWebSocket) !== 'undefined') {\n",
       "        return MozWebSocket;\n",
       "    } else {\n",
       "        alert('Your browser does not have WebSocket support.' +\n",
       "              'Please try Chrome, Safari or Firefox ≥ 6. ' +\n",
       "              'Firefox 4 and 5 are also supported but you ' +\n",
       "              'have to enable WebSockets in about:config.');\n",
       "    };\n",
       "}\n",
       "\n",
       "mpl.figure = function(figure_id, websocket, ondownload, parent_element) {\n",
       "    this.id = figure_id;\n",
       "\n",
       "    this.ws = websocket;\n",
       "\n",
       "    this.supports_binary = (this.ws.binaryType != undefined);\n",
       "\n",
       "    if (!this.supports_binary) {\n",
       "        var warnings = document.getElementById(\"mpl-warnings\");\n",
       "        if (warnings) {\n",
       "            warnings.style.display = 'block';\n",
       "            warnings.textContent = (\n",
       "                \"This browser does not support binary websocket messages. \" +\n",
       "                    \"Performance may be slow.\");\n",
       "        }\n",
       "    }\n",
       "\n",
       "    this.imageObj = new Image();\n",
       "\n",
       "    this.context = undefined;\n",
       "    this.message = undefined;\n",
       "    this.canvas = undefined;\n",
       "    this.rubberband_canvas = undefined;\n",
       "    this.rubberband_context = undefined;\n",
       "    this.format_dropdown = undefined;\n",
       "\n",
       "    this.image_mode = 'full';\n",
       "\n",
       "    this.root = $('<div/>');\n",
       "    this._root_extra_style(this.root)\n",
       "    this.root.attr('style', 'display: inline-block');\n",
       "\n",
       "    $(parent_element).append(this.root);\n",
       "\n",
       "    this._init_header(this);\n",
       "    this._init_canvas(this);\n",
       "    this._init_toolbar(this);\n",
       "\n",
       "    var fig = this;\n",
       "\n",
       "    this.waiting = false;\n",
       "\n",
       "    this.ws.onopen =  function () {\n",
       "            fig.send_message(\"supports_binary\", {value: fig.supports_binary});\n",
       "            fig.send_message(\"send_image_mode\", {});\n",
       "            if (mpl.ratio != 1) {\n",
       "                fig.send_message(\"set_dpi_ratio\", {'dpi_ratio': mpl.ratio});\n",
       "            }\n",
       "            fig.send_message(\"refresh\", {});\n",
       "        }\n",
       "\n",
       "    this.imageObj.onload = function() {\n",
       "            if (fig.image_mode == 'full') {\n",
       "                // Full images could contain transparency (where diff images\n",
       "                // almost always do), so we need to clear the canvas so that\n",
       "                // there is no ghosting.\n",
       "                fig.context.clearRect(0, 0, fig.canvas.width, fig.canvas.height);\n",
       "            }\n",
       "            fig.context.drawImage(fig.imageObj, 0, 0);\n",
       "        };\n",
       "\n",
       "    this.imageObj.onunload = function() {\n",
       "        this.ws.close();\n",
       "    }\n",
       "\n",
       "    this.ws.onmessage = this._make_on_message_function(this);\n",
       "\n",
       "    this.ondownload = ondownload;\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._init_header = function() {\n",
       "    var titlebar = $(\n",
       "        '<div class=\"ui-dialog-titlebar ui-widget-header ui-corner-all ' +\n",
       "        'ui-helper-clearfix\"/>');\n",
       "    var titletext = $(\n",
       "        '<div class=\"ui-dialog-title\" style=\"width: 100%; ' +\n",
       "        'text-align: center; padding: 3px;\"/>');\n",
       "    titlebar.append(titletext)\n",
       "    this.root.append(titlebar);\n",
       "    this.header = titletext[0];\n",
       "}\n",
       "\n",
       "\n",
       "\n",
       "mpl.figure.prototype._canvas_extra_style = function(canvas_div) {\n",
       "\n",
       "}\n",
       "\n",
       "\n",
       "mpl.figure.prototype._root_extra_style = function(canvas_div) {\n",
       "\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._init_canvas = function() {\n",
       "    var fig = this;\n",
       "\n",
       "    var canvas_div = $('<div/>');\n",
       "\n",
       "    canvas_div.attr('style', 'position: relative; clear: both; outline: 0');\n",
       "\n",
       "    function canvas_keyboard_event(event) {\n",
       "        return fig.key_event(event, event['data']);\n",
       "    }\n",
       "\n",
       "    canvas_div.keydown('key_press', canvas_keyboard_event);\n",
       "    canvas_div.keyup('key_release', canvas_keyboard_event);\n",
       "    this.canvas_div = canvas_div\n",
       "    this._canvas_extra_style(canvas_div)\n",
       "    this.root.append(canvas_div);\n",
       "\n",
       "    var canvas = $('<canvas/>');\n",
       "    canvas.addClass('mpl-canvas');\n",
       "    canvas.attr('style', \"left: 0; top: 0; z-index: 0; outline: 0\")\n",
       "\n",
       "    this.canvas = canvas[0];\n",
       "    this.context = canvas[0].getContext(\"2d\");\n",
       "\n",
       "    var backingStore = this.context.backingStorePixelRatio ||\n",
       "\tthis.context.webkitBackingStorePixelRatio ||\n",
       "\tthis.context.mozBackingStorePixelRatio ||\n",
       "\tthis.context.msBackingStorePixelRatio ||\n",
       "\tthis.context.oBackingStorePixelRatio ||\n",
       "\tthis.context.backingStorePixelRatio || 1;\n",
       "\n",
       "    mpl.ratio = (window.devicePixelRatio || 1) / backingStore;\n",
       "\n",
       "    var rubberband = $('<canvas/>');\n",
       "    rubberband.attr('style', \"position: absolute; left: 0; top: 0; z-index: 1;\")\n",
       "\n",
       "    var pass_mouse_events = true;\n",
       "\n",
       "    canvas_div.resizable({\n",
       "        start: function(event, ui) {\n",
       "            pass_mouse_events = false;\n",
       "        },\n",
       "        resize: function(event, ui) {\n",
       "            fig.request_resize(ui.size.width, ui.size.height);\n",
       "        },\n",
       "        stop: function(event, ui) {\n",
       "            pass_mouse_events = true;\n",
       "            fig.request_resize(ui.size.width, ui.size.height);\n",
       "        },\n",
       "    });\n",
       "\n",
       "    function mouse_event_fn(event) {\n",
       "        if (pass_mouse_events)\n",
       "            return fig.mouse_event(event, event['data']);\n",
       "    }\n",
       "\n",
       "    rubberband.mousedown('button_press', mouse_event_fn);\n",
       "    rubberband.mouseup('button_release', mouse_event_fn);\n",
       "    // Throttle sequential mouse events to 1 every 20ms.\n",
       "    rubberband.mousemove('motion_notify', mouse_event_fn);\n",
       "\n",
       "    rubberband.mouseenter('figure_enter', mouse_event_fn);\n",
       "    rubberband.mouseleave('figure_leave', mouse_event_fn);\n",
       "\n",
       "    canvas_div.on(\"wheel\", function (event) {\n",
       "        event = event.originalEvent;\n",
       "        event['data'] = 'scroll'\n",
       "        if (event.deltaY < 0) {\n",
       "            event.step = 1;\n",
       "        } else {\n",
       "            event.step = -1;\n",
       "        }\n",
       "        mouse_event_fn(event);\n",
       "    });\n",
       "\n",
       "    canvas_div.append(canvas);\n",
       "    canvas_div.append(rubberband);\n",
       "\n",
       "    this.rubberband = rubberband;\n",
       "    this.rubberband_canvas = rubberband[0];\n",
       "    this.rubberband_context = rubberband[0].getContext(\"2d\");\n",
       "    this.rubberband_context.strokeStyle = \"#000000\";\n",
       "\n",
       "    this._resize_canvas = function(width, height) {\n",
       "        // Keep the size of the canvas, canvas container, and rubber band\n",
       "        // canvas in synch.\n",
       "        canvas_div.css('width', width)\n",
       "        canvas_div.css('height', height)\n",
       "\n",
       "        canvas.attr('width', width * mpl.ratio);\n",
       "        canvas.attr('height', height * mpl.ratio);\n",
       "        canvas.attr('style', 'width: ' + width + 'px; height: ' + height + 'px;');\n",
       "\n",
       "        rubberband.attr('width', width);\n",
       "        rubberband.attr('height', height);\n",
       "    }\n",
       "\n",
       "    // Set the figure to an initial 600x600px, this will subsequently be updated\n",
       "    // upon first draw.\n",
       "    this._resize_canvas(600, 600);\n",
       "\n",
       "    // Disable right mouse context menu.\n",
       "    $(this.rubberband_canvas).bind(\"contextmenu\",function(e){\n",
       "        return false;\n",
       "    });\n",
       "\n",
       "    function set_focus () {\n",
       "        canvas.focus();\n",
       "        canvas_div.focus();\n",
       "    }\n",
       "\n",
       "    window.setTimeout(set_focus, 100);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._init_toolbar = function() {\n",
       "    var fig = this;\n",
       "\n",
       "    var nav_element = $('<div/>')\n",
       "    nav_element.attr('style', 'width: 100%');\n",
       "    this.root.append(nav_element);\n",
       "\n",
       "    // Define a callback function for later on.\n",
       "    function toolbar_event(event) {\n",
       "        return fig.toolbar_button_onclick(event['data']);\n",
       "    }\n",
       "    function toolbar_mouse_event(event) {\n",
       "        return fig.toolbar_button_onmouseover(event['data']);\n",
       "    }\n",
       "\n",
       "    for(var toolbar_ind in mpl.toolbar_items) {\n",
       "        var name = mpl.toolbar_items[toolbar_ind][0];\n",
       "        var tooltip = mpl.toolbar_items[toolbar_ind][1];\n",
       "        var image = mpl.toolbar_items[toolbar_ind][2];\n",
       "        var method_name = mpl.toolbar_items[toolbar_ind][3];\n",
       "\n",
       "        if (!name) {\n",
       "            // put a spacer in here.\n",
       "            continue;\n",
       "        }\n",
       "        var button = $('<button/>');\n",
       "        button.addClass('ui-button ui-widget ui-state-default ui-corner-all ' +\n",
       "                        'ui-button-icon-only');\n",
       "        button.attr('role', 'button');\n",
       "        button.attr('aria-disabled', 'false');\n",
       "        button.click(method_name, toolbar_event);\n",
       "        button.mouseover(tooltip, toolbar_mouse_event);\n",
       "\n",
       "        var icon_img = $('<span/>');\n",
       "        icon_img.addClass('ui-button-icon-primary ui-icon');\n",
       "        icon_img.addClass(image);\n",
       "        icon_img.addClass('ui-corner-all');\n",
       "\n",
       "        var tooltip_span = $('<span/>');\n",
       "        tooltip_span.addClass('ui-button-text');\n",
       "        tooltip_span.html(tooltip);\n",
       "\n",
       "        button.append(icon_img);\n",
       "        button.append(tooltip_span);\n",
       "\n",
       "        nav_element.append(button);\n",
       "    }\n",
       "\n",
       "    var fmt_picker_span = $('<span/>');\n",
       "\n",
       "    var fmt_picker = $('<select/>');\n",
       "    fmt_picker.addClass('mpl-toolbar-option ui-widget ui-widget-content');\n",
       "    fmt_picker_span.append(fmt_picker);\n",
       "    nav_element.append(fmt_picker_span);\n",
       "    this.format_dropdown = fmt_picker[0];\n",
       "\n",
       "    for (var ind in mpl.extensions) {\n",
       "        var fmt = mpl.extensions[ind];\n",
       "        var option = $(\n",
       "            '<option/>', {selected: fmt === mpl.default_extension}).html(fmt);\n",
       "        fmt_picker.append(option)\n",
       "    }\n",
       "\n",
       "    // Add hover states to the ui-buttons\n",
       "    $( \".ui-button\" ).hover(\n",
       "        function() { $(this).addClass(\"ui-state-hover\");},\n",
       "        function() { $(this).removeClass(\"ui-state-hover\");}\n",
       "    );\n",
       "\n",
       "    var status_bar = $('<span class=\"mpl-message\"/>');\n",
       "    nav_element.append(status_bar);\n",
       "    this.message = status_bar[0];\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.request_resize = function(x_pixels, y_pixels) {\n",
       "    // Request matplotlib to resize the figure. Matplotlib will then trigger a resize in the client,\n",
       "    // which will in turn request a refresh of the image.\n",
       "    this.send_message('resize', {'width': x_pixels, 'height': y_pixels});\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.send_message = function(type, properties) {\n",
       "    properties['type'] = type;\n",
       "    properties['figure_id'] = this.id;\n",
       "    this.ws.send(JSON.stringify(properties));\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.send_draw_message = function() {\n",
       "    if (!this.waiting) {\n",
       "        this.waiting = true;\n",
       "        this.ws.send(JSON.stringify({type: \"draw\", figure_id: this.id}));\n",
       "    }\n",
       "}\n",
       "\n",
       "\n",
       "mpl.figure.prototype.handle_save = function(fig, msg) {\n",
       "    var format_dropdown = fig.format_dropdown;\n",
       "    var format = format_dropdown.options[format_dropdown.selectedIndex].value;\n",
       "    fig.ondownload(fig, format);\n",
       "}\n",
       "\n",
       "\n",
       "mpl.figure.prototype.handle_resize = function(fig, msg) {\n",
       "    var size = msg['size'];\n",
       "    if (size[0] != fig.canvas.width || size[1] != fig.canvas.height) {\n",
       "        fig._resize_canvas(size[0], size[1]);\n",
       "        fig.send_message(\"refresh\", {});\n",
       "    };\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_rubberband = function(fig, msg) {\n",
       "    var x0 = msg['x0'] / mpl.ratio;\n",
       "    var y0 = (fig.canvas.height - msg['y0']) / mpl.ratio;\n",
       "    var x1 = msg['x1'] / mpl.ratio;\n",
       "    var y1 = (fig.canvas.height - msg['y1']) / mpl.ratio;\n",
       "    x0 = Math.floor(x0) + 0.5;\n",
       "    y0 = Math.floor(y0) + 0.5;\n",
       "    x1 = Math.floor(x1) + 0.5;\n",
       "    y1 = Math.floor(y1) + 0.5;\n",
       "    var min_x = Math.min(x0, x1);\n",
       "    var min_y = Math.min(y0, y1);\n",
       "    var width = Math.abs(x1 - x0);\n",
       "    var height = Math.abs(y1 - y0);\n",
       "\n",
       "    fig.rubberband_context.clearRect(\n",
       "        0, 0, fig.canvas.width, fig.canvas.height);\n",
       "\n",
       "    fig.rubberband_context.strokeRect(min_x, min_y, width, height);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_figure_label = function(fig, msg) {\n",
       "    // Updates the figure title.\n",
       "    fig.header.textContent = msg['label'];\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_cursor = function(fig, msg) {\n",
       "    var cursor = msg['cursor'];\n",
       "    switch(cursor)\n",
       "    {\n",
       "    case 0:\n",
       "        cursor = 'pointer';\n",
       "        break;\n",
       "    case 1:\n",
       "        cursor = 'default';\n",
       "        break;\n",
       "    case 2:\n",
       "        cursor = 'crosshair';\n",
       "        break;\n",
       "    case 3:\n",
       "        cursor = 'move';\n",
       "        break;\n",
       "    }\n",
       "    fig.rubberband_canvas.style.cursor = cursor;\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_message = function(fig, msg) {\n",
       "    fig.message.textContent = msg['message'];\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_draw = function(fig, msg) {\n",
       "    // Request the server to send over a new figure.\n",
       "    fig.send_draw_message();\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_image_mode = function(fig, msg) {\n",
       "    fig.image_mode = msg['mode'];\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.updated_canvas_event = function() {\n",
       "    // Called whenever the canvas gets updated.\n",
       "    this.send_message(\"ack\", {});\n",
       "}\n",
       "\n",
       "// A function to construct a web socket function for onmessage handling.\n",
       "// Called in the figure constructor.\n",
       "mpl.figure.prototype._make_on_message_function = function(fig) {\n",
       "    return function socket_on_message(evt) {\n",
       "        if (evt.data instanceof Blob) {\n",
       "            /* FIXME: We get \"Resource interpreted as Image but\n",
       "             * transferred with MIME type text/plain:\" errors on\n",
       "             * Chrome.  But how to set the MIME type?  It doesn't seem\n",
       "             * to be part of the websocket stream */\n",
       "            evt.data.type = \"image/png\";\n",
       "\n",
       "            /* Free the memory for the previous frames */\n",
       "            if (fig.imageObj.src) {\n",
       "                (window.URL || window.webkitURL).revokeObjectURL(\n",
       "                    fig.imageObj.src);\n",
       "            }\n",
       "\n",
       "            fig.imageObj.src = (window.URL || window.webkitURL).createObjectURL(\n",
       "                evt.data);\n",
       "            fig.updated_canvas_event();\n",
       "            fig.waiting = false;\n",
       "            return;\n",
       "        }\n",
       "        else if (typeof evt.data === 'string' && evt.data.slice(0, 21) == \"data:image/png;base64\") {\n",
       "            fig.imageObj.src = evt.data;\n",
       "            fig.updated_canvas_event();\n",
       "            fig.waiting = false;\n",
       "            return;\n",
       "        }\n",
       "\n",
       "        var msg = JSON.parse(evt.data);\n",
       "        var msg_type = msg['type'];\n",
       "\n",
       "        // Call the  \"handle_{type}\" callback, which takes\n",
       "        // the figure and JSON message as its only arguments.\n",
       "        try {\n",
       "            var callback = fig[\"handle_\" + msg_type];\n",
       "        } catch (e) {\n",
       "            console.log(\"No handler for the '\" + msg_type + \"' message type: \", msg);\n",
       "            return;\n",
       "        }\n",
       "\n",
       "        if (callback) {\n",
       "            try {\n",
       "                // console.log(\"Handling '\" + msg_type + \"' message: \", msg);\n",
       "                callback(fig, msg);\n",
       "            } catch (e) {\n",
       "                console.log(\"Exception inside the 'handler_\" + msg_type + \"' callback:\", e, e.stack, msg);\n",
       "            }\n",
       "        }\n",
       "    };\n",
       "}\n",
       "\n",
       "// from http://stackoverflow.com/questions/1114465/getting-mouse-location-in-canvas\n",
       "mpl.findpos = function(e) {\n",
       "    //this section is from http://www.quirksmode.org/js/events_properties.html\n",
       "    var targ;\n",
       "    if (!e)\n",
       "        e = window.event;\n",
       "    if (e.target)\n",
       "        targ = e.target;\n",
       "    else if (e.srcElement)\n",
       "        targ = e.srcElement;\n",
       "    if (targ.nodeType == 3) // defeat Safari bug\n",
       "        targ = targ.parentNode;\n",
       "\n",
       "    // jQuery normalizes the pageX and pageY\n",
       "    // pageX,Y are the mouse positions relative to the document\n",
       "    // offset() returns the position of the element relative to the document\n",
       "    var x = e.pageX - $(targ).offset().left;\n",
       "    var y = e.pageY - $(targ).offset().top;\n",
       "\n",
       "    return {\"x\": x, \"y\": y};\n",
       "};\n",
       "\n",
       "/*\n",
       " * return a copy of an object with only non-object keys\n",
       " * we need this to avoid circular references\n",
       " * http://stackoverflow.com/a/24161582/3208463\n",
       " */\n",
       "function simpleKeys (original) {\n",
       "  return Object.keys(original).reduce(function (obj, key) {\n",
       "    if (typeof original[key] !== 'object')\n",
       "        obj[key] = original[key]\n",
       "    return obj;\n",
       "  }, {});\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.mouse_event = function(event, name) {\n",
       "    var canvas_pos = mpl.findpos(event)\n",
       "\n",
       "    if (name === 'button_press')\n",
       "    {\n",
       "        this.canvas.focus();\n",
       "        this.canvas_div.focus();\n",
       "    }\n",
       "\n",
       "    var x = canvas_pos.x * mpl.ratio;\n",
       "    var y = canvas_pos.y * mpl.ratio;\n",
       "\n",
       "    this.send_message(name, {x: x, y: y, button: event.button,\n",
       "                             step: event.step,\n",
       "                             guiEvent: simpleKeys(event)});\n",
       "\n",
       "    /* This prevents the web browser from automatically changing to\n",
       "     * the text insertion cursor when the button is pressed.  We want\n",
       "     * to control all of the cursor setting manually through the\n",
       "     * 'cursor' event from matplotlib */\n",
       "    event.preventDefault();\n",
       "    return false;\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._key_event_extra = function(event, name) {\n",
       "    // Handle any extra behaviour associated with a key event\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.key_event = function(event, name) {\n",
       "\n",
       "    // Prevent repeat events\n",
       "    if (name == 'key_press')\n",
       "    {\n",
       "        if (event.which === this._key)\n",
       "            return;\n",
       "        else\n",
       "            this._key = event.which;\n",
       "    }\n",
       "    if (name == 'key_release')\n",
       "        this._key = null;\n",
       "\n",
       "    var value = '';\n",
       "    if (event.ctrlKey && event.which != 17)\n",
       "        value += \"ctrl+\";\n",
       "    if (event.altKey && event.which != 18)\n",
       "        value += \"alt+\";\n",
       "    if (event.shiftKey && event.which != 16)\n",
       "        value += \"shift+\";\n",
       "\n",
       "    value += 'k';\n",
       "    value += event.which.toString();\n",
       "\n",
       "    this._key_event_extra(event, name);\n",
       "\n",
       "    this.send_message(name, {key: value,\n",
       "                             guiEvent: simpleKeys(event)});\n",
       "    return false;\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.toolbar_button_onclick = function(name) {\n",
       "    if (name == 'download') {\n",
       "        this.handle_save(this, null);\n",
       "    } else {\n",
       "        this.send_message(\"toolbar_button\", {name: name});\n",
       "    }\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.toolbar_button_onmouseover = function(tooltip) {\n",
       "    this.message.textContent = tooltip;\n",
       "};\n",
       "mpl.toolbar_items = [[\"Home\", \"Reset original view\", \"fa fa-home icon-home\", \"home\"], [\"Back\", \"Back to  previous view\", \"fa fa-arrow-left icon-arrow-left\", \"back\"], [\"Forward\", \"Forward to next view\", \"fa fa-arrow-right icon-arrow-right\", \"forward\"], [\"\", \"\", \"\", \"\"], [\"Pan\", \"Pan axes with left mouse, zoom with right\", \"fa fa-arrows icon-move\", \"pan\"], [\"Zoom\", \"Zoom to rectangle\", \"fa fa-square-o icon-check-empty\", \"zoom\"], [\"\", \"\", \"\", \"\"], [\"Download\", \"Download plot\", \"fa fa-floppy-o icon-save\", \"download\"]];\n",
       "\n",
       "mpl.extensions = [\"eps\", \"jpeg\", \"pdf\", \"png\", \"ps\", \"raw\", \"svg\", \"tif\"];\n",
       "\n",
       "mpl.default_extension = \"png\";var comm_websocket_adapter = function(comm) {\n",
       "    // Create a \"websocket\"-like object which calls the given IPython comm\n",
       "    // object with the appropriate methods. Currently this is a non binary\n",
       "    // socket, so there is still some room for performance tuning.\n",
       "    var ws = {};\n",
       "\n",
       "    ws.close = function() {\n",
       "        comm.close()\n",
       "    };\n",
       "    ws.send = function(m) {\n",
       "        //console.log('sending', m);\n",
       "        comm.send(m);\n",
       "    };\n",
       "    // Register the callback with on_msg.\n",
       "    comm.on_msg(function(msg) {\n",
       "        //console.log('receiving', msg['content']['data'], msg);\n",
       "        // Pass the mpl event to the overriden (by mpl) onmessage function.\n",
       "        ws.onmessage(msg['content']['data'])\n",
       "    });\n",
       "    return ws;\n",
       "}\n",
       "\n",
       "mpl.mpl_figure_comm = function(comm, msg) {\n",
       "    // This is the function which gets called when the mpl process\n",
       "    // starts-up an IPython Comm through the \"matplotlib\" channel.\n",
       "\n",
       "    var id = msg.content.data.id;\n",
       "    // Get hold of the div created by the display call when the Comm\n",
       "    // socket was opened in Python.\n",
       "    var element = $(\"#\" + id);\n",
       "    var ws_proxy = comm_websocket_adapter(comm)\n",
       "\n",
       "    function ondownload(figure, format) {\n",
       "        window.open(figure.imageObj.src);\n",
       "    }\n",
       "\n",
       "    var fig = new mpl.figure(id, ws_proxy,\n",
       "                           ondownload,\n",
       "                           element.get(0));\n",
       "\n",
       "    // Call onopen now - mpl needs it, as it is assuming we've passed it a real\n",
       "    // web socket which is closed, not our websocket->open comm proxy.\n",
       "    ws_proxy.onopen();\n",
       "\n",
       "    fig.parent_element = element.get(0);\n",
       "    fig.cell_info = mpl.find_output_cell(\"<div id='\" + id + \"'></div>\");\n",
       "    if (!fig.cell_info) {\n",
       "        console.error(\"Failed to find cell for figure\", id, fig);\n",
       "        return;\n",
       "    }\n",
       "\n",
       "    var output_index = fig.cell_info[2]\n",
       "    var cell = fig.cell_info[0];\n",
       "\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_close = function(fig, msg) {\n",
       "    var width = fig.canvas.width/mpl.ratio\n",
       "    fig.root.unbind('remove')\n",
       "\n",
       "    // Update the output cell to use the data from the current canvas.\n",
       "    fig.push_to_output();\n",
       "    var dataURL = fig.canvas.toDataURL();\n",
       "    // Re-enable the keyboard manager in IPython - without this line, in FF,\n",
       "    // the notebook keyboard shortcuts fail.\n",
       "    IPython.keyboard_manager.enable()\n",
       "    $(fig.parent_element).html('<img src=\"' + dataURL + '\" width=\"' + width + '\">');\n",
       "    fig.close_ws(fig, msg);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.close_ws = function(fig, msg){\n",
       "    fig.send_message('closing', msg);\n",
       "    // fig.ws.close()\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.push_to_output = function(remove_interactive) {\n",
       "    // Turn the data on the canvas into data in the output cell.\n",
       "    var width = this.canvas.width/mpl.ratio\n",
       "    var dataURL = this.canvas.toDataURL();\n",
       "    this.cell_info[1]['text/html'] = '<img src=\"' + dataURL + '\" width=\"' + width + '\">';\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.updated_canvas_event = function() {\n",
       "    // Tell IPython that the notebook contents must change.\n",
       "    IPython.notebook.set_dirty(true);\n",
       "    this.send_message(\"ack\", {});\n",
       "    var fig = this;\n",
       "    // Wait a second, then push the new image to the DOM so\n",
       "    // that it is saved nicely (might be nice to debounce this).\n",
       "    setTimeout(function () { fig.push_to_output() }, 1000);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._init_toolbar = function() {\n",
       "    var fig = this;\n",
       "\n",
       "    var nav_element = $('<div/>')\n",
       "    nav_element.attr('style', 'width: 100%');\n",
       "    this.root.append(nav_element);\n",
       "\n",
       "    // Define a callback function for later on.\n",
       "    function toolbar_event(event) {\n",
       "        return fig.toolbar_button_onclick(event['data']);\n",
       "    }\n",
       "    function toolbar_mouse_event(event) {\n",
       "        return fig.toolbar_button_onmouseover(event['data']);\n",
       "    }\n",
       "\n",
       "    for(var toolbar_ind in mpl.toolbar_items){\n",
       "        var name = mpl.toolbar_items[toolbar_ind][0];\n",
       "        var tooltip = mpl.toolbar_items[toolbar_ind][1];\n",
       "        var image = mpl.toolbar_items[toolbar_ind][2];\n",
       "        var method_name = mpl.toolbar_items[toolbar_ind][3];\n",
       "\n",
       "        if (!name) { continue; };\n",
       "\n",
       "        var button = $('<button class=\"btn btn-default\" href=\"#\" title=\"' + name + '\"><i class=\"fa ' + image + ' fa-lg\"></i></button>');\n",
       "        button.click(method_name, toolbar_event);\n",
       "        button.mouseover(tooltip, toolbar_mouse_event);\n",
       "        nav_element.append(button);\n",
       "    }\n",
       "\n",
       "    // Add the status bar.\n",
       "    var status_bar = $('<span class=\"mpl-message\" style=\"text-align:right; float: right;\"/>');\n",
       "    nav_element.append(status_bar);\n",
       "    this.message = status_bar[0];\n",
       "\n",
       "    // Add the close button to the window.\n",
       "    var buttongrp = $('<div class=\"btn-group inline pull-right\"></div>');\n",
       "    var button = $('<button class=\"btn btn-mini btn-primary\" href=\"#\" title=\"Stop Interaction\"><i class=\"fa fa-power-off icon-remove icon-large\"></i></button>');\n",
       "    button.click(function (evt) { fig.handle_close(fig, {}); } );\n",
       "    button.mouseover('Stop Interaction', toolbar_mouse_event);\n",
       "    buttongrp.append(button);\n",
       "    var titlebar = this.root.find($('.ui-dialog-titlebar'));\n",
       "    titlebar.prepend(buttongrp);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._root_extra_style = function(el){\n",
       "    var fig = this\n",
       "    el.on(\"remove\", function(){\n",
       "\tfig.close_ws(fig, {});\n",
       "    });\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._canvas_extra_style = function(el){\n",
       "    // this is important to make the div 'focusable\n",
       "    el.attr('tabindex', 0)\n",
       "    // reach out to IPython and tell the keyboard manager to turn it's self\n",
       "    // off when our div gets focus\n",
       "\n",
       "    // location in version 3\n",
       "    if (IPython.notebook.keyboard_manager) {\n",
       "        IPython.notebook.keyboard_manager.register_events(el);\n",
       "    }\n",
       "    else {\n",
       "        // location in version 2\n",
       "        IPython.keyboard_manager.register_events(el);\n",
       "    }\n",
       "\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._key_event_extra = function(event, name) {\n",
       "    var manager = IPython.notebook.keyboard_manager;\n",
       "    if (!manager)\n",
       "        manager = IPython.keyboard_manager;\n",
       "\n",
       "    // Check for shift+enter\n",
       "    if (event.shiftKey && event.which == 13) {\n",
       "        this.canvas_div.blur();\n",
       "        // select the cell after this one\n",
       "        var index = IPython.notebook.find_cell_index(this.cell_info[0]);\n",
       "        IPython.notebook.select(index + 1);\n",
       "    }\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_save = function(fig, msg) {\n",
       "    fig.ondownload(fig, null);\n",
       "}\n",
       "\n",
       "\n",
       "mpl.find_output_cell = function(html_output) {\n",
       "    // Return the cell and output element which can be found *uniquely* in the notebook.\n",
       "    // Note - this is a bit hacky, but it is done because the \"notebook_saving.Notebook\"\n",
       "    // IPython event is triggered only after the cells have been serialised, which for\n",
       "    // our purposes (turning an active figure into a static one), is too late.\n",
       "    var cells = IPython.notebook.get_cells();\n",
       "    var ncells = cells.length;\n",
       "    for (var i=0; i<ncells; i++) {\n",
       "        var cell = cells[i];\n",
       "        if (cell.cell_type === 'code'){\n",
       "            for (var j=0; j<cell.output_area.outputs.length; j++) {\n",
       "                var data = cell.output_area.outputs[j];\n",
       "                if (data.data) {\n",
       "                    // IPython >= 3 moved mimebundle to data attribute of output\n",
       "                    data = data.data;\n",
       "                }\n",
       "                if (data['text/html'] == html_output) {\n",
       "                    return [cell, data, j];\n",
       "                }\n",
       "            }\n",
       "        }\n",
       "    }\n",
       "}\n",
       "\n",
       "// Register the function which deals with the matplotlib target/channel.\n",
       "// The kernel may be null if the page has been refreshed.\n",
       "if (IPython.notebook.kernel != null) {\n",
       "    IPython.notebook.kernel.comm_manager.register_target('matplotlib', mpl.mpl_figure_comm);\n",
       "}\n"
      ],
      "text/plain": [
       "<IPython.core.display.Javascript object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<img src=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAABEwAAAK8CAYAAAD1QyqZAAAgAElEQVR4XuydB9guS1LX65izIhhWjjlgQAwYQFdEUTGAaQ0Y7lEMqGu4imkNiAF0zV7Dqmvea0BlTSAoJkRUVEyIARXj1TXnnI7P/9zpe+a+z/d9VdXz9rw1Pb95nn1gz1fTU/2r6tmu/9vd88i4IAABCEAAAhCAAAQgAAEIQAACEIAABF5H4BE8IAABCEAAAhCAAAQgAAEIQAACEIAABF5PAMGEjIAABCAAAQhAAAIQgAAEIAABCEAAAhcEEExICQhAAAIQgAAEIAABCEAAAhCAAAQggGBCDkAAAhCAAAQgAAEIQAACEIAABCAAgYcJsMKEDDkCgZ9tZh9tZj35+oPN7LeY2Vc1s380qLNfxcz+oZl9uJn91is84wPN7E+Z2fc2s4+/QntHaqL1/duY2afewPHPY2afZWa/y8w+9gbP55H7EXh3M/snyzj7pP0ey5MgAAEIQAACEIAABI5CoKcAPUrf8PP2BL6emf00M1Px+x5m9m8XIeDnm9nfTLiHYJKAdRDTN5vZf7tDYLq1YPIDzOxtZvaVzOw/Liy/mJn9ZDP75mb2zczs3RxxTKLLj1j+815LP/+6mf0EM9P/bZfsfpKZ/Sgze4OZ/V0z+wWLWLM1jF9k8fG7mdnXNzP14e+b2duX//zfiwf8jKV/6uOXNbOfY2Yad3dd72lmv9zMvoOZqQ8S99S3fxBwWqLlV17ZKQf0LvjVZvaOe+6XPx9pZt9lET4/n5m9YmZ/xsx+k5l9+uq+JpCum/rXyzN+kZl98sUzXjKzN5rZ+wZ8xwQCEIAABCAAAQhA4GQEEExOFvAdu/s9l8Lv3y1FjVZgaCXGDzUz/bL7YWb2+4P+qEDSf/5H0H5t9nnN7POb2f80s6cd90duYYVJhNLrbT7bzP6NmUkgWV8qwL+Amf0vM/t/+WY33/HXzOwvLGJHa6zFV6sRJArI54dWE2mVkYQXCQB/3sy+qJl9IzP7HWb2x1YeShx5i5n9BjP7S2YmcUOiwPczs4/b2JP3XlbK/Akz+xQz+09m9sFm9j0Wv37QRfsaG/9iEXRkd59gIuHlr5jZlzSzX2pm/3sRS/S/Jd9wEUUfcl2Cyb9f7pWdhKIfZmZfy8w+YmGxvl8C1R82sy++MBEnjWWtGPvuZvZ1zexbm9mnLTc1weRnLau+5NeXMzP9uwTcDzWzT1w94OuY2d8ysw8ysz+5kTm3QwACEIAABCAAAQhMRgDBZLKAFunOV1+KNRWYH2Bm+oW3XVppol+Gv6KZvY/zq7QKzf9apE8PuTGzYKKVCloFcO3rPsHk2s/JtCdRQ2LAtzMzCQ3t+oLLqhIJCt9kETfuE0y+j5n9bjOTYPiQIKhVGhIRteLjxywP0vv4Ty9igHJqvQpEqzn+zrKF5LJPEjEkBGgb0XqcSSi4XMn1mxex52suK06avZ4nMUPjU+P1PsHkp5jZL1xW2ki80PW1zUzx1AqOn+4A1zNk+yEruy+zvAf+6SKAtD9pJY/8FxetUlP/15f+XcKrVs40X5pg8k3N7DNXxmrrX5rZ713ErHU7f8PM/qqZPckkC7YQgAAEIAABCEAAAvMTQDCZP8a36OGvW36hl1giceTy0r+rMPz1ZvYjlz+2bTf6Ffhnmtl3Wgo4FbF3bcn5wkvh9v3NTAWttgVoa4OW6q+LvbvOMGlF21vN7Jctws0/X56z3hbwpZcCUL+46xdtrXj4s8uqgPXWioxg8tUWv/WL9hdahKWft/yK3ji1bSkqBr+Bmf2Q5Rd2FfE/2sxUWLZLha/68S3N7Estqza0RUFbQtqWEtn+wGUlgH6R/+/LqgNtM1m3pTNDVDBr9YG2XEgcUEH/NZZCVr5fXlpBoRU8stUlIeEFM9MKB61C+Fwz+1Vm9mtXN15uy9CflA/q931bcnSei1ZjyH+JaH/EzH6qmf2zVbta2fG9zEzbYH7NInyor79tsb3chnLZF+WNniEBQisn7ro8weQzlgJfW1u0WkZ5epfopy1J8lH5rhUO7dLqkt9pZt9qtdVE7fztZYWU/n0tQGo1jlZgvP+So+u/3eW/hJU/ZGbf1cw+4Q4DTzD5i8s9Wvmxvv6omUkoVa48dN0lmMhegocEVI3ldmk7n7bvaRxIhIpc9wkm+t+6/2Bmf2DJ73VbegcobzXeR61Ci/iODQQgAAEIQAACEIBAMQIIJsUCMok7KmK1pUIiw32Xfl3XNhutNNHVRBEVj3/PzFSAKT91nsRdgokKKP2a/7KZqUjVsnwVyhIYIoKJtvdIYNAZCBJLJEpInNF5D+1XeRXH2hqhX6Xlr36xlxChglqFu+7TFRVMdL+EFq3a+JXL9gWJE3qmCv22IqGJBvrlWwWchACd4/DjlxUG2vogIUDFsn51V5EpTloBoZUL+vVeAsM/XvzT+RQSZX7PIkzoF/0fa2b/ZemzCkldEkzEUNuY1G+tBNCv8l9i2cahIrn9ki97nUWhAljCyy9Z2lBBLX7q5/9ZVj5odYRWUUgg0KWtFBJR9Px2sKqeo+0qdwkmrQjWsyUmiOOLi2+KWfNfnL7vItJoW43stVrkTWYmgWIt2tyVl3q+iuaHzrN4SDARJ/miWGgLjBgrV5Q7EmLEv13ahiNxRFtN1kW6RAetmPhxC6Nmr60jEh+1akuM1L7ipHEg8UPbeS7P57irjz98EcG+xbJd6NLmIcFEwo1WG2mViniuL+WXhE4x+M/3DfolXy5XmOg9IOFO4/3Lr+79c8s2Hwlv9wlYl49quaK4KwfVpsaOYqG+f8eLbVG6X9unfvsyDuUbFwQgAAEIQAACEIAABJ4RQDAhEa5NQMWNisY/uBTG97Wvv6vQawVWE0W0rUCrRtbXpWDyjc3sL5vZr1hWTTRbfQ1HBVNEMFGxv14BIxFBRZsOn9RBnLokRKhQW5+lIXFEIoUKfRWJuqKCiVZtSPRYrx5QQa2vsmgsqljWs5poIOFJhXIrQCWCqOiWWCDBRcKJthI89DUd9VOrPHSmg36tb5dWgOhefX2o/bsEEwlPWvWj1T/tUoz+1QUb/U1CibZnqP8q5HVpRYXEnPWl1SBaCaP+teu+LTmXgolWr2jVkJ6vbRbtHBud9aGzKH7u0ge1K8FEApT62mKjf9c2G3Ftq2Au3Hvtvyr+ElokXt13PSSYtC09OtxYYpHyUKt8FC/5/p2XlTFqW74rtmsm+neJaVqRolVDWmGxvtSGztlQf1T4Kwd0JpAK/vV2nPt8l8CmmCtGOjNEPl5eDwkm7W+XfNVGWzGj7Tmf8wA/CWz6u3zWJYFE23y0KkmCWtuepL/p/COJfuK6viQyrVeiKN/aKp67Dn3VvTr3RGKnVhtdXlqdI3FGYtta1HqgG/wJAhCAAAQgAAEIQOAMBBBMzhDlffv4eBEe9IutiqD7Lv1dRZPsJQw0UWR9gGO791Iw0TkJEixU9Gk1SruakBIRTFRgaTvE+tIv0hIXdP7E5aVf87UiRWNGW2N0+KcO0NQVFUxUKEpM0naN9aXVBzoAVCtNJCQ00UD/tj4TQs8WKwksKpi1gkd+/MalKL/rrBF9vUSHc4pVW4nRnq2DMtXet1/+QYLJ+y0illYIrS+tftHKCwkwbUWEzoiQnVYr3HVJPJPgoV/2JcqIX9smFBVMWjF71woRbVNRHJsQ0gQTrShYb03Rl1CUi1o98tAlfjqYVf7edz0kmEgIa4ePiqPEF11tlYniry+y6FIO6cBTrVRaX1rFoa1D8lni2uWlrVzagqMVOfqSz3rljtO9Z9ur1DeJTfd9RvchwUSrwSSMaSuUzitZX1qhpdVaEjd0cO59113bsWQrsVPbzdZimwQdbfkS1/WlbTVaUdOutdDSBBO1pa8O6dKKJG1J06oTiSK/76I9iTzKJd2j1UFcEIAABCAAAQhAAAIQeEYAwYREuDaB7AoT2Wt7QRNFVASuz9WQf5eCiVY/6Msa+pV5/Su5VkKoII8IJiqQdE7K+pJgIDFAB0zqUvGq1QEq1iVOSDRpl85M+bbLf4kKJlodoV+wLw+XVPGnIlBbaVQMN8FERagKyfWlglyCgIo8XRJD9MlVFZrasqHzKSRGNWFCBaDOdrnvkviibUy61H8VxZerHvQ3nSOhVQw6K0W/xretIyrqVdy3S38XfwkdWi2xviS2tJUoUcGkPfeur5hIxJEAodVButqWHK2gWF/Rz1JLMNGWH+XWfddDgkn7m7bgXJ73om0sKtrFRDnbs8Kk+aRVEsohrRbRMyNfE9JqIIkcH2VmH/NA//ZYYaKtY9q+o/GklU76/5V7WtmzPmdGX9ORwHK5wkRnnWgM6NI2qrsEk8tDXzWWxUu5ovG6FgTbl3Ii27YeQMefIAABCEAAAhCAAARmI4BgMltEa/RHZ3tIHLjrkNDmoYpKrT7QChNdrahVQaPPzT5U8F5DMLk8R0HPU9Gmq33qVoWctnao2FVhpi0CKk61FWj9SdxbCibyVytTJLrorBAJFu9aVopoK4sO4NXnWiUO3XXoqc4R0Rkwrf8qmFXEXl76YpG2xejsDYkk2i6iwlsCSzvLRSKKzi/RliWdFyLhS4WptqJopYtEJxXAukYJJiq6taLjofy5b5TIX53BojNP7rseEky+wrJiRzwlGK0vbbHRyoy2ykYctfVMvkbOMGltSfySCKZ81IoJjYWHBDHdp1UXyuGI7S3OMGnbq37icghz66tWl0jMe+gME7GLCCZqU+NWAqjye/31IK2Q0mHOOhNJ5xVxQQACEIAABCAAAQhA4BkBBBMSYQSBtvR/fVbH+jlt68JdX8mJCCbX2JITEUy0tUAiSVtJ0vogIUIHczZhJSqY3LclR4W0Curslpy7YteKP21ZkuDTVhboMNe2ReG+mLev5NwlmOgeHTCqFR0SSXSOhrb4NAb6u4QUndOyXkmif5cvitlaMNGBtjrrY32/bC/PMHloS44OCNaqkPWWnC2CiUSIdzczbe267/K+kiOxSufeaKXU+tLXl3TWjIQniW7a/qHzci6/kiMRRduC7vrClFbb6G9adaRVMCr+JQJcbt1aP1dC2juXFUwSBLzVKN5XcnSQrkSKy6/kfMqy6uiu1Ulrf+77So5yT3mn3GnnkeiwYolyD50tkhFMxFvc19ul5BuHvo74XwHahAAEIAABCEAAAhMQQDCZIIgFu6ADPnUeiFaRqPBTYdwunSOhrSMSGbS0XmeG6MqsMNFZGjo/Y8uhrxHBRAfLartQ26IjP9vBq+0zuPq3qGDSDn1df6FEBbS2xWjLQPTQ17YNRluQJBistyXpQEwJGfpUqsQStSmhRmKHtoSsVzNo/CseLT6eYKKzXVR86/BMiV2XWxj0JRIdRCoe7Qs9Wh2g7U86r2MtmGgVhj6rrINr19d9h75qG4eKdB3eqUsrZnQOx+Whr1sEE7Wl82TEsD3ncnh5gklbxaDVPhJgdEmEUJ7rc89aTaFLK6t0/ozExXbQqeKhvNLKLAkH6xVBWqWjbVs6LFnCSfubtj/pEFYdVKztWetLY09fm9JKDfG6r0/rezzBpIl76y0vEuO0YkNfShK/h677BJMWT61EEkNdyk2JYuqrxuCl4CdeEoAiK0y0mk0+iqtEWY3rdmmsaPvbu/FZYSd6/BkCEIAABCAAAQicjACCyckCvmN3JSzo13BtXdFhkBJPVEjrqx4qyvRJ1fXhixnBRN34+GXrxPqzwjrYVAW42lIhqasdArku1h/6lVv3tFUPrRjV2Rg6t0MrQPRrtAQJbd/IrjBpnxWWUCBhQatX9FUXbTvQNpD7PiusFQW6V0KJVrfIXkKJPs+rX821jUDFpD7PqsNNxUDFcttq0w6VVR9UdOurO+KhQ2tVsLdPAnuCifzWthxdOotDW1Daf9e/qXCW+COBRoKKtpvokFFt+5HP6xioyNVWEhX7Wq2jdvQFmIc+K6xDVHWOSvussO65/KzwFsGkCXEfbGZaMbG+JGpoO436LL+VuzoTQ5c+kdzOjJFv+nf1XYW4/l1fHdKqHK2WkZDYLp0pIlFLMdDKDcVTgopyTGeptEtimkQn5e2HXpy/IRvlkgp+8W2H3UoY0LP0ZRyJKWuBQPcoTvpPu5Q3ukdx1XYrndGjeOjSGGsCmMQk9U//V3mj1TQ6Q0fnkSjv1oftXiB89l/vG3v6m1YdSWCTyNc+Iyxmn7CszNGnrsVJfxNPvWN0ls/6ENo23pVXeufo0nknWrmj+N719SE9V6vJHjqk+q6+8G8QgAAEIAABCEAAApMTQDCZPMA37p4EBhVfKoIlkmglgwoxfTFFKzzWV1YwUWGnglPCi4rCP74UTirWJRDoc7e6tggmOlRW20lUbKlY1jYUFZ8qunRlBRPdo9UD8k3nT0iAUNGqlQ067LVdTTRQ37QKRyKTClQVsFrV0Q5OVYGsbTf6stB7LiKKimT5rK+wrC+tDtGv9+0ATQk+slGx3X659wQTtde+biTe7es66+eooNc2ColXWhWis0xUROsMjbVgImFBQpqEHfWtrdi5SzBR+9pOorjqqzLasvHJS7z1lZ92SdjaIpioHfFTUX558Ot9X3fRPet+tRhLTNBBtVrZoBUe8l3tri8JISr2tWJHK3D0xSdtr5HQeHlp1Zb6eteXkPQe11hbCyCN4x1NPfun9cHI+u/tk9J32Wt1RzvfR3/X6hitltIqGvVBf1NuSfjyrocEE4mHiuGHL/+3taVPD0uUkZgk1nqmWGjFjr4QpRVr7brrs8I6T0nn6kjE03/Wq6zaF3I0Hi/HjNcX/g4BCEAAAhCAAAQgMDkBBJPJA3yy7ukXbv36ra0ndxWdJ8NBdzsIaJWBVr/oDJLLzzB3NMctxQlo+49EO60+WQspxd3GPQhAAAIQgAAEIACBPQggmOxBmWeMIKBPx+pTuutLv06r4NXWn8tPE4/wgTbnI6DVC1qpoa0/WqnDNS8BHfCrrUZavaTzcLggAAEIQAACEIAABCDwOgIIJiTEUQl89PKrsLb46NBTHRqp/+g8CG1x4IIABCAAAQhAAAIQgAAEIAABCHQTQDDpRseNNyag8zMkmuhMCx2wqXM9dDilVgWsvxpzYzd5PAQgAAEIQAACEIAABCAAAQgckQCCyRGjhs8QgAAEIAABCEAAAhCAAAQgAAEIDCWAYDIUL41DAAIQgAAEIAABCEAAAhCAAAQgcEQCCCZHjBo+QwACEIAABCAAAQhAAAIQgAAEIDCUAILJULw0DgEIQAACEIAABCAAAQhAAAIQgMARCZxBMHl6xMDgMwQgAAEIQGByAmeYg0weQroHAQhAAAIQmJvAGSYrCCZz5zC9gwAEIACBYxI4wxzkmJHBawhAAAIQgAAEnhGYfrLyyosvIpiQ7BCAAAQgAIFiBB6/9NL0c5BiyHEHAhCAAAQgAIEkgeknKwgmyYzAHAIQgAAEILADAQSTHSDzCAhAAAIQgAAENhFAMNmEj5shAAEIQAACEOghgGDSQ417IAABCEAAAhDYkwCCyQXtx+98w578edaVCLzypnelWiLOKVwYQ2ATAcbnJnyHuTkdZ7bkHCa2OAoBCEAAAhA4KwEEEwSTKXI/PVFHGJsi7nTiGAQYn8eI01Yv03FGMNmKnPshAAEIQAACEBhMAMEEwWRwiu3TfHqijmCyT2B4CgTMjPF5jjRIxxnB5ByJQS8hAAEIQAACByaAYIJgcuD0fe56eqKOYDJF3OnEMQgwPo8Rp61epuOMYLIVOfdDAAIQgAAEIDCYAIIJgsngFNun+fREHcFkn8DwFAiwwuQ0OZB+DyOYnCY36CgEIAABCEDgqAQQTBBMjpq7r/M7PVFHMJki7nTiGAQYn8eI01Yv03FGMNmKnPshAAEIQAACEBhMAMEEwWRwiu3TfHqijmCyT2B4CgRYYXKaHEi/hxFMTpMbdBQCEIAABCBwVAIIJggmR81dVphMETk6cQYC6UIaQfOQaZGOM4LJIeOM0xCAAAQgAIEzEUAwQTCZIt/TE3UKsiniTieOQYDxeYw4bfUyHWcEk63IuR8CEIAABCAAgcEEEEwQTAan2D7NpyfqCCb7BIanQIAtOafJgfR7GMHkNLlBRyEAAQhAAAJHJYBggmBy1Nx9nd/piTqCyRRxpxPHIMD4PEactnqZjjOCyVbk3A8BCEAAAhCAwGACCCYIJoNTbJ/m0xN1BJN9AsNTIMAKk9PkQPo9jGBymtygoxCAAAQgAIGjEkAwQTA5au6ywmSKyNGJMxBIF9IImodMi3ScEUwOGWechgAEIAABCJyJAIIJgskU+Z6eqFOQTRF3OnEMAozPY8Rpq5fpOCOYbEXO/RCAAAQgAAEIDCaAYIJgMjjF9mk+PVFHMNknMDwFAmzJOU0OpN/DCCanyQ06CgEIQAACEDgqAQSTjYJJdoKoxz1OFuvZZ1RrX30+eh+y/hPn2Csxm6uxVp9bZePW48/oZ1Rrn9yOZWE2l04ZZwSTWDJhBQEIQAACEIDAzQggmCCYDBczEExi47tawZQt+PaIc4wkgkmGUzbvEExidLPjJxuH0e3vEmcEk1gyYQUBCEAAAhCAwM0IIJggmCCYBIZftpjZpdgotlIJwSSQSBOsMCO3ifN9BLLvyccIJrFkwgoCEIAABCAAgZsRQDBBMEEwCQy/bCFAURmA2iEexFp9bpWNW/ZX+z1EotF9yLZPbseyMJtL2TiMbn+XOCOYxJIJKwhAAAIQgAAEbkYAwQTBBMEkMPyyxcwuxQYrTNzIZeOWLUIRTNwQPDMYHYfR7c/QhyyjXd5hCCaxAYQVBCAAAQhAAAI3I4BggmAyvJih2IiN72xBky3uR7e/R5xjJJ9bzdDn0X3Itr9LIX1CMTAbh9Hjf5c4I5hkX2nYQwACEIAABCCwMwEEk42Cyc7x4nH3EBhdbAAeAhDoJ8D47Gd3pDvTcUYwOVJ48RUCEIAABCBwSgIIJggmUyR+eqKe/AV7Ckh0AgI3IsD4vBH4nR+bjjOCyc4R4nEQgAAEIAABCGQJIJggmGRzpqR9eqKOYFIyjjg1JwHG55xxvexVOs4IJudIDHoJAQhAAAIQODABBBMEkwOn73PX0xN1BJMp4k4njkGA8XmMOG31Mh1nBJOtyLkfAhCAAAQgAIHBBBBMEEwGp9g+zacn6ggm+wSGp0Bgh6/kALkGgfR7GMGkRuDwYkYCv9XMPtDMvsqVOvepZvYeZvbeV2rvSM2o77rE8xbX9zGzX2dmX8nM/sstHOCZZQgoB/+UmX0bM2t5GXXuR5rZTzezr2lm/zN6E3avEkAwQTCZYiykJ+oIJlPEnU4cgwDj8xhx2uplOs4IJluR73n/Dzaz37J6oCbc/8TMPsXMfp6Z/cs9neFZLgEEExfR6wy+rplJmBC3f3Rx6y0Fk89rZp9tZr/HzD565dfnMbOPMDMVwV/DzP6rmf2VZSz+uQv/VSBrjL7RzL70Mm5/p5n9EjP7bzlMd1p/ITP7CWb2wiLQ/Xszkw8/28z+5sUdX8rMfpGZfQ8z+yJm9hfN7Ccuvl/Blemb2CKYKE7K7Z9vZr9yelJX7iCCCYLJlVPqNs2lJ+oIJrcJFE89JQHG5znCno4zgsmREqMJJj/LzP6hmWnyrQJMRdI/XlYeXKP4OhKTyr4imOSi873M7Pfe88v9F1ia+l+5Jq9i/d3N7PeZ2Vc0s3+2avGXmtlHmtlvN7M/Y2YSIn7EsgrlWy5ChMx132eZ2X9cVqn8OzN7fzPTeP5DZvbdruDlO83su5rZb1iEj69gZj/azL6wmX395f2gx0jkka/fwMx+sZn9GzN78+Lj+5rZ37uCL7M3sUUwEZtfaGbf18y+qpk9nR3WNfuHYIJgcs18ullb6Yk6gsnNYsWDz0eA8XmOmKfjjGBypMRogsk3NbPPvKNw+/5m9rvu6dAXXX4B36u/+uX67OLNrIKJhDoJF//vysn0kGBy5UelmvuDy6qQb7W66/OZ2X8ysz9sZt979e8qgv/BsnrgxeXftQXjYxdBc73a47eZ2ZOlba0IaZc4/HEz+w93ePnlzeybmNknrv72ntp1u6xW+cmrf9eWkT+5iDq/fPl3reD53YvPH7/825cxs79rZp9sZnqHjLr2fgeN6sdWwUTClN7fH7TEZ5Sf07WLYHIR0scU0odM8vREnTgfMs44fUwCjM9jxi3rdTrOCCZZxLe0v08w+S5LAfUzlqXezU4Te/2SqQLs85vZuy3Oq8DS9gDdp1/F/76Z6dfy37zqXCsKPmz5NfqHmNkXN7M/sfxy/U9Xtu1sjR9kZirMVNC93cx+/GKjX7D1a7e2LfxbM/v9ZiZfLwvCb75sedCv71pR8Llm9pvM7KXVs762mX2MmX3bZTuBtkr83OWX+mamvqpI/YHLL+faKvG3zeznmNkfW4xUeP4CM/v2ZqaCUb/6a2uCitz1dpDvtLT1jReB4NPM7Kfcsc1BqxDkl/oonh+1rByInmESYbTm/KvM7BuZ2b9YfrHW+Rrr68cuW0VUwGvrllj+MjPTNpB2ZfLg+y0F/4eb2RvM7JuZ2V9aVkmo8F9fH2xmf8TMPnTJy69sZj91KRB1BoiENBXyKu4b68vtZq29dk7EXVtyvuwSww8xsy9pZp+z9HHtj86P0WosPUsCh/x4vKz4EHP14aFL4pBWhmgLhfKnXVq5oX78GjP7Mat/lyig52j1xluWf3/r8lzlmVZ0tEv/Lr++xErM1BjTarG/ZWbf4UJ01Fj902b27mb21RbRSm1pTCi/1Za2+LSr/fuPWla26N+1regDzEwrUNaC169fxou2Cz10toZWqGiFm7YiyZ+/sIztT1rO81AcdW19B32xZYvhb1zG5DpGip8Y/cwl/pHx3jjpXaGcUvvazijRSO8iXZE8ld19goneX8oRvb/kk3JL76E/e0eC6T2olUlNVHPSkD+LAILJRR4gmBxzYKQn6ggmxww0Xh+SAOPzkKRv6UkAACAASURBVGFLO52OM4JJmvENb7hPMPlxi6igsxRU+DQ7FV3/eikKVMhpKfiXW37d1FJwLd/X3yUKaDm/zkD4FUv/WlHwN5Zl41otoQJVIogKjW9oZv99sVUx+15mprMePm4570HnqeiXeZ2hoHMf9Iu5/rvsVMDprAdtW/jfSxsSLvSr+buWMywkBHwdM9O5Fvqbrq+3FB/aFqGiWEKIfjHXL/9vWoQY2enX/J9mZiq2JIKoIJWI89cXBrJREaP2JDyoaFff9Bz5K1FEl7Y66Tl/dFlJoFUz8l2FosSKVuyrsNWv8+It0UkFrYpo/eqvwsw79DXKSJx1FoZWN6jw1aoA9V/bsn7oSvD64YtgpWJQApGK/vdZeLUCLZsH6ptWlbzDzL7gsoJCufF3FuFtPSzEQAKSnqH4SrBTcav4i4l4iKOEBcVXwoMEAPmmXJY4IQFAl/xXLl0KJhIs/vIiUP3qRRTRSg/lgnK0iWxNMPmri+CnnFfuS/T6H8tzWw7eNbSVo5++jI9PuDD4jEVE0jactiVHQpkKcglKWmmi6zsu+aHtNxoLKpa/xRIjsdK4W18SNJRzOlhU23Xkn3JPZxVpe43a1/hpl4pzPUt5ISFDfZUgonNKJFBJ7GvipLbc6D/f+eKZyh+NF+WJ4nrfpXeI2ImFfNTWHr0/lGNabXMpmGx5B0lQ+HZmJmHv/64ckjAkPxRbvYsi4139UozEUmKuxu5XX3zX33RF8lR2dwkmEnD1DlBOatxJjJK4KNFKOan30PpSXkvA1nuJK0gAweQCFIJJMHOKmaUn6ggmxSKIOzMTYHzOHN3nfUvHGcHkSInRhBAVESr+VaSooNOv3CqoVExLTGh2KvQ0uV8XGyqKVCyp8FLh1i5t5VHho9UDEkJaUaD2JFz858VQRamKdRW37dBCFbPfelnRIMGmXfpFXQWy/q622y/aWm2iIlerVnSIrYQWFXH69VpCzHrliebIbZ+/RBcJG9qS1H4F19/VTz3ray0P/mvLc7Xy4K5Lgoe2QFz+Ir+2ldChVTQ6U0NFaLskAmglgxi0f1eBqn8XJ61G0CXxRQWufgl/SDCJMlKbjbMO6NRqEV1aiaNf+VUg65d3FYR/YBESHvqaTjYPVJCrvSaS6dkSNn7S0ve2pUT+SODQmR8qwnVJ3Fjfp397PzP788uWlJcXu4e25FwKJso/iXtaRfQ7lvslHGgFhnJbPJSzTTBRrmt8ND8lEErAaatg7kmVZ30QK7Wp1UzrS6uJtL1FgkS7xEnjSzmyviQYabWBWLRLhb7+/a5LfomhzibRyi2typJQIvFFfby8JNBo9ZBEgHapeNcYkPjYLn3hRz632LR/l88SPNS+hJC7LuW4xrOETR0Y2y6JQBL9JC5eCiZb3kESIuWL3h1asdQuvfsUx/bFJG+86z4xk8ipHJbI0q71+yWap5eCidpQvBV7+dreV2pPW7C04kx9WV96T0qQ1XubK0gAweQCFIJJMHOKmaUn6ggmxSKIOzMTYHzOHN3nfUvHGcHkSIlx37YFFeX6lbsVOs1OhZZWBKyLA209UbHflqG3v6mwknih1QpafdGKAm1bUaG3bkMiig6xVHGlS8WsCmCt5FgfyqltHCriVIzp19d2qajWyhb9yqoiWb+yavn6eoXLZVy0VUDbGbQd4HL7ibZWaCm8BAP5Jn+0vF5Fyl2HWGqFhFY36PkqWtbnR7TnqiBUwapfji9/cVeBrhURKsAlMP1zM9P2Cq1qWV8qlrSy5yHBJMqocZZAJsFHq2vapZVFv3bZCqBVD1oNpBUeEm3u2nKiuiObByqKtZ1hfWl1gYrVH7ZsndLflEdafaBtORKMLi+JGsoT+aDYyNe2wiIjmCjXtTJAqw/WW0u0hUziXxNCmmDytmXrSPNHv+6LwVr4u8PdZ6sptJqh5dbaRgKCtt4ol7RVTdu8tA1HK2a0qmC9/UbCjv4jAUTijbbDaQWCVtRIPLzraiucJNxJANIqKq1SuetSLmqsiqlyQGKO8lErdZQHWk2jS+KpinWNmfWlPFcflPcS3O66dL6Jcl/jqm1tk53Gpvp0l2Cy5R0kAVV91/YtsdAlwUPjUauoJGTp8sa7RMl/taw6atsE7+nia//8UJ5eCiYSYrTiR33V1qT1pZg0YWSdp22blt4PZz/ryYvFa39HMLlAhWASzp1ShumJOoJJqfjhzNwEGJ9zx7f1Lh1nBJMjJUYTQrRCQ9sx/s/ya75+3VxPxpudlvZrGXq7tDrD+/Tw91x+zW5FQVsFsuakLStqS8vNW8GiL4Gsf93Wv6t4VMGgf2/bE1o7WpUh/7VaROesaCuPCjutIrnr0i/oWknx0KVf+tWu+q3VAxIWtCpAv05rFYNEnnapcNK5LSogVWDqV3OJS+3X+FYo3/c8Fck6N6OtlFhviWn3SHCRTw8JJlFGjbPOJJEYtL5asSvxRRy10kUFrcQE/bot4ULCVTtLoScPdDhpWwmyfrYKchW17Rd02UhIk5Ck+OrSL+0q3iUQyKd13SORTjmmKyOYaCuQimDFen01EUdborTyqgkm4izhY31pJYBWRqzPJrmMd8sD5bdWV7RL21+UayrWdV5MuyRcSCjTWT46L0WXRBxtvdEKqHUb6ru2VGnbzHq119oHrYxQHx/6mk47v0XijXK6XVr1Jf8kjkhQ07VlhYliqFVFEgt1Lsz6kvgkHy9XmGx5B6l9bSvS9i2JUxIW9D7R2JU41VZzeeNdZ4tojK9Flss4679H8/RSMGkH6d7VZvs3iUprYbZtbdIKk8vVVw+1c+q/IZhchB/B5JjjIT1RRzA5ZqDx+pAEGJ+HDFva6XScEUzSjG94w31nmFy6dJ+digydEaKzAS4P6mxtSFRQIZoVTN5j+fV37UtUDIgIJk2Y0KGW920ZkKDStg6pQNH5DyrkVcDrME2txGi/SstPCTlaiaHVECouVZBJfFAh3HzXr8PrLQ2tfxID2soabS2pJJjIR/1yrdUe6ru2Cajg1AoRrRTpyQNtxWpfVVnHWIKDVitJIBF75Y5WeIh1u8RcYom20IiVCl2JFRJ3JFS1InukYHLX9iv5ILFEfbjvuu9sjyZSXa62UDvaMiIWWq2lSwKjtp1pddD6aquY7hMKW1EtwUvPkWChlQmXlwQnHY4sQeZyRZVYa8WPVrfo2nKGSY9gcvlFr0zuyd+2ouQHLHmls0d0Hsj6y0Sye2i8RwWTaJ5eCiZtVZNyTCuu7rokfK3PytE5KuqTxilXkACCyQUoBJNg5hQzS0/UEUyKRRB3ZibA+Jw5us/7lo4zgsmREmOrYKKiTb9yqkj1Ph+a3ZJzl2Dy0HYTFdZaTRLdktNWRVxuEYrET+eRtFUx2lpx16WVASp2dFaEist2Vst9W0taGyO35KwZ6XkSaCJbci77py1QWu0i8UQsVLhl8+A+wUSrWXSwpwQSrV5qZ220M0fki86k0fPbShL9m87f0WoHiXdNMNGWE4ky7cs4635cnmFy35acJr5dbsnpFUzaoa8S39bbYVpuX56vIZ/FQyuQJPLp0gow8W7/vfWrrUy4qw2tTpE4oi1o+qKVVqxoZYW23qnYXl9NyFAstPKmXaovJdxovKuo16UzebRd6PIrOa2Af+grOfdtydEhx9p+dNeWnEvBJPMOav3QdhcJvRKQJDxcxuIy3y/He3RLTjRPLwUT9VEizl2xue/9xKGvkTf3hQ2CyQUQBJOOLCpwS3qijmBSIGq4cBYCjM9zRDodZwSTIyXGVsFEfdU2ABU+73vHIZYqLHS2iC7v0Nf1l0ja524vDxltB5rqDAKdY9IOQ9QSe50p0bb76KwCbR3xDn3VV0N0boWeowJqfa19VwF3ucVB57aoEJedlsFrC1M710Ht6Nk6i0RbmCQO6JwNbTXRahOtALj8ksr6edc49NVjJB8fOvRVW130H/l5V/+1tUGHxWqbkorobB7cJ5jIL61KUsEswUQrdSRKrbeIKRYSG7TKpF0SMOTTushuX5O56xyN+w59VS5rRYsubZORnbblXB762iuYtM8Kq1iXeNEujZ/PvPBff9MWLJ0bIwFCea5LKzy0QkQHx2orXbskLunwWW33Ue61q33lSCLJRy7/qFpR23q0NUpijfK5XU1oulwtI2FB55GstyM1QWkdT4mdWnkiEaoJK6vmX/t/26Gv6o+27rXroUNfLwUT3RPNvda+zrhRrui52n4jkXI9Hr3xrnYih75G8/RSMNG7Q3FVziv+EgLX1/pd0f5dz9J5MDrDhitIAMHkAlRFwSQ7CQ3GvttsBkYz9KE7gNw4NYEZcnuGPoxOshkYPUYwGZ0m12z/GoKJih5tXdEkXp9Y1a/h+lVZE319fUf/v67LzwqryNG9Ekp0DoOK0nZY4X2Cidppn8zVtgIVzfqssM5UuPyssFZyqCBS4ahnSRDRGSn69K/+pkufoNVXN1SYyHediyKf3n8p0uWTrvYZWn0lRGcr6FBZfdFGh2uqQNGXeHTApYpO9V/ba1SkSxjRihcdzKlLxbjO5JCNto9ITNJ5EzqwU+eB6JwMXSr09YWR9llhMdS5FtnPCnuM1p8V1pdOVKSpANbWD/VPTHSp39pGJB/FQisP5KvaV4GuK5sHDwkm2pKj7T4SoLQ15LIIlCii7QfiL0aKl3JNZ0ast+Rou4aYSXDQwb76EpKEJK20ue+zwtpW1T4NrdhJsLnrs8K9golYKS9VlOtTwOtLPJUzEj70/6uQV9y1okeCSvtSjop89UNFshjo/2q7lFaWaBuIBJJ2aeuYDnGW0KHtQE1k1N+1OkMrcJTP6nc7YFnP03jS+BDrduirYq6VLRIZ2wG0akNjSKKjzjzRv2s8Kq8lblx+3eeiy6YtcRLexERnA7XPCrdYNlHsoXdVNPfas5tQI0FMZ7FcHljrjXe1Iz/Vb+WUxCydwaLzbTSW9T7QFc3Tuz4rrH/TwdbKVb2/dPi0BEyJtFptpBVP7Wpim8aA3kNcQQIIJhegZpiEBmPfbTYDoxn60B1AbpyawAy5PUMfRifZDIwQTEZnyVXbv4ZgIoe0vUW/lqt4VpGqAk4HVaoIb0V3Kwr0a7YKLhVvKuZU+KlgWX+a8yHBRM/TIbUq3lTkScDQ9gx9eWf9+WDZafuDfq3W1gX9avu5iz/rr4jowEnZ6Bd7FbEqULTCQ19baUKHCnj1TWc66Is4KkAlfKhAbCsw9Gv8By2/7ksw0VYGHZipLQvrSxz0C718UlsqhLQKRT5JmGiXfnH/mOVATPmtz8XqF37d/9Chr+3+CKPGWV/jkEigr3OoWFS/dMBpuySeSKCQ2KTtCRIhxFz+qXhrVyYPHhJM9EWWdnZG+8rSmqFWtegzyCoatWJDQo6+TqMVDepT25Kje/TFHW0x0cG2Ku7b9pxLwaTlsbatqF2tCFKxr+coF9rVDn3dIphITFNuySetOmqXRAJ9VlmrMnQYrwQM5cZH3XGWhQ4tlniomClvVbCrQNfKiXY4bmtXYoYO011/Drz9TTmoZ6233uhv+uqPnisBQH5qFZG2vGmcXR7QKlvljM7vUR8kUKkfWjHjXYqJxp9EHh02qzNplLsSIzR22qoa710Vyb21LxIktUpNopWeub688d5sNR60vUk5pTzUe0GiaVs5FM3TuwQTPUPCi2Ig0U7jTqKlxGl9lUjvzXYpZ/VeVW6uBTGP/en/jmBykQIzTEJHZ/UMjGbow+g40/4xCcyQ2zP0YXT2zMAIwWR0lhy2/VYUPFQoH7ZzOA6BBAGJBFoZo+JaBTHX6wlIaNBKFomEHzsIjlbxaEuTxLkjXxK8dHCtRJOXjtyRW/iOYLJRMOnZLpOd6PY8Y2QyZf0f6UtrO8tohj7swZVnHI/ADLk9Qx9GZ84MjBBMRmfJYdtHMDls6HB8AAFtfdJ2EG1duTyjYsDjyjapFSmXn8Ft2+7uWl10jY5oq5NWg0iMeegT0Nd41ug2dDiyVv3okGltD+JKEEAwQTBJpMurplNM1Ase+poVfdKB44ZTEGB8jglztfE5RZw5w2RMsh6/VQST48eQHkDg2gS01Ub/+aRFOJJIou0lOsOlnTV0rWdq65G26Wmbls5X0Za+uz7xfa3n0U5xAggmCCbpFJ1ioo5gko47NxyDAONzTJwQTHyuWUasMPGZntQCweSkgafbEHiAgA6H1rkrOq9D58boDB2d76LtONdeedPOQdF5STpoVgfecp2YAIIJgkk6/SnI0shCN2SLjVCjGJ2OAONzTMirjc8p4swKkzHJSqsQgAAEIAABCFyNAIIJgkk6maaYqLPCJB13bjgGAcbnmDghmPhcs4xYYeIzxQICEIAABCAAgdsSQDBBMElnIAVZGlnohmyxEWoUo9MRYHyOCXm18TlFnFlhMiZZaRUCEIAABCAAgasRQDBBMEkn0xQTdVaYpOPODccgwPgcEycEE59rlhErTHymWEAAAhCAAAQgcFsCCCYIJukMpCBLIwvdkC02Qo1idDoCjM8xIa82PqeI8yQrTJ4+ffp0TNbRKgQgAAEIQAACvQQePXp0Fa3jKo30dmKP+1558cXURCY7Ce2ZRO/xjJFss/6P9KW1nY3DDH3YgyvPOB6BGXJ7hj6MzpwZGM2ywgTBZHS20z4EIAABCEAgTwDBJMgMwSQIKmE2xUSdLTmJiGN6JAKMzzHRyoqyY7x43uoUcWaFyeg0oX0IQAACEIDAaQkgmARDj2ASBJUwm2KijmCSiDimRyLA+BwTLQQTn2uWEStMfKZYQAACEIAABCDQRwDBJMgNwSQIKmFGQZaAlTDNFhuJpjE9EQHG55hgVxufU8SZFSZjkpVWIQABCEAAAhAwBJNgEiCYBEElzHom6tliI/uMau0L5+g+JEKG6YkI7JF3o59RrX2lT/YdMzrlsoxG+9PDiBUme0SFZ0AAAhCAAATOSQDBJBh3BJMgqIRZz0Q9W2xkn1GtfQSTREJhelUCo8fOHrk9ug/Z9nvEgKsG9Y7Gevow2qf0e/ikK0yePHkyOhS0P4DAO97xjlSrxDmFC2MIbCLA+NyE7zA3Z+OMYBIMLYJJEFTCrGeinp5IJ88Yqdb+HkVlImSYnohAdnxmx84euT26D9n2EUxiAyibS2ddYUIhHcunalbZiTpxrhZB/JmZAONz5ug+71s2zggmwbxAMAmCSpjtUWxkn5GeqA8WZPYoKhMhw/REBEaPnT1ye3Qfsu0jmMQGUPo9zAqTGFisShDITtQRTEqEDSdOQoDxeY5AZ+OMYBLMCwSTIKiE2R7FRvYZ6Yk6gkki4pgeicDosYNgUiMbsnHew+v0exjBZI+w8IwrEchO1BFMrgSeZiAQIMD4DECawCQbZwSTYNARTIKgEmY9E/X0RHqwoJHtQ9b/PYrKRMgwPRGBGXJ7dB+y7St9et4BI9Oupw8j/elhxJac0RGh/WsSyE7UEUyuSZ+2IPAwAcbnOTIkG2cEk2BeIJgEQSXMppioJwWZBJ5u02oFWXdHuPGmBBifY/BXG59TxJkVJmOSlVaHEMhO1BFMhoSBRiFwJwHG5zkSIxtnBJNgXiCYBEElzKaYqCOYJCKO6ZEIMD7HRAvBxOeaZcQKE58pFnUIZCfqCCZ1Yocn8xNgfM4fY/UwG2cEk2BeIJgEQSXMKMgSsBKm2WIj0TSmJyLA+BwT7Grjc4o4s8JkTLLS6hAC2Yk6gsmQMNAoBO4kwPg8R2Jk44xgEsyL0YJJ0I1NZkzUfXxZRjMUGz4VLM5IYIbcnqEPo3NvBkasMBmdJbR/TQLZiTqCyTXp0xYEHibA+DxHhmTjjGASzAsEkyCohNkUE3W25CQijumRCDA+x0QrK8qO8eJ5q1PEmRUmo9OE9q9IIDtRRzC5InyagoBDgPF5jhTJxhnBJJgXCCZBUAmzKSbqCCaJiGN6JAKMzzHRQjDxuWYZscLEZ4pFHQLZiTqCSZ3Y4cn8BBif88dYPczGGcEkmBcIJkFQCTMKsgSshGm22Eg0jemJCDA+xwS72vicIs6sMBmTrLQ6hEB2oo5gMiQMNAqBOwkwPs+RGNk4I5gE8wLBJAgqYTbFRJ0VJomIY3okAozPMdFCMPG5ZhmxwsRn2vOLmu7JFuvZSWi19ns4VetDNgbEOTZ+snGOtfrcKhu3Hn9GP6Na++R2LAuzuXTGOCOYxHLJEEyCoBJmFGQJWAnTbLGRaBrTExFgfI4JdrXxOUWcWWESStbsJJdiI4T18KIScR4T51irCCYZTrzDYrSynBBMfK4IJj6jZxYIJkFQCbMpJuqsMElEHNMjEWB8jokWgonPNcuIFSY+U1lkJ9EU0jGuFBs+p2qMesZDtg8+lddbZMdnjz+jn1Gtfd5hsSzM5tIZ44xgEsslBJMgp4wZBVmGVtw2W2zEW8byTAQYn2OiXW18ThFnVpiEkjU7yaXYCGFlhUkAU7WCDMEkELQJtuTxDiPO9xHI/u8hgkkslxBMgpwyZlNM1Flhkgk5tgciwPgcEywEE59rlhErTHymPQUixUaMazUxIFsIEOcxcY61+twqG7ds3vW8A7LPGN2HbPvkdiwLibPPCcHEZ/TMgi05QVAJMwqyBKyEabbYSDSN6YkIMD7HBLva+JwizqwwCSUrxUYIU3rrEsWGz7Uaoz3EA5/K6y2y4zPLdI8+j+5Dtn0Ek1gWZnMpG4fR7e8RZwSTWC4hmAQ5ZcymmKizwiQTcmwPRIDxOSZYCCY+1ywjVpj4TLGoQ2B0sVGnp3gCgeMRYHweL2Y9HmfjjGASpMwKkyCohBkFWQJWwjRbbCSaxvREBBifY4JdbXxOEWdWmIxJVlodQiA7Uc/+OjvEaRqFwEkIMD7PEehsnBFMgnkxg2AS7CpmEIAABCAAgZsRyIpKrDC5Wah4cAeB7EQdwaQDMrdAoJMA47MT3MFuy8YZwSQYYASTICjMIAABCEAAAhsIIJjE4FFIxzhVs8pO1IlztQjiz8wEGJ8zR/d537JxRjAJ5gWCSRAUZhCAAAQgAIENBBBMYvAopGOcqlllJ+rEuVoE8WdmAozPmaOLYDI8uggmwxHzAAhAAAIQgIAhmMSSgEI6xqmaFQVZtYjgDwT6C2new8fMnux7mBUmwTgjmARBYQYBCEAAAhDYQADBJAaPiXqMUzWr7ESdOFeLIP7MTIDxOXN0+4UxBJNgXiCYBEFhBgEIQAACENhAAMEkBo9COsapmhUFWbWI4A8E+gtp3sPHzJ7sexjBJBhnBJMgKMwgAAEIQAACGwggmMTgMVGPcapmlZ2oE+dqEcSfmQkwPmeObr8whmASzAsEkyAozCAAAQhAAAIbCCCYxOBRSMc4VbOiIKsWEfyBQH8hzXv4mNmTfQ8jmATjjGASBIUZBCAAAQhAYAMBBJMYPCbqMU7VrLITdeJcLYL4MzMBxufM0e0XxhBMgnmBYBIEhRkEIAABCEBgAwEEkxg8CukYp2pWFGTVIoI/EOgvpHkPHzN7su9hBJNgnBFMgqAwgwAEIAABCGwggGASg8dEPcapmlV2ok6cq0UQf2YmwPicObr9whiCSTAvEEyCoDCDAAQgAAEIbCCAYBKDRyEd41TNioKsWkTwBwL9hTTv4WNmT/Y9jGASjDOCSRAUZhCAAAQgAIENBBBMYvCYqMc4VbPKTtSJc7UI4s/MBBifM0e3XxhDMAnmBYJJEBRmEIAABCAAgQ0EEExi8CikY5yqWVGQVYsI/kCgv5DmPXzM7Mm+hxFMgnFGMAmCwgwCEIAABCCwgQCCSQweE/UYp2pW2Yk6ca4WQfyZmQDjc+bo9gtjCCbBvEAwCYLCDAIQgAAEILCBAIJJDB6FdIxTNSsKsmoRwR8I9BfSvIePmT3Z9zCCSTDOowWT7ARRbj9+5xuC3r9qln1GtfZn6EM2BsQ5luLZXI21+twqG7cef0Y/o1r75HYsC7O5dMo4v/TSoxjN2lZPnz59mvGQiXqGVh3b7ESdONeJHZ7MT4DxOX+M1cNsnBFMgnmBYOKDGj1RRzDxYzADoz36ECOJYJLhlB3/CCYxuggmPqfHCCY+pI4JohrNFuvZSWjI8Q1GWf83PCp8a5bRDH0Iw8HwVARmyO0Z+jA66WZghGASzBIEEx9UtmDKFgJ7FNKj+5Btn6LSz7seRrFWEUwynMjtGK0sp+x7slr7PeMz3QcEk1DyZQt1BJMQ1rRRNg4zFBtpSNxwCgIz5PYMfRidbDMwQjAJZgmCiQ8qPclNbilCMPFjMAOjPfoQI4lgkuGUHf+7FNLJd0y2D1kxY4/cHt2HbPu7xBnBJDRUs4U6gkkIa9ooG4cZio00JG44BYEZcnuGPoxOthkYIZgEswTBxAeVnUhTbPhMdyk2TlhUxsgjmGQ4Zcc/uR2jm31PZuMwuv1d4oxgEkqmbKGOYBLCmjbKxmGGYiMNiRtOQWCG3J6hD6OTbQZGCCbBLEEw8UGNnqjz66wfgxkY7dGHGEkEkwyn7PjfpZA+oRiYjQOCSSbLx9qOPvQ1W6gjmIyJdzYOMxQbY0jS6tEJzJDbM/RhdB7NwAjBJJglowWToBuYQQACEIAABKYmkBZ9WGESyodsoY5gEsKaNsrGYYZiIw2JG05BYIbcnqEPo5NtBkYIJsEsQTAJgsIMAhCAAAQgsIEAgkkMXnYSmi3UEUxicchaZeOQjXPWnx77bB96nsE98xOYIbdn6MPoTJuBEYJJMEsQTIKgMIMABCAAAQhsIIBgEoOXnYT2FLl7PCPW2z6rrP99T8ndlY3DDH3IEcL6LARmyO0Z+jA632ZghGASzBIEkyAozCAAAQhAAAIbCCCYxOBlJ6HZQl1e7PGMWG/7rLL+9z0ld1c2DjP0IUcI67MQmCG3Z+jD6HybgRGCSTBLEEyCoDCDAAQgAAEIbCCAYBKDl52EZgt1BJNYHLJW2Thk45z1p8c+24eeZ3DP/ARmyO0Z+jA6sdIiBgAAIABJREFU02ZghGASzBIEkyAozCAAAQhAAAIbCCCYxOBlJ6E9Re4ez4j1ts8q63/fU3J3ZeMwQx9yhLA+C4EZcnuGPozOtxkYIZgEswTBJAgKMwhAAAIQgMAGAggmMXjZSWi2UJcXezwj1ts+q6z/fU/J3ZWNwwx9yBHC+iwEZsjtGfowOt9mYIRgEswSBJMgKMwgAAEIQAACGwggmMTgZSeh2UL9rIJJltPoOIxuf4Y4x0YMVtUIzJDbo/uQbV8xzr7DRudFTx9G+5RlhGASjAiCSRAUZhCAAAQgAIENBBBMYvCyk9DsBHGGQjrLqKfYyD4jG4fR7c8Q59iIwaoagRlye3Qfsu33vMNG50VPH0b7lH0PI5gEI4JgEgSFGQQgAAEIQGADAQSTGLzsJDQ7QZyhkM4y6ik2ss/IxmF0+zPEOTZisKpGYIbcHt2HbPs977DRedHTh9E+Zd/DCCbBiCCYBEFhBgEIQAACENhAAMEkBi87Cc1OEGcopLOMeoqN7DOycRjd/gxxjo0YrKoRmCG3R/ch237PO2x0XvT0YbRP2fcwgkkwIggmQVCYQQACEIAABDYQQDCJwctOQrMTxBkK6SyjnmIj+4xsHEa3P0OcYyMGq2oEZsjt0X3Itt/zDhudFz19GO1T9j2MYBKMCIJJEBRmEIAABCAAgQ0EEExi8LKT0OwEcYZCOsuop9jIPiMbh9HtzxDn2IjBqhqBGXJ7dB+y7fe8w0bnRU8fRvuUfQ8jmAQjgmASBIUZBCAAAQhAYAMBBJMYvOwkNDtBnKGQzjKKkd9mlY3DDH3YRoy7ZyUwQ27P0IfR+TUDIwSTYJYgmARBYQYBCEAAAhDYQADBJAYvOwnNFuoIJrE4ZK2yccjGOetPj322Dz3P4J75CcyQ2zP0YXSmzcAIwSSYJVnBJNjsJrPH73zDpvuvfXN6klvMf/HI9uHaDO9q7+hx3oPRGZ9BXvhRr8ao5x1TrQ8l35EvvfTIz4b6Fk+fPn2a8TI7Ce0pcvd4RqbPWdus/9n2e+yzcZihDz2cuGd+AjPk9gx9GJ1pMzBCMAlmCYKJDyo7ka5WCPQUMz6V7RbVOGXjvJ0ALSCk9eVAtbHT846p1oeK4/8xgklogGQLdTWanej2PCPkfKdR1v/Ox6RuyzKaoQ8pQBifhsAMuT1DH0Yn3AyMEEyCWYJg4oPKTqSrFQI9xYxPZbtFNU7ZOG8nQAsIJn05UG3s9LxjqvWh4vhHMImNj2yhjmAS45q1ysZhhmIjywj7cxCYIbdn6MPobJuBEYJJMEsQTHxQ2Yl0tUKgp5jxqWy3qMYpG+ftBGgBwaQvB6qNnZ53TLU+VBz/CCax8ZEt1BFMYlyzVtk4zFBsZBlhfw4CM+T2DH0YnW0zMEIwCWYJgokPKjuRrlYI9BQzPpXtFtU4ZeO8nQAtIJj05UC1sdPzjqnWh4rjH8Gkb3yMuCsrBozwYd3mDBP1GfowOs60f0wCM+T2DH0YnT0zMEIwCWYJgokPKjuRrlYI9BQzPpXtFtU4ZeO8nQAtIJj05UC1sdPzjqnWh4rjH8Gkb3yMuAvBxKeaZTRDseFTweKMBGbI7Rn6MDr3ZmCEYBLMEgQTH1R2Il2tEOgpZnwq2y2qccrGeTsBWkAw6cuBamOn5x1TrQ8Vxz+CSd/4GHFXVgwY4cO6zRkm6jP0YXScaf+YBGbI7Rn6MDp7ZmCEYBLMEgQTH1R2Il2tEOgpZnwq2y2qccrGeTsBWkAw6cuBamOn5x1TrQ8Vxz+CSd/4GHEXgolPNctohmLDp4LFGQnMkNsz9GF07s3ACMEkmCUIJj6o7ES6WiHQU8z4VLZbVOOUjfN2ArSAYNKXA9XGTs87plofKo5/BJO+8THirqwYMMKHdZszTNRn6MPoONP+MQnMkNsz9GF09szACMEkmCUIJj6o7ES6WiHQU8z4VLZbVOOUjfN2ArSAYNKXA9XGTs87plofKo5/BJO+8THiLgQTn2qW0QzFhk8FizMSmCG3Z+jD6NybgRGCSTBLEEx8UNmJdLVCoKeY8alst6jGKRvn7QRoAcGkLweqjZ2ed0y1PlQc/wgmfeNjxF1ZMWCED+s2Z5ioz9CH0XGm/WMSmCG3Z+jD6OyZgRGCSTBLEEx8UNmJdLVCoKeY8alst6jGKRvn7QRoAcGkLweqjZ2ed0y1PlQc/wgmfeNjxF0IJj7VLKMZig2fChZnJDBDbs/Qh9G5NwMjBJNgliCY+KCyE+lqhUBPMeNT2W5RjVM2ztsJ0AKCSV8OVBs7Pe+Yan2oOP4RTPrGx4i7smLACB/Wbc4wUZ+hD6PjTPvHJDBDbs/Qh9HZMwMjBJNgliCY+KCyE+lqhUBPMeNT2W5RjVM2ztsJ0AKCSV8OVBs7Pe+Yan2oOP4RTPrGx4i7EEx8qllGMxQbPhUszkhghtyeoQ+jc28GRggmwSxBMPFBZSfS1QqBnmLGp7LdohqnbJy3E6AFBJO+HKg2dnreMdX6UHH8I5j0jY8Rd2XFgBE+rNucYaI+Qx9Gx5n2j0lghtyeoQ+js2cGRggmwSyZQTDJTnSzE/XR7e9RbGT7EEyfTWbV4lCR0SbAB72ZvPADV41RxXfYaEZ+lLZbIJhsZ3itFhBMfJJZRjMUGz4VLM5IYIbcnqEPo3NvBkYIJsEsQTDxQWUL6exEvWKx4VPZbpHlNDoO2fa3E6CFuwiQF35eVGNU8R02mpEfpe0WCCbbGdICBCAAAQhA4CwEssI1gkkwMxBMfFDZQjo7Ua9YbPhUtltkOY2OQ7b97QRoAcGkLweqjZ2K77DRjPoil7sLwSTHC2sIQAACEIDAmQkgmAyKPoKJDzZbSGcn6hWLDZ/Kdossp9FxyLa/nQAtIJj05UC1sVPxHTaaUV/kcnchmOR4YQ0BCEAAAhA4MwEEk0HRRzDxwWYL6exEvWKx4VPZbpHlNDoO2fa3E6AFBJO+HKg2diq+w0Yz6otc7i4EkxwvrCEAAQhAAAJnJoBgMij6CCY+2GwhnZ2oVyw2fCrbLbKcRsch2/52ArSAYNKXA9XGTsV32GhGfZHL3YVgkuOFNQQgAAEIQODMBBBMBkV/BsFkEJrXms0W0tmJ+mj/e4qZPXyqxikb5z0YnfEZ5IUf9WqMet4x1fpQcfwjmPhjAQsIQAACEIAABF4lgGAyKBMQTHyw2Yl0tUKgp5jxqWy3qMYpG+ftBGjhLgLkhZ8X1Rj1vGOq9aHi+Ecw8ccCFhCAAAQgAAEIIJgMzQEEEx9vdiJdrRDoKWZ8KtstqnHKxnk7AVpAMOnLgWpjp+cdU60PFcc/gknf+OAuCEAAAhCAwBkJsMJkUNQRTHyw2Yl0tUKgp5jxqWy3qMYpG+ftBGgBwaQvB6qNnZ53TLU+VBz/CCZ944O7IAABCEAAAmckgGAyKOoIJj7Y7ES6WiHQU8z4VLZbVOOUjfN2ArSAYNKXA9XGTs87plofKo5/BJO+8cFdEIAABCAAgTMSQDAZFHUEEx9sdiJdrRDoKWZ8KtstqnHKxnk7AVpAMOnLgWpjp+cdU60PFcc/gknf+OAuCEAAAhCAwBkJIJgMijqCiQ82O5GuVgj0FDM+le0W1Thl47ydAC0gmPTlQLWx0/OOqdaHiuMfwaRvfHAXBCAAAQhA4IwEEEwGRR3BxAebnUhXKwR6ihmfynaLapyycd5OgBYQTPpyoNrY6XnHVOtDxfGPYNI3PrgLAhCAAAQgcEYCCCaDoo5g4oPNTqSrFQI9xYxPZbtFNU7ZOG8nQAsIJn05UG3s9LxjqvWh4vhHMImNj+wEUa0+efIk1vhilX1GtfbVjaP3Ies/cY6leDZXY60+t8rGrcef0c+o1j65HcvCbC6dMc6PHj16FKP5sNVVGrmGI6PaQDDxyWYn0tUKgZ5ixqey3aIap2yctxOgBQSTvhyoNnZ63jHV+lBx/COYxMZHdpJLsRHjSrHhc6rGaA9hzKfyeovs+Mwy3aPPo/uQbZ93WCwLs7mUjcPo9veIM4JJLJcMwcQHlZ1IVysEeooZn8p2i2qcsnHeToAWEEz6cqDa2Ol5x1TrQ8Xxj2ASGx/ZSe4ek9DRE+ls+xSVY3IpG4dsrmbb3yPOMZLPrWbo8+g+ZNvnHRbLwuz4ycZhdPt7xBnBJJZLCCYBTtmJdLVCoKeYCWDZbFKNUzbOmwHQwJ0EyAs/Maox6nnHVOtDxfGPYOKPhZ4CcY9J6OiJdLb9Hk7ZZ5yx2KjGaI84x0YlgkmGU3bs8A6L0a02PivGGcEklksIJgFO2Yl0tUKgp5gJYNlsUo1TNs6bAdAAgklnDlQbOz3vmGp9qDj+EUxiA6TiJLTaRH2PQjobh9GMKCpj4ycbh1irCCYZTtmxQ27H6GZzOxuH0e3vEWcEk1guIZgEOGUn0tUKgZ5iJoBls0k1Ttk4bwZAAwgmnTlQbez0vGOq9aHi+EcwiQ2Q7CR3j0no6Il0tn0EkzG5lI1DNlez7e8R5xhJBJMMp2xe8A6L0c2On2wcRre/R5wRTGK5hGAS4JSdSFcrBHqKmQCWzSbVOGXjvBkADSCYdOZAtbHT846p1oeK4x/BpHOAcBsEIAABCEDghASyog+CSTBJOPTVB5WdSFcrBHqKGZ/KdotqnLJx3k6AFu4iQF74eVGNUc87plofKo5/BBN/LGABAQhAAAIQgMCrBBBMRmXC47c+HdX0LO1mJ9LVCoFZ4kA/IACBugR4Tw6IzStveTSg1d2bfPr0aWqekV3mvHuHeCAEIAABCECgIAEEk1FBQTBxyVIIuIgwgAAETk6A9+SABEAwGQCVJiEAAQhAAAJzEkAwGRVXBBOXLIWAiwgDCEDg5AR4Tw5IAASTAVBpEgIQgAAEIDAnAQSTUXFFMHHJUgi4iDDYmcBnvPenpp74fp/9gSl7jCGQJcB7MkssYI9gEoCECQQgAAEIQAACIoBgMioPEExcshQCLiIMdiaAYLIzcB7nEuA96SLKGyCY5JlxBwQgAAEIQOCkBBBMRgUewcQlSyHgIsJgZwIIJjsD53EuAd6TLqK8AYJJnhl3QAACEIAABE5KAMFkVOARTFyyFAIuIgx2IJAVSe5zie05OwTrhI/gPTkg6AgmA6DSJAQgAAEIQGBOAggmo+KKYOKSpRBwEWGwAwEEkx0g84huArwnu9HdfyOCyQCoNAkBCEAAAhCYkwCCyai4Ipi4ZCkEXEQY7EAAwWQHyDyimwDvyW50CCYXBJ48eTIAJk1CAAIQgAAE5iaAYDIqvggmLlkKARcRBoMIrEWS+7bSZG3WrrI9Z1DgTtgs78kBQWeFyQCoNAkBCEAAAhCYkwCCyai4Ipi4ZCkEXEQYDCKQFUMiogqCyaBgnbxZ3pMDEgDBZABUmoQABCAAAQjMSQDBZFRcEUxcshQCLiIMBhFAMBkElmavToD35NWRmiGYDIBKkxCAAAQgAIE5CSCYjIorgolLlkLARYTBIAL3nVuyXklyLZtBXaDZkxDgPTkg0AgmA6DSJAQgAAEIQGBOAggmo+KKYOKSpRBwEWEwiMC1xJBIO4O6QLMnIcB7ckCgEUwGQKVJCEAAAhCAwJwEEExGxRXBxCVLIeAiwmAQgYjQcS2bQV2g2ZMQ4D05INAIJgOg0iQEIAABCEBgTgIIJqPiOlgwyU6i1c3H73xDqrfZZ1RrX509eh+y/hPnWIq/8jmf85ph5EDXiM36yY/f671ijixW2bFDbsfwZsdPNg6j2yfOsTinrRBM0si4AQIQgAAEIHBWAggmoyKPYOKSpdhwEaUFHwQTn+mzIhTBxAU1enxm2ye33ZA9M6gm+uwR5xiZldVJBZM0p44bnjx50nHXuFuyk9xq/otMtg/jaD5vuRqnioz2iEO1Z5AXfkSqMep5x1TrQ8Xx/+jRo0d+NvgWV2nEf8wNLRBMXPjZiXS2EHhWGL/pXa4fa4PsM6q1P0PBlI1BT5wRTPxhQW7fnlFPbmfHzwxx9iN1YYFgkkYWveHoE+lq/vcUM9FYbbGrxqliwbSF71HvJS/8yFVj1POOqdaHiuMfwcQfC69aIJi4pEZP1Ck23BA8Mxgdh9Ht9/RhLZjEKOWs2JLj88rmBWKgz3QGRj19iJFZWSGYpJFFbzj6RLqa/z3FTDRWW+yqcapYMG3he9R7yQs/ctUY9bxjqvWh4vhHMPHHAoJJkFG2YMr+ctpTSGefMboP2fZ7io3sM6ox6okzgok/SKvlBbntx2wGRj19iJFBMElz6rjh6BPpav73FDMdYUvfUo1TxYIpDXWCG8gLP4jVGPW8Y6r1oeL4RzDxxwKCSZDR6IKsp5CuJgZkGfUUG9lnVGPUE2cEE3+QVssLctuP2QyMevoQI4NgkubUccPRJ9LV/O8pZjrClr6lGqeKBVMa6gQ3kBd+EKsx6nnHVOtDxfGPYOKPhV0Ek6gble1GF2SV+45vtyVw3yeDr+XVfV/VuVb7tHMeArwnB8SaLTkDoL7a5NEn0tX87ylmhgV31XA1ThULpj3iUO0Z5IUfkWqMet4x1fpQcfwjmPhjAcEkyIhCIAgKs6sTQDC5OlIaHESA9+QAsAgmA6AimIyCWrEYoGAaFe1jt0te+PGrxgjBxI9ZjwWCSZTa4ENfo25UtqMQqByduX1DMJk7vjP1jvfkgGgimAyAimAyCiqCiU+2IiPf6/ksqokBFfOiGiMEkzHjEMEkyhXBxCVFIeAiwmAQAQSTQWBp9uoEeE9eHakZgskAqAgmo6BS9PlkKzLyvZ7PopoYUDEvqjFCMBkzDhFMolwRTFxSFAIuIgwGEUAwGQSWZq9OgPfk1ZEimAxA2pqsVgxkC6Zq/vcUMwPD+1rT1Thl47wHozM+g7zwo16NUc87plofKo5/BBN/LLxqgWDikqIQcBFhMIgAgskgsDR7dQK8J6+OFMFkAFIEk3FQKxYDFEzj4n3klskLP3rVGCGY+DHrsUAwiVJDMHFJUQi4iDAYRADBZBBYmr06Ad6TV0eKYDIAKYLJOKgIJj7biox8r+ezqCYGVMyLaowQTMaMQwSTKFcEE5cUhYCLCINBBBBMBoGl2asT4D15daQIJgOQ9gom2YImW2yMbn+PYiPbh4Hhfa3panGoyGiPOFR7BnnhR6Qao4rvsNGM/Chtt0AwiTJEMHFJUQi4iDAYRADBZBBYmr06Ad6TV0eKYDIAKYJJHOoMxcDoPoxuPx4tLDMERsdtdPuZvvbaju5Dtn0Ek95IPnwfgkmUK4KJS4pCwEWEwQ4E1uLJ+332B772xGv9+w5d4BETE+A9OSC4fCVnANRXm8xO1rMrA6q1X7HYGBbcVcPV4pDNoz0YnfEZ5IUf9WqMKr7DRjPyo7TdAsEkyhDBxCVFIeAiwmAHAtcSRu5rZ4cu8IiJCfCeHBBcBJMBUBFMolBnKAZG92F0+9FYYZcjMDpuo9vP9bbPenQfsu0jmPTF0bsLwcQj1P6OYOKSohBwEWGwAwEEkx0g84huArwnu9HdfyOCyQCoCCZRqNmCpuLqidF9GN1+NFbY5QiMjtvo9nO97bMe3Yds+wgmfXH07kIw8QghmEQJGYVAGBWGEIDASQnwnhwQeASTAVARTKJQswUNgolPtiIj3+v5LEbn9uj294jI6D5k20cwGRN1BJMoV1aYuKQoBFxEGEAAAicnwHtyQAIgmAyA2ieYDHNkaThbSPcUG9X6MNoftV+NUzbOezA64zPICz/q1RjtIZj4VLZZVBz/CCbRmCKYuKQoBFxEGEAAAicnwHtyQAIgmAyAimAyCmrFYqBa0VeR0ah8qNwueeFHpxojBBM/Zj0WCCZBaq+8+OLToOluZo/f+YbdnhV5ULYQiLSJjRlxJgvuIkBenCMviLMf58cvvfTIt6pv8fTp03LzjGrFQLaQruZ/TzGzR+ZW45SN8x6MzvgM8sKPejVGPe+Yan2oOP4RTPyx8MwCwcQHhWDiM+qxoGDqoTb/PeTF/DFWD4mzH2cEE59Rr8XRJ9LV/O8pZnpjl7mvGqeKBVOG5yy25IUfyWqMet4x1fpQcfwjmPhjAcEkyuhN7wpaYpYhQMGUoXUeW/LiHLEmzn6cEUx8Rr0WR59IV/O/p5jpjV3mvmqcKhZMGZ6z2JIXfiSrMep5x1TrQ8Xxj2DijwUEkygjBJMgqZwZBVOO11msyYtzRJo4+3FGMPEZ9VocfSJdzf+eYqY3dpn7qnGqWDBleM5iS174kazGqOcdU60PFcc/gok/FhBMoowQTIKkcmYUTDleZ7EmL84RaeLsxxnBxGfUa3H0iXQ1/3uKmd7YZe6rxqliwZThOYsteeFHshqjnndMtT5UHP8IJv5YQDCJMkIwCZLKmVEw5XidxZq8OEekibMfZwQTn1GvxdEn0tX87ylmemOXua8ap4oFU4bnLLbkhR/Jaox63jHV+lBx/COY+GMBwSTKCMEkSCpnRsGU43UWa/LiHJEmzn6cEUx8Rr0WR59IV/O/p5jpjV3mvmqcKhZMGZ6z2JIXfiSrMep5x1TrQ8Xxj2DijwUEkygjBJMgqZwZBVOO11msyYtzRJo4+3FGMPEZ9VocfSJdzf+eYqY3dpn7qnGqWDBleM5iS174kazGqOcdU60PFcc/gok/FhBMoowQTIKkcmYUTDleZ7EmL84RaeLsxxnBxGfUa3H0iXQ1/3uKmd7YZe6rxqliwZThOYsteeFHshqjnndMtT5UHP8IJv5YQDCJMkIwCZLKmVEw5XidxZq8OEekibMfZwQTn1GvxdEn0tX87ylmemOXua8ap4oFU4bnLLbkhR/Jaox63jHV+lBx/COY+GMBwSTKCMEkSCpnRsGU43UWa/LiHJEmzn6cEUx8Rr0WR59IV/O/p5jpjV3mvmqcKhZMGZ6z2JIXfiSrMep5x1TrQ8Xxj2DijwUEkygjBJMgqZwZBVOO11msyYtzRJo4+3FGMPEZ9VocfSJdzf+eYqY3dpn7qnGqWDBleM5iS174kazGqOcdU60PFcc/gok/FhBMoowQTIKkcmYUTDleZ7EmL84RaeLsxxnBxGfUa3H0iXQ1/3uKmd7YZe6rxqliwZThOYsteeFHshqjnndMtT5UHP8IJv5YQDCJMkIwCZLKmVEw5XidxZq8OEekibMfZwQTn1GvxdEn0tX87ylmemOXua8ap4oFU4bnLLbkhR/Jaox63jHV+lBx/COY+GMBwSTKCMEkSCpnRsGU43UWa/LiHJEmzn6cEUx8Rr0WR59IV/O/p5jpjV3mvmqcKhZMGZ6z2JIXfiSrMep5x1TrQ8Xxj2DijwUEkygjBJMgqZwZBVOO11msyYtzRJo4+3FGMPEZ9VocfSJdzf+eYqY3dpn7qnGqWDBleM5iS174kazGqOcdU60PFcc/gok/FqYRTF5JChrZiXq2/SD605uNjkO19s8a8Oz4qRa3rP9njXO238TZJ4Zg4jPqtTj6RLqa/z3FTG/sMvdV41SxYMrwnMWWvPAjWY1RzzumWh8qjn8EE38sIJhEGSUFmWCzpzc7esGU9f+sAc8KDlmu1do/a5yz/SbOPrFZBJMXXnjhqd/bc1tkJ9LVCoFzR2/O3n/ER3xEqmNvf/vbU/YYQyBLgPdklphv//LLLz/yrXyLqzTiP+Z2Fq+8+GK5iQwT6dvlw55PPnqcs/7vybbSs6oJGtm4Zf2vxL6yL6PjMLr9PdgimOxBucYzKARqxAEvnhNAMCEbqhHgPXn9iCCYBJkimPigKJh8Rj0Wowuaau33MJrhnuz4qRa3rP8zxGyPPhBnnzKCic9oFgsKgVkiOU8/EEzmieUsPeE9ef1IIpgEmSKY+KAomHxGPRZHL5iy/vcwmuGe7PjJcq3W/gwx26MPxNmnjGDiM5rFgkJglkjO0w8Ek3liOUtPeE9eP5IIJkGmCCY+qGxB5reIhQgcvWDK+n/WqGfHT5ZrtfbPGudsv4mzTwzBxGc0iwWFwCyRnKcfCCbzxHKWnvCevH4kEUyCTGcQTIJd7TbLFmTdDzrZjdmCaTSebJyr+T+aT2/7R+ea9b+X09nuqzZ+KsYZweQ8o4JC4DyxPkpPEUyOEqnz+Ml78vqxRjAJMkUw8UFVnEj7Xte3OHrBVM3/qhHPjp9qXLP+V41DNb+Isx8RBBOf0SwWFAKzRPJ4/bhPGMl+9eZa7RyPIB7vRYD35PVJI5gEmSKY+KAomHxGPRZHL5iq+d8Tgz3uyY6falyz/u/BdIZnEGc/iggmPqNZLCgEZonk8fpxLaHjWu0cjyAe70WA9+T1SSOYBJkimPigKJh8Rj0WRy+YqvnfE4M97smOn2pcs/7vwXSGZxBnP4oIJj6jWSwoBGaJ5PH6cS2h41rtHI8gHu9FgPfk9UkjmASZIpj4oCiYfEY9FkcvmKr53xODPe7Jjp9qXLP+78F0hmcQZz+KCCY+o1ksKARmieTx+nEtoeNa7RyPIB7vRYD35PVJI5gEmSKY+KAomHxGPRZHL5iq+d8Tgz3uyY6falyz/u/BdIZnEGc/iggmPqNZLCgEZonk8fpxLaHjWu0cjyAe70WA9+T1SSOYBJkimPigKJh8Rj0WRy+YqvnfE4M97smOn2pcs/7vwXSGZxBnP4oIJj6jWSwoBGaJ5PH6cS2h41rtHI8gHu9FgPfk9UkjmASZIpj4oCiYfEY9FkcvmKr53xODPe7Jjp9qXLP+78F0hmcQZz+KCCY+I1lkJ9G658mTJ7HGF6vsM6q138OpWh+yMSDOsRT/9E//9DsNr/WVnDe+8Y0xRxarbN6R2zG82fGTjcPo9olzLM5ZKwSTIDEEEx8UBZPPqMfi6AVTNf97YrDHPdnxU41r1v89mM7wDOLsRxHBxGfUM4mmkI5xrVYwZQsy4hyLM4KJzymbe6PHDrntx2wGRj19iJF5boVgEiSGYOKDomDyGfVYHL1gquZ/Twz2uCc7fqpxzfq/B9MZnkGc/SgimPiMEExijHo4jS5+LXN4AAAgAElEQVT6RrffU2xUK4yzjHrijGDij6FqeUFu+zGbgVFPH2JkEEyynAzBxEdGweQz6rE4esFUzf+eGOxxT3b8VOOa9X8PpjM8gzj7UUQw8Rn1FIg9k9BqBdMehXT2GdUYEefY+EEw8TmR27dn1POuP+M7zI/U6y1YYRIkhmDig6Jg8hn1WBy9YKrmf08M9rgnO36qcc36vwfTGZ5BnP0oIpj4jHom0RTSMa5nLDaqFcbZGPSMBwQTfzxUywveYX7MZmDU04cYmedWCCZBYggmPigKJp9Rj8XRC6Zq/vfEYI97suOnGtes/3swneEZxNmPIoKJz6inQOyZhFYrmPYopLPPqMaIOMfGD4KJz4ncvj2jnnf9Gd9hfqReb4FgEiSGYOKDomDyGfVYHL1gquZ/Twz2uCc7fqpxzfq/B9MZnkGc/SgimPiMZrEYXZDNwol+XJ/AfZ8DvtaTsl/budZzaWc+Arwnrx9TBJMgUwQTHxQFk8+ox+LoBVM1/3tisMc92fFTjWvW/z2YzvAM4uxHEcHEZzSLBYXALJE8Xj8QTI4Xs7N6zHvy+pFHMIkyffzWp1HTHrueYqPaRLqn39xzPALZXCVPYzGGa4wTVrclkM1TeTv8HfDKWx7dlsp1nv7CCy8MnWdcx8vbtkIhcFv+Z346gsmZo3+svvOevH68EEyiTBFMoqSwm5xAtmAaXixNwhuukwRy8m5k8xTBJJ4QCCY+KwoBnxEWYwggmIzhSqvXJ8B78vpMEUyiTBFMoqSwm5xAtmBCMIklBFxjnLC6LYFsniKYxOOFYOKzohDwGWExhgCCyRiutHp9Arwnr88UwSTKFMEkSgq7yQlkCyYEk1hCwDXGCavbEsjmKYJJPF4IJj4rCgGfERZjCCCYjOFKq9cnwHvy+kwRTKJMEUyipLCbnEC2YEIwiSUEXGOcsLotgWyeIpjE44Vg4rOiEPAZYTGGAILJGK60en0CvCevzxTBJMoUwSRKCrvJCWQLJgSTWELANcYJq9sSyOYpgkk8XggmPisKAZ8RFmMIIJiM4Uqr1yfAe/L6TBFMokwRTKKksJucQLZgQjCJJQRcY5ywui2BbJ4imMTjhWDis6IQ8BlhMYYAgskYrrR6fQK8J6/PFMEkyhTBJEoKu8kJZAsmBJNYQsA1xgmr2xLI5imCSTxeCCY+KwoBnxEWYwggmIzhSqvXJ8B78vpMEUyiTBFMoqSwm5xAtmBCMIklBFxjnLC6LYFsniKYxOOFYOKzohDwGWExhgCCyRiutHp9Arwnr88UwSTKFMEkSgq7yQlkCyYEk1hCwDXGCavbEsjmKYJJPF4IJj4rCgGfERZjCCCYjOFKq9cnwHvy+kwRTKJMEUyipLCbnEC2YEIwiSUEXGOcsLotgWyeIpjE44Vg4rOiEPAZYQEBCJybAO/J68cfwSTKFMEkSgq7yQlkCyYEk1hCwDXGCavbEsjmKYJJPF4IJj4rCgGfERYQgMC5CfCevH78EUyiTBFMoqSwm5xAtmBCMIklBFxjnLC6LYFsniKYxOOFYOKzohDwGWEBAQicmwDvyevHH8EkyhTBJEoKu8kJZAsmBJNYQsA1xgmr2xLI5imCSTxeCCY+KwoBnxEWEIDAuQnwnrx+/BFMokwRTKKksJucQLZgQjCJJQRcY5ywui2BbJ4imMTjhWDis6IQ8BlhAQEInJsA78nrxx/BJMp0AsEkO9HNFrqj21eoRj+jWvs9xUa1PmTzKDokZ7M7etyy/p81t7OcsuOnWvs9cU6P7Vfe8ih9T8EbEEz8oFAI+IywgAAEzk2A9+T1449gEmWKYOKSGj1RRzBxQ/DMYHQcRrcf6+V8VqO5Vmu/p5Cu1oesmDHD+MzGoCfO6dE9iWDy9OnTp+m+D77hyZMng5+Qaz5bCORaP681cT5v7B/qOXlxjrwgzn6cHz16dJUfZq7SiO/uDS0QTFz42Yk0xYaL9JlBltPoOIxuP0ZlPqvRXKu1f9bcrhaH0e+XnjinRzeCSRpZ9AYm0lFSx7YjzseO3yjvyYtRZGu1S5z9eCCY+IxetUAwcUmNLgTkwOhnVGu/p9io1odsQeYm2qQGR49b1v+z5naWU3b8VGu/J87pIY5gkkYWvYGJdJTUse2I87HjN8p78mIU2VrtEmc/HggmPiMEkyCj0RN1BJNYIEbHYXT7sV7OZzWaa7X2ewrpan3Iihm8wwaNWwSTQWDNmEgPQ1uqYeJcKhxlnCEvyoRiqCPE2ceLYOIzQjAJMhpdzFBsxAIxOg6j24/1cj6r0VyrtY9gEsvhrCgzQ5xjZFZWCCZpZNEbmEhHSR3bjjgfO36jvCcvRpGt1S5x9uOBYOIzmkYwiXYVOwg8RGB0QXZN+p/45ONSzX3IOz4sZX9N4yNxvWa/aetYBLJ52iOMpYkgmKSRRW9gIh0ldWw74nzs+I3ynrwYRbZWu8TZjweCic8IwSTKCLtTEMgWTNlfyK8JEcHkmjRpCwL5c6QQTOJZw1dyfFZ8Jcdn1GNBwdRDbf57yIv5Y6weEmc/zggmPiMEkygj7E5BAMFkTJiPxHUMAVo9AoFsniKYxKOKYOKzQjDxGfVYUDD1UJv/HvJi/hgjmMRijGAS4zTFV3KiXcUOAg8RyBZMe68wya4qua+ve2/Pqc6VUQEBEcjmKYJJPG8QTHxWCCY+ox4LCuMeavPfQ17MH2MEk1iMEUxinBBMopywm55AtmBCMImlRHWusV5gNTuBbJ4imMQzAsHEZ4Vg4jPqsaAw7qE2/z3kxfwxRjCJxRjBJMYJwSTKCbvpCWQLJgSTWEpU5xrrBVazE8jmKYJJPCMQTHxWCCY+ox4LCuMeavPfQ17MH2MEk1iMEUxinBBMopywm55AtmDaQzBZb8O5bytN1mYdyD2251TkOn0y08E0gWyeIpjEESOY+KwQTHxGPRYUxj3U5r+HvJg/xggmsRgjmMQ4IZhEOWE3PYFswYRgEkuJilxjnmN1JgLZPEUwiWfHDIJJVtDIFmTZ9uP0z205Og7V2j9rtLPjp1rcsv6fNc7ZfhNnnxiCic/oVYvHb30aNe2xKzkJ7ekI90xPIJurCCaxlKjINeY5VmcikM1TBJN4diCY+KwomHxGPRZHL5iy/vcwmuGe7PjJcq3W/gwx26MPxNmnjGDiM0IwiTLC7hQEsgXT3oLJOgjrrTT3fT0nazMqyBW5juor7R6XQDZPEUzisUYw8VllCzK/RSxE4OgFU9b/s0Y9O36yXKu1f9Y4Z/tNnH1iCCY+IwSTKCPsTkEgWzAhmMTSoiLXmOdYnYlANk8RTOLZgWDis8oWZH6LWCCYnCcHsuOHQvocuUGc/TgjmPiMEEyijLA7BYFswYRgEkuLilxjnmN1JgLZPEUwiWcHgonPKlvw+S1igWBynhzIjh8K6XPkBnH244xg4jNCMIkywu4UBLIF096CCV/JOUUa0skbEciOfwSTeKAQTHxW2YLPbxELBJPz5EB2/FBInyM3iLMfZwQTnxGCSZQRdqcgkC2YEExiaVGRa8xzrM5EIJunCCbx7JhBMIn3ts8yW/D1PeV8d2ULptGEsnGu5v9oPr3tH51r1v9eTme7r9r4qRhnBJPoqOArOVFS2E1OIFswIZjEEqIi15jnWJ2JQDZPEUzi2YFg4rOqOJH2va5vcfSCqZr/VSOeHT/VuGb9rxqHan4RZz8iCCY+o2cWr7z44tDPCgfdeJ3ZHoVoxq+eiXSm/bPaHj3Oe/h/3xdwrpUz923zuVb7z94xb3pXqrk9uGYcyvqfafvMtsTZj/7jl1565FvVt0Aw8WNEweQz6rE4esFUzf+eGOxxT3b8VOOa9X8PpjM8gzj7UUQw8RkhmEQZJQu+YLOnNzt6wbSH/wgmtx8mCCZjYrDH+Ml4XjHOCCaZCOZsmUjneB3V+uhxruZ/1TzICg7VuGb9rxqHan4RZz8iCCY+IwSTKCMEkyCpnNnRC6Y9/EcwyeXUCOuKhfSIfu7d5h7jJ9OninFGMMlEMGfLRDrH66jWR49zNf+r5kFWcKjGNet/1ThU84s4+xFBMPEZIZhEGSGYBEnlzI5eMO3hP4JJLqdGWFcspEf0c+829xg/mT5VjDOCSSaCOVsm0jleR7U+epyr+V81D7KCQzWuWf+rxqGaX8TZjwiCic8IwSTKCMEkSCpndvSCaQ//EUxyOTXCumIhPaKfe7e5x/jJ9KlinBFMMhHM2TKRzvE6qvXR41zN/6p5kBUcqnHN+l81DtX8Is5+RBBMfEYIJlFGCCZBUjmzoxdMe/iPYJLLqRHWFQvpEf3cu809xk+mTxXjjGCSiWDOlol0jtdRrY8e52r+V82DrOBQjWvW/6pxqOYXcfYjgmDiM0IwiTJCMAmSypkdvWDaw38Ek1xOjbCuWEiP6Ofebe4xfjJ9qhhnBJNMBHO2TKRzvI5qffQ4V/O/ah5kBYdqXLP+V41DNb+Isx8RBBOfEYJJlBGCSZBUzuzoBdMe/iOY5HJqhHXFQnpEP/duc4/xk+lTxTgjmGQimLNlIp3jdVTro8e5mv9V8yArOFTjmvW/ahyq+UWc/YggmPiMEEyijBBMgqRyZkcvmPbwH8Ekl1MjrCsW0iP6uXebe4yfTJ8qxhnBJBPBnC0T6Ryvo1ofPc7V/K+aB1nBoRrXrP9V41DNL+LsRwTBxGeEYBJlhGASJJUzO3rBtIf/CCa5nBphXbGQHtHPvdvcY/xk+lQxzggmmQjmbJlI53gd1froca7mf9U8yAoO1bhm/a8ah2p+EWc/IggmPiMEkygjBJMgqZzZ0QumPfxHMMnl1AjrioX0iH7u3eYe4yfTp4pxRjDJRDBny0Q6x+uo1kePczX/q+ZBVnCoxjXrf9U4VPOLOPsRQTDxGSGYRBkhmARJ5cyOXjDt4T+CSS6nRlhXLKRH9HPvNvcYP5k+VYwzgkkmgjlbJtI5Xke1Pnqcq/lfNQ+ygkM1rln/q8ahml/E2Y8IgonPCMEkygjBJEgqZ3b0gmkP/xFMcjk1wrpiIT2in3u3ucf4yfSpYpwRTDIRzNkykc7xOqr10eNczf+qeZAVHKpxzfpfNQ7V/CLOfkQQTHxGCCZRRggmQVI5s6MXTHv7vxZPPuQdH/Ya7Gv9ey56cetsIbo3V68nWf+99vj7qwSIs58JCCY+o14LJtK95I5139HjXM3/qtHPCg7VuGb9rxqHan4RZz8iCCY+IwSTKCMEkyCpnNnRC6a9/b+WMHJfO7noxa2zgsPeXL2eZP332uPvCCbRHEAwiZLK2zGRzjM74h1Hj3M1/6vmQFZwqMY163/VOFTzizj7EUEw8RkhmEQZIZgESeXMjl4Y7+0/gkkuv65ljWByLZKvb2fv8eP1omKcEUy8qPX/nYl0P7sj3Xn0OFfzv2rss4JDNa5Z/6vGoZpfxNmPCIKJz2gawSQ70c1O1LPtB9Gf3mx0HKq1f9aAZ8dPtbhl/T9rnLP9Js4+MQQTn1GvBRPpXnLHuu/oca7mf9XoZwWHalyz/leNQzW/iLMfEQQTnxGCSZQRK0yCpHJmRy+Ysv7n6MxjnRUcslyrtT9P5Mb2hDj7fGcRTF544YWnfm/7LXqKjWoT6f7ec+eRCGRzlTyNRReuMU5Y3ZZANk/l7eh3wMsvv/zoGlSu0sg1HBnVxisvvjh0ItPjNxPpHmrHu+focc76f7wIXcfjaoJGNm5Z/69Dbf5WRsdhdPt7RAjBJEa54iQ05jlWZyOQzdXRxdIs/OE6SyTn7kc2TxFMCuUDgokfDAomn1GPxeiCplr7PYxmuCc7fqrFLev/DDHbow/E2aeMYOIzkkXFSWjMc6zORiCbqwgmsQyBa4wTVrclkM1TBJPbxut1T0cw8YNBweQz6rE4esGU9b+H0Qz3ZMdPlmu19meI2R59IM4+ZQQTnxGCSYwRVjUIZAsmBJNY3OAa44TVbQlk8xTB5LbxQjB55xtSEcgWZKnGT2x89IIp6/9ZQ50dP1mu1do/a5yz/SbOPjEEE58RgkmMEVY1CGQLJgSTWNzgGuOE1W0JZPMUweS28ZpOMBmNM1uQjfZnlvazBdPofhPn0YRj7ZMXMU5HtyLOfgQRTHxGCCYxRljVIJAtmBBMYnGDa4wTVrclkM1TBJPbxgvBJMmfQjoJLGhOwRQEdTIz8uIcASfOfpwRTHxGCCYxRljVIJAtmBBMYnGDa4wTVrclkM1TBJPbxgvBJMkfwSQJLGhOwRQEdTIz8uIcASfOfpwRTHxGCCYxRljVIJAtmBBMYnGDa4wTVrclkM1TBJPbxgvBJMkfwSQJLGhOwRQEdTIz8uIcASfOfpwRTHxGCCYxRljVIJAtmBBMYnGDa4wTVrclkM1TBJPbxgvBJMkfwSQJLGhOwRQEdTIz8uIcASfOfpwRTHxGCCYxRljVIJAtmBBMYnGDa4wTVrclkM1TBJPbxgvBJMkfwSQJLGhOwRQEdTIz8uIcASfOfpwRTHxGCCYxRljVIJAtmBBMYnGDa4wTVrclkM1TBJPbxgvBJMkfwSQJLGhOwRQEdTIz8uIcASfOfpwRTHxGCCYxRljVIJAtmBBMYnGDa4wTVrclkM1TBJPbxgvBJMkfwSQJLGhOwRQEdTIz8uIcASfOfpwRTHxGewkm2YluttAd3X4Pp2p9yDLqKTayz6jGKDZi5rM6etyy/p81t7Ocqo3PrP89cc6O7pdffvlR9p677K/SyDUcGdXGKy+++HRU273tMpHuJXes+4jzseK1l7fkxV6kb/sc4uzzRzDxGfUIAT2T0OxEt9pEvYdTtT5kY3DWOMdGzVxW2dwgt/34V2PEO8yPWY8FgkmQGoKJD4oVJj6jHgsKph5q899DXswfY/WQOPtxRjDxGfVMos9aSFNU+vl0dEZ+D+e0OHrcsv7zDovlcTXRZ484x8g8t0IwCRJDMPFBIZj4jHosKJh6qM1/D3kxf4wRTGIxRjCJcdpjEpp9RrWJeo+wVK0P2RictaiMjZq5rLK5QW778a/GiHeYH7MeCwSTIDUEEx8UgonPqMeCwriH2vz3kBfzxxjBJBZjBJMYp2yxdNZCOsupWsGU9f+scY6NmrmssrlBbvvxr8YIwcSPWY8FgkmQGoKJDwrBxGfUY0Fh3ENt/nvIi/ljjGASizGCSYxTtlg6ayGd5VStYMr6f9Y4x0bNXFbZ3CC3/fhXY4Rg4sesxwLBJEgNwcQHhWDiM+qxoDDuoTb/PeTF/DFGMInFGMEkxilbLPUU0jFPsILAwwSyuZotWq/J/81vfnOqube97W0p+2saH4nrNftNW8cikM3TPf63CsEkmEMIJj4oBBOfUY8FhXEPtfnvIS/mjzGCSSzGCCYxThUnoTHPsTobgWyuIpjEMuRIXGM9wmpGAtk8RTCplAWP31rus8KV8MiXrGBSreCrxhN/IACB+QjwnhwQ01fe8mhAq7s3+cILLwydZ1SchO4OmQcegkA2VxFMYmE9EtdYj7CakUA2TxFMKmUBgokbDQoBFxEGEIDAyQnwnhyQAAgmIagVJ6EhxzE6HYFsriKYxFLkSFxjPcJqRgLZPEUwqZQFCCZuNCgEXEQYQAACJyfAe3JAAiCYhKBWnISGHMfodASyuYpgEkuRI3GN9QirGQlk8xTBpFIWIJi40aAQcBFhsDOBz3rrp7lPfJ+3fMBrNmv79b+7jWAAgSAB3pNBUBkzBJMQrYqT0JDjGJ2OQDZXEUxiKXIkrrEeYTUjgWyeIphUygIEEzcaFAIuIgx2JoBgsjNwHucS4D3pIsobIJiEmFWchIYcx+h0BLK5uodgct/XcLJfvblWOz1JUZFrTz+4Z24C2TxFMKmUDwgmbjQoBFxEGOxMAMFkZ+A8ziXAe9JFlDdAMAkxqzgJDTmO0ekIZHMVwSSWIhW5xjzH6kwEsnmKYFIpOxBM3GhQCLiIMNiBQEQkuc8NtufsEKCTP4L35IAEQDAJQa04CQ05jtHpCGRzFcEkliIVucY8x+pMBLJ5imBSKTsQTNxoUAi4iDDYgQCCyQ6QeUQ3Ad6T3ejuvxHBJAS14iQ05DhGpyOQzVUEk1iKVOQa8xyrMxHI5imCSaXsQDBxo0Eh4CLCYAcCCCY7QOYR3QR4T3ajQzDZiK7iJHRjl7h9UgLZXEUwiSVCRa4xz7E6E4FsniKYVMoOBBM3GhQCLiIMdiAQEUzu23qzdo+v5OwQrBM+gvfkgKCzwiQEteIkNOQ4RqcjkM1VBJNYilTkGvMcqzMRyOYpgkml7EAwcaNBIeAiwmAHAggmO0DmEd0EeE92o7v/RgSTENSKk9CQ4xidjkA2VxFMYilSkWvMc6zORCCbpwgmlbIDwcSNBoWAiwiDHQggmOwAmUd0E+A92Y0OwWQjuoqT0I1d4vZJCWRzFcEklggVucY8x+pMBLJ5imBSKTsQTNxoUAi4iDC4IYGIkLJ2jy05NwzWxI/mPTkguKwwCUGtOAkNOY7R6QhkcxXBJJYiFbnGPMfqTASyeYpgUik7EEzcaFAIuIgwuCEBBJMbwufRrxHgPTkgGRBMQlArTkJDjmN0OgLZXEUwiaVIRa4xz7E6E4FsniKYVMoOBBM3GhQCLiIMbkgAweSG8Hk0gsnIHEAwCdGtOAkNOY7R6QhkcxXBJJYiFbnGPMfqTASyeYpgUik7BgsmWbFBaB6/8w0pQtlnVGtfnT16H7L+E+dYimdzdd3qfULKektONm49/ox+RrX2ye0xuT1DnGNkVlYIJiFkFSehIccxOh2BbK4imMRSpCLXmOdYnYlANk8RTCplB4KJG43RE3UEEzcEzwxGx2F0+3v0AcHEz6Ws6JPNCwQTPwYzMOrpQ4zMfILJ06dPn6b7PviGPQrRTBd6JtKZ9s9qe/Q47+H/m9/85qHp8ba3vW1o+2o8O3724JrpdNb/TNtntiXOfvQfPXr0yLfyLa7SiP+YG1ogmLjwswVTtiDbo5Ae3Yds+z3FRvYZ2TiMbn+POCOYuMN5+Ao2ctuPwQyMevoQI4NgkubUcQMT6Q5oB7zl6HHew38Ek9snNoLJmBjsMX4ynleMM4JJNIIIJi6pGQrp0X3Itt9TbGSfgWDyaXfmNltyxm75I7fdV+ozg2rjM/t+6elDjAyCSZpTxw1MpDugHfCWo8d5D/8RTG6f2BUL6dtT2e7BHuMn42XFOCOYRCOIYOKSyk6ks4WAHBj9jGrt9xQb1fpQMc7rZOYMk7uHdjZu2bwjt91XKoJJDJHZJGeYsCXHD3jFibTvdX2LoxdMe/iPYHL7PGb8j4nBHuMn43nFOCOYRCOIYOKSyhZM2YIMwcQNwTOD0XEY3f4efUAw8XMpOz6zeYFg4sdgBkY9fYiRWVkhmKSRRW9gIh0ldWy7o8d5D/8RTG6f4xUL6dtT2e7BHuMn42XFOCOYRCM4WDCJulHZLlswZQuyyn3Ht/oE+Kxw/RidwUPekwOijGAyAOqrTTKRHoa2VMNHj/Me/iOY3D5lKxbSt6ey3YM9xk/Gy4pxRjCJRhDBxCVFIeAiwuCGBBBMbgifR79GgPfkgGRAMBkAFcFkGNSCDR+9YNrDfwST2yduxUL69lS2e7DH+Ml4WTHOCCbRCCKYuKQoBFxEGNyQAILJDeHzaASTkTmAYDKMLhPpYWhLNXz0OO/hP4LJ7VO2YiF9eyrbPdhj/GS8rBhnBJNoBBFMXFIIJi4iDG5IAMHkhvB5NILJyBxAMBlGl4n0MLSlGj56nPfwH8Hk9ilbsZC+PZXtHuwxfjJeVowzgkk0gggmLikEExcRBjsQyAoj97m0/qzwDm7ziJMQ4D05INAIJgOgvtokE+lhaEs1fPQ47+E/gsntU7ZiIX17Kts92GP8ZLysGGcEk2gEEUxcUhQCLiIMdiCAYLIDZB7RTYD3ZDe6+29EMBkAFcFkGNSCDR+9YNrDfwST2yduxUL69lS2e7DH+Ml4WTHOCCbRCCKYuKQoBFxEGOxAAMFkB8g8opsA78ludAgmA9B5TTKR9gjN8fejx3kP/xFMbp/rFQvp21PZ7sEe4yfjZcU4I5hEI4hg4pKiEHARYbADgS2CCdtwdgjQyR/Be3JAArDCZADUV5tkIj0MbamGjx7nPfxHMLl9ylYspG9PZbsHe4yfjJcV44xgEo0ggolLikLARYTBDgQQTHaAzCO6CfCe7EZ3/40IJgOg9gkm2YludqKebX8YmMkaHh2Hau1PFr5wd7Ljp1rcsv6HwZzckDj7CYBg4jN61QLBxCVFIeAiwmAHAggmO0DmEd0EeE92o0MwGYDOa5KJtEdojr8fPc5Z/+eIWr4XWcEhy7Va+3lC57yDOPtxRzDxGSGYBBlRCARBYbYbgYh4wjac3cLBg8yM9+SANGCFyQCorzbJRHoY2lINHz3OWf9Lwd/RmWqCRjZuWf93RHvoR42Ow+j294CPYBKlzAoTlxSFgIsIg50JIJjsDJzHuQR4T7qI8gYIJnlmwTtGT3RHtx/s5unNRsehWvtnDXhWcKgWt6z/Z41ztt/E2SeGYOIzetUCwcQlRSHgIsJgZwIIJjsD53EuAd6TLqK8AYJJnlnwDibSQVAHNzt6nLP+Hzxc3e5nBYcs12rtd4M62Y3E2Q84gonPCMEkyIhCIAgKMwhA4LQEeE8OCD2CyQCorzbJRHoY2lINHz3OWf9Lwd/RmWqCRjZuWf93RHvoR42Ow+j294CPYBKlzAoTlxSFgIsIAwhA4OQEeE8OSAAEkwFQ+wSTYY4sDVMwjSGcLWjGePG8VeI8mnCsffIixunoVsTZjyCCic/oVQsEE5cUhYCLCAMIQODkBHhPDkgABJMBUBFMhkEt2DAFU8GgFHCJvCgQhB1cIM4+ZAQTn9Ezi1defPFp0HQ3s8fvfMNuz4o8aIZCgD5EIj3eJhuH8R7lnlBtbD57h73pXalO0IcUrrDx0eOQ9T8MZoPh45deerTh9jK3Pn36tNw8gwlRAOcAACAASURBVIl0mfQY6ghxHor3sI2TF4cNXcpx4uzjQjDxGSGYRBlRkAVJjTXLFjQzFMZjieZbn4EpfcjHPXLH0cdn1v8Ik602CCZbCd5/PxPpcWwrtUycK0Wjji/kRZ1YjPSEOPt0EUx8RggmUUYIJkFSY82yBc0MhfFYovnWZ2BKH/Jxj9xx9PGZ9T/CZKsNgslWgggm4wgeo2UKpmPEaW8vyYu9id/mecTZ545g4jNCMIkyQjAJkhprli1oZiiMxxLNtz4DU/qQj3vkjqOPz6z/ESZbbRBMthJEMBlH8BgtUzAdI057e0le7E38Ns8jzj53BBOfEYJJlBGCSZDUWLNsQTNDYTyWaL71GZjSh3zcI3ccfXxm/Y8w2WqDYLKVIILJOILHaJmC6Rhx2ttL8mJv4rd5HnH2uSOY+IwQTKKMEEyCpMaaZQuaGQrjsUTzrc/AlD7k4x654+jjM+t/hMlWGwSTrQQRTMYRPEbLFEzHiNPeXpIXexO/zfOIs88dwcRnhGASZYRgEiQ11ixb0MxQGI8lmm99Bqb0IR/3yB1HH59Z/yNMttogmGwliGAyjuAxWqZgOkac9vaSvNib+G2eR5x97ggmPiMEkygjBJMgqbFm2YJmhsJ4LNF86zMwpQ/5uEfuOPr4zPofYbLVBsFkK0EEk3EEj9EyBdMx4rS3l+TF3sRv8zzi7HNHMPEZIZhEGSGYBEmNNcsWNDMUxmOJ5lufgSl9yMc9csfRx2fW/wiTrTYIJlsJIpiMI3iMlimYjhGnvb0kL/YmfpvnEWefO4KJzwjBJMoIwSRIaqxZtqCZoTAeSzTf+gxM6UM+7pE7jj4+s/5HmGy1QTDZShDBZBzBY7RMwXSMOO3tJXmxN/HbPI84+9wRTHxGCCZRRggmQVJjzbIFzQyF8Vii+dZnYEof8nGP3HH08Zn1P8Jkqw2CyVaCCCbjCB6jZQqmY8Rpby/Ji72J3+Z5xNnnjmDiM0IwiTJCMAmSGmuWLWhmKIzHEs23PgNT+pCPe+SOo4/PrP8RJlttEEy2EkQwGUfwGC1TMB0jTnt7SV7sTfw2zyPOPncEE58RgkmUEYJJkNRYs2xBM0NhPJZovvUZmNKHfNwjdxx9fGb9jzDZaoNgspUggsk4gsdomYLpGHHa20vyYm/it3kecfa5I5j4jBBMoowQTIKkxpplC5oZCuOxRPOtz8CUPuTjHrnj6OMz63+EyVYbBJOtBBFMxhE8RssUTMeI095ekhd7E7/N84izzx3BxGeEYBJlhGASJDXWLFvQzFAYjyWab30GpvQhH/fIHUcfn1n/I0y22iCYbCWIYDKO4DFapmA6Rpz29pK82Jv4bZ5HnH3uCCY+o2kEk+xEN1swjW5fgRj9jGrtq8/V4jCaUU+cg8N4N7NszHr6nH3G6Lhl2ye3Y+k4Q5xjPe23QjDpZ+fdyUTaIzTH34nzHHG8di/Ii2sTrdkecfbjgmDiM0IwiTLaYYVJtiibodg4eh+y/veIB8EU3c1sjz5nn1Ft7CCYxNJxhjjHetpvNYtg8sILLzztp3COO9/xjnekOlqtEEg5j/EhCLz88stX8fOFF164Sjs0AgHek9fPgZdffvnRNVq9SiPXcGRUG6+8+GK5iczRJ9JZ/3sK6ewzKCr9ETSaUU+cfa/3tcjmXU+fs88YHbds+wgmsZycIc6xnvZbIZj0szvanRQCR4vY/P4imMwf46P1kPfk9SOGYBJkimDig8oWTNlCgKLSj8EMjHr6ECOznxW5HWOd5TT6HTO6/Z7cPjqjWCZss0Iw2cbvSHdTCBwpWv+/vTvI0SQ5rjxedQHtCfQpBMxSSwHaa83WEaijqI+gA3A/gJZaDsBTENBeF8hBVpOs6mZnhT0P9/gsPH67wYy5udn/WcaYP2Z2PaNWhskzdL5Tl76T89VimBSZMkyOQXlszGf0nvHuD6a0/pFH5TH5ayOu6Dm9Y/XPZ5rfbNdmcgeda52ORzFMxtnd7aSHwN0U279ehsn+Gt+tQ9/J+YoxTIpMGSbHoNIHU/oQGHlIp3es7iHN71F5PHcdI9K5M9s1FdOfn1SH1fmfqnNN3fEohsk4u7ud9BC4m2L718sw2V/ju3XoOzlfMYZJkekOhkmx1eGwKx4bw8UVD+qhCGpxWKrD4nLi9OlDPb5g4EDKVA8DkAtH7q5DWn8ByekQhslphLdJ4CFwG6keUyjD5DFS36ZR38n5UjFMikwZJseg0kXag+yY6UjEE3UY4bTyjNleQ/eJs91tllIN1kzCL7MyTK6g3OMOD4EeOqjiK4GPDJOP/tWbNB5rBFICvpMpseN4hskxoy8RDJNjUOki3e0h8EXnC/5p5GOS5yKe2MM5YvNPm+35TP18rmGaZk2/L2n+kXiGyQi1e57xELinbjtXnRogafzO7PS2hoDv5HyuDJMiU4bJMah0kfaoPGY6EvFEHUY4rTxjttfQfeJsd5ulVIM1k/DLrAyTKyj3uMNDoIcOqvhKIDVA0nisEUgJ+E6mxI7jGSbHjL5EMEyOQaWLdLeHwBed/YbJsdAXRKQ6XFBSdIXZjnCVg9O5oEMZbTkw1aCc+EQgw+QEvJsd9RC4mWAPKDc1QNL4ByDU4mQCvpOTgX769IlhUmTKMDkGlS7SHjPHTEcinqjDCKeVZ8z2GrpPnO1us5RqsGYSfpmVYXIF5R53eAj00EEVXwmkBkgajzUCKQHfyZTYcTzD5JjRlwiGyTGodJHu9hD4orPfMDkW+oKIVIcLSoquMNsRrnJwOhd0KKMtB6YalBOfCGSYnIB3s6MeAjcT7AHlpgZIGv8AhFqcTMB3cjJQv2FSB8owOWaVLtIeM8dMRyKeqMMIp5VnzPYauk+c7W6zlGqwZhJ+mZVhUqOcLtHvWX/88cda8r9EpXd0y//ext17SOunc23E01n9KOtHhsnb21utkL9EjdSTzkZ6R7f8Zrs2Uk/UuUbma5TfMCkSY5gcg0oX6W4PgfcO9XCs8xURqQ5X1JTcYbYTWvXYdC7oUGdbjUw1qOY9E8cwqdFLHzMeGzWuT3xspLPUjdEVxhjD5PjnZ/Vc+IYda7ADo5EeamQYJiknf5JTIJYu0h4zBagDIU/UYQDT0iNmew3eJ852t1lKNVgzCb/MyjCpUU4fuSNLaHrH6gdTmv+Kh3Q3RnSu/fyMzNJvZfYbJl+ppEzTnx2zvWa2Ux066lwjwzBJOTFMCsTSRbrbQ+C9RT0UhL4gJNXhgpKiK8x2hKscnM4FHcpoy4GpBuXEJwIZJjV46ZLrsVHjuvoxsDo/ndfo/FFWhgnDpDZxP0elP//vZ9JvfXpHt/yjnBId/ElOkZY/yTkGlS7SHjPHTEcinqjDCKeVZ8z2GrpPnO1us5RqsGYSfpmVYVKjnC65I0toeke3Rd1jY80sPVFnhsnxLK2eC9+wYw12YDTSQ43M1yiGSZEYw+QYVLpId3sIvHeoh2Odr4hIdbiipuQOs53Qqsemc0GHOttqZKpBNe+ZOIZJjV5qZowsoekdqx9MaX6GyZpZSnVYPUdX6MwwOZ6l1XPhG3aswQ6MRnqokWGYpJz8SU6BWLpIe8wUoA6EPFGHAUxLj5jtNXifONvdZinVYM0k/DIrw+QKyj3uuOIh3aNTVdydwEd/evNRX7///e/v3rL6mxDwnZwvhN8wKTL1GybHoNJFuttD4L1DPRzrfEVEqsMVNSV3mO2EVj02nQs61NlWI1MNqnnPxDFMztC711kPgXvp9eRqGSZPVv+1vftOzufPMCkyTQ2Tjot6sdVHh6WPgY4679DDo4dwUfM7zMUOPSySd6u0sc4//fR5BwC///3v33boY2UPHgIr6co9kwDDZCZNuRICvpMJrVosw6TGKf6TnI4P6WKrjw6LF/U//q4drx16aAd1g4J2mIsdethglJa3EOvMMFmuSZcLPAS6KKGOIwIMkyNC/t9XEfCdnE+WYVJk6jdMiqBuHhYv6gyTmyv+nPLN9nO0vnun8awyTO4uebl+D4EyKoEvJsAwebEAD77ed3K++AyTIlOGSRHUzcPiRZ1hcnPFn1O+2X6O1nfvNJ5VhsndJS/X7yFQRiXwxQQYJi8W4MHX+07OF59hUmTKMCmCunlYvKgzTG6u+HPKN9vP0fruncazyjC5u+Tl+j0EyqgEXkSAMXIRaNeUCfhOllGVAxkmRVQMkyKom4fFizrD5OaKP6d8s/0cre/eaTyrDJO7S16u30OgjErgRQQYJheBdk2ZgO9kGVU5kGFSRMUwKYK6eVi8qDNMbq74c8o328/R+u6dxrPKMLm75OX6PQTKqAReRIBhchFo15QJ+E6WUZUDGSZFVAyTIqibh8WLOsPk5oo/p3yz/Ryt795pPKsMk7tLXq7fQ6CMSuBFBBgmF4F2TZmA72QZVTmQYVJExTApgrp5WLyoM0xurvhzyjfbz9H67p3Gs8owubvk5fo9BMqoBF5EgGFyEWjXlAn4TpZRlQMZJkVUDJMiqJuHxYs6w+Tmij+nfLP9HK3v3mk8qwyTu0tert9DoIxK4EUEGCYXgXZNmYDvZBlVOZBhUkTFMCmCunlYvKgzTG6u+HPKN9vP0fruncazyjC5u+Tl+j0EyqgEXkSAYXIRaNeUCfhOllGVAxkmRVQMkyKom4fFizrD5OaKP6d8s/0cre/eaTyrDJO7S16u30OgjEogAgg8lIDv5HzhGSZFpgyTIqibh8WLOsPk5oo/p3yz/Ryt795pPKsMk7tLXq7fQ6CMSiACCDyUgO/kfOEZJkWmDJMiqJuHxYs6w+Tmij+nfLP9HK3v3mk8qwyTu0tert9DoIxKIAIIPJSA7+R84RkmRaYMkyKom4fFizrD5OaKP6d8s/0cre/eaTyrDJO7S16u30OgjEogAgg8lIDv5HzhGSZFpgyTIqibh8WLOsPk5oo/p3yz/Ryt795pPKsMk7tLXq7fQ6CMSiACCDyUgO/kfOEZJkWmqw2TdEF8L/uH8LGe3tEt/3vPd+8hrZ/OtR/QdFZrWb9GpbqN1LP6jm75zXZtCtNZeqTODJPaMG0Q5SGwgYhaQACBpQR8J+fjZZgUmTJMjkGtXtQZJsca7MDoih5qJBkmCaf0559hUqPLMDnm9MMmhsnb29vbcbfXRvz444/XXnhw2w4PAT30GKlUhx5Vf62i28/me2UpUz2smaq765DWv4biL7N+/vz584x7piSZUciqHAyTY7Lpgyl9CFzxkF7dQ5rfo/J47kYY1bIyTBJOZrtGK+WUfie75R/5+Yx7YJjUhm8gqtuDJl2ku9XvUTkwhIuOpLO0qIzhtGZ7GN13D6ZzQYf5OqQazK/g7zMyTIqUGSbHoOIlN/yTIobJsQY7MLqihxpJhknCKf35v+QhHX5j0h5SM+OK2V7dQ5r/Ep0ZJsmPahTb7TGQLtLd6meYROO3NDidpaXFDCQ32wPQCkfSuaBDAWoYkmoQph8KZ5gUsTFMjkGli7THxjHTSx4bD3xU1sgzTBJO6c+/2a7RTb+TqQ6r81+iM8OkNkwDUd0eA+ki3a1+hsnAEC46ks7SojKG05rtYXTfPZjOBR3m65BqML+Cv8/IMClSZpgcg1q9qL9XsPqObvkveWwwTA6He/VcmO1DCb4ErNZhdf4dekgZXfINY5jUfoAGoro9BtJFulv9DJOBIVx0JJ2lRWUMpzXbw+gYJr8i0G2WOv5sMkyKP2+rDZNiGcIWE0gfA+n/Oru4/EselVf04I75BMz2fKYyriEQzyrDZI0Qnz59uvsi3a1+hsmyUY0Td3yUJU2Y7YRWPTadCzrU2VYjUw2qec/EMUyK9BgmRVA3D4sX9fC3M67As0MPV3B62h07zMUOPTxt7kb6jXVmmIxgLp3p9hhIF+lu9TNMSmN3SVA6S5cUFVxitgNYQWg6F3QI4BZDUw2KaU+FMUyK+BgmRVA3D4sXdYbJzRV/Tvlm+zla373TeFYZJssk7/YYSBfpbvUzTJaNapw4naX4gsUHzPYawOlc0GG+DqkG8yv4+4wMkyJlhkkR1M3D4kWdYXJzxZ9Tvtl+jtZ37zSeVYbJMsm7PQbSRbpb/QyTZaMaJ05nKb5g8QGzvQZwOhd0mK9DqsH8Chgmw0wZJsPobnUwXtQZJrfS98nFmu0nq3+v3uNZZZgsE7jbYyBdpLvVzzBZNqpx4nSW4gsWHzDbawCnc0GH+TqkGsyvgGEyzJRhMozuVgfjRZ1hcit9n1ys2X6y+vfqPZ5Vhskygbs9BtJFulv9DJNloxonTmcpvmDxAbO9BnA6F3SYr0OqwfwKGCbDTBkmw+hudTBe1Bkmt9L3ycWa7Serf6/e41llmCwTuNtjIF2ku9XPMFk2qnHidJbiCxYfMNtrAKdzQYf5OqQazK+AYTLMlGEyjO5WB+NFnWFyK32fXKzZfrL69+o9nlWGyTKB08dAuuh2y3+FodGN0XvP3XRYzWhE52U/ZIOJU81Gek7vWK1bmt9s14ZrB51rnY5H+Y++FtkxTIqgbh4WL+oMk5sr/pzyzfZztL57p/GsMkyWSX73RTqt36OyNkrpwzXVYXX+EZ1rZK6LSpmO9JzesVq3ND/DpDaPO+hc63Q8imFSZMcwKYK6eVi8qDNMbq74c8o328/R+u6dxrPKMFkm+d0X6bR+j8raKKUP11SH1flHdK6RuS4qZTrSc3rHat3S/AyT2jzuoHOt0/EohkmRHcOkCOrmYfGizjC5ueLPKd9sP0fru3cazyrDZJnkd1+k0/o9KmujlD5cUx1W5x/RuUbmuqiU6UjP6R2rdUvzM0xq87iDzrVOx6MYJkV2DJMiqJuHxYs6w+Tmij+nfLP9HK3v3mk8qwyTZZLffZFO6/eorI1S+nBNdVidf0TnGpnrolKmIz2nd6zWLc3PMKnN4w461zodj2KYFNkxTIqgbh4WL+oMk5sr/pzyzfZztL57p/GsMkyWSX73RTqt36OyNkrpwzXVYXX+EZ1rZK6LSpmO9JzesVq3ND/DpDaPO+hc63Q8imFSZMcwKYK6eVi8qDNMbq74c8o328/R+u6dxrPKMFkmebpILyvkL4nTB1O3+q94tK7W4Kk9XME1ucNsJ7Tqsb4xdVarIlMNVtXxbV6GSZEyw6QI6uZh8aLOMLm54s8p32w/R+u7dxrPKsNkmeTdHmXpIt2t/qeaDTvosOyHbDDxDkz1MCj+wbG7fyfT+tdQ/GVWhkmR8g6GSbqEFtEMh/3AbBhmt/vBbrOa8u4422kPHePTueioQ9rDah12YPQDw2TZmHR70KSLdLf6GSbLRjVOnM5SfMHiA2Z7DeB0LugwX4dUg/kV/H1GhkmRMsOkCCoI22JRb2j6BBK0De32qExBdZzttIeO8elcdNQh7WG1DjswYpism5Juj4F0ke5WP8Nk3aymmdNZSvOvjjfbawinc0GH+TqkGsyvgGEyzJRhMozuw4NbLOoMk/mD8enTp26PyrTJjrOd9tAxPp2LjjqkPazWYQdGDJN1U9LtMZAu0t3qZ5ism9U0czpLaf7V8WZ7DeF0LugwX4dUg/kVMEyGmTJMhtExTOaj2z5jt0dlCrzjIzTtoWN8OhcddUh7WK3DDowYJuumpNtjIF2ku9XPMFk3q2nmdJbS/KvjzfYawulc0GG+DqkG8ytgmAwzZZgMo2OYzEe3fcZuj8oUeMdHaNpDx/h0LjrqkPawWocdGDFM1k1Jt8dAukh3q59hsm5W08zpLKX5V8eb7TWE07mgw3wdUg3mV8AwGWbKMBlGxzCZj277jN0elSnwjo/QtIeO8elcdNQh7WG1DjswYpism5Juj4F0ke5WP8Nk3aymmdNZSvOvjjfbawinc0GH+TqkGsyvgGEyzJRhMoyOYTIf3fYZuz0qU+AdH6FpDx3j07noqEPaw2oddmDEMFk3Jd0eA+ki3a1+hsm6WU0zp7OU5l8db7bXEE7ngg7zdUg1mF8Bw2SYKcNkGB3DZD667TN2e1SmwDs+QtMeOsanc9FRh7SH1TrswIhhsm5Kuj0G0kW6W/0Mk3WzmmZOZynNvzrebK8hnM4FHebrkGowvwKGyTBThskwOobJfHTbZ+z2qEyBd3yEpj10jE/noqMOaQ+rddiBEcNk3ZR0ewyki3S3+hkm62Y1zZzOUpp/dbzZXkM4nQs6zNch1WB+BQyTYaYMk2F0DJP56LbP2O1RmQLv+AhNe+gYn85FRx3SHlbrsAMjhsm6Ken2GEgX6W71M0zWzWqaOZ2lNP/qeLO9hnA6F3SYr0OqwfwKGCbDTBkmw+gYJvPRbZ+x26MyBd7xEZr20DE+nYuOOqQ9rNZhB0YMk3VT0u0xkC7S3epnmKyb1TRzOktp/tXxZnsN4XQu6DBfh1SD+RUwTIaZMkyG0TFM5qPbPmO3R2UKvOMjNO2hY3w6Fx11SHtYrcMOjBgm66ak22MgXaS71c8wWTeraeZ0ltL8q+PN9hrC6VzQYb4OqQbzK2CYDDNlmAyjY5jMR7d9xm6PyhR4x0do2kPH+HQuOuqQ9rBahx0YMUzWTUm3x0C6SHern2GyblbTzOkspflXx5vtNYTTuaDDfB1SDeZXwDAZZsowGUbHMJmPbvuM3R6VKfCOj9C0h47x6Vx01CHtYbUOOzBimKybkm6PgXSR7lY/w2TdrKaZ01lK86+ON9trCKdzQYf5OqQazK+AYTLMlGEyjI5hMh/d9hm7PSpT4B0foWkPHePTueioQ9rDah12YMQwWTcl3R4D6SLdrX6GybpZTTOns5TmXx1vttcQTueCDvN1SDWYXwHDZJjpasNkZIlOF92RO4aBFQ6m9b+nTHtI7+iW/73nu/eQ1j+ic2HcLg25ouf0DrN9PAKrGXWc7dVzdMk37KefPh+r2z/i7e3trVuV3R4D6SLdrX6GSZ8JT2epT+U/V2K21yiSzgUd5uuQajC/AobJMFOGyTC6Dw+mi/rIYyO9Y/WDKc1/yWPjj7+LxE17SDUY0Tlq4ILgK3pO71itW5rfbF8wiIUrVs/RJTo/1DDpuKgXRu7xIeljoKPOO/Tw+EFcAGCHudihhwXSbpcy1fnz589T/oeZKUk6q8Ewma9OuqiPPKTTO9JH3+r8lzw2GCbThzudC7Ndk6Dbz+cVOtfIjEelPaQaXPINY5iMD4CTlxNIF3WGyeUSuXCQgNkeBOfY5QTSWWWYFCVimBRBBWHpou5RWYObPmhSHVbnH9G5Rua6qJTpSM/pHat1S/Nf8pBmBh4O/eo5ukRnhsmhzgL6EEgXdYZJH+1U8n0CZtuE3IVAOqsMk6KyDJMiqCAsXdQ9Kmtw04drqsPq/CM618hcF5UyHek5vWO1bmn+Sx7SDJPDoV89R5fozDA51FlAHwLpos4w6aOdShgmvybQ8efTnB4TSL/DDJNjpl8iGCZFUEFYuqh7VNbgpg/XVIfV+Ud0rpG5LiplOtJzesdq3dL8lzykGSaHQ796ji7RmWFyqLOAPgTSRb3jg2yHHvpMxD6V7DAXO/Swz0St6yTVmWFS1GK1YVIs41TYyIPm1IUHh9NFfWUtf82dMurYwxWcVt+R6rC6njS/uUiJ1eLTueioQ9pDjcx41A6MnvrPCnd8SI9P4nNOpot6R5136OE5E3ddpzvMxQ49XKf4fW9KdWaYFLVmmBRBBWFbLOrh/4Id4Hl0aLdHZSpGx9lOe+gYn85FRx3SHlbrsAMjhsnqKZF/JoF0UWeYzKQv10oCZnslXblnEkhnlWFSpM8wKYIKwrZY1BkmgeL10G6PynrlP0d2nO20h47x6Vx01CHtYbUOOzBimKyeEvlnEkgXdYbJTPpyrSRgtlfSlXsmgXRWGSZF+gyTIqggbItFnWESKF4P7faorFfOMElZJfHpXOzwjUn4jMTuwIhhMqK8M68ikC7qDJNXKeXelIDZTomJfxWBdFYZJkWlGCZFUEHYFos6wyRQvB6aPozrma+J7Djb13S+9pZ0LjrqkPawlmjP34ZKGTFMVk+J/DMJpIs6w2QmfblWEjDbK+nKPZNAOqsMkyJ9hkkRVBC2w2OmYw+BBG1D0wdTt0bMxRpF0rnoqEPawxqSX7PuwIhhsnpK5J9JIF3UGSYz6cu1koDZXklX7pkE0lllmBTpM0yKoIKwLRZ1v2ESKF4P7faorFf+c2TH2U576BifzkVHHdIeVuuwAyOGyeopkX8mgXRRZ5jMpC/XSgJmeyVduWcSSGeVYVKkzzApggrCtljUGSaB4vXQbo/KeuUMk5RVEp/OxQ7fmITPSOwOjBgmNeXTBfE9a/pYT+/olv+957v3kNZP59rPTzqrtaxfo1LdRupZfUe3/Ga7NoXpLD1RZ4ZJbZY+MUyKoIKwLRZ1hkmgeD00fRjXM18T2XG2r+l87S3pXHTUIe1hLdGevw2VMmKY1KYkXXI9NmpcPTaOOXVjdIUxdkzllxHpz2fK9IqeV/eQ5vcNq01hOkupDqvzX6Ezw6Q2SwyTIqckbIfHTMceEg26xqYPpm59mIs1iqRz0VGHtIc1JL9m3YERw6Q2JemSe8USunqRTvN7VK6ZpVSHdFbT/FfoXCP5NWqHnlf3kOb3DatNYfrzk+qwOv8VOjNMarPEMClySsK2WNT9hkkieTm226OyXPhfAjvOdtpDx/h0LjrqkPawWocdGDFMalOSLrlXLKGrF+k0/xUP6VSHtIc0P51rPz+pDrWsDJOEk9mu0Uo5pbPdLf8V3zCGSW32GCZFTknYFos6wySRvBzb7VFZLpxhkqKK4tO52OEbEwEaCN6BEcOkJny65F6xhHZb1Bkma2bpiTrXSDJMEk6+YTVaKaduP59p/Vf8/1UMk9rsMUyKnJKwLRZ1hkkieTk2fRiXE18U2HG2L2p96TXpXHTUIe1hKdCm/6JTyohhUpuSjktot0WdYbJmlp6oc40kwyTh5BtWo5Vy6vbzmdbPMKnNxSVR/qOv8zHv8Jjp2MN8p0yQjwAAIABJREFUpa7PmD6Yrq/w+zeaizWKpHPRUYe0hzUkv2bdgRHDpDYlHZfQbos6w2TNLD1R5xpJhknCyTesRivl1O3nM62fYVKbi0uiGCbzMW+xqPsNk/mD8enTp26PyrTJjrOd9tAxPp2LjjqkPazWYQdGDJPVUyL/TALpYyB9zMys9aNcO/RwBaen3bHDXOzQw9PmbqTfVGd/klOkzDApggrCtljUGSaB4vXQbo/KeuU/R3ac7bSHjvHpXHTUIe1htQ47MGKYrJ4S+WcSSBd1hslM+nKtJGC2V9KVeyaBdFYZJkX6DJMiqCBsh0U9aLdtKB3mS4PpfKYjGelwTG0HRgyTY51F9CGQLuoMkz7aqeT7BMy2CbkLgXRWGSZFZRkmRVBB2A6LetBu21A6zJcG0/lMRzLS4ZjaDowYJsc6i+hDIF3UGSZ9tFMJw+TXBDr+fJrTYwLpd5hhcsz0SwTDpAgqCNthUQ/abRtKh/nSYDqf6UhGOhxT24ERw+RYZxF9CKSLescH2Q499JmIfSrZYS526GGfiVrXSaozw6SoxWrDZOTv2tNFd+SOIp6hsLT+oUvCQ90YheUPhdNhCNt3D2E6n+lIRjocU9uBEcPkWGcRfQikizrDpI92Kvk+AbNtQu5CIJ1VhklRWYZJEVQQtsOiHrTbNpQO86XBdD7TkYx0OKa2AyOGybHOIvoQSBd1hkkf7VTCMPk1gY4/n+b0mED6HWaYHDP9EsEwKYIKwnZY1IN224bSYb40mM5nOpKRDsfUdmDEMDnWWUQfAumi3vFBtkMPfSZin0p2mIsdethnotZ1kurMMClqwTApggrCdljUg3bbhtJhvjSYzmc6kpEOx9R2YMQwOdZZRB8C6aLOMOmjnUq+T8Bsm5C7EEhnlWFSVJZhUgQVhO2wqAfttg2lw3xpMJ3PdCQjHY6p7cCIYXKss4g+BNJFnWHSRzuVMEx+TaDjz6c5PSaQfocZJsdMv0QwTIqggrAdFvWg3bahdJgvDabzmY5kpMMxtR0YMUyOdRbRh0C6qHd8kO3QQ5+J2KeSHeZihx72mah1naQ6M0yKWjBMiqCCsB0W9aDdtqF0mC8NpvOZjmSkwzG1HRgxTI51FtGHQLqoM0z6aKeS7xMw2ybkLgTSWWWYFJVlmBRBBWE7LOpBu21D6TBfGkznMx3JSIdjajswYpgc6yyiD4F0UWeY9NFOJQyTXxPo+PNpTo8JpN9hhskx0y8RDJMiqCBsh0U9aLdtKB3mS4PpfKYjGelwTG0HRgyTY51F9CGQLuodH2Q79NBnIvapZIe52KGHfSZqXSepzgyTohYMkyKoIGyHRT1ot20oHeZLg+l8piMZ6XBMbQdGDJNjnUX0IZAu6gyTPtqp5PsEzLYJuQuBdFYZJkVlGSZFUEHYDot60G7bUDrMlwbT+UxHMtLhmNoOjBgmxzqL6EMgXdQZJn20UwnD5NcEOv58mtNjAul3mGFyzPRLBMOkCCoI22FRD9ptG0qH+dJgOp/pSEY6HFPbgRHD5FhnEX0IpIt6xwfZDj30mYh9KtlhLnboYZ+JWtdJqjPDpKgFw6QIKggbWdT//K//E9zw6VN6R5o/KqZp8GpGaf4vBmWocze0V/Sc3nF3piMar2aU5u842yM9jGiRnElnlWGS0BX7agLpos4webVi7q8SMNtVUuJeTSCdVYZJUTGGSRFUEDayqMeL9B9/F1R0/4d61OxfglMdVmvQ8VGZck2ZjvSc3pHqlvbcMX41ozT/iM6ruY70sLqmdFYZJqsVqedPl9B65rFIZsMYtyec6jarKfOOs5320DE+nYuOOqQ9rNZhB0YMk+KUMEyKoIKwkUU9XqQZJoeKpDqs1qDjo/IQ4q8CUqYjPad3pLqlPXeMX80ozT+i82quIz2srimdVYbJakXq+S3qx6xSRh0fG8dd9o9IdejWkblYo0g6Fx11SHtYQ/Jr1h0YMUyKU8IwKYIKwkYW9XiRZpgcKpLqsFqDjo/KQ4gMkxTRJfFm+xhzyug44/mI+Bvz00+fz9/6+gxvb29vSRU7LKFJvyOxOzDq2MOIFt3OdHtUpnzMRUqsFp/ORUcd0h5qZMajdmDEMCnqzzApggrCRhb1eJFmmBwqkuqwWgOGyaFkXwJW61aronfUakZp/o6zPdLDatXjbwzDZLUk5fwW9WNUKaOOj43jLvtHpDp068hcrFEknYuOOqQ9rCH5NesOjBgmxSlhmBRBBWEji3q8SDNMDhVJdVitQcdH5SHEXwWkTEd6Tu9IdUt77hi/mlGaf0Tn1VxHelhdUzqr/iRntSL1/Bb1Y1Ypo46PjeMu+0ekOnTryFysUSSdi446pD2sIckw+S2uW/w67PcGhmEy/8dph0V9PpXrM9JhPnNM5zMdyUiHY2o7MGKYHOt8VYRF/Zh0yqjjg+y4y/4RqQ7dOjIXaxRJ56KjDmkPa0gyTBgmhclKl9D0f1F7L+GKOwqtDoek9Q9fFBwc0SFI3zKUDvNlwXQ+05GMdDimtgMjhsmxzldFWNSPSaeMOj7IjrvsH5Hq0K0jc7FGkXQuOuqQ9rCGJMOEYVKYrHQJHXmoX3FHodXhkLT+4YuCgyM6BOlbhtJhviyYzmc6kpEOx9R2YMQwOdb5qgiL+jHplFHHB9lxl/0jUh26dWQu1iiSzkVHHdIe1pBkmDBMCpO1wxJaaPNUCEan8E07TIdpKP+WCNP5TEcy0uGY2g6MGCbHOl8VYVE/Jp0y6vggO+6yf0SqQ7eOzMUaRdK56KhD2sMakgwThklhsnZYQgttngrB6BS+aYfpMA0lw2Q+ylMZzfYxvh0YMUyOdb4qwqJ+TDpl1PFBdtxl/4hUh24dmYs1iqRz0VGHtIc1JBkmDJPCZO2whBbaPBWC0Sl80w7TYRpKhsl8lKcymu1jfDswYpgc63xVhEX9mHTKqOOD7LjL/hGpDt06MhdrFEnnoqMOaQ9rSDJMGCaFydphCS20eSoEo1P4ph2mwzSUDJP5KE9lNNvH+HZgxDA51vmqCIv6MemUUccH2XGX/SNSHbp1ZC7WKJLORUcd0h7WkGSYMEwKk7XDElpo81QIRqfwTTtMh2koGSbzUZ7KaLaP8e3AiGFyrPN7xMgSnT4GRu6oVT8WldY/wim9I2W0Ov97z6vv6JZ/ROexCVx3KmU60nN6h9k+1ns1oxGdj6s+F7F6jq74hn3+/PnzOQo/n56SZEYhq3L8+Q9/eEty77CEJv2OxGI0Qm3+GTpgOp9Aj4xm+1iHHRgxTI51Hl2ir1h0a9WPRaX1j3BK71j9YErzX/HY6MZoROexCVx3KmU60nN6Rzp7q/Ob7XXzl2TeQWeGSVFxhkkRVBC2w6IetNs2lA7zpcF0PtORjHQ4prYDI4bJsc4jj6UrHhu1ysej0kV9hFN6h0flsZ6rGY3ofFz1tRHp3I30nN6xWrc0/xXfsG6MRnRePbmrGV2hM8OkOCUMkyKoIGyHRT1ot20oHeZLg+l8piMZ6XBMbQdGDJNjnUeX6CsW3Vr1Y1Fp/SOc0jvSR9/q/Fc8Nlb3kOYf0XlsAteduqLn9A6zfaz3akYdZ3v1HF3xDWOYHM/2lwiGSRFUELbDoh602zaUDvOlwXQ+05GMdDimtgMjhsmxzqNL9BWLbq36sai0/hFO6R2rH0xp/iseG90Yjeg8NoHrTqVMR3pO70hnb3V+s71u/pLMO+jMMCkqzjApggrCdljUg3bbhtJhvjSYzmc6kpEOx9R2YMQwOdZ55LF0xWOjVvl4VLqoj3BK7/CoPNZzNaMRnY+rvjYinbuRntM7VuuW5r/iG9aN0YjOqyd3NaMrdGaYFKeEYVIEFYTtsKgH7bYNpcN8aTCdz3QkIx2Oqe3AiGFyrPNVESMPmpW1pYv6ylr+mjtl1LGHKzitviPVYXU9aX5zkRKrxadz0VGHtIcamfGoHRgxTIr6M0yKoIKwHRb1oN22oXSYLw2m85mOZKTDMbUdGDFMjnW+KsKifkw6ZdTxsXHcZf+IVIduHZmLNYqkc9FRh7SHNSS/Zt2BEcOkOCU7GCbFVoUhgAACCCDwMgJ//tf/ie5mmES4lgZb1I/xpow6PjaOu+wfkerQrSNzsUaRdC466pD2sIYkw+S3uH5eDfvV+Rkmr1bA/QgggAACTyDAMKmpbFE/5rQDo449HJPvH9HtUZkSMxcpsVp8OhcddUh7qJEZj9qBkd8wKerPMCmCEoYAAggggMAJAgyTGrwdltBap+NROzDq2MO4In1OdntUpmTMRUqsFp/ORUcd0h5qZMajdmDEMCnqzzApghKGAAIIIIDACQIMkxq8HZbQWqfjUTsw6tjDuCJ9TnZ7VKZkzEVKrBafzkVHHdIeamTGo3ZgxDAp6s8wKYIShgACCCCAwAkCDJMavB2W0Fqn41E7MOrYw7gifU52e1SmZMxFSqwWn85FRx3SHmpkxqN2YMQwKerPMCmCEoYAAggggMAJAgyTGrwdltBap+NROzDq2MO4In1OdntUpmTMRUqsFp/ORUcd0h5qZMajdmDEMCnqzzApghKGAAIIIIDACQIMkxq8HZbQWqfjUTsw6tjDuCJ9TnZ7VKZkzEVKrBafzkVHHdIeamTGo3ZgxDAp6s8wKYIShgACCCCAwAkCDJMavB2W0Fqn41E7MOrYw7gifU52e1SmZMxFSqwWn85FRx3SHmpkxqN2YMQwKerPMCmCEoYAAggggMAJAgyTGrwdltBap+NROzDq2MO4In1OdntUpmTMRUqsFp/ORUcd0h5qZMajdmDEMCnqzzApghKGAAIIIIDACQIMkxq8HZbQWqfjUTsw6tjDuCJ9TnZ7VKZkzEVKrBafzkVHHdIeamTGo3ZgxDAp6s8wKYIShgACCCCAwAkCDJMavB2W0Fqn41E7MOrYw7gifU52e1SmZMxFSqwWn85FRx3SHmpkxqN2YMQwKerPMCmCEoYAAggggMAJAgyTGrwdltBap+NROzDq2MO4In1OdntUpmTMRUqsFp/ORUcd0h5qZMajdmDEMCnqzzApghKGAAIIIIDACQIMkxq8HZbQWqfjUTsw6tjDuCJ9TnZ7VKZkzEVKrBafzkVHHdIeamTGo3ZgxDAp6s8wKYIShgACCCCAwAkCDJMavB2W0Fqn41E7MOrYw7gifU52e1SmZMxFSqwWn85FRx3SHmpkxqN2YMQwKerPMCmCEoYAAggggMAJAgyTGrwdltBap+NROzDq2MO4In1OdntUpmTMRUqsFp/ORUcd0h5qZMajdmDEMCnqzzApghKGAAIIIIDACQIMkxq8HZbQWqfjUTsw6tjDuCJ9TnZ7VKZkzEVKrBafzkVHHdIeamTGo3ZgxDAp6r/aMEkXxPeyf/jj74rV/xyW3tEt/w49pBrQuTbi6azWsn6NSnUbqWf1Hd3ym+3aFKaz9Eidf/rpc41m76i3t7e3pMJ0CR1Zoq+4I+k5jU3rT/OPxI/oMHJPpzN0mK8GpvOZjmSkwzG1HRgxTI51/tls+MMfokVm9ZLrsVETbrUOq/PTeY3OtawMk4RT+lA32zW6q78xq/NfojPDpDRMIw/1dNEduaNU/GBQWv/gNdGxboyi4geD6TAI7jvHMJ3PdCQjHY6p7cCIYXKsM8Okyuhf/6cY+XNYuqh/Ma4W39Et/winbj101Dka1AvmzmzXFDHbx5y6MbrkG8YwOR6MT58+jTzU00V35I5S8YNBaf2D10THujGKih8MpsMgOIbJfHCTM5rtY6A7MGKYHOvMMKkyWmxmeFTWhOj2YGKYPEO3dO4ueUg3+7NF37BFPwsMkxLYkYd6uuiO3FEqfjAorX/wmuhYN0ZR8YPBdBgExzCZD25yRrN9DHQHRgyTY50ZJlVGDJNDUh6Vh4h+/nm7YJZqlfwcdUU9q+/olp9hUpvA1HB8pM4Mk9IwjTzU00V35I5S8YNBaf2D10THujGKih8MpsMgOIbJfHCTM5rtY6A7MGKYHOvMMKkyuuCR2+0xsPox41FZG75Uh1rWr1Gr5+4KU2Z1D2l+s12bwnS2Ux1W579EZ4ZJaZhGHurpojtyR6n4waC0/sFromPdGEXFDwbTYRAcw2Q+uMkZzfYx0B0YMUyOdb7EMCmWIQwBBBBAAIGtCcSmD8OkNA8jD/V00R25o1T8YFBa/+A10bFujKLiB4PpMAiOYTIf3OSMZvsY6A6MGCbHOjNMioyEIYAAAgggcJYAw6RGMF1CRx7qV9xR63YsKq1/7Jbs1IgO2Q39oukwXxNM5zMdyUiHY2o7MGKYHOvMMCkyEoYAAggggMBZAgyTGsF0CR15qF9xR63bsai0/rFbslMjOmQ39Iumw3xNMJ3PdCQjHY6p7cCIYXKsM8OkyEgYAggggAACZwkwTGoE0yV05KF+xR21bsei0vrHbslOjeiQ3dAvmg7zNcF0PtORjHQ4prYDI4bJsc4MkyIjYQgggAACCJwlwDCpEUyX0JGH+hV31Lodi0rrH7slOzWiQ3ZDv2g6zNcE0/lMRzLS4ZjaDowYJsc6M0yKjIQhgAACCCBwlgDDpEYwXUJHHupX3FHrdiwqrX/sluzUiA7ZDf2i6TBfE0znMx3JSIdjajswYpgc68wwKTIShgACCCCAwFkCDJMawXQJHXmoX3FHrduxqLT+sVuyUyM6ZDf0i6bDfE0wnc90JCMdjqntwIhhcqwzw6TISBgCCCCAAAJnCTBMagTTJXTkoX7FHbVux6LS+t9vSTmld6T5xzrvdWo1ozT/iM69iH76dEXP6R1m+3hKUkapBh1ne6SHY5LnIlIdGCZF3n/+wx/eiqFfwn744++ScLEIIIAAAggg8OnTJ4ZJbQzSJTRdEN+ruOKOWrdjUWn9I4+N9I4RHca673NqNaM0/4jOfWj+XMkVPad3mO3jKUkZpRp0nO2RHo5JnotIdWCYFHkzTIqghCGAAAIIIHCCAMOkBi9dQtMFceRRNnJHrduxqJTRyGMjvaMbozGy2anVjNL8IzpnHa+PvqLn9A6zfax7yijVoONsj/RwTPJcRKoDw6TIm2FSBCUMAQQQQACBEwQYJjV46RKaLogMkz461CrpG7V6VtP8HR+VqXpX9JzeMfKNSfvuFr+aUZq/42yP9LBa53RWGSZFRRgmRVDCEEAAAQQQOEGAYVKDly6h6YLIMOmjQ62SvlGrZzXN3/FRmap3Rc/pHSPfmLTvbvGrGaX5O872SA+rdU5nlWFSVIRhUgQlDAEEEEAAgRMEGCY1eOkSmi6IDJM+OtQq6Ru1elbT/B0flal6V/Sc3jHyjUn77ha/mlGav+Nsj/SwWud0VhkmRUUYJkVQwhBAAAEEEDhBgGFSg5cuoemC+FTDpEZ/PGpEh/HbepxMZ/WKqu+uA6ZXTMnxHXR4BiOGybHOXyIYJkVQwhBAAAEEEDhBgGFSg5cu6iMPxCvuqHU7FpXWP3ZLdmpEh+yGftF0mK8JpvOZjmSkwzG1HRgxTI51HjJMimmFIYAAAggggMAJAj/89NPnE8fbHH17e3tLikmX0JGH+hV3JD2nsWn9af6R+BEdRu7pdIYO89XAdD7TkYx0OKa2AyOGybHODJMiI2EIIIAAAghcTYBhUiM+8lBPF92RO2rVj0Wl9Y/dkp3qxiirfiyaDmPcvncK0/lMRzLS4ZjaDowYJsc6M0yKjIQhgAACCCBwNQGGSY34yEM9XXRH7qhVPxaV1j92S3aqG6Os+rFoOoxxY5jM5zY7o9k+JroDI4bJsc4MkyIjYQgggAACCFxNgGFSIz7yUE8X3ZE7atWPRaX1j92SnerGKKt+LJoOY9wYJvO5zc5oto+J7sCIYXKsM8OkyEgYAggggAACVxNgmFxN/OP7upkBOyzqfdQdr4QO4+w+OonpfKYjGelwTG0HRgyTY50ZJkVGwhBAAAEEELiaAMPkauIMkzPEu5lKZ3qpnt3hwVTt9ao4TK8i/f176HCsww6MGCbHOv81Ivqv19fTikQAAQQQQACBEwT8Kzkn4M082s0M2GFRn6nPq3LRYT55TOczHclIh2NqOzBimBzrzDCpMxKJAAIIIIDA1QQYJlcT/+A+hsmxEN0YHVd8PmKHB9N5CnMzYDqX52g2OhyT24ERw+RYZ4ZJnZFIBBBAAAEEribAMLmaOMNkmDjDZBjd1IN312GHR+hUQV+UjA7H4HdgxDA51vkWhsl//df/+Vsn//zP/+9v/+dv/+8/avXb+DqOuZEf1f9+y116mEtENgQQ6Ejg//7jP/5mWf/ypz91LPcpNTFMmijd7RG6w6LeRNpTZdDhFL7fPIzpfKYjGelwTG0HRgyTY50ZJnVGw5EMk2F0DiKAwIUEGCYXwq5fxTCps1oayTA5xtuN0XHF5yN2eDCdpzA3A6ZzeY5mo8MxuR0YMUyOdWaY1BkNRzJMhtE5iAACFxJgmFwIu34Vw6TOamlkNzNgh0V9qWAXJafDfNCYzmc6kpEOx9R2YMQwOda5rWFy5s9wPmr7yj/PWVH/e19X9lAfH5EIIHBXAt+aJB/96c1HRsp7z/5cZ7nyDJPliGsXMEyOOXVjdFzx+YgdHkznKczNgOlcnqPZ6HBMbgdGDJNjnRkmdUZRJMMkwiUYAQReRIBh8iLw9WsZJnVWSyO7mQE7LOpLBbsoOR3mg8Z0PtORjHQ4prYDI4bJsc4MkzqjKJJhEuESjAACLyLAMHkR+Pq1DJM6q6WRDJNjvN0YHVd8PmKHB9N5CnMzYDqX52g2OhyT24ERw+RY57aGybelf/QvydzlX8z53r+Ec5ce6qMkEgEEdiPwPVOlYrjsxuPifhgmFwP/6LpuZsAOi3oTaU+VQYdT+H7zMKbzmY5kpMMxtR0YMUyOdWaY1BkNRzJMhtE5iAACDQgwTF4qAsPkpfhdjgACCCCAwH0IpOY+w6Su7Vs99PpIv2Hylbn/6Ov18+dGBJ5OgGHy0glgmLwUv8sRQAABBBC4DwGGyTqtbmmYfISjm6nwvd8wuUsP60ZPZgQQ6E6g+mc31bju/Tarj2HSTBDlIIAAAggg0JUAw2SdMgyTdWw/MUwWwpUaAQSWE6gaIdW45QXvdQHDZC89dYMAAggggMAyAgyTZWg/MUzWsWWYLGQrNQIIrCdQNUKqcesr3uoGhslWcmoGAQQQQACBdQQYJuvY3tIw6fanNx/JU/2Pvq6TV2YEEEAgI1A1P76N+/aGf/nTn7ILRX9EgGFiNhBAAAEEEECgRIBhUsI0FMQwGcJWO8QwqXEShQACfQgwTNpowTBpI4VCEEAAAQQQ6E2AYbJOH4bJOrbf/ZOcu/yWzEI8UiOAQEMCDJM2ojBM2kihEAQQQAABBHoTYJis04dhso4tw2QhW6kRQGAegYpJ8tGf4HyvCn+ec0ojhskpfA4jgAACCCDwHAIMk3VaM0zWsWWYLGQrNQIIzCPAMJnHcmImhslEmFIhgAACCCCwMwGGyTp1GSbr2DJMFrKVGgEE5hFgmMxjOTETw2QiTKkQQAABBBDYmQDDZJ26tzRMPsLR7b8L8r3/6Otdelg3ejIjgEAXApV/8caf5FyuFsPkcuQuRAABBBBA4J4EGCbrdGOYrGP73d8wYZgsBC81AghEBBgmEa6rghkmBdLpgvie8scffyxk/hqS3tEt/3snd+8hrZ/OtRFPZ7WW9bqfHbNdUyT9+UnnYnV+Oq/R+fPnz1P2jClJai2+LIphshC93zBZCFdqBBCYRoBhMg3lzERb7CBvb2/RnrF6UfeQro3oah1W56fzGp1rWRkmCafUbDDbNbqrvzGr81+hM8OkNkvvUdEiU087J3LEcPitm1/1pzqz6n/v6VU9zFFSFgQQ6EYg/RObX/+LNxWTpVvPN6uHYVIQzGOjAMlvmJQgpbO0+sGU5n9vcnUPJZDfBF1Rz+o7uuW/4iGdzt5qRlfM9uoe0vxX6MwwqX/RGCZ1VnEkwyRG5gACCFxEgGFyEejxaxgmBXYdl1CPjWPhVjO64rGxuoc0/xWPymNlfxmR/nx27Hl1D2l+s12bwnSWUh1W579CZ4ZJbZbeoxgmdVZxJMMkRuYAAghcRIBhchHo8WsYJgV26ZJ7xRK6epFO81/xkE51SHtI89O58MMz8N/zqWX9GpXqls6F2a4pslqH1fnpvEZnhkmNa3vD5Ns2zpgPHf6c5Uz97xw69FAfK5EIINCdwFnDpHt/G9THMCmImC7qHtIFqAMP6VSH9GGc5qfzGp1rWRkmCSezXaOVclr9jVmd/4pvGMOkNnsMkzqn05EMk9MIJUAAgYkEGCYTYa5JxTBZw1VWBBBAAAEEtiOQmkoMk/oItP6TnG/bOGM4dPjtjDP1v3Po0EN9rEQigEB3AgyT7gp9Ypi0l0iBCCCAAAII9CDAMFmnA8NkHdtfZGaYXATaNQggUCLAMClhemUQw+SV9N2NAAIIIIDAjQgwTNaJdRvD5FsEFfOh+29k7NDDurGUGQEEriTgnwi+knb5LoZJGZVABBBAAAEEnk2AYbJOf4bJOrbfzcwweRF41yKAwN8RYJi0HAqGSUtZFIUAAggggEA/AgyTdZowTNaxZZi8iK1rEUAgI8AwyXhdFM0wuQi0axBAAAEEELg7AYbJOgVvaZiswyEzAggggAACLQgwTFrIoAgEEEAAAQT6E2CYrNOIYbKOrcwIIIAAAgiMEmCYjJJzDgEEEEAAgYcRYJisE5xhso6tzAgggAACCIwSYJiMknMOAQQQQACBhxFgmKwTnGGyjq3MCCCAAAIIjBJgmIyScw4BBBBAAIGHEWCYrBOcYbKOrcwIIIAAAgiMEmCYjJJzDgEEEEAAgYcRYJisE5xhso6tzAgggAACCIwSYJiMknMOAQQQQACBhxFgmKwTnGGyjq3MCCCAAAIIjBJgmIyScw4BBBBAAIGHEWCYrBOcYbKOrcwIIIAAAgiMEmCYjJJzDgEEEEAAgYcRYJisE5wY4U2+AAAOYElEQVRhso6tzAgggAACCIwSYJiMknMOAQQQQACBhxFgmKwTnGGyjq3MCCCAAAIIjBJgmIyScw4BBBBAAIGHEWCYrBOcYbKOrcwIIIAAAgiMEmCYjJJzDgEEEEAAgYcRYJisE5xhso6tzAgggAACCIwSYJiMknMOAQQQQACBhxFgmKwTnGGyjq3MCCBwgsB//8c/HZ7+p3//78MYAQjclADD5KbCKRsBBBBAAIGrCTBM1hFnmKxjKzMCCJwgwDA5Ac/RHQg80jDZQTg9IIAAAggg0J3A58+fp+wZU5I0h8UwaS6Q8hB4KgGGyVOV1/dfCGyxg7y9vdkzjDQCCCCAAALNCDBM6oJYZOqsRCKAwGICH5kk3/7pTSVmcZnSI3AFAYbJFZTdgQACCCCAwAMJMEzqojNM6qxEIoDAYgIVM6QSs7hM6RG4ggDD5ArK7kAAAQQQQOCBBBgmddEZJnVWIhFAYDGBihlSiVlcpvQIXEGAYXIFZXcggAACCCDwQAIMk7roDJM6K5EIILCAwBkD5MzZBa1IicBMAgyTmTTlQgABBBBAAIG/EWCY1IeBYVJnJRIBBBYQOGN6nDm7oBUpEZhJYAvD5NOnT7ffM/73f//3N3X9h3/4h5l6L821Qw9LAUmOAAKXEfjP//zP37zr3/7t3y6rwUVfCEzZM6YkaS7I7ReZ5nyVhwACBwTOmB5nzhIGgeYEdtlBbr9n7GA27NBD859X5SGAQJEAw6QIan3YlD1jSpL1vZ664faLzKnuHUYAgZcTOGN6nDn78sYVgMD3Ceyyg9x+z9jBbNihBx8MBBDYgwDDpI2OU/aMKUnaIPntQm6/yDTnqzwEEDggcMb0OHOWMAg0J7DLDnL7PWMHs2GHHpr/vCoPAQSKBBgmRVDrw6bsGVOSrO/11A23X2ROde8wAgi8nMAZ0+PM2Zc3rgAEvk9glx3k9nvGDmbDDj34YCCAwB4EGCZtdJyyZ0xJ0gbJbxdy+0WmOV/lIYDAAYEzpseZs4RBoDmBXXaQ2+8ZO5gNO/TQ/OdVeQggUCTAMCmCWh82Zc+YkmR9r6duuP0ic6p7hxFA4OUEzpgeZ86+vHEFIPB9ArvsILfZMz4yFdJBfdW/njOr/vd+X9VDylo8AggggMAwgSl7xpQkwy1cc/A2i8w1ONyCAAJXEzhjepw5e3Wf7kMgJLDLDnKbPWOW4fAqs2FW/QyT8CdVOAIIIHBPAlP2jClJmvO7zSLTnKPyEEBgAoGKAVKJmVCKFAi8msAuO8ht9oxZhgPD5NU/Ou5HAAEEECgQmLJnTElSKPaVIbdZZF4Jyd0IIHANgYoZUom5plq3ILCUwC47yG32DIbJ13l+lemz9CdKcgQQQACBbwlM2TOmJGmuy20WmeYclYcAAhMIVMyQSsyEUqRA4NUEdtlBbrNnMEwYJq/+oXc/AgggcCGBKXvGlCQXNj1y1W0WmZHmnEEAgfsS+MgY+bajf/r3/75vgypH4PsEdtlBbrNnMEwYJj5KCCCAwIMITNkzpiRpDv02i0xzjspDAIHJBBgmk4FKdzcCT9hB7qaJehFAAAEEEEDgGwJPWFYYJkYeAQRaEmCYtJRFUdcReMIOch1NNyGAAAIIIIDAdAJPWFYYJtPHRkIEEEAAAQROE3jCDnIakgQIIIAAAggg8DoCT1hWGCavmy83I4AAAggg8BGBJ+wg1EcAAQQQQACBGxN4wrLCMLnxgCodAQQQQGBbAk/YQbYVT2MIIIAAAgg8gcATlhWGyRMmWY8IIIAAAncj8IQd5G6aqBcBBBBAAAEEviHwhGWFYWLkEUAAAQQQ6EfgCTtIP+oqQgABBBBAAIEygScsKwyT8jgIRAABBBBA4DICT9hBLoPpIgQQQAABBBCYT+AJywrDZP7cyIgAAggggMBZAk/YQc4ych4BBBBAAAEEXkjgCcsKw+SFA+ZqBBBAAAEEPiDwhB2E+AgggAACCCBwYwJPWFYYJjceUKUjgAACCGxL4Ak7yLbiaQwBBBBAAIEnEHjCssIwecIk6xEBBBBA4G4EnrCD3E0T9SKAAAIIIIDANwSesKwwTIw8AggggAAC/Qg8YQfpR11FCCCAAAIIIFAm8IRlhWFSHgeBCCCAAAIIXEbgCTvIZTBdhAACCCCAAALzCTxhWWGYzJ8bGRFAAAEEEDhL4Ak7yFlGziOAAAIIIIDACwk8YVlhmLxwwFyNAAIIIIDABwSesIMQHwEEEEAAAQRuTOAJywrD5MYDqnQEEEAAgW0JPGEH2VY8jSGAAAIIIPAEAk9YVhgmT5hkPSKAAAII3I3AE3aQu2miXgQQQAABBBD4hsATlhWGiZFHAAEEEECgH4En7CD9qKsIAQQQQAABBMoEnrCsMEzK4yAQAQQQQACBywg8YQe5DKaLEEAAAQQQQGA+gScsKwyT+XMjIwIIIIAAAmcJPGEHOcvIeQQQQAABBBB4IYEnLCsMkxcOmKsRQAABBBD4gMATdhDiI4AAAggggMCNCTxhWWGY3HhAlY4AAgggsC2BJ+wg24qnMQQQQAABBJ5A4AnLCsPkCZOsRwQQQACBuxF4wg5yN03UiwACCCCAAALfEHjCssIwMfIIIIAAAgj0I/CEHaQfdRUhgAACCCCAQJnAE5YVhkl5HAQigAACCCBwGYEn7CCXwXQRAggggAACCMwn8IRlhWEyf25kRAABBBBA4CyBJ+wgZxk5jwACCCCAAAIvJPCEZYVh8sIBczUCCCCAAAIfEHjCDkJ8BBBAAAEEELgxgScsKwyTGw+o0hFAAAEEtiXwhB1kW/E0hgACCCCAwBMIPGFZYZg8YZL1iAACCCBwNwJP2EHupol6EUAAAQQQQOAbAk9YVhgmRh4BBBBAAIF+BJ6wg/SjriIEEEAAAQQQKBN4wrLCMCmPg0AEEEAAAQQuI/CEHeQymC5CAAEEEEAAgfkEnrCsMEzmz42MCCCAAAIInCXwhB3kLCPnEUAAAQQQQOCFBJ6wrDBMXjhgrkYAAQQQQOADAk/YQYiPAAIIIIAAAjcm8IRlhWFy4wFVOgIIIIDAtgSesINsK57GEEAAAQQQeAKBJywrDJMnTLIeEUAAAQTuRuAJO8jdNFEvAggggAACCHxD4AnLCsPEyCOAAAIIINCPwBN2kH7UVYQAAggggAACZQJPWFYYJuVxEIgAAggggMBlBJ6wg1wG00UIIIAAAgggMJ/AE5YVhsn8uZERAQQQQACBswSesIOcZeQ8AggggAACCLyQwBOWFYbJCwfM1QgggAACCHxA4Ak7CPERQAABBBBA4MYEnrCsMExuPKBKRwABBBDYlsATdpBtxdMYAggggAACTyDwhGWFYfKESdYjAggggMDdCDxhB7mbJupFAAEEEEAAgW8IPGFZYZgYeQQQQAABBPoReMIO0o+6ihBAAAEEEECgTOAJywrDpDwOAhFAAAEEELiMwBN2kMtguggBBBBAAAEE5hN4wrLCMJk/NzIigAACCCBwlsATdpCzjJxHAAEEEEAAgRcSeMKywjB54YC5GgEEEEAAgQ8IPGEHIT4CCCCAAAII3JjAE5YVhsmNB1TpCCCAAALbEnjCDrKteBpDAAEEEEDgCQSesKwwTJ4wyXpEAAEEELgbgSfsIHfTRL0IIIAAAggg8A2BJywrDBMjjwACCCCAQD8CT9hB+lFXEQIIIIAAAgiUCTxhWWGYlMdBIAIIIIAAApcReMIOchlMFyGAAAIIIIDAfAJPWFYYJvPnRkYEEEAAAQTOEnjCDnKWkfMIIIAAAggg8EIClpUXwnc1AggggAACCCCAAAIIIIAAAgj0JMAw6amLqhBAAAEEEEAAAQQQQAABBBBA4IUEGCYvhO9qBBBAAAEEEEAAAQQQQAABBBDoSYBh0lMXVSGAAAIIIIAAAggggAACCCCAwAsJMExeCN/VCCCAAAIIIIAAAggggAACCCDQkwDDpKcuqkIAAQQQQAABBBBAAAEEEEAAgRcSYJi8EL6rEUAAAQQQQAABBBBAAAEEEECgJwGGSU9dVIUAAggggAACCCCAAAIIIIAAAi8kwDB5IXxXI4AAAggggAACCCCAAAIIIIBATwIMk566qAoBBBBAAAEEEEAAAQQQQAABBF5IgGHyQviuRgABBBBAAAEEEEAAAQQQQACBngQYJj11URUCCCCAAAIIIIAAAggggAACCLyQAMPkhfBdjQACCCCAAAIIIIAAAggggAACPQkwTHrqoioEEEAAAQQQQAABBBBAAAEEEHghAYbJC+G7GgEEEEAAAQQQQAABBBBAAAEEehJgmPTURVUIIIAAAggggAACCCCAAAIIIPBCAgyTF8J3NQIIIIAAAggggAACCCCAAAII9CTAMOmpi6oQQAABBBBAAAEEEEAAAQQQQOCFBBgmL4TvagQQQAABBBBAAAEEEEAAAQQQ6EmAYdJTF1UhgAACCCCAAAIIIIAAAggggMALCTBMXgjf1QgggAACCCCAAAIIIIAAAggg0JMAw6SnLqpCAAEEEEAAAQQQQAABBBBAAIEXEmCYvBC+qxFAAAEEEEAAAQQQQAABBBBAoCcBhklPXVSFAAIIIIAAAggggAACCCCAAAIvJMAweSF8VyOAAAIIIIAAAggggAACCCCAQE8CDJOeuqgKAQQQQAABBBBAAAEEEEAAAQReSIBh8kL4rkYAAQQQQAABBBBAAAEEEEAAgZ4EGCY9dVEVAggggAACCCCAAAIIIIAAAgi8kADD5IXwXY0AAggggAACCCCAAAIIIIAAAj0JMEx66qIqBBBAAAEEEEAAAQQQQAABBBB4IQGGyQvhuxoBBBBAAAEEEEAAAQQQQAABBHoSYJj01EVVCCCAAAIIIIAAAggggAACCCDwQgIMkxfCdzUCCCCAAAIIIIAAAggggAACCPQk8P8BR7Tz1f4haB0AAAAASUVORK5CYII=\" width=\"1100\">"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Saving figure preprocessing_plot\n"
     ]
    }
   ],
   "source": [
    "plt.figure(figsize=(11, 7))\n",
    "plt.subplot(121)\n",
    "plt.title(\"Original observation (160×210 RGB)\")\n",
    "plt.imshow(obs)\n",
    "plt.axis(\"off\")\n",
    "plt.subplot(122)\n",
    "plt.title(\"Preprocessed observation (88×80 greyscale)\")\n",
    "plt.imshow(img.reshape(88, 80), interpolation=\"nearest\", cmap=\"gray\")\n",
    "plt.axis(\"off\")\n",
    "save_fig(\"preprocessing_plot\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Build DQN"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note: instead of using `tf.contrib.layers.convolution2d()` or `tf.contrib.layers.conv2d()` (as in the first version of the book), we now use the `tf.layers.conv2d()`, which did not exist when this chapter was written. This is preferable because anything in contrib may change or be deleted without notice, while `tf.layers` is part of the official API. As you will see, the code is mostly the same, except that the parameter names have changed slightly:\n",
    "* the `num_outputs` parameter was renamed to `filters`,\n",
    "* the `stride` parameter was renamed to `strides`,\n",
    "* the `_fn` suffix was removed from parameter names that had it (e.g., `activation_fn` was renamed to `activation`),\n",
    "* the `weights_initializer` parameter was renamed to `kernel_initializer`,\n",
    "* the weights variable was renamed to `\"kernel\"` (instead of `\"weights\"`), and the biases variable was renamed from `\"biases\"` to `\"bias\"`,\n",
    "* and the default `activation` is now `None` instead of `tf.nn.relu`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "reset_graph()\n",
    "\n",
    "input_height = 88\n",
    "input_width = 80\n",
    "input_channels = 1\n",
    "conv_n_maps = [32, 64, 64]\n",
    "conv_kernel_sizes = [(8,8), (4,4), (3,3)]\n",
    "conv_strides = [4, 2, 1]\n",
    "conv_paddings = [\"SAME\"] * 3 \n",
    "conv_activation = [tf.nn.relu] * 3\n",
    "n_hidden_in = 64 * 11 * 10  # conv3 has 64 maps of 11x10 each\n",
    "n_hidden = 512\n",
    "hidden_activation = tf.nn.relu\n",
    "n_outputs = env.action_space.n  # 9 discrete actions are available\n",
    "initializer = tf.contrib.layers.variance_scaling_initializer()\n",
    "\n",
    "def q_network(X_state, name):\n",
    "    prev_layer = X_state\n",
    "    with tf.variable_scope(name) as scope:\n",
    "        for n_maps, kernel_size, strides, padding, activation in zip(\n",
    "                conv_n_maps, conv_kernel_sizes, conv_strides,\n",
    "                conv_paddings, conv_activation):\n",
    "            prev_layer = tf.layers.conv2d(\n",
    "                prev_layer, filters=n_maps, kernel_size=kernel_size,\n",
    "                strides=strides, padding=padding, activation=activation,\n",
    "                kernel_initializer=initializer)\n",
    "        last_conv_layer_flat = tf.reshape(prev_layer, shape=[-1, n_hidden_in])\n",
    "        hidden = tf.layers.dense(last_conv_layer_flat, n_hidden,\n",
    "                                 activation=hidden_activation,\n",
    "                                 kernel_initializer=initializer)\n",
    "        outputs = tf.layers.dense(hidden, n_outputs,\n",
    "                                  kernel_initializer=initializer)\n",
    "    trainable_vars = tf.get_collection(tf.GraphKeys.TRAINABLE_VARIABLES,\n",
    "                                       scope=scope.name)\n",
    "    trainable_vars_by_name = {var.name[len(scope.name):]: var\n",
    "                              for var in trainable_vars}\n",
    "    return outputs, trainable_vars_by_name"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "X_state = tf.placeholder(tf.float32, shape=[None, input_height, input_width,\n",
    "                                            input_channels])\n",
    "online_q_values, online_vars = q_network(X_state, name=\"q_networks/online\")\n",
    "target_q_values, target_vars = q_network(X_state, name=\"q_networks/target\")\n",
    "\n",
    "copy_ops = [target_var.assign(online_vars[var_name])\n",
    "            for var_name, target_var in target_vars.items()]\n",
    "copy_online_to_target = tf.group(*copy_ops)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'/conv2d/bias:0': <tf.Variable 'q_networks/online/conv2d/bias:0' shape=(32,) dtype=float32_ref>,\n",
       " '/conv2d/kernel:0': <tf.Variable 'q_networks/online/conv2d/kernel:0' shape=(8, 8, 1, 32) dtype=float32_ref>,\n",
       " '/conv2d_1/bias:0': <tf.Variable 'q_networks/online/conv2d_1/bias:0' shape=(64,) dtype=float32_ref>,\n",
       " '/conv2d_1/kernel:0': <tf.Variable 'q_networks/online/conv2d_1/kernel:0' shape=(4, 4, 32, 64) dtype=float32_ref>,\n",
       " '/conv2d_2/bias:0': <tf.Variable 'q_networks/online/conv2d_2/bias:0' shape=(64,) dtype=float32_ref>,\n",
       " '/conv2d_2/kernel:0': <tf.Variable 'q_networks/online/conv2d_2/kernel:0' shape=(3, 3, 64, 64) dtype=float32_ref>,\n",
       " '/dense/bias:0': <tf.Variable 'q_networks/online/dense/bias:0' shape=(512,) dtype=float32_ref>,\n",
       " '/dense/kernel:0': <tf.Variable 'q_networks/online/dense/kernel:0' shape=(7040, 512) dtype=float32_ref>,\n",
       " '/dense_1/bias:0': <tf.Variable 'q_networks/online/dense_1/bias:0' shape=(9,) dtype=float32_ref>,\n",
       " '/dense_1/kernel:0': <tf.Variable 'q_networks/online/dense_1/kernel:0' shape=(512, 9) dtype=float32_ref>}"
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "online_vars"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "learning_rate = 0.001\n",
    "momentum = 0.95\n",
    "\n",
    "with tf.variable_scope(\"train\"):\n",
    "    X_action = tf.placeholder(tf.int32, shape=[None])\n",
    "    y = tf.placeholder(tf.float32, shape=[None, 1])\n",
    "    q_value = tf.reduce_sum(online_q_values * tf.one_hot(X_action, n_outputs),\n",
    "                            axis=1, keep_dims=True)\n",
    "    error = tf.abs(y - q_value)\n",
    "    clipped_error = tf.clip_by_value(error, 0.0, 1.0)\n",
    "    linear_error = 2 * (error - clipped_error)\n",
    "    loss = tf.reduce_mean(tf.square(clipped_error) + linear_error)\n",
    "\n",
    "    global_step = tf.Variable(0, trainable=False, name='global_step')\n",
    "    optimizer = tf.train.MomentumOptimizer(learning_rate, momentum, use_nesterov=True)\n",
    "    training_op = optimizer.minimize(loss, global_step=global_step)\n",
    "\n",
    "init = tf.global_variables_initializer()\n",
    "saver = tf.train.Saver()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note: in the first version of the book, the loss function was simply the squared error between the target Q-Values (`y`) and the estimated Q-Values (`q_value`). However, because the experiences are very noisy, it is better to use a quadratic loss only for small errors (below 1.0) and a linear loss (twice the absolute error) for larger errors, which is what the code above computes. This way large errors don't push the model parameters around as much. Note that we also tweaked some hyperparameters (using a smaller learning rate, and using Nesterov Accelerated Gradients rather than Adam optimization, since adaptive gradient algorithms may sometimes be bad, according to this [paper](https://arxiv.org/abs/1705.08292)). We also tweaked a few other hyperparameters below (a larger replay memory, longer decay for the $\\epsilon$-greedy policy, larger discount rate, less frequent copies of the online DQN to the target DQN, etc.)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from collections import deque\n",
    "\n",
    "replay_memory_size = 500000\n",
    "replay_memory = deque([], maxlen=replay_memory_size)\n",
    "\n",
    "def sample_memories(batch_size):\n",
    "    indices = np.random.permutation(len(replay_memory))[:batch_size]\n",
    "    cols = [[], [], [], [], []] # state, action, reward, next_state, continue\n",
    "    for idx in indices:\n",
    "        memory = replay_memory[idx]\n",
    "        for col, value in zip(cols, memory):\n",
    "            col.append(value)\n",
    "    cols = [np.array(col) for col in cols]\n",
    "    return cols[0], cols[1], cols[2].reshape(-1, 1), cols[3], cols[4].reshape(-1, 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "eps_min = 0.1\n",
    "eps_max = 1.0\n",
    "eps_decay_steps = 2000000\n",
    "\n",
    "def epsilon_greedy(q_values, step):\n",
    "    epsilon = max(eps_min, eps_max - (eps_max-eps_min) * step/eps_decay_steps)\n",
    "    if np.random.rand() < epsilon:\n",
    "        return np.random.randint(n_outputs) # random action\n",
    "    else:\n",
    "        return np.argmax(q_values) # optimal action"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {},
   "outputs": [],
   "source": [
    "n_steps = 4000000  # total number of training steps\n",
    "training_start = 10000  # start training after 10,000 game iterations\n",
    "training_interval = 4  # run a training step every 4 game iterations\n",
    "save_steps = 1000  # save the model every 1,000 training steps\n",
    "copy_steps = 10000  # copy online DQN to target DQN every 10,000 training steps\n",
    "discount_rate = 0.99\n",
    "skip_start = 90  # Skip the start of every game (it's just waiting time).\n",
    "batch_size = 50\n",
    "iteration = 0  # game iterations\n",
    "checkpoint_path = \"./my_dqn.ckpt\"\n",
    "done = True # env needs to be reset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A few variables for tracking progress:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "loss_val = np.infty\n",
    "game_length = 0\n",
    "total_max_q = 0\n",
    "mean_max_q = 0.0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And now the main training loop!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Restoring parameters from ./my_dqn.ckpt\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[2017-09-25 13:55:15,610] Restoring parameters from ./my_dqn.ckpt\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\r",
      "Iteration 1270171\tTraining step 315001/4000000 (7.9)%\tLoss 2.651937\tMean Max-Q 30.964941   "
     ]
    }
   ],
   "source": [
    "with tf.Session() as sess:\n",
    "    if os.path.isfile(checkpoint_path + \".index\"):\n",
    "        saver.restore(sess, checkpoint_path)\n",
    "    else:\n",
    "        init.run()\n",
    "        copy_online_to_target.run()\n",
    "    while True:\n",
    "        step = global_step.eval()\n",
    "        if step >= n_steps:\n",
    "            break\n",
    "        iteration += 1\n",
    "        print(\"\\rIteration {}\\tTraining step {}/{} ({:.1f})%\\tLoss {:5f}\\tMean Max-Q {:5f}   \".format(\n",
    "            iteration, step, n_steps, step * 100 / n_steps, loss_val, mean_max_q), end=\"\")\n",
    "        if done: # game over, start again\n",
    "            obs = env.reset()\n",
    "            for skip in range(skip_start): # skip the start of each game\n",
    "                obs, reward, done, info = env.step(0)\n",
    "            state = preprocess_observation(obs)\n",
    "\n",
    "        # Online DQN evaluates what to do\n",
    "        q_values = online_q_values.eval(feed_dict={X_state: [state]})\n",
    "        action = epsilon_greedy(q_values, step)\n",
    "\n",
    "        # Online DQN plays\n",
    "        obs, reward, done, info = env.step(action)\n",
    "        next_state = preprocess_observation(obs)\n",
    "\n",
    "        # Let's memorize what happened\n",
    "        replay_memory.append((state, action, reward, next_state, 1.0 - done))\n",
    "        state = next_state\n",
    "\n",
    "        # Compute statistics for tracking progress (not shown in the book)\n",
    "        total_max_q += q_values.max()\n",
    "        game_length += 1\n",
    "        if done:\n",
    "            mean_max_q = total_max_q / game_length\n",
    "            total_max_q = 0.0\n",
    "            game_length = 0\n",
    "\n",
    "        if iteration < training_start or iteration % training_interval != 0:\n",
    "            continue # only train after warmup period and at regular intervals\n",
    "        \n",
    "        # Sample memories and use the target DQN to produce the target Q-Value\n",
    "        X_state_val, X_action_val, rewards, X_next_state_val, continues = (\n",
    "            sample_memories(batch_size))\n",
    "        next_q_values = target_q_values.eval(\n",
    "            feed_dict={X_state: X_next_state_val})\n",
    "        max_next_q_values = np.max(next_q_values, axis=1, keepdims=True)\n",
    "        y_val = rewards + continues * discount_rate * max_next_q_values\n",
    "\n",
    "        # Train the online DQN\n",
    "        _, loss_val = sess.run([training_op, loss], feed_dict={\n",
    "            X_state: X_state_val, X_action: X_action_val, y: y_val})\n",
    "\n",
    "        # Regularly copy the online DQN to the target DQN\n",
    "        if step % copy_steps == 0:\n",
    "            copy_online_to_target.run()\n",
    "\n",
    "        # And save regularly\n",
    "        if step % save_steps == 0:\n",
    "            saver.save(sess, checkpoint_path)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can interrupt the cell above at any time to test your agent using the cell below. You can then run the cell above once again, it will load the last parameters saved and resume training."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Restoring parameters from ./my_dqn.ckpt\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[2017-09-25 13:53:39,307] Restoring parameters from ./my_dqn.ckpt\n"
     ]
    }
   ],
   "source": [
    "frames = []\n",
    "n_max_steps = 10000\n",
    "\n",
    "with tf.Session() as sess:\n",
    "    saver.restore(sess, checkpoint_path)\n",
    "\n",
    "    obs = env.reset()\n",
    "    for step in range(n_max_steps):\n",
    "        state = preprocess_observation(obs)\n",
    "\n",
    "        # Online DQN evaluates what to do\n",
    "        q_values = online_q_values.eval(feed_dict={X_state: [state]})\n",
    "        action = np.argmax(q_values)\n",
    "\n",
    "        # Online DQN plays\n",
    "        obs, reward, done, info = env.step(action)\n",
    "\n",
    "        img = env.render(mode=\"rgb_array\")\n",
    "        frames.append(img)\n",
    "\n",
    "        if done:\n",
    "            break"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "application/javascript": [
       "/* Put everything inside the global mpl namespace */\n",
       "window.mpl = {};\n",
       "\n",
       "\n",
       "mpl.get_websocket_type = function() {\n",
       "    if (typeof(WebSocket) !== 'undefined') {\n",
       "        return WebSocket;\n",
       "    } else if (typeof(MozWebSocket) !== 'undefined') {\n",
       "        return MozWebSocket;\n",
       "    } else {\n",
       "        alert('Your browser does not have WebSocket support.' +\n",
       "              'Please try Chrome, Safari or Firefox ≥ 6. ' +\n",
       "              'Firefox 4 and 5 are also supported but you ' +\n",
       "              'have to enable WebSockets in about:config.');\n",
       "    };\n",
       "}\n",
       "\n",
       "mpl.figure = function(figure_id, websocket, ondownload, parent_element) {\n",
       "    this.id = figure_id;\n",
       "\n",
       "    this.ws = websocket;\n",
       "\n",
       "    this.supports_binary = (this.ws.binaryType != undefined);\n",
       "\n",
       "    if (!this.supports_binary) {\n",
       "        var warnings = document.getElementById(\"mpl-warnings\");\n",
       "        if (warnings) {\n",
       "            warnings.style.display = 'block';\n",
       "            warnings.textContent = (\n",
       "                \"This browser does not support binary websocket messages. \" +\n",
       "                    \"Performance may be slow.\");\n",
       "        }\n",
       "    }\n",
       "\n",
       "    this.imageObj = new Image();\n",
       "\n",
       "    this.context = undefined;\n",
       "    this.message = undefined;\n",
       "    this.canvas = undefined;\n",
       "    this.rubberband_canvas = undefined;\n",
       "    this.rubberband_context = undefined;\n",
       "    this.format_dropdown = undefined;\n",
       "\n",
       "    this.image_mode = 'full';\n",
       "\n",
       "    this.root = $('<div/>');\n",
       "    this._root_extra_style(this.root)\n",
       "    this.root.attr('style', 'display: inline-block');\n",
       "\n",
       "    $(parent_element).append(this.root);\n",
       "\n",
       "    this._init_header(this);\n",
       "    this._init_canvas(this);\n",
       "    this._init_toolbar(this);\n",
       "\n",
       "    var fig = this;\n",
       "\n",
       "    this.waiting = false;\n",
       "\n",
       "    this.ws.onopen =  function () {\n",
       "            fig.send_message(\"supports_binary\", {value: fig.supports_binary});\n",
       "            fig.send_message(\"send_image_mode\", {});\n",
       "            if (mpl.ratio != 1) {\n",
       "                fig.send_message(\"set_dpi_ratio\", {'dpi_ratio': mpl.ratio});\n",
       "            }\n",
       "            fig.send_message(\"refresh\", {});\n",
       "        }\n",
       "\n",
       "    this.imageObj.onload = function() {\n",
       "            if (fig.image_mode == 'full') {\n",
       "                // Full images could contain transparency (where diff images\n",
       "                // almost always do), so we need to clear the canvas so that\n",
       "                // there is no ghosting.\n",
       "                fig.context.clearRect(0, 0, fig.canvas.width, fig.canvas.height);\n",
       "            }\n",
       "            fig.context.drawImage(fig.imageObj, 0, 0);\n",
       "        };\n",
       "\n",
       "    this.imageObj.onunload = function() {\n",
       "        this.ws.close();\n",
       "    }\n",
       "\n",
       "    this.ws.onmessage = this._make_on_message_function(this);\n",
       "\n",
       "    this.ondownload = ondownload;\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._init_header = function() {\n",
       "    var titlebar = $(\n",
       "        '<div class=\"ui-dialog-titlebar ui-widget-header ui-corner-all ' +\n",
       "        'ui-helper-clearfix\"/>');\n",
       "    var titletext = $(\n",
       "        '<div class=\"ui-dialog-title\" style=\"width: 100%; ' +\n",
       "        'text-align: center; padding: 3px;\"/>');\n",
       "    titlebar.append(titletext)\n",
       "    this.root.append(titlebar);\n",
       "    this.header = titletext[0];\n",
       "}\n",
       "\n",
       "\n",
       "\n",
       "mpl.figure.prototype._canvas_extra_style = function(canvas_div) {\n",
       "\n",
       "}\n",
       "\n",
       "\n",
       "mpl.figure.prototype._root_extra_style = function(canvas_div) {\n",
       "\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._init_canvas = function() {\n",
       "    var fig = this;\n",
       "\n",
       "    var canvas_div = $('<div/>');\n",
       "\n",
       "    canvas_div.attr('style', 'position: relative; clear: both; outline: 0');\n",
       "\n",
       "    function canvas_keyboard_event(event) {\n",
       "        return fig.key_event(event, event['data']);\n",
       "    }\n",
       "\n",
       "    canvas_div.keydown('key_press', canvas_keyboard_event);\n",
       "    canvas_div.keyup('key_release', canvas_keyboard_event);\n",
       "    this.canvas_div = canvas_div\n",
       "    this._canvas_extra_style(canvas_div)\n",
       "    this.root.append(canvas_div);\n",
       "\n",
       "    var canvas = $('<canvas/>');\n",
       "    canvas.addClass('mpl-canvas');\n",
       "    canvas.attr('style', \"left: 0; top: 0; z-index: 0; outline: 0\")\n",
       "\n",
       "    this.canvas = canvas[0];\n",
       "    this.context = canvas[0].getContext(\"2d\");\n",
       "\n",
       "    var backingStore = this.context.backingStorePixelRatio ||\n",
       "\tthis.context.webkitBackingStorePixelRatio ||\n",
       "\tthis.context.mozBackingStorePixelRatio ||\n",
       "\tthis.context.msBackingStorePixelRatio ||\n",
       "\tthis.context.oBackingStorePixelRatio ||\n",
       "\tthis.context.backingStorePixelRatio || 1;\n",
       "\n",
       "    mpl.ratio = (window.devicePixelRatio || 1) / backingStore;\n",
       "\n",
       "    var rubberband = $('<canvas/>');\n",
       "    rubberband.attr('style', \"position: absolute; left: 0; top: 0; z-index: 1;\")\n",
       "\n",
       "    var pass_mouse_events = true;\n",
       "\n",
       "    canvas_div.resizable({\n",
       "        start: function(event, ui) {\n",
       "            pass_mouse_events = false;\n",
       "        },\n",
       "        resize: function(event, ui) {\n",
       "            fig.request_resize(ui.size.width, ui.size.height);\n",
       "        },\n",
       "        stop: function(event, ui) {\n",
       "            pass_mouse_events = true;\n",
       "            fig.request_resize(ui.size.width, ui.size.height);\n",
       "        },\n",
       "    });\n",
       "\n",
       "    function mouse_event_fn(event) {\n",
       "        if (pass_mouse_events)\n",
       "            return fig.mouse_event(event, event['data']);\n",
       "    }\n",
       "\n",
       "    rubberband.mousedown('button_press', mouse_event_fn);\n",
       "    rubberband.mouseup('button_release', mouse_event_fn);\n",
       "    // Throttle sequential mouse events to 1 every 20ms.\n",
       "    rubberband.mousemove('motion_notify', mouse_event_fn);\n",
       "\n",
       "    rubberband.mouseenter('figure_enter', mouse_event_fn);\n",
       "    rubberband.mouseleave('figure_leave', mouse_event_fn);\n",
       "\n",
       "    canvas_div.on(\"wheel\", function (event) {\n",
       "        event = event.originalEvent;\n",
       "        event['data'] = 'scroll'\n",
       "        if (event.deltaY < 0) {\n",
       "            event.step = 1;\n",
       "        } else {\n",
       "            event.step = -1;\n",
       "        }\n",
       "        mouse_event_fn(event);\n",
       "    });\n",
       "\n",
       "    canvas_div.append(canvas);\n",
       "    canvas_div.append(rubberband);\n",
       "\n",
       "    this.rubberband = rubberband;\n",
       "    this.rubberband_canvas = rubberband[0];\n",
       "    this.rubberband_context = rubberband[0].getContext(\"2d\");\n",
       "    this.rubberband_context.strokeStyle = \"#000000\";\n",
       "\n",
       "    this._resize_canvas = function(width, height) {\n",
       "        // Keep the size of the canvas, canvas container, and rubber band\n",
       "        // canvas in synch.\n",
       "        canvas_div.css('width', width)\n",
       "        canvas_div.css('height', height)\n",
       "\n",
       "        canvas.attr('width', width * mpl.ratio);\n",
       "        canvas.attr('height', height * mpl.ratio);\n",
       "        canvas.attr('style', 'width: ' + width + 'px; height: ' + height + 'px;');\n",
       "\n",
       "        rubberband.attr('width', width);\n",
       "        rubberband.attr('height', height);\n",
       "    }\n",
       "\n",
       "    // Set the figure to an initial 600x600px, this will subsequently be updated\n",
       "    // upon first draw.\n",
       "    this._resize_canvas(600, 600);\n",
       "\n",
       "    // Disable right mouse context menu.\n",
       "    $(this.rubberband_canvas).bind(\"contextmenu\",function(e){\n",
       "        return false;\n",
       "    });\n",
       "\n",
       "    function set_focus () {\n",
       "        canvas.focus();\n",
       "        canvas_div.focus();\n",
       "    }\n",
       "\n",
       "    window.setTimeout(set_focus, 100);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._init_toolbar = function() {\n",
       "    var fig = this;\n",
       "\n",
       "    var nav_element = $('<div/>')\n",
       "    nav_element.attr('style', 'width: 100%');\n",
       "    this.root.append(nav_element);\n",
       "\n",
       "    // Define a callback function for later on.\n",
       "    function toolbar_event(event) {\n",
       "        return fig.toolbar_button_onclick(event['data']);\n",
       "    }\n",
       "    function toolbar_mouse_event(event) {\n",
       "        return fig.toolbar_button_onmouseover(event['data']);\n",
       "    }\n",
       "\n",
       "    for(var toolbar_ind in mpl.toolbar_items) {\n",
       "        var name = mpl.toolbar_items[toolbar_ind][0];\n",
       "        var tooltip = mpl.toolbar_items[toolbar_ind][1];\n",
       "        var image = mpl.toolbar_items[toolbar_ind][2];\n",
       "        var method_name = mpl.toolbar_items[toolbar_ind][3];\n",
       "\n",
       "        if (!name) {\n",
       "            // put a spacer in here.\n",
       "            continue;\n",
       "        }\n",
       "        var button = $('<button/>');\n",
       "        button.addClass('ui-button ui-widget ui-state-default ui-corner-all ' +\n",
       "                        'ui-button-icon-only');\n",
       "        button.attr('role', 'button');\n",
       "        button.attr('aria-disabled', 'false');\n",
       "        button.click(method_name, toolbar_event);\n",
       "        button.mouseover(tooltip, toolbar_mouse_event);\n",
       "\n",
       "        var icon_img = $('<span/>');\n",
       "        icon_img.addClass('ui-button-icon-primary ui-icon');\n",
       "        icon_img.addClass(image);\n",
       "        icon_img.addClass('ui-corner-all');\n",
       "\n",
       "        var tooltip_span = $('<span/>');\n",
       "        tooltip_span.addClass('ui-button-text');\n",
       "        tooltip_span.html(tooltip);\n",
       "\n",
       "        button.append(icon_img);\n",
       "        button.append(tooltip_span);\n",
       "\n",
       "        nav_element.append(button);\n",
       "    }\n",
       "\n",
       "    var fmt_picker_span = $('<span/>');\n",
       "\n",
       "    var fmt_picker = $('<select/>');\n",
       "    fmt_picker.addClass('mpl-toolbar-option ui-widget ui-widget-content');\n",
       "    fmt_picker_span.append(fmt_picker);\n",
       "    nav_element.append(fmt_picker_span);\n",
       "    this.format_dropdown = fmt_picker[0];\n",
       "\n",
       "    for (var ind in mpl.extensions) {\n",
       "        var fmt = mpl.extensions[ind];\n",
       "        var option = $(\n",
       "            '<option/>', {selected: fmt === mpl.default_extension}).html(fmt);\n",
       "        fmt_picker.append(option)\n",
       "    }\n",
       "\n",
       "    // Add hover states to the ui-buttons\n",
       "    $( \".ui-button\" ).hover(\n",
       "        function() { $(this).addClass(\"ui-state-hover\");},\n",
       "        function() { $(this).removeClass(\"ui-state-hover\");}\n",
       "    );\n",
       "\n",
       "    var status_bar = $('<span class=\"mpl-message\"/>');\n",
       "    nav_element.append(status_bar);\n",
       "    this.message = status_bar[0];\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.request_resize = function(x_pixels, y_pixels) {\n",
       "    // Request matplotlib to resize the figure. Matplotlib will then trigger a resize in the client,\n",
       "    // which will in turn request a refresh of the image.\n",
       "    this.send_message('resize', {'width': x_pixels, 'height': y_pixels});\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.send_message = function(type, properties) {\n",
       "    properties['type'] = type;\n",
       "    properties['figure_id'] = this.id;\n",
       "    this.ws.send(JSON.stringify(properties));\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.send_draw_message = function() {\n",
       "    if (!this.waiting) {\n",
       "        this.waiting = true;\n",
       "        this.ws.send(JSON.stringify({type: \"draw\", figure_id: this.id}));\n",
       "    }\n",
       "}\n",
       "\n",
       "\n",
       "mpl.figure.prototype.handle_save = function(fig, msg) {\n",
       "    var format_dropdown = fig.format_dropdown;\n",
       "    var format = format_dropdown.options[format_dropdown.selectedIndex].value;\n",
       "    fig.ondownload(fig, format);\n",
       "}\n",
       "\n",
       "\n",
       "mpl.figure.prototype.handle_resize = function(fig, msg) {\n",
       "    var size = msg['size'];\n",
       "    if (size[0] != fig.canvas.width || size[1] != fig.canvas.height) {\n",
       "        fig._resize_canvas(size[0], size[1]);\n",
       "        fig.send_message(\"refresh\", {});\n",
       "    };\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_rubberband = function(fig, msg) {\n",
       "    var x0 = msg['x0'] / mpl.ratio;\n",
       "    var y0 = (fig.canvas.height - msg['y0']) / mpl.ratio;\n",
       "    var x1 = msg['x1'] / mpl.ratio;\n",
       "    var y1 = (fig.canvas.height - msg['y1']) / mpl.ratio;\n",
       "    x0 = Math.floor(x0) + 0.5;\n",
       "    y0 = Math.floor(y0) + 0.5;\n",
       "    x1 = Math.floor(x1) + 0.5;\n",
       "    y1 = Math.floor(y1) + 0.5;\n",
       "    var min_x = Math.min(x0, x1);\n",
       "    var min_y = Math.min(y0, y1);\n",
       "    var width = Math.abs(x1 - x0);\n",
       "    var height = Math.abs(y1 - y0);\n",
       "\n",
       "    fig.rubberband_context.clearRect(\n",
       "        0, 0, fig.canvas.width, fig.canvas.height);\n",
       "\n",
       "    fig.rubberband_context.strokeRect(min_x, min_y, width, height);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_figure_label = function(fig, msg) {\n",
       "    // Updates the figure title.\n",
       "    fig.header.textContent = msg['label'];\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_cursor = function(fig, msg) {\n",
       "    var cursor = msg['cursor'];\n",
       "    switch(cursor)\n",
       "    {\n",
       "    case 0:\n",
       "        cursor = 'pointer';\n",
       "        break;\n",
       "    case 1:\n",
       "        cursor = 'default';\n",
       "        break;\n",
       "    case 2:\n",
       "        cursor = 'crosshair';\n",
       "        break;\n",
       "    case 3:\n",
       "        cursor = 'move';\n",
       "        break;\n",
       "    }\n",
       "    fig.rubberband_canvas.style.cursor = cursor;\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_message = function(fig, msg) {\n",
       "    fig.message.textContent = msg['message'];\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_draw = function(fig, msg) {\n",
       "    // Request the server to send over a new figure.\n",
       "    fig.send_draw_message();\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_image_mode = function(fig, msg) {\n",
       "    fig.image_mode = msg['mode'];\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.updated_canvas_event = function() {\n",
       "    // Called whenever the canvas gets updated.\n",
       "    this.send_message(\"ack\", {});\n",
       "}\n",
       "\n",
       "// A function to construct a web socket function for onmessage handling.\n",
       "// Called in the figure constructor.\n",
       "mpl.figure.prototype._make_on_message_function = function(fig) {\n",
       "    return function socket_on_message(evt) {\n",
       "        if (evt.data instanceof Blob) {\n",
       "            /* FIXME: We get \"Resource interpreted as Image but\n",
       "             * transferred with MIME type text/plain:\" errors on\n",
       "             * Chrome.  But how to set the MIME type?  It doesn't seem\n",
       "             * to be part of the websocket stream */\n",
       "            evt.data.type = \"image/png\";\n",
       "\n",
       "            /* Free the memory for the previous frames */\n",
       "            if (fig.imageObj.src) {\n",
       "                (window.URL || window.webkitURL).revokeObjectURL(\n",
       "                    fig.imageObj.src);\n",
       "            }\n",
       "\n",
       "            fig.imageObj.src = (window.URL || window.webkitURL).createObjectURL(\n",
       "                evt.data);\n",
       "            fig.updated_canvas_event();\n",
       "            fig.waiting = false;\n",
       "            return;\n",
       "        }\n",
       "        else if (typeof evt.data === 'string' && evt.data.slice(0, 21) == \"data:image/png;base64\") {\n",
       "            fig.imageObj.src = evt.data;\n",
       "            fig.updated_canvas_event();\n",
       "            fig.waiting = false;\n",
       "            return;\n",
       "        }\n",
       "\n",
       "        var msg = JSON.parse(evt.data);\n",
       "        var msg_type = msg['type'];\n",
       "\n",
       "        // Call the  \"handle_{type}\" callback, which takes\n",
       "        // the figure and JSON message as its only arguments.\n",
       "        try {\n",
       "            var callback = fig[\"handle_\" + msg_type];\n",
       "        } catch (e) {\n",
       "            console.log(\"No handler for the '\" + msg_type + \"' message type: \", msg);\n",
       "            return;\n",
       "        }\n",
       "\n",
       "        if (callback) {\n",
       "            try {\n",
       "                // console.log(\"Handling '\" + msg_type + \"' message: \", msg);\n",
       "                callback(fig, msg);\n",
       "            } catch (e) {\n",
       "                console.log(\"Exception inside the 'handler_\" + msg_type + \"' callback:\", e, e.stack, msg);\n",
       "            }\n",
       "        }\n",
       "    };\n",
       "}\n",
       "\n",
       "// from http://stackoverflow.com/questions/1114465/getting-mouse-location-in-canvas\n",
       "mpl.findpos = function(e) {\n",
       "    //this section is from http://www.quirksmode.org/js/events_properties.html\n",
       "    var targ;\n",
       "    if (!e)\n",
       "        e = window.event;\n",
       "    if (e.target)\n",
       "        targ = e.target;\n",
       "    else if (e.srcElement)\n",
       "        targ = e.srcElement;\n",
       "    if (targ.nodeType == 3) // defeat Safari bug\n",
       "        targ = targ.parentNode;\n",
       "\n",
       "    // jQuery normalizes the pageX and pageY\n",
       "    // pageX,Y are the mouse positions relative to the document\n",
       "    // offset() returns the position of the element relative to the document\n",
       "    var x = e.pageX - $(targ).offset().left;\n",
       "    var y = e.pageY - $(targ).offset().top;\n",
       "\n",
       "    return {\"x\": x, \"y\": y};\n",
       "};\n",
       "\n",
       "/*\n",
       " * return a copy of an object with only non-object keys\n",
       " * we need this to avoid circular references\n",
       " * http://stackoverflow.com/a/24161582/3208463\n",
       " */\n",
       "function simpleKeys (original) {\n",
       "  return Object.keys(original).reduce(function (obj, key) {\n",
       "    if (typeof original[key] !== 'object')\n",
       "        obj[key] = original[key]\n",
       "    return obj;\n",
       "  }, {});\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.mouse_event = function(event, name) {\n",
       "    var canvas_pos = mpl.findpos(event)\n",
       "\n",
       "    if (name === 'button_press')\n",
       "    {\n",
       "        this.canvas.focus();\n",
       "        this.canvas_div.focus();\n",
       "    }\n",
       "\n",
       "    var x = canvas_pos.x * mpl.ratio;\n",
       "    var y = canvas_pos.y * mpl.ratio;\n",
       "\n",
       "    this.send_message(name, {x: x, y: y, button: event.button,\n",
       "                             step: event.step,\n",
       "                             guiEvent: simpleKeys(event)});\n",
       "\n",
       "    /* This prevents the web browser from automatically changing to\n",
       "     * the text insertion cursor when the button is pressed.  We want\n",
       "     * to control all of the cursor setting manually through the\n",
       "     * 'cursor' event from matplotlib */\n",
       "    event.preventDefault();\n",
       "    return false;\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._key_event_extra = function(event, name) {\n",
       "    // Handle any extra behaviour associated with a key event\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.key_event = function(event, name) {\n",
       "\n",
       "    // Prevent repeat events\n",
       "    if (name == 'key_press')\n",
       "    {\n",
       "        if (event.which === this._key)\n",
       "            return;\n",
       "        else\n",
       "            this._key = event.which;\n",
       "    }\n",
       "    if (name == 'key_release')\n",
       "        this._key = null;\n",
       "\n",
       "    var value = '';\n",
       "    if (event.ctrlKey && event.which != 17)\n",
       "        value += \"ctrl+\";\n",
       "    if (event.altKey && event.which != 18)\n",
       "        value += \"alt+\";\n",
       "    if (event.shiftKey && event.which != 16)\n",
       "        value += \"shift+\";\n",
       "\n",
       "    value += 'k';\n",
       "    value += event.which.toString();\n",
       "\n",
       "    this._key_event_extra(event, name);\n",
       "\n",
       "    this.send_message(name, {key: value,\n",
       "                             guiEvent: simpleKeys(event)});\n",
       "    return false;\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.toolbar_button_onclick = function(name) {\n",
       "    if (name == 'download') {\n",
       "        this.handle_save(this, null);\n",
       "    } else {\n",
       "        this.send_message(\"toolbar_button\", {name: name});\n",
       "    }\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.toolbar_button_onmouseover = function(tooltip) {\n",
       "    this.message.textContent = tooltip;\n",
       "};\n",
       "mpl.toolbar_items = [[\"Home\", \"Reset original view\", \"fa fa-home icon-home\", \"home\"], [\"Back\", \"Back to  previous view\", \"fa fa-arrow-left icon-arrow-left\", \"back\"], [\"Forward\", \"Forward to next view\", \"fa fa-arrow-right icon-arrow-right\", \"forward\"], [\"\", \"\", \"\", \"\"], [\"Pan\", \"Pan axes with left mouse, zoom with right\", \"fa fa-arrows icon-move\", \"pan\"], [\"Zoom\", \"Zoom to rectangle\", \"fa fa-square-o icon-check-empty\", \"zoom\"], [\"\", \"\", \"\", \"\"], [\"Download\", \"Download plot\", \"fa fa-floppy-o icon-save\", \"download\"]];\n",
       "\n",
       "mpl.extensions = [\"eps\", \"jpeg\", \"pdf\", \"png\", \"ps\", \"raw\", \"svg\", \"tif\"];\n",
       "\n",
       "mpl.default_extension = \"png\";var comm_websocket_adapter = function(comm) {\n",
       "    // Create a \"websocket\"-like object which calls the given IPython comm\n",
       "    // object with the appropriate methods. Currently this is a non binary\n",
       "    // socket, so there is still some room for performance tuning.\n",
       "    var ws = {};\n",
       "\n",
       "    ws.close = function() {\n",
       "        comm.close()\n",
       "    };\n",
       "    ws.send = function(m) {\n",
       "        //console.log('sending', m);\n",
       "        comm.send(m);\n",
       "    };\n",
       "    // Register the callback with on_msg.\n",
       "    comm.on_msg(function(msg) {\n",
       "        //console.log('receiving', msg['content']['data'], msg);\n",
       "        // Pass the mpl event to the overriden (by mpl) onmessage function.\n",
       "        ws.onmessage(msg['content']['data'])\n",
       "    });\n",
       "    return ws;\n",
       "}\n",
       "\n",
       "mpl.mpl_figure_comm = function(comm, msg) {\n",
       "    // This is the function which gets called when the mpl process\n",
       "    // starts-up an IPython Comm through the \"matplotlib\" channel.\n",
       "\n",
       "    var id = msg.content.data.id;\n",
       "    // Get hold of the div created by the display call when the Comm\n",
       "    // socket was opened in Python.\n",
       "    var element = $(\"#\" + id);\n",
       "    var ws_proxy = comm_websocket_adapter(comm)\n",
       "\n",
       "    function ondownload(figure, format) {\n",
       "        window.open(figure.imageObj.src);\n",
       "    }\n",
       "\n",
       "    var fig = new mpl.figure(id, ws_proxy,\n",
       "                           ondownload,\n",
       "                           element.get(0));\n",
       "\n",
       "    // Call onopen now - mpl needs it, as it is assuming we've passed it a real\n",
       "    // web socket which is closed, not our websocket->open comm proxy.\n",
       "    ws_proxy.onopen();\n",
       "\n",
       "    fig.parent_element = element.get(0);\n",
       "    fig.cell_info = mpl.find_output_cell(\"<div id='\" + id + \"'></div>\");\n",
       "    if (!fig.cell_info) {\n",
       "        console.error(\"Failed to find cell for figure\", id, fig);\n",
       "        return;\n",
       "    }\n",
       "\n",
       "    var output_index = fig.cell_info[2]\n",
       "    var cell = fig.cell_info[0];\n",
       "\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_close = function(fig, msg) {\n",
       "    var width = fig.canvas.width/mpl.ratio\n",
       "    fig.root.unbind('remove')\n",
       "\n",
       "    // Update the output cell to use the data from the current canvas.\n",
       "    fig.push_to_output();\n",
       "    var dataURL = fig.canvas.toDataURL();\n",
       "    // Re-enable the keyboard manager in IPython - without this line, in FF,\n",
       "    // the notebook keyboard shortcuts fail.\n",
       "    IPython.keyboard_manager.enable()\n",
       "    $(fig.parent_element).html('<img src=\"' + dataURL + '\" width=\"' + width + '\">');\n",
       "    fig.close_ws(fig, msg);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.close_ws = function(fig, msg){\n",
       "    fig.send_message('closing', msg);\n",
       "    // fig.ws.close()\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.push_to_output = function(remove_interactive) {\n",
       "    // Turn the data on the canvas into data in the output cell.\n",
       "    var width = this.canvas.width/mpl.ratio\n",
       "    var dataURL = this.canvas.toDataURL();\n",
       "    this.cell_info[1]['text/html'] = '<img src=\"' + dataURL + '\" width=\"' + width + '\">';\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.updated_canvas_event = function() {\n",
       "    // Tell IPython that the notebook contents must change.\n",
       "    IPython.notebook.set_dirty(true);\n",
       "    this.send_message(\"ack\", {});\n",
       "    var fig = this;\n",
       "    // Wait a second, then push the new image to the DOM so\n",
       "    // that it is saved nicely (might be nice to debounce this).\n",
       "    setTimeout(function () { fig.push_to_output() }, 1000);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._init_toolbar = function() {\n",
       "    var fig = this;\n",
       "\n",
       "    var nav_element = $('<div/>')\n",
       "    nav_element.attr('style', 'width: 100%');\n",
       "    this.root.append(nav_element);\n",
       "\n",
       "    // Define a callback function for later on.\n",
       "    function toolbar_event(event) {\n",
       "        return fig.toolbar_button_onclick(event['data']);\n",
       "    }\n",
       "    function toolbar_mouse_event(event) {\n",
       "        return fig.toolbar_button_onmouseover(event['data']);\n",
       "    }\n",
       "\n",
       "    for(var toolbar_ind in mpl.toolbar_items){\n",
       "        var name = mpl.toolbar_items[toolbar_ind][0];\n",
       "        var tooltip = mpl.toolbar_items[toolbar_ind][1];\n",
       "        var image = mpl.toolbar_items[toolbar_ind][2];\n",
       "        var method_name = mpl.toolbar_items[toolbar_ind][3];\n",
       "\n",
       "        if (!name) { continue; };\n",
       "\n",
       "        var button = $('<button class=\"btn btn-default\" href=\"#\" title=\"' + name + '\"><i class=\"fa ' + image + ' fa-lg\"></i></button>');\n",
       "        button.click(method_name, toolbar_event);\n",
       "        button.mouseover(tooltip, toolbar_mouse_event);\n",
       "        nav_element.append(button);\n",
       "    }\n",
       "\n",
       "    // Add the status bar.\n",
       "    var status_bar = $('<span class=\"mpl-message\" style=\"text-align:right; float: right;\"/>');\n",
       "    nav_element.append(status_bar);\n",
       "    this.message = status_bar[0];\n",
       "\n",
       "    // Add the close button to the window.\n",
       "    var buttongrp = $('<div class=\"btn-group inline pull-right\"></div>');\n",
       "    var button = $('<button class=\"btn btn-mini btn-primary\" href=\"#\" title=\"Stop Interaction\"><i class=\"fa fa-power-off icon-remove icon-large\"></i></button>');\n",
       "    button.click(function (evt) { fig.handle_close(fig, {}); } );\n",
       "    button.mouseover('Stop Interaction', toolbar_mouse_event);\n",
       "    buttongrp.append(button);\n",
       "    var titlebar = this.root.find($('.ui-dialog-titlebar'));\n",
       "    titlebar.prepend(buttongrp);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._root_extra_style = function(el){\n",
       "    var fig = this\n",
       "    el.on(\"remove\", function(){\n",
       "\tfig.close_ws(fig, {});\n",
       "    });\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._canvas_extra_style = function(el){\n",
       "    // this is important to make the div 'focusable\n",
       "    el.attr('tabindex', 0)\n",
       "    // reach out to IPython and tell the keyboard manager to turn it's self\n",
       "    // off when our div gets focus\n",
       "\n",
       "    // location in version 3\n",
       "    if (IPython.notebook.keyboard_manager) {\n",
       "        IPython.notebook.keyboard_manager.register_events(el);\n",
       "    }\n",
       "    else {\n",
       "        // location in version 2\n",
       "        IPython.keyboard_manager.register_events(el);\n",
       "    }\n",
       "\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._key_event_extra = function(event, name) {\n",
       "    var manager = IPython.notebook.keyboard_manager;\n",
       "    if (!manager)\n",
       "        manager = IPython.keyboard_manager;\n",
       "\n",
       "    // Check for shift+enter\n",
       "    if (event.shiftKey && event.which == 13) {\n",
       "        this.canvas_div.blur();\n",
       "        // select the cell after this one\n",
       "        var index = IPython.notebook.find_cell_index(this.cell_info[0]);\n",
       "        IPython.notebook.select(index + 1);\n",
       "    }\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_save = function(fig, msg) {\n",
       "    fig.ondownload(fig, null);\n",
       "}\n",
       "\n",
       "\n",
       "mpl.find_output_cell = function(html_output) {\n",
       "    // Return the cell and output element which can be found *uniquely* in the notebook.\n",
       "    // Note - this is a bit hacky, but it is done because the \"notebook_saving.Notebook\"\n",
       "    // IPython event is triggered only after the cells have been serialised, which for\n",
       "    // our purposes (turning an active figure into a static one), is too late.\n",
       "    var cells = IPython.notebook.get_cells();\n",
       "    var ncells = cells.length;\n",
       "    for (var i=0; i<ncells; i++) {\n",
       "        var cell = cells[i];\n",
       "        if (cell.cell_type === 'code'){\n",
       "            for (var j=0; j<cell.output_area.outputs.length; j++) {\n",
       "                var data = cell.output_area.outputs[j];\n",
       "                if (data.data) {\n",
       "                    // IPython >= 3 moved mimebundle to data attribute of output\n",
       "                    data = data.data;\n",
       "                }\n",
       "                if (data['text/html'] == html_output) {\n",
       "                    return [cell, data, j];\n",
       "                }\n",
       "            }\n",
       "        }\n",
       "    }\n",
       "}\n",
       "\n",
       "// Register the function which deals with the matplotlib target/channel.\n",
       "// The kernel may be null if the page has been refreshed.\n",
       "if (IPython.notebook.kernel != null) {\n",
       "    IPython.notebook.kernel.comm_manager.register_target('matplotlib', mpl.mpl_figure_comm);\n",
       "}\n"
      ],
      "text/plain": [
       "<IPython.core.display.Javascript object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<img src=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAoAAAAHgCAYAAAA10dzkAAAgAElEQVR4Xu3dO5IlSXJm4cAyUiTXMAR4gAbfC8khewlNTiyk+KEHPAisoUR6GTkSVYjq7Kz7cNdrpm5q+oEZGZSbmer59aYdeLz+5c3/IIAAAggggAACCLQi8C+tutUsAggggAACCCCAwBsBNAQIIIAAAggggEAzAgSwWeDaRQABBBBAAAEECKAZQAABBBBAAAEEmhEggM0C1y4CCCCAAAIIIEAAzQACCCCAAAIIINCMAAFsFrh2EUAAAQQQQAABAmgGEEAAAQQQQACBZgQIYLPAtYsAAggggAACCBBAM4AAAggggAACCDQjQACbBa5dBBBAAAEEEECAAJoBBBBAAAEEEECgGQEC2Cxw7SKAAAIIIIAAAgTQDCCAAAIIIIAAAs0IEMBmgWsXAQQQQAABBBAggGYAAQQQQAABBBBoRoAANgtcuwgggAACCCCAAAE0AwgggAACCCCAQDMCBLBZ4NpFAAEEEEAAAQQIoBlAAAEEEEAAAQSaESCAzQLXLgIIIIAAAgggQADNAAIIIIAAAggg0IwAAWwWuHYRQAABBBBAAAECaAYQQAABBBBAAIFmBAhgs8C1iwACCCCAAAIIEEAzgAACCCCAAAIINCNAAJsFrl0EEEAAAQQQQIAAmgEEEEAAAQQQQKAZAQLYLHDtIoAAAggggAACBNAMIIAAAggggAACzQgQwGaBaxcBBBBAAAEEECCAZgABBBBAAAEEEGhGgAA2C1y7CCCAAAIIIIAAATQDCCCAAAIIIIBAMwIEsFng2kUAAQQQQAABBAigGUAAAQQQQAABBJoRIIDNAtcuAggggAACCCBAAM0AAggggAACCCDQjAABbBa4dhFAAAEEEEAAAQJoBhBAAAEEEEAAgWYECGCzwLWLAAIIIIAAAggQQDOAAAIIIIAAAgg0I0AAmwWuXQQQQAABBBBAgACaAQQQQAABBBBAoBkBAtgscO0igAACCCCAAAIE0AwggAACCCCAAALNCBDAZoFrFwEEEEAAAQQQIIBmAAEEEEAAAQQQaEaAADYLXLsIIIAAAggggAABNAMIIIAAAggggEAzAgSwWeDaRQABBBBAAAEECKAZQAABBBBAAAEEmhEggM0C1y4CCCCAAAIIIEAAzQACCCCAAAIIINCMAAFsFrh2EUAAAQQQQAABAmgGEEAAAQQQQACBZgQIYLPAtYsAAggggAACCBBAM9CJwPdOzeoVAQQuIeBevQS7Q88SMKhniXm+LIFfv30jgGXTUzgCNQh8fX93r9aIqn2VBrX9CPQBQAD7ZK1TBK4iQACvIu/cswQI4Flini9L4JEAfv3lS9m+Xi3817/8/eEWndm8yrby+kdz0X0mHrLxBrDy2LeqnQC2irt3swTwdv4EsPfn4l73BPD+XBBAn5kdCBDAHVLUwyECBJAAHhoUD/1GgAASQB+FvQkQwL3z1d0PBAggAfSBOE6AABLA49PiyYoECGDF1NQcIkAACWBocJouIoAEsOnot2mbALaJWqMEkAD6FBwnQAAJ4PFp8WRFAgSwYmpqDhEggAQwNDhNFxFAAth09Nu0TQDbRK1RAkgAfQqOEyCABPD4tHiyIgECWDE1NYcIRAUwehHusO4D9KPf+bZDj9Hfabd779H+PmYmunaLdX4PYOjfZ4vyCRDAfOZOvIgAARz/BnCLCzv4S8B37z3aHwH0p+Au+ifesScJEMCTwDxelwABJIC3CHgDeH4unjGLyuMW67wBrHtJNKucADYLvHO7BPD8Re9LwMHvA3vwVnELyXny1nT3Hv0lkM43yT69E8B9stTJEwJRAdwdrD8Ft3vCsf6iEhc7rdYqAlgrL9XeJkAATUYbAgRw/BvANsPTsFECGHz760vADT8tNVsmgDVzU3WAAAEkgIGxabuEABLAtsPfpHEC2CRobb69EUAC6HNwnAABJIDHp8WTFQkQwIqpqTlEgAASwNDgNF1EAAlg09Fv0zYBbBO1RgkgAfQpOE6AABLA49PiyYoECGDF1NQcIkAACWBocJouIoAEsOnot2mbALaJWqMEkAD6FBwnQAAJ4PFp8WRFAgSwYmpqDhEggAQwNDhNFxFAAth09Nu0TQDbRK3RbAF89guWI4k8+xNckT2f1XnFmZE+Oq/JzmjGeR/5PZvFSMaPao1Krl8EHUnCmtUIEMDVElHPNAIEcJ03gDMu+mmDU2DjGUIWlaNXcM2YCwL4SiLW7kyAAO6crt7+iQABJIC7fiQI4P1kCeCuU6+vVwkQwFcJWl+GAAEkgGWG9WShBJAAnhwZjyPwRgANQRsCBJAA7jrsBJAA7jrb+ppHgADOY2vnxQgQQAK42EgOK4cAEsBhw2SjNgQIYJuoNUoACeCunwICSAB3nW19zSNAAOextfNiBAggAVxsJIeVQwAJ4LBhslEbAgSwTdQa3V0Ao7+249mv3sj+KUqTep5AdkbPhHPWLJ4n8/aWzub93b0aCcqadAIGNR25A68iQADHvwFc6aK/aq5WODddcn758rDtleYinQ0BXOEjoYYDBAjgAUge2YMAASSAe0zyn7tIlxwCeHeUvhLAXT9m2/VFALeLVEP3CBBAArjrp4MA3k82nQ0B3PVjtl1fBHC7SDXUVQCjyb/yPYCzzozu23Xds+/Ji3CJfhk3ctbnmmezGNk7nQ0BjMRkzQUECOAF0B15DYHd3wBGqT67dLMv0GgfnddlZzTjvI/8ns1iJOMZtT6UYwIYicmaCwgQwAugO/IaAgTwNvdnl272BXrNdNQ+NTujGecRwNozqPp6BAhgvcxUHCRAAAlgcHSWXzZDyHwJ+H7s3gAu/5FQ4AECBPAAJI/sQYAAEsA9JvnPXRDA+8mms/El4F0/Ztv1RQC3i1RD9wgQQAK466cjXXKe/BqYKOdn344Q2TedDQGMxGTNBQQI4AXQHXkNAQJIAK+ZvPmnpksOAbwbqt8DOH/enTCGAAEcw9EuBQgQQAJYYExDJRJAXwIODY5FrQkQwNbx92qeABLAXSeeABLAXWdbX/MIEMB5bO28GIFsAVysfeUggMAgAn4KeBBI21xKgABeit/hmQQIYCZtZyGwLwECuG+2nTojgJ3Sbt4rAWw+ANpHYBABAjgIpG0uJUAAL8Xv8EwCBDCTtrMQ2JcAAdw3206dEcBOaTfvlQA2HwDtIzCIAAEcBNI2lxIggJfid3gmAQKYSdtZCOxLgADum22nzghgp7Sb90oAmw+A9hEYRIAADgJpm0sJEMBL8Ts8kwABzKTtLAT2JUAA9822U2cEsFPazXslgM0HQPsIDCJAAAeBtM2lBAjgpfgdnkkgKoAP/7F/8DdRd1j3kc+jvzKxQ4/Rv6Kxe+/R/j5mJrp2i3Xv7+7VzH/YnRUmYFDD6CysRoAA3k7s0aVLAO9P+RayMuH/gCGABLDa3dC1XgLYNfmGfRNAAniLgDeA5+fiGbPWcuwNYMPbpWbLBLBmbqoOEIgKYOAoSxBAYGMCvgdw43AbtUYAG4XdvVUC2H0C9I/AGAIEcAxHu1xLgABey9/piQQIYCJsRyGwMQECuHG4jVojgI3C7t4qAew+AfpHYAwBAjiGo12uJUAAr+Xv9EQCBDARtqMQ2JgAAdw43EatEcBGYXdvlQB2nwD9IzCGAAEcw9Eu1xIggNfyd3oiAQKYCNtRCGxMgABuHG6j1ghgo7C7t0oAu0+A/hEYQ4AAjuFol2sJEMBr+Ts9kQABTITtKAQ2JkAANw63UWsEsFHY3Vt9JIBRNs/+IkJk3+hfUYic9bHm2Z+Ci+6bzSZa50rrqjCbUeezWZxx5ozZ/+ovgaz0kVLLAwIE0Hi0IUAAb0c94xL8OKnKhb3SB6AKsxl1EsCVJlEtHQgQwA4p6/E3AgSQAK7+UZghVjMEf0adBHD16VTfbgQI4G6J6ucuAQJIAFf/eMwQKwJ4P/UpbHwJePWPmfr+hwABNAptCBBAArj6sBPAv9+NqAwbArj6x0x9BNAMdCNAAAng6jNfRnJ++TIF5Q4/AOWHQKaMhk0nEPAGcAJUW65JgAASwDUn8x9VEUBvAFefUfXtQ4AA7pOlTp4QIIAEcPUPCQEkgKvPqPr2IUAA98lSJwQwNAMzvhH+o5AqMhOCNmlRFWYz6vxA6kvAkwbLtgjcIEAAjUUbAtlvAKOX2UrrXhmOR5Iwo8dXal1lbRVmzwRwRr5l2PghkFU+Tup4QoAAGpE2BAjgOl8CniEIOwxyGcl58kMgM/Itw4YA7vBRbNEDAWwRsyZ/+/LSt2/fR5PIvpSyz3uFV6VaX+lz5NoqzLwBvJ+6nwIe+Ymw10wCBHAmXXsvRSBbAKPNR9+ezDgvuufHumeSENl71vcrRmqZsaYKsxl1/vZ/pP3FD4HMmCt7InCLAAE0F20IEMB1vgQcHToCeJ7cDGYE0BvA85NoxWoECOBqiahnGgECSACnDdegjWeIFQG8H84UNr4HcNCnwTazCRDA2YTtvwwBAkgAlxnGO4UQQF8CXn1G1bcPAQK4T5Y6eUKAABLA1T8kBJAArj6j6tuHAAHcJ0udEMDQDMz4MthHIVVkJgRt0qIqzGbU+YHUD4FMGizbInCDAAE0Fm0IeAPoDeDqwz5DrGYI/ow6CeDq06m+3QgQwN0S1c9dAgSQAK7+8ZghVgTwfupT2PghkNU/Zur7HwIE0Ci0IUAACeDqw04AfQ/g6jOqvn0IEMB9stTJMwJf/zb8L4E8OzLy37O/DypSozUIzCCwxez/+lf36ozhsOdwAgZ1OFIbLkuAAC4bjcIQ+CBAAM0BAnkECGAeayddTYAAXp2A8xF4SIAAGhAE8ggQwDzWTrqaAAG8OgHnI0AAzQACixAggIsEoYwEAgQwAbIjEIgT8AYwzs5KBM4SIIBniXm+LgECWDc7lbcgQABbxKzJRQgQwEWCUEYCAQKYANkRCMQJEMA4OysROEuAAJ4l5vm6BAhg3exU3oIAAWwRsyYXIUAAFwlCGQkECGACZEcgECdAAOPsrETgLAECeJaY5+sSCApg9FLaYd1H2I/+OsUOPUb/+sbuvUf7+5iZ6Noq6x7+I+gXQde9I5pVTgCbBd66XQJ4M/5nfw+VAN7+1FSRlew6CaC/BNL6ninUPAEsFJZSXyRAAAngDQLeAI4VXAJIAF/8l9ryJAIEMAm0YxYgEBTA7Mqjb2yy63QeAqMJbDH7vgQ8eizsN4kAAZwE1rYLEiCAC4aiJAT+QYAAmgYE8ggQwDzWTrqaAAG8OgHnI/CQAAE0IAjkESCAeayddDUBAnh1As5HgACaAQQWIUAAFwlCGQkECGACZEcgECfgDWCcnZUInCVAAM8S83xdAgSwbnYqb0GAALaIWZOLECCAiwShjAQCBDABsiMQiBMggHF2ViJwlgABPEvM83UJEMC62am8BQEC2CJmTS5CgAAuEoQyEggQwATIjkAgToAAxtlZicBZAgTwLDHPlyXw67dv30cXH/0rEo/qePan2Ub3MGs/bM6Txew+sypsvr6/u1fPj74VFxAwqBdAd+Q1BAhgLvcqF3YulcenYUYAV5pHtexNgADuna/ufiBAAHPHgcyc540ZATw/NVYgECNAAGPcrCpIgADmhkZmzvPGjACenxorEIgRIIAxblYVJEAAc0MjM+d5Y0YAz0+NFQjECBDAGDerChIggLmhkZnzvDEjgOenxgoEYgQIYIybVQUJEMDc0MjMed6YEcDzU2MFAjECBDDGzaqCBAhgbmhk5jxvzAjg+amxAoEYAQIY42ZVQQIEMDc0MnOeN2YE8PzUWIFAjAABjHGzqiCBbAGM/lWDDr8IegabGfI0Y8yzezdPX+7GOIONXwQ941NjzxkECOAMqvZckgABzI3lkZBlS1Bu549Py+59huRcwTN7nqI9EsAoOeuyCRDAbOLOu4wAAcxFn31hewN4O18C6A1g7iffaVUIEMAqSanzZQIE8GWEpzYggOeFLJvZqUAvfrgKG28ALx4Uxx8mQAAPo/JgdQLZAhjl1eGNzQw2u78BnMEsuucV62bkO+OzRgCvmA5nRggQwAg1a0oSIIC5sWVf2DPOm0Es+j2A0VpmSE60llfWzch3BhsC+ErK1mYSIICZtJ11KQECmIs/+8Kecd4MYgQwRnVGvgQwloVVexAggHvkqIsDBAjgAUgDH8m+sGecNxDHH1sRwBjVGfkSwFgWVu1BgADukaMuDhAggAcgDXwk+8Kecd5AHATwRZgz8iWAL4ZieWkCBLB0fIo/Q4AAnqH1+rPZF/aM816n8OcdvAGMUZ2RLwGMZWHVHgQI4B456uIAAQJ4ANLAR7Iv7BnnDcThDeCLMGfkSwBfDMXy0gQIYOn4FH+GAAE8Q+v1Z7Mv7BnnvU7BG8BRDGfkSwBHpWOfigQIYMXU1Bwj8PVv3yMLs79kF6mxy5odstihh13mbUoWv/7VvbrLgGzeh0HdPGDt/UCAAJYfhykXdjKVHXpIRjbtuClZEMBpedl4LAECOJan3VYmQABXTudQbVMu7EMnj3tohx7G0bh2pylZEMBrQ3X6YQIE8DAqD5YnQADLRzjlwk6mskMPycimHTclCwI4LS8bjyVAAMfytNvKBAjgyukcqm3KhX3o5HEP7dDDOBrX7jQlCwJ4bahOP0yAAB5G5cHyBAhg+QinXNjJVHboIRnZtOOmZEEAp+Vl47EECOBYnnZbmQABXDmdQ7VNubAPnTzuoR16GEfj2p2mZEEArw3V6YcJEMDDqDxYngABLB/hlAs7mcoOPSQjm3bclCwI4LS8bDyWAAEcy9NuKxMggCunc6i2KRf2oZPHPbRDD+NoXLvTlCwI4LWhOv0wAQJ4GJUHyxNIFsDo5bLSuo/MH/0FhpVqnfGXImbMfBVm0To/mEXXRtdFc5pyHgGMxmFdMgECmAzccRcSIIA34T+6BAng+HmNSkeVdQTQXwIZ/6mx4wwCBHAGVXuuSYAAEsAFJrOKyEXrJIAEcIGPmRIOECCAByB5ZBMCBJAALjDKUbGqso4AEsAFPmZKOECAAB6A5JFNCCQL4CbUlmojKkErNbFDDyvxfKWWKVn4HsBXIrE2kQABTITtqIsJEMCLA3j9+CkX9utlndphhx5ONbzww1OyIIALJ660HwkQQPPQhwABLJ/1lAs7mcoOPSQjm3bclCwI4LS8bDyWAAEcy9NuKxMggCunc6i2KRf2oZPHPbRDD+NoXLvTlCwI4LWhOv0wAQJ4GJUHyxMggOUjnHJhJ1PZoYdkZNOOm5IFAZyWl43HEiCAY3nabWUCBHDldA7VNuXCPnTyuId26GEcjWt3mpIFAbw2VKcfJkAAD6PyYHkCBLB8hFMu7GQqO/SQjGzacVOyIIDT8rLxWAIEcCxPu61MgACunM6h2qZc2IdOHvfQDj2Mo3HtTlOyIIDXhur0wwQI4GFUHixPgACWj3DKhZ1MZYcekpFNO25KFgRwWl42HkuAAI7labeFCfz67dv30eV9/eXL6C3fHl1Kww+buGE2mxnnzcAzRToeFGqe7sOZwebr+7t7dcYHx57DCRjU4UhtuCoBApibzAwhy5anGcSye5ghOTO4PNsze56e1XPvvxPAKDnrsgkQwGzizruMAAHMRZ99Yc84bwYxAhijOiPfGXJMAGP5WpVPgADmM3fiRQQIYC747At7xnkziBHAGNUZ+RLAWBZW7UGAAO6Roy4OECCAByANfCT7wp5x3kAcf2xFAGNUZ+RLAGNZWLUHAQK4R466OECAAB6ANPCR7At7xnkDcRDAF2HOyJcAvhiK5aUJEMDS8Sn+DAECeIbW689mX9gzznudwp938AYwRnVGvgQwloVVexAggHvkqIsDBAjgAUgDH8m+sGecNxCHN4AvwpyRLwF8MRTLSxMggKXjU/wZAtkCGH3TM+NSOsNp1LOPLuwZbGYIwigWP+6T3XuHeYrmNIONnwKOpmFdNgECmE3ceZcRIIC56Angbd4EMDaHMwSfAMaysGoPAgRwjxx1cYAAATwAaeAjBJAADhynNwI4kqa9EHh7I4CmoA0BApgbNQEkgCMnjgCOpGkvBAigGWhEIFsAo2hnfFkqWssr66pc2K/0OHotZveJVmHjewBHfyrsN4uAN4CzyNp3OQIEMDeSKhd2LpXHp2FGAFeaR7XsTYAA7p2v7n4gQABzx4HMnOeNGQE8PzVWIBAjQABj3KwqSIAA5oZGZs7zxowAnp8aKxCIESCAMW5WFSRAAHNDIzPneWNGAM9PjRUIxAgQwBg3qwoSIIC5oZGZ87wxI4Dnp8YKBGIECGCMm1UFCRDA3NDIzHnemBHA81NjBQIxAgQwxs2qggQIYG5oZOY8b8wI4PmpsQKBGAECGONmVUECBDA3NDJznjdmBPD81FiBQIwAAYxxs6oiga9/+16h7Ojfiq3QmxoReERgi9n/9a/uVWNegoBBLRGTIocQIIBDMNoEgVkECOAssvZF4M8ECKCp6EOAAPbJWqclCRDAkrEpuigBAlg0OGUHCBDAADRLEMgjQADzWDsJAQJoBvoQIIB9stZpSQIEsGRsii5KgAAWDU7ZAQIEMADNEgTyCBDAPNZOQoAAmoE+BAhgn6x1WpIAASwZm6KLEiCARYNTdoAAAQxAswSBPAIEMI+1kxAggGagD4GgAEYvpR3WfQzHo19OnN1jn2Ht2Wl0nqK0ppzn9wBG47AumQABTAbuuAsJEMCb8B9dggTwwnltePQUIXvAccp5BLDh5NZsmQDWzE3VEQIEkABG5saaNAJThIwApuXnoFoECGCtvFT7CgECSABfmR9rpxMggNMROwCBPwgQQMPQh0BQALMBZV+C2f05D4F7BLaYfV8CNuBFCBDAIkEpcwABAjgAoi0QmEeAAM5ja2cEfiZAAM1EHwIEsE/WOi1JgACWjE3RRQkQwKLBKTtAgAAGoFmCQB4BApjH2kkIEEAz0IcAAeyTtU5LEiCAJWNTdFECBLBocMoOECCAAWiWIJBHgADmsXYSAgTQDPQhQAD7ZK3TkgQIYMnYFF2UAAEsGpyyAwQIYACaJQjkESCAeaydhAABNAN9CBDAPlnrtCQBAlgyNkUXJUAAiwan7PMEfv327fv5VY9XfP3ly+gt37IvwVf+FnC0+dV6jPZxb93uczGjvw+WO8zF1/d39+roD5T9phAwqFOw2nRFAgTwdioEcPy0zhCkleRoRn8EcPwc2hGBRwQIoPloQ4AAEsCsYZ8hSAQw9217dFa8AYySsy6bAAHMJu68ywgQQAKYNXwEMEZ6JcmNdfD2RgCj5KzLJkAAs4k77zICBJAAZg0fAYyRJoAxblYhECFAACPUrClJgAASwKzBJYAx0gQwxs0qBCIECGCEmjUlCRBAApg1uAQwRpoAxrhZhUCEAAGMULOmJAECSACzBpcAxkgTwBg3qxCIECCAEWrWlCRAAAlg1uASwBhpAhjjZhUCEQIEMELNmpIEsgUwepmttO4j6Ecys1Kt0TpnDDMBjFGNzlPstMe/eDq6p58CjpKzLpsAAcwm7rzLCBDA8W8Aoxf2SutmDCQBjFGNzkXsNAIY5WbdHgQI4B456uIAAQJIAA+MyZBHCGAMIwGMcbMKgQgBAhihZk1JAtkCGIW02iW4u8xEc3q0bndmM/r74Lna7Edmw5eAI9SsuYIAAbyCujMvIUAAx78BjAa5w0VPAKPp31+3w1wQwPFzYcc5BAjgHK52XZAAASSAWWM54w3ZSnI0oz9vALOm0zkI/E6AAJqENgQIIAHMGvYZgkQAvwyP7xHT6GHeAEbJWZdNgABmE3feZQQIIAHMGj4CGCO9kuTGOnh7I4BRctZlEyCA2cSddxkBAkgAs4aPAMZIE8AYN6sQiBAggBFq1pQkQAAJYNbgEsAYaQIY42YVAhECBDBCzZqSBAggAcwaXAIYI00AY9ysQiBCgABGqFlTkgABJIBZg0sAY6QJYIybVQhECBDACDVrShJ4JIAzLuwqkJ79JOQMNtkXfZUsHtWZzSz7vEoZPWTz/u5erRRm41oNauPwu7VOAPu+Adxh1rOFLPu8ShkRwEppqfUeAQJoNtoQIIAEsPKwZwtZxnn//bf/91sk/+uv//7b//v5//8xp8//tlJ2BHClNNQSJUAAo+SsK0eAABLAckP7Q8EZQvYjn5nnHRG/n7NaSQQJYOVPkto/CRBAs9CGAAEkgJWHfaaQ3eIy8zwCWHkS1b4LAQK4S5L6eEqAABLAp0Oy8AMzhSxbAD/P+/lLvh9v+W59GfjjeW8AFx5OpZUkQABLxqboCAECSAAjc7PKGgL4+/cJrvA/vgS8QgpqeJUAAXyVoPVlCBBAAlhmWG8U2kEAf2x7pTd+P8dBACt/ktT+SYAAmoU2BAggAaw87ARwnfQI4DpZqCROgADG2VlZjEBUAKMX7w7rPiJ+9Iugd+gx+ouuo71HPzY7nnfrewCjfCLrokwJYIS2NasRIICrJaKeaQQI4Pg3gFMu0F++3J2B7PMeDWO0luiA73geAYxOg3UIvE6AAL7O0A5FCBBAAniLgDeA5+ciyuznkwhgkX88lbklAQK4ZayaukUgKoC703z0ZunZl4CjbLLfZkXrXGldNrOM8+79ypcP7n4IZKXpU8uOBAjgjqnq6SYBAnj+TQ8BXOfDlCFkP3abcR4BXGe+VNKPAAHsl3nbjgkgAaw8/BlCtpIA/pzVSm8E/RBI5U+S2j8JEECz0IYAASSAlYedAPpF0JXnV+3rESCA62WiokkECCABnDRaKdvuKICf4B59KfjzGW8AU8bMIY0IEMBGYXdvlQASwMqfAQLoDWDl+VX7egQI4HqZqGgSAQJIACeNVsq2OwvgozeBK735+6zT9wCmjLxDJhMggJMB234dAgSQAK4zjecr6SCA56lcs4IAXsPdqWMJEMCxPO22MAECSAAXHs+npRHAp4jSHiCAaagdNJEAAZwI19ZrEcgWwGe/YNiC/+oAABGPSURBVDlCZ9RfYPjx7Gd1zjgz0vsra571+Mret9buzmxWfzNymlErARz9ibHfFQQI4BXUnXkJAQK4zhvA7AGYIRaPepghHSsxm9XfjJxm1EoAs6fReTMIEMAZVO25JAECSACzBnOGdGTV/nlO9pecP84lgNkpO68zAQLYOf1mvRNAApg18gQwRpoAxrhZhUCEAAGMULOmJAECSACzBpcAxkgTwBg3qxCIECCAEWrWlCRAAAlg1uASwBhpAhjjZhUCEQIEMELNmpIECCABzBpcAhgjTQBj3KxCIEKAAEaoWVOSAAEkgFmDSwBjpAlgjJtVCEQIEMAINWtKEiCABDBrcAlgjDQBjHGzCoEIAQIYoWZNSQJRAYz+Oozsy2xWnY9kZsaZ2efNGGYCGKO6w2fm6/u7ezUWv1XJBAxqMnDHXUeAAI5/A0gAbzMlgLHPOQGMcbMKgQgBAhihZk1JAgSQAGYNLgGMkSaAMW5WIRAhQAAj1KwpSSAqgNFmsy+zWXXuLjNRbo/W7c5sVn87fGZ8CXjGJ8qeMwgQwBlU7bkkAQI4/g3gkkHfKGqGWBDA8enPyGmGrPpbwOOzt2M+AQKYz9yJFxEggAQwa/RmSEdW7Z/nRL+/85U6CeAr9KxF4BwBAniOl6cLEyCABDBrfAlgjDQBjHGzCoEIAQIYoWZNSQIEkABmDS4BjJEmgDFuViEQIUAAI9SsKUmAABLArMElgDHSBDDGzSoEIgQIYISaNSUJEEACmDW4BDBGmgDGuFmFQIQAAYxQs6YkAQJIALMGlwDGSBPAGDerEIgQIIARataUJEAACWDW4BLAGGkCGONmFQIRAgQwQs2akgQIYEwAs8OeIU8zxOIRlx16yO7v47wZOWVn4RdBZ/+L4bwoAQIYJWddOQIEkABmDW22dGT19XnOjP4IYHaKzutOgAB2n4BG/RNAApg17jMEacbbsSiPGf0RwGga1iEQI0AAY9ysKkiAABLArLGdIUgEMJZedha+BBzLyap8AgQwn7kTLyJAAAlg1uhlS0dWX74E/DsBfws4e+KcN4MAAZxB1Z5LEiCABDBrMAlgjPSMt5zZWXgDGMveqnwCBDCfuRMvIkAACWDW6GVLR1Zf3gB6A5g9a86bR4AAzmNr58UIEEACmDWSBDBG2hvAGDerEIgQIIARataUJEAACWDW4BLAGGkCGONmFQIRAgQwQs2akgR2F8CH35j+y5e7mc24dF8ZkEfyVKVHAhibgBmzmJ2F7wGMZW9VPgECmM/ciRcRIID13wASwIs+PD8dO0OqPo4ggGvkq4oeBAhgj5x1+XG5fPv2/R6IGRda9mVWRY6eDaM3gOuL+ozPCwF89snw3xEYS4AAjuVpt4UJEMD1xeKjQgK4fk4E8O93/6XzJeCFLwGl/RMBAmgg2hDYXQCjQc54Uxmt5ZkARvfN7nGGIGX38Ij1jP68AYxOt3UIxAgQwBg3qwoSIIDrv1kigPc/WAQw9o/ODFn1l0BiWVi1FgECuFYeqplIgAASwInj9U9bZ0tHVl+f58zozxvA7BSd150AAew+AY36J4AEMGvcZwiSN4Cx9LKz8D2AsZysyidAAPOZO/EiAgSQAGaNXrZ0ZPXlDeDvBHwJOHvinDeDAAGcQdWeSxIggAQwazAJYIz0jLec2Vl4AxjL3qp8AgQwn7kTLyJAAAlg1uhlS0dWX94AegOYPWvOm0eAAM5ja+fFCBBAApg1kgQwRtobwBg3qxCIECCAEWrWlCSQLYAlISkaAQSeEvA9gE8ReaAAAQJYICQljiFAAMdwtAsC3QkQwO4TsEf/BHCPHHVxgAABPADJIwgg8JQAAXyKyAMFCBDAAiEpcQwBAjiGo10Q6E6AAHafgD36J4B75KiLAwQI4AFIHkEAgacECOBTRB4oQIAAFghJiWMIEMAxHO2CQHcCBLD7BOzRPwHcI0ddHCBAAA9A8ggCCDwlQACfIvJAAQIEsEBIShxDgACO4WgXBLoTIIDdJ2CP/gngHjnq4gABAngAkkcQQOApAQL4FJEHChAggAVCUuIYAlEBfPiP/S9f7ha3w7qP5h79VYsdeoz+1Y5o72Om2S5XEiCAV9J39igCBHAUSfssT4AA3o7o2Z/fIoDnuUWlcvkPkQJ/I0AADcIOBAjgDinq4RABAnheZLwBvD9a3gAe+tht+RAB3DLWdk0RwHaR922YABLAWwSib+sIYON/S/7y97vNf31/d6/2HY1SnRvUUnEp9hUCUQF85UxrEUBgPwLeAO6XaceOCGDH1Jv2TACbBq9tBAYTIICDgdruEgIE8BLsDr2CAAG8grozEdiPAAHcL9OOHRHAjqk37ZkANg1e2wgMJkAABwO13SUECOAl2B16BQECeAV1ZyKwHwECuF+mHTsigB1Tb9ozAWwavLYRGEyAAA4GartLCBDAS7A79AoCBPAK6s5EYD8CBHC/TDt2RAA7pt60ZwLYNHhtIzCYAAEcDNR2lxAggJdgd+gVBAjgFdSdicB+BAjgfpl27IgAdky9ac+PBLApEm0jgMBgAv4SyGCgtptGgABOQ2vj1QgQwNUSUQ8C+xEggPtlumtHBHDXZPX1JwIE0FAggMBsAgRwNmH7jyJAAEeRtE8FAt8rFKlGBBAoTcC9Wjq+PsUb1D5Z6/TtjQCaAgQQmE3AvTqbsP2HEDCoQzDapAgBAlgkKGXuR+D//uu//tHUf/zXf+3X4D86cq/unO5GvRnUjcLUylMCBPApIg8gMIcAAZzD1a4IRAkQwCg56yoSIIAVU1NzaQKf4vfjW78fZfCjuc3eCLpXS09sn+INap+sdep7AM0AAksQ+FkKb0niEoXGinCvxrhZlUzAoCYDd9ylBLwBvBS/wxH4nQABNAkIXE+AAF6fgQryCBDAPNZOQuAugXtv/D7+9xt8Odi9avZLEDCoJWJS5CACBHAQSNsg8AoBAvgKPWsRGEOAAI7haJcaBAhgjZxUuTGBW2/5NvsJYffqxvO7U2sGdac09fKMAAF8Rsh/R2AyAQI4GbDtEThIgAAeBOWxLQgQwC1i1ERFAkd+HcyPfRX+XkD3asUBbVizQW0YeuOWCWDj8LV+LYFbX+b9+fcBEsBrM3J6LwIEsFfe3bslgN0nQP+XESCAl6F3MAI3CRBAg9GJAAHslLZelyBw5C2fHwJZIipFNCNAAJsF3rxdAth8ALSfT4AA5jN3IgJHCBDAI5Q8swsBArhLkvooQ+CIAJZp5lih7tVjnDx1MQGDenEAjk8lQABTcTsMgX/82bdbLAr/pO+jaN2rBr8EAYNaIiZFDiJAAAeBtA0CZwls9n1+BPDsAHh+OQIEcLlIFDSRAAGcCNfWCCDwGwH3qkEoQcCglohJkYMIEMBBIG2DAAJ3CbhXDUcJAga1REyKHESAAA4CaRsEECCAZqA2AQJYOz/VnyNAAM/x8jQCCJwn4F49z8yKCwgY1AugO/IyAgTwMvQORqANAfdqm6hrN2pQa+en+nMECOA5Xp5GAIHzBNyr55lZcQEBg3oBdEdeRoAAXobewQi0IeBebRN17UYNau38VH+OAAE8x6vt0//5f/7tcO//9r//80/PHll/a93nRrfWP3r+cLEezCDgXs2g7IyXCRjUlxHaoBABAlgorCtLPSJwn/URwCuTWvJs9+qSsSjqZwIG1Ux0IkAAO6U9udefJfFDBG/9734u49Ez0T0nt2r7cwTcq+d4efoiAgb1IvCOvYQAAbwE+56HRmWNAO45Dz905V7dPuI9GjSoe+Soi2MECOAxTp46QGCkAL765vBAuR7JI+BezWPtpBcIGNQX4FlajgABLBfZugUTwHWzubgy9+rFATj+GAGDeoyTp/YgQAD3yDGtiyM/DBL9IZDPJkZ872AaEAcdIeBePULJM5cTMKiXR6CARAIEMBH2DkcRwB1STO/BvZqO3IERAgY1Qs2aqgQIYNXkFqz7yPft3Sp75JeOF8SipLc396opKEHAoJaISZGDCBDAQSBt83boV74cFcDP56JyKI+lCLhXl4pDMfcIGFSz0YkAAeyU9uReR74BJICTw8rd3r2ay9tpQQIGNQjOspIECGDJ2PKLPvK9f59VRX8IxJ+Cy8816UT3ahJox7xGwKC+xs/qWgQIYK28LquWAF6GfoeD3as7pNigB4PaIGQt/kGAABoGBBCYTcC9Opuw/YcQMKhDMNqkCAECWCQoZSJQmIB7tXB4nUo3qJ3S1isBNAMIIDCbgHt1NmH7DyFgUIdgtEkRAgSwSFDKRKAwAfdq4fA6lW5QO6WtVwJoBhBAYDYB9+pswvYfQsCgDsFokyIECGCRoJSJQGEC7tXC4XUq3aB2SluvBNAMIIDAbALu1dmE7T+EgEEdgtEmRQgQwCJBKROBwgTcq4XD61S6Qe2Utl4JoBlAAIHZBNyrswnbfwgBgzoEo02KECCARYJSJgKFCbhXC4fXqXSD2iltvRJAM4AAArMJuFdnE7b/EAIGdQhGmxQhQACLBKVMBAoTcK8WDq9T6Qa1U9p6JYBmAAEEZhNwr84mbP8hBAzqEIw2KUKAABYJSpkIFCbgXi0cXqfSDWqntPVKAM0AAgjMJuBenU3Y/kMIGNQhGG1ShAABLBKUMhEoTMC9Wji8TqUb1E5p65UAmgEEEJhNwL06m7D9hxAwqEMw2qQIAQJYJChlIlCYgHu1cHidSjeondLWKwE0AwggMJuAe3U2YfsPIWBQh2C0SRECBLBIUMpEoDAB92rh8DqVblA7pa1XAmgGEEBgNgH36mzC9h9CwKAOwWiTIgQIYJGglIlAYQLu1cLhdSrdoHZKW68E0AwggMBsAu7V2YTtP4SAQR2C0SZFCBDAIkEpE4HCBNyrhcPrVLpB7ZS2XgmgGUAAgdkE3KuzCdt/CAGDOgSjTYoQIIBFglImAoUJuFcLh9epdIPaKW29EkAzgAACswm4V2cTtv8QAgZ1CEabFCFAAIsEpUwEChNwrxYOr1PpBrVT2npFAAEEEEAAAQTe3t4IoDFAAAEEEEAAAQSaESCAzQLXLgIIIIAAAgggQADNAAIIIIAAAggg0IwAAWwWuHYRQAABBBBAAAECaAYQQAABBBBAAIFmBAhgs8C1iwACCCCAAAIIEEAzgAACCCCAAAIINCNAAJsFrl0EEEAAAQQQQIAAmgEEEEAAAQQQQKAZAQLYLHDtIoAAAggggAACBNAMIIAAAggggAACzQgQwGaBaxcBBBBAAAEEECCAZgABBBBAAAEEEGhGgAA2C1y7CCCAAAIIIIAAATQDCCCAAAIIIIBAMwIEsFng2kUAAQQQQAABBAigGUAAAQQQQAABBJoRIIDNAtcuAggggAACCCBAAM0AAggggAACCCDQjAABbBa4dhFAAAEEEEAAAQJoBhBAAAEEEEAAgWYECGCzwLWLAAIIIIAAAggQQDOAAAIIIIAAAgg0I0AAmwWuXQQQQAABBBBAgACaAQQQQAABBBBAoBkBAtgscO0igAACCCCAAAIE0AwggAACCCCAAALNCBDAZoFrFwEEEEAAAQQQIIBmAAEEEEAAAQQQaEaAADYLXLsIIIAAAggggAABNAMIIIAAAggggEAzAgSwWeDaRQABBBBAAAEECKAZQAABBBBAAAEEmhEggM0C1y4CCCCAAAIIIEAAzQACCCCAAAIIINCMAAFsFrh2EUAAAQQQQAABAmgGEEAAAQQQQACBZgQIYLPAtYsAAggggAACCBBAM4AAAggggAACCDQjQACbBa5dBBBAAAEEEECAAJoBBBBAAAEEEECgGQEC2Cxw7SKAAAIIIIAAAgTQDCCAAAIIIIAAAs0IEMBmgWsXAQQQQAABBBAggGYAAQQQQAABBBBoRoAANgtcuwgggAACCCCAAAE0AwgggAACCCCAQDMCBLBZ4NpFAAEEEEAAAQQIoBlAAAEEEEAAAQSaESCAzQLXLgIIIIAAAgggQADNAAIIIIAAAggg0IwAAWwWuHYRQAABBBBAAIH/D3MrcFinnSaAAAAAAElFTkSuQmCC\" width=\"640\">"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.animation.FuncAnimation at 0x7f79dd797a90>"
      ]
     },
     "execution_count": 70,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "plot_animation(frames)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Extra material"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Preprocessing for Breakout"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here is a preprocessing function you can use to train a DQN for the Breakout-v0 Atari game:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def preprocess_observation(obs):\n",
    "    img = obs[34:194:2, ::2] # crop and downsize\n",
    "    return np.mean(img, axis=2).reshape(80, 80) / 255.0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[2017-09-25 13:54:27,989] Making new env: Breakout-v0\n"
     ]
    }
   ],
   "source": [
    "env = gym.make(\"Breakout-v0\")\n",
    "obs = env.reset()\n",
    "for step in range(10):\n",
    "    obs, _, _, _ = env.step(1)\n",
    "\n",
    "img = preprocess_observation(obs)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/javascript": [
       "/* Put everything inside the global mpl namespace */\n",
       "window.mpl = {};\n",
       "\n",
       "\n",
       "mpl.get_websocket_type = function() {\n",
       "    if (typeof(WebSocket) !== 'undefined') {\n",
       "        return WebSocket;\n",
       "    } else if (typeof(MozWebSocket) !== 'undefined') {\n",
       "        return MozWebSocket;\n",
       "    } else {\n",
       "        alert('Your browser does not have WebSocket support.' +\n",
       "              'Please try Chrome, Safari or Firefox ≥ 6. ' +\n",
       "              'Firefox 4 and 5 are also supported but you ' +\n",
       "              'have to enable WebSockets in about:config.');\n",
       "    };\n",
       "}\n",
       "\n",
       "mpl.figure = function(figure_id, websocket, ondownload, parent_element) {\n",
       "    this.id = figure_id;\n",
       "\n",
       "    this.ws = websocket;\n",
       "\n",
       "    this.supports_binary = (this.ws.binaryType != undefined);\n",
       "\n",
       "    if (!this.supports_binary) {\n",
       "        var warnings = document.getElementById(\"mpl-warnings\");\n",
       "        if (warnings) {\n",
       "            warnings.style.display = 'block';\n",
       "            warnings.textContent = (\n",
       "                \"This browser does not support binary websocket messages. \" +\n",
       "                    \"Performance may be slow.\");\n",
       "        }\n",
       "    }\n",
       "\n",
       "    this.imageObj = new Image();\n",
       "\n",
       "    this.context = undefined;\n",
       "    this.message = undefined;\n",
       "    this.canvas = undefined;\n",
       "    this.rubberband_canvas = undefined;\n",
       "    this.rubberband_context = undefined;\n",
       "    this.format_dropdown = undefined;\n",
       "\n",
       "    this.image_mode = 'full';\n",
       "\n",
       "    this.root = $('<div/>');\n",
       "    this._root_extra_style(this.root)\n",
       "    this.root.attr('style', 'display: inline-block');\n",
       "\n",
       "    $(parent_element).append(this.root);\n",
       "\n",
       "    this._init_header(this);\n",
       "    this._init_canvas(this);\n",
       "    this._init_toolbar(this);\n",
       "\n",
       "    var fig = this;\n",
       "\n",
       "    this.waiting = false;\n",
       "\n",
       "    this.ws.onopen =  function () {\n",
       "            fig.send_message(\"supports_binary\", {value: fig.supports_binary});\n",
       "            fig.send_message(\"send_image_mode\", {});\n",
       "            if (mpl.ratio != 1) {\n",
       "                fig.send_message(\"set_dpi_ratio\", {'dpi_ratio': mpl.ratio});\n",
       "            }\n",
       "            fig.send_message(\"refresh\", {});\n",
       "        }\n",
       "\n",
       "    this.imageObj.onload = function() {\n",
       "            if (fig.image_mode == 'full') {\n",
       "                // Full images could contain transparency (where diff images\n",
       "                // almost always do), so we need to clear the canvas so that\n",
       "                // there is no ghosting.\n",
       "                fig.context.clearRect(0, 0, fig.canvas.width, fig.canvas.height);\n",
       "            }\n",
       "            fig.context.drawImage(fig.imageObj, 0, 0);\n",
       "        };\n",
       "\n",
       "    this.imageObj.onunload = function() {\n",
       "        this.ws.close();\n",
       "    }\n",
       "\n",
       "    this.ws.onmessage = this._make_on_message_function(this);\n",
       "\n",
       "    this.ondownload = ondownload;\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._init_header = function() {\n",
       "    var titlebar = $(\n",
       "        '<div class=\"ui-dialog-titlebar ui-widget-header ui-corner-all ' +\n",
       "        'ui-helper-clearfix\"/>');\n",
       "    var titletext = $(\n",
       "        '<div class=\"ui-dialog-title\" style=\"width: 100%; ' +\n",
       "        'text-align: center; padding: 3px;\"/>');\n",
       "    titlebar.append(titletext)\n",
       "    this.root.append(titlebar);\n",
       "    this.header = titletext[0];\n",
       "}\n",
       "\n",
       "\n",
       "\n",
       "mpl.figure.prototype._canvas_extra_style = function(canvas_div) {\n",
       "\n",
       "}\n",
       "\n",
       "\n",
       "mpl.figure.prototype._root_extra_style = function(canvas_div) {\n",
       "\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._init_canvas = function() {\n",
       "    var fig = this;\n",
       "\n",
       "    var canvas_div = $('<div/>');\n",
       "\n",
       "    canvas_div.attr('style', 'position: relative; clear: both; outline: 0');\n",
       "\n",
       "    function canvas_keyboard_event(event) {\n",
       "        return fig.key_event(event, event['data']);\n",
       "    }\n",
       "\n",
       "    canvas_div.keydown('key_press', canvas_keyboard_event);\n",
       "    canvas_div.keyup('key_release', canvas_keyboard_event);\n",
       "    this.canvas_div = canvas_div\n",
       "    this._canvas_extra_style(canvas_div)\n",
       "    this.root.append(canvas_div);\n",
       "\n",
       "    var canvas = $('<canvas/>');\n",
       "    canvas.addClass('mpl-canvas');\n",
       "    canvas.attr('style', \"left: 0; top: 0; z-index: 0; outline: 0\")\n",
       "\n",
       "    this.canvas = canvas[0];\n",
       "    this.context = canvas[0].getContext(\"2d\");\n",
       "\n",
       "    var backingStore = this.context.backingStorePixelRatio ||\n",
       "\tthis.context.webkitBackingStorePixelRatio ||\n",
       "\tthis.context.mozBackingStorePixelRatio ||\n",
       "\tthis.context.msBackingStorePixelRatio ||\n",
       "\tthis.context.oBackingStorePixelRatio ||\n",
       "\tthis.context.backingStorePixelRatio || 1;\n",
       "\n",
       "    mpl.ratio = (window.devicePixelRatio || 1) / backingStore;\n",
       "\n",
       "    var rubberband = $('<canvas/>');\n",
       "    rubberband.attr('style', \"position: absolute; left: 0; top: 0; z-index: 1;\")\n",
       "\n",
       "    var pass_mouse_events = true;\n",
       "\n",
       "    canvas_div.resizable({\n",
       "        start: function(event, ui) {\n",
       "            pass_mouse_events = false;\n",
       "        },\n",
       "        resize: function(event, ui) {\n",
       "            fig.request_resize(ui.size.width, ui.size.height);\n",
       "        },\n",
       "        stop: function(event, ui) {\n",
       "            pass_mouse_events = true;\n",
       "            fig.request_resize(ui.size.width, ui.size.height);\n",
       "        },\n",
       "    });\n",
       "\n",
       "    function mouse_event_fn(event) {\n",
       "        if (pass_mouse_events)\n",
       "            return fig.mouse_event(event, event['data']);\n",
       "    }\n",
       "\n",
       "    rubberband.mousedown('button_press', mouse_event_fn);\n",
       "    rubberband.mouseup('button_release', mouse_event_fn);\n",
       "    // Throttle sequential mouse events to 1 every 20ms.\n",
       "    rubberband.mousemove('motion_notify', mouse_event_fn);\n",
       "\n",
       "    rubberband.mouseenter('figure_enter', mouse_event_fn);\n",
       "    rubberband.mouseleave('figure_leave', mouse_event_fn);\n",
       "\n",
       "    canvas_div.on(\"wheel\", function (event) {\n",
       "        event = event.originalEvent;\n",
       "        event['data'] = 'scroll'\n",
       "        if (event.deltaY < 0) {\n",
       "            event.step = 1;\n",
       "        } else {\n",
       "            event.step = -1;\n",
       "        }\n",
       "        mouse_event_fn(event);\n",
       "    });\n",
       "\n",
       "    canvas_div.append(canvas);\n",
       "    canvas_div.append(rubberband);\n",
       "\n",
       "    this.rubberband = rubberband;\n",
       "    this.rubberband_canvas = rubberband[0];\n",
       "    this.rubberband_context = rubberband[0].getContext(\"2d\");\n",
       "    this.rubberband_context.strokeStyle = \"#000000\";\n",
       "\n",
       "    this._resize_canvas = function(width, height) {\n",
       "        // Keep the size of the canvas, canvas container, and rubber band\n",
       "        // canvas in synch.\n",
       "        canvas_div.css('width', width)\n",
       "        canvas_div.css('height', height)\n",
       "\n",
       "        canvas.attr('width', width * mpl.ratio);\n",
       "        canvas.attr('height', height * mpl.ratio);\n",
       "        canvas.attr('style', 'width: ' + width + 'px; height: ' + height + 'px;');\n",
       "\n",
       "        rubberband.attr('width', width);\n",
       "        rubberband.attr('height', height);\n",
       "    }\n",
       "\n",
       "    // Set the figure to an initial 600x600px, this will subsequently be updated\n",
       "    // upon first draw.\n",
       "    this._resize_canvas(600, 600);\n",
       "\n",
       "    // Disable right mouse context menu.\n",
       "    $(this.rubberband_canvas).bind(\"contextmenu\",function(e){\n",
       "        return false;\n",
       "    });\n",
       "\n",
       "    function set_focus () {\n",
       "        canvas.focus();\n",
       "        canvas_div.focus();\n",
       "    }\n",
       "\n",
       "    window.setTimeout(set_focus, 100);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._init_toolbar = function() {\n",
       "    var fig = this;\n",
       "\n",
       "    var nav_element = $('<div/>')\n",
       "    nav_element.attr('style', 'width: 100%');\n",
       "    this.root.append(nav_element);\n",
       "\n",
       "    // Define a callback function for later on.\n",
       "    function toolbar_event(event) {\n",
       "        return fig.toolbar_button_onclick(event['data']);\n",
       "    }\n",
       "    function toolbar_mouse_event(event) {\n",
       "        return fig.toolbar_button_onmouseover(event['data']);\n",
       "    }\n",
       "\n",
       "    for(var toolbar_ind in mpl.toolbar_items) {\n",
       "        var name = mpl.toolbar_items[toolbar_ind][0];\n",
       "        var tooltip = mpl.toolbar_items[toolbar_ind][1];\n",
       "        var image = mpl.toolbar_items[toolbar_ind][2];\n",
       "        var method_name = mpl.toolbar_items[toolbar_ind][3];\n",
       "\n",
       "        if (!name) {\n",
       "            // put a spacer in here.\n",
       "            continue;\n",
       "        }\n",
       "        var button = $('<button/>');\n",
       "        button.addClass('ui-button ui-widget ui-state-default ui-corner-all ' +\n",
       "                        'ui-button-icon-only');\n",
       "        button.attr('role', 'button');\n",
       "        button.attr('aria-disabled', 'false');\n",
       "        button.click(method_name, toolbar_event);\n",
       "        button.mouseover(tooltip, toolbar_mouse_event);\n",
       "\n",
       "        var icon_img = $('<span/>');\n",
       "        icon_img.addClass('ui-button-icon-primary ui-icon');\n",
       "        icon_img.addClass(image);\n",
       "        icon_img.addClass('ui-corner-all');\n",
       "\n",
       "        var tooltip_span = $('<span/>');\n",
       "        tooltip_span.addClass('ui-button-text');\n",
       "        tooltip_span.html(tooltip);\n",
       "\n",
       "        button.append(icon_img);\n",
       "        button.append(tooltip_span);\n",
       "\n",
       "        nav_element.append(button);\n",
       "    }\n",
       "\n",
       "    var fmt_picker_span = $('<span/>');\n",
       "\n",
       "    var fmt_picker = $('<select/>');\n",
       "    fmt_picker.addClass('mpl-toolbar-option ui-widget ui-widget-content');\n",
       "    fmt_picker_span.append(fmt_picker);\n",
       "    nav_element.append(fmt_picker_span);\n",
       "    this.format_dropdown = fmt_picker[0];\n",
       "\n",
       "    for (var ind in mpl.extensions) {\n",
       "        var fmt = mpl.extensions[ind];\n",
       "        var option = $(\n",
       "            '<option/>', {selected: fmt === mpl.default_extension}).html(fmt);\n",
       "        fmt_picker.append(option)\n",
       "    }\n",
       "\n",
       "    // Add hover states to the ui-buttons\n",
       "    $( \".ui-button\" ).hover(\n",
       "        function() { $(this).addClass(\"ui-state-hover\");},\n",
       "        function() { $(this).removeClass(\"ui-state-hover\");}\n",
       "    );\n",
       "\n",
       "    var status_bar = $('<span class=\"mpl-message\"/>');\n",
       "    nav_element.append(status_bar);\n",
       "    this.message = status_bar[0];\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.request_resize = function(x_pixels, y_pixels) {\n",
       "    // Request matplotlib to resize the figure. Matplotlib will then trigger a resize in the client,\n",
       "    // which will in turn request a refresh of the image.\n",
       "    this.send_message('resize', {'width': x_pixels, 'height': y_pixels});\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.send_message = function(type, properties) {\n",
       "    properties['type'] = type;\n",
       "    properties['figure_id'] = this.id;\n",
       "    this.ws.send(JSON.stringify(properties));\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.send_draw_message = function() {\n",
       "    if (!this.waiting) {\n",
       "        this.waiting = true;\n",
       "        this.ws.send(JSON.stringify({type: \"draw\", figure_id: this.id}));\n",
       "    }\n",
       "}\n",
       "\n",
       "\n",
       "mpl.figure.prototype.handle_save = function(fig, msg) {\n",
       "    var format_dropdown = fig.format_dropdown;\n",
       "    var format = format_dropdown.options[format_dropdown.selectedIndex].value;\n",
       "    fig.ondownload(fig, format);\n",
       "}\n",
       "\n",
       "\n",
       "mpl.figure.prototype.handle_resize = function(fig, msg) {\n",
       "    var size = msg['size'];\n",
       "    if (size[0] != fig.canvas.width || size[1] != fig.canvas.height) {\n",
       "        fig._resize_canvas(size[0], size[1]);\n",
       "        fig.send_message(\"refresh\", {});\n",
       "    };\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_rubberband = function(fig, msg) {\n",
       "    var x0 = msg['x0'] / mpl.ratio;\n",
       "    var y0 = (fig.canvas.height - msg['y0']) / mpl.ratio;\n",
       "    var x1 = msg['x1'] / mpl.ratio;\n",
       "    var y1 = (fig.canvas.height - msg['y1']) / mpl.ratio;\n",
       "    x0 = Math.floor(x0) + 0.5;\n",
       "    y0 = Math.floor(y0) + 0.5;\n",
       "    x1 = Math.floor(x1) + 0.5;\n",
       "    y1 = Math.floor(y1) + 0.5;\n",
       "    var min_x = Math.min(x0, x1);\n",
       "    var min_y = Math.min(y0, y1);\n",
       "    var width = Math.abs(x1 - x0);\n",
       "    var height = Math.abs(y1 - y0);\n",
       "\n",
       "    fig.rubberband_context.clearRect(\n",
       "        0, 0, fig.canvas.width, fig.canvas.height);\n",
       "\n",
       "    fig.rubberband_context.strokeRect(min_x, min_y, width, height);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_figure_label = function(fig, msg) {\n",
       "    // Updates the figure title.\n",
       "    fig.header.textContent = msg['label'];\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_cursor = function(fig, msg) {\n",
       "    var cursor = msg['cursor'];\n",
       "    switch(cursor)\n",
       "    {\n",
       "    case 0:\n",
       "        cursor = 'pointer';\n",
       "        break;\n",
       "    case 1:\n",
       "        cursor = 'default';\n",
       "        break;\n",
       "    case 2:\n",
       "        cursor = 'crosshair';\n",
       "        break;\n",
       "    case 3:\n",
       "        cursor = 'move';\n",
       "        break;\n",
       "    }\n",
       "    fig.rubberband_canvas.style.cursor = cursor;\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_message = function(fig, msg) {\n",
       "    fig.message.textContent = msg['message'];\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_draw = function(fig, msg) {\n",
       "    // Request the server to send over a new figure.\n",
       "    fig.send_draw_message();\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_image_mode = function(fig, msg) {\n",
       "    fig.image_mode = msg['mode'];\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.updated_canvas_event = function() {\n",
       "    // Called whenever the canvas gets updated.\n",
       "    this.send_message(\"ack\", {});\n",
       "}\n",
       "\n",
       "// A function to construct a web socket function for onmessage handling.\n",
       "// Called in the figure constructor.\n",
       "mpl.figure.prototype._make_on_message_function = function(fig) {\n",
       "    return function socket_on_message(evt) {\n",
       "        if (evt.data instanceof Blob) {\n",
       "            /* FIXME: We get \"Resource interpreted as Image but\n",
       "             * transferred with MIME type text/plain:\" errors on\n",
       "             * Chrome.  But how to set the MIME type?  It doesn't seem\n",
       "             * to be part of the websocket stream */\n",
       "            evt.data.type = \"image/png\";\n",
       "\n",
       "            /* Free the memory for the previous frames */\n",
       "            if (fig.imageObj.src) {\n",
       "                (window.URL || window.webkitURL).revokeObjectURL(\n",
       "                    fig.imageObj.src);\n",
       "            }\n",
       "\n",
       "            fig.imageObj.src = (window.URL || window.webkitURL).createObjectURL(\n",
       "                evt.data);\n",
       "            fig.updated_canvas_event();\n",
       "            fig.waiting = false;\n",
       "            return;\n",
       "        }\n",
       "        else if (typeof evt.data === 'string' && evt.data.slice(0, 21) == \"data:image/png;base64\") {\n",
       "            fig.imageObj.src = evt.data;\n",
       "            fig.updated_canvas_event();\n",
       "            fig.waiting = false;\n",
       "            return;\n",
       "        }\n",
       "\n",
       "        var msg = JSON.parse(evt.data);\n",
       "        var msg_type = msg['type'];\n",
       "\n",
       "        // Call the  \"handle_{type}\" callback, which takes\n",
       "        // the figure and JSON message as its only arguments.\n",
       "        try {\n",
       "            var callback = fig[\"handle_\" + msg_type];\n",
       "        } catch (e) {\n",
       "            console.log(\"No handler for the '\" + msg_type + \"' message type: \", msg);\n",
       "            return;\n",
       "        }\n",
       "\n",
       "        if (callback) {\n",
       "            try {\n",
       "                // console.log(\"Handling '\" + msg_type + \"' message: \", msg);\n",
       "                callback(fig, msg);\n",
       "            } catch (e) {\n",
       "                console.log(\"Exception inside the 'handler_\" + msg_type + \"' callback:\", e, e.stack, msg);\n",
       "            }\n",
       "        }\n",
       "    };\n",
       "}\n",
       "\n",
       "// from http://stackoverflow.com/questions/1114465/getting-mouse-location-in-canvas\n",
       "mpl.findpos = function(e) {\n",
       "    //this section is from http://www.quirksmode.org/js/events_properties.html\n",
       "    var targ;\n",
       "    if (!e)\n",
       "        e = window.event;\n",
       "    if (e.target)\n",
       "        targ = e.target;\n",
       "    else if (e.srcElement)\n",
       "        targ = e.srcElement;\n",
       "    if (targ.nodeType == 3) // defeat Safari bug\n",
       "        targ = targ.parentNode;\n",
       "\n",
       "    // jQuery normalizes the pageX and pageY\n",
       "    // pageX,Y are the mouse positions relative to the document\n",
       "    // offset() returns the position of the element relative to the document\n",
       "    var x = e.pageX - $(targ).offset().left;\n",
       "    var y = e.pageY - $(targ).offset().top;\n",
       "\n",
       "    return {\"x\": x, \"y\": y};\n",
       "};\n",
       "\n",
       "/*\n",
       " * return a copy of an object with only non-object keys\n",
       " * we need this to avoid circular references\n",
       " * http://stackoverflow.com/a/24161582/3208463\n",
       " */\n",
       "function simpleKeys (original) {\n",
       "  return Object.keys(original).reduce(function (obj, key) {\n",
       "    if (typeof original[key] !== 'object')\n",
       "        obj[key] = original[key]\n",
       "    return obj;\n",
       "  }, {});\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.mouse_event = function(event, name) {\n",
       "    var canvas_pos = mpl.findpos(event)\n",
       "\n",
       "    if (name === 'button_press')\n",
       "    {\n",
       "        this.canvas.focus();\n",
       "        this.canvas_div.focus();\n",
       "    }\n",
       "\n",
       "    var x = canvas_pos.x * mpl.ratio;\n",
       "    var y = canvas_pos.y * mpl.ratio;\n",
       "\n",
       "    this.send_message(name, {x: x, y: y, button: event.button,\n",
       "                             step: event.step,\n",
       "                             guiEvent: simpleKeys(event)});\n",
       "\n",
       "    /* This prevents the web browser from automatically changing to\n",
       "     * the text insertion cursor when the button is pressed.  We want\n",
       "     * to control all of the cursor setting manually through the\n",
       "     * 'cursor' event from matplotlib */\n",
       "    event.preventDefault();\n",
       "    return false;\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._key_event_extra = function(event, name) {\n",
       "    // Handle any extra behaviour associated with a key event\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.key_event = function(event, name) {\n",
       "\n",
       "    // Prevent repeat events\n",
       "    if (name == 'key_press')\n",
       "    {\n",
       "        if (event.which === this._key)\n",
       "            return;\n",
       "        else\n",
       "            this._key = event.which;\n",
       "    }\n",
       "    if (name == 'key_release')\n",
       "        this._key = null;\n",
       "\n",
       "    var value = '';\n",
       "    if (event.ctrlKey && event.which != 17)\n",
       "        value += \"ctrl+\";\n",
       "    if (event.altKey && event.which != 18)\n",
       "        value += \"alt+\";\n",
       "    if (event.shiftKey && event.which != 16)\n",
       "        value += \"shift+\";\n",
       "\n",
       "    value += 'k';\n",
       "    value += event.which.toString();\n",
       "\n",
       "    this._key_event_extra(event, name);\n",
       "\n",
       "    this.send_message(name, {key: value,\n",
       "                             guiEvent: simpleKeys(event)});\n",
       "    return false;\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.toolbar_button_onclick = function(name) {\n",
       "    if (name == 'download') {\n",
       "        this.handle_save(this, null);\n",
       "    } else {\n",
       "        this.send_message(\"toolbar_button\", {name: name});\n",
       "    }\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.toolbar_button_onmouseover = function(tooltip) {\n",
       "    this.message.textContent = tooltip;\n",
       "};\n",
       "mpl.toolbar_items = [[\"Home\", \"Reset original view\", \"fa fa-home icon-home\", \"home\"], [\"Back\", \"Back to  previous view\", \"fa fa-arrow-left icon-arrow-left\", \"back\"], [\"Forward\", \"Forward to next view\", \"fa fa-arrow-right icon-arrow-right\", \"forward\"], [\"\", \"\", \"\", \"\"], [\"Pan\", \"Pan axes with left mouse, zoom with right\", \"fa fa-arrows icon-move\", \"pan\"], [\"Zoom\", \"Zoom to rectangle\", \"fa fa-square-o icon-check-empty\", \"zoom\"], [\"\", \"\", \"\", \"\"], [\"Download\", \"Download plot\", \"fa fa-floppy-o icon-save\", \"download\"]];\n",
       "\n",
       "mpl.extensions = [\"eps\", \"jpeg\", \"pdf\", \"png\", \"ps\", \"raw\", \"svg\", \"tif\"];\n",
       "\n",
       "mpl.default_extension = \"png\";var comm_websocket_adapter = function(comm) {\n",
       "    // Create a \"websocket\"-like object which calls the given IPython comm\n",
       "    // object with the appropriate methods. Currently this is a non binary\n",
       "    // socket, so there is still some room for performance tuning.\n",
       "    var ws = {};\n",
       "\n",
       "    ws.close = function() {\n",
       "        comm.close()\n",
       "    };\n",
       "    ws.send = function(m) {\n",
       "        //console.log('sending', m);\n",
       "        comm.send(m);\n",
       "    };\n",
       "    // Register the callback with on_msg.\n",
       "    comm.on_msg(function(msg) {\n",
       "        //console.log('receiving', msg['content']['data'], msg);\n",
       "        // Pass the mpl event to the overriden (by mpl) onmessage function.\n",
       "        ws.onmessage(msg['content']['data'])\n",
       "    });\n",
       "    return ws;\n",
       "}\n",
       "\n",
       "mpl.mpl_figure_comm = function(comm, msg) {\n",
       "    // This is the function which gets called when the mpl process\n",
       "    // starts-up an IPython Comm through the \"matplotlib\" channel.\n",
       "\n",
       "    var id = msg.content.data.id;\n",
       "    // Get hold of the div created by the display call when the Comm\n",
       "    // socket was opened in Python.\n",
       "    var element = $(\"#\" + id);\n",
       "    var ws_proxy = comm_websocket_adapter(comm)\n",
       "\n",
       "    function ondownload(figure, format) {\n",
       "        window.open(figure.imageObj.src);\n",
       "    }\n",
       "\n",
       "    var fig = new mpl.figure(id, ws_proxy,\n",
       "                           ondownload,\n",
       "                           element.get(0));\n",
       "\n",
       "    // Call onopen now - mpl needs it, as it is assuming we've passed it a real\n",
       "    // web socket which is closed, not our websocket->open comm proxy.\n",
       "    ws_proxy.onopen();\n",
       "\n",
       "    fig.parent_element = element.get(0);\n",
       "    fig.cell_info = mpl.find_output_cell(\"<div id='\" + id + \"'></div>\");\n",
       "    if (!fig.cell_info) {\n",
       "        console.error(\"Failed to find cell for figure\", id, fig);\n",
       "        return;\n",
       "    }\n",
       "\n",
       "    var output_index = fig.cell_info[2]\n",
       "    var cell = fig.cell_info[0];\n",
       "\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_close = function(fig, msg) {\n",
       "    var width = fig.canvas.width/mpl.ratio\n",
       "    fig.root.unbind('remove')\n",
       "\n",
       "    // Update the output cell to use the data from the current canvas.\n",
       "    fig.push_to_output();\n",
       "    var dataURL = fig.canvas.toDataURL();\n",
       "    // Re-enable the keyboard manager in IPython - without this line, in FF,\n",
       "    // the notebook keyboard shortcuts fail.\n",
       "    IPython.keyboard_manager.enable()\n",
       "    $(fig.parent_element).html('<img src=\"' + dataURL + '\" width=\"' + width + '\">');\n",
       "    fig.close_ws(fig, msg);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.close_ws = function(fig, msg){\n",
       "    fig.send_message('closing', msg);\n",
       "    // fig.ws.close()\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.push_to_output = function(remove_interactive) {\n",
       "    // Turn the data on the canvas into data in the output cell.\n",
       "    var width = this.canvas.width/mpl.ratio\n",
       "    var dataURL = this.canvas.toDataURL();\n",
       "    this.cell_info[1]['text/html'] = '<img src=\"' + dataURL + '\" width=\"' + width + '\">';\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.updated_canvas_event = function() {\n",
       "    // Tell IPython that the notebook contents must change.\n",
       "    IPython.notebook.set_dirty(true);\n",
       "    this.send_message(\"ack\", {});\n",
       "    var fig = this;\n",
       "    // Wait a second, then push the new image to the DOM so\n",
       "    // that it is saved nicely (might be nice to debounce this).\n",
       "    setTimeout(function () { fig.push_to_output() }, 1000);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._init_toolbar = function() {\n",
       "    var fig = this;\n",
       "\n",
       "    var nav_element = $('<div/>')\n",
       "    nav_element.attr('style', 'width: 100%');\n",
       "    this.root.append(nav_element);\n",
       "\n",
       "    // Define a callback function for later on.\n",
       "    function toolbar_event(event) {\n",
       "        return fig.toolbar_button_onclick(event['data']);\n",
       "    }\n",
       "    function toolbar_mouse_event(event) {\n",
       "        return fig.toolbar_button_onmouseover(event['data']);\n",
       "    }\n",
       "\n",
       "    for(var toolbar_ind in mpl.toolbar_items){\n",
       "        var name = mpl.toolbar_items[toolbar_ind][0];\n",
       "        var tooltip = mpl.toolbar_items[toolbar_ind][1];\n",
       "        var image = mpl.toolbar_items[toolbar_ind][2];\n",
       "        var method_name = mpl.toolbar_items[toolbar_ind][3];\n",
       "\n",
       "        if (!name) { continue; };\n",
       "\n",
       "        var button = $('<button class=\"btn btn-default\" href=\"#\" title=\"' + name + '\"><i class=\"fa ' + image + ' fa-lg\"></i></button>');\n",
       "        button.click(method_name, toolbar_event);\n",
       "        button.mouseover(tooltip, toolbar_mouse_event);\n",
       "        nav_element.append(button);\n",
       "    }\n",
       "\n",
       "    // Add the status bar.\n",
       "    var status_bar = $('<span class=\"mpl-message\" style=\"text-align:right; float: right;\"/>');\n",
       "    nav_element.append(status_bar);\n",
       "    this.message = status_bar[0];\n",
       "\n",
       "    // Add the close button to the window.\n",
       "    var buttongrp = $('<div class=\"btn-group inline pull-right\"></div>');\n",
       "    var button = $('<button class=\"btn btn-mini btn-primary\" href=\"#\" title=\"Stop Interaction\"><i class=\"fa fa-power-off icon-remove icon-large\"></i></button>');\n",
       "    button.click(function (evt) { fig.handle_close(fig, {}); } );\n",
       "    button.mouseover('Stop Interaction', toolbar_mouse_event);\n",
       "    buttongrp.append(button);\n",
       "    var titlebar = this.root.find($('.ui-dialog-titlebar'));\n",
       "    titlebar.prepend(buttongrp);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._root_extra_style = function(el){\n",
       "    var fig = this\n",
       "    el.on(\"remove\", function(){\n",
       "\tfig.close_ws(fig, {});\n",
       "    });\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._canvas_extra_style = function(el){\n",
       "    // this is important to make the div 'focusable\n",
       "    el.attr('tabindex', 0)\n",
       "    // reach out to IPython and tell the keyboard manager to turn it's self\n",
       "    // off when our div gets focus\n",
       "\n",
       "    // location in version 3\n",
       "    if (IPython.notebook.keyboard_manager) {\n",
       "        IPython.notebook.keyboard_manager.register_events(el);\n",
       "    }\n",
       "    else {\n",
       "        // location in version 2\n",
       "        IPython.keyboard_manager.register_events(el);\n",
       "    }\n",
       "\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._key_event_extra = function(event, name) {\n",
       "    var manager = IPython.notebook.keyboard_manager;\n",
       "    if (!manager)\n",
       "        manager = IPython.keyboard_manager;\n",
       "\n",
       "    // Check for shift+enter\n",
       "    if (event.shiftKey && event.which == 13) {\n",
       "        this.canvas_div.blur();\n",
       "        // select the cell after this one\n",
       "        var index = IPython.notebook.find_cell_index(this.cell_info[0]);\n",
       "        IPython.notebook.select(index + 1);\n",
       "    }\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_save = function(fig, msg) {\n",
       "    fig.ondownload(fig, null);\n",
       "}\n",
       "\n",
       "\n",
       "mpl.find_output_cell = function(html_output) {\n",
       "    // Return the cell and output element which can be found *uniquely* in the notebook.\n",
       "    // Note - this is a bit hacky, but it is done because the \"notebook_saving.Notebook\"\n",
       "    // IPython event is triggered only after the cells have been serialised, which for\n",
       "    // our purposes (turning an active figure into a static one), is too late.\n",
       "    var cells = IPython.notebook.get_cells();\n",
       "    var ncells = cells.length;\n",
       "    for (var i=0; i<ncells; i++) {\n",
       "        var cell = cells[i];\n",
       "        if (cell.cell_type === 'code'){\n",
       "            for (var j=0; j<cell.output_area.outputs.length; j++) {\n",
       "                var data = cell.output_area.outputs[j];\n",
       "                if (data.data) {\n",
       "                    // IPython >= 3 moved mimebundle to data attribute of output\n",
       "                    data = data.data;\n",
       "                }\n",
       "                if (data['text/html'] == html_output) {\n",
       "                    return [cell, data, j];\n",
       "                }\n",
       "            }\n",
       "        }\n",
       "    }\n",
       "}\n",
       "\n",
       "// Register the function which deals with the matplotlib target/channel.\n",
       "// The kernel may be null if the page has been refreshed.\n",
       "if (IPython.notebook.kernel != null) {\n",
       "    IPython.notebook.kernel.comm_manager.register_target('matplotlib', mpl.mpl_figure_comm);\n",
       "}\n"
      ],
      "text/plain": [
       "<IPython.core.display.Javascript object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<img src=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAABEwAAAK8CAYAAAD1QyqZAAAgAElEQVR4XuzdBbQlz34W7BrcNbhDggdICBCC2yXB3d1dBncL+sEfd3cLLgkXtxAgJEDwBHcI7nq/9Sbd0Hdzzpna01125um17kruTHdV9fOrM/fUu6t7vygOAgQIECBAgAABAgQIECBAgACBdxJ4wYMAAQIECBAgQIAAAQIECBAgQOCdBQQmZgQBAgQIECBAgAABAgQIECBA4EZAYGJKECBAgAABAgQIECBAgAABAgQEJuYAAQIECBAgQIAAAQIECBAgQOBpATtMzBACBAgQIECAAAECBAgQIECAwI2AwMSUIECAAAECBAgQIECAAAECBAgITMwBAgQIECBAgAABAgQIECBAgMDTAnaYmCEECBAgQIAAAQIECBAgQIAAgRsBgYkpQYAAAQIECBAgQIAAAQIECBAQmJgDBAgQIECAAAECBAgQIECAAIGnBewwMUMIECBAgAABAgQIECBAgAABAjcCAhNTggABAgQIECBAgAABAgQIECAgMDEHCBAgQIAAAQIECBAgQIAAAQJPC9hhYoYQIECAAAECBAgQIECAAAECBG4EBCamBAECBAgQIECAAAECBAgQIEBAYGIOECBAgAABAgQIECBAgAABAgSeFrDDxAwhQIAAAQIECBAgQIAAAQIECNwICExMCQIECBAgQIAAAQIECBAgQICAwMQcIECAAAECBAgQIECAAAECBAg8LWCHiRlCgAABAgQIECBAgAABAgQIELgREJiYEgQIECBAgAABAgQIECBAgAABgYk5QIAAAQIECBAgQIAAAQIECBB4WsAOEzOEAAECBAgQIECAAAECBAgQIHAjIDAxJQgQIECAAAECBAgQIECAAAECAhNzgAABAgQIECBAgAABAgQIECDwtIAdJmYIAQIECBAgQIAAAQIECBAgQOBGQGBiShAgQIAAAQIECBAgQIAAAQIEBCbmAAECBAgQIECAAAECBAgQIEDgaQE7TMwQAgQIECBAgAABAgQIECBAgMCNgMDElCBAgAABAgQIECBAgAABAgQICEzMAQIECBAgQIAAAQIECBAgQIDA0wJ2mJghBAgQIECAAAECBAgQIECAAIEbAYGJKUGAAAECBAgQIECAAAECBAgQEJiYAwQIECBAgAABAgQIECBAgACBpwXsMDFDCBAgQIAAAQIECBAgQIAAAQI3AgITU4IAAQIECBAgQIAAAQIECBAgIDAxBwgQIECAAAECBAgQIECAAAECTwvYYWKGECBAgAABAgQIECBAgAABAgRuBAQmpgQBAgQIECBAgAABAgQIECBAQGBiDhAgQIAAAQIECBAgQIAAAQIEnhaww8QMIUCAAAECBAgQIECAAAECBAjcCAhMTAkCBAgQIECAAAECBAgQIECAgMDEHCBAgAABAgQIECBAgAABAgQIPC1gh4kZQoAAAQIECBAgQIAAAQIECBC4ERCYmBIECBAgQIAAAQIECBAgQIAAAYGJOUCAAAECBAgQIECAAAECBAgQeFrADhMzhAABAgQIECBAgAABAgQIECBwIyAwMSUIECBAgAABAgQIECBAgAABAgITc4AAAQIECBAgQIAAAQIECBAg8LSAHSZmCAECBAgQIECAAAECBAgQIEDgRkBgYkoQIECAAAECBAgQIECAAAECBAQm5gABAgQIECBAgAABAgQIECBA4GkBO0zMEAIECBAgQIAAAQIECBAgQIDAjYDAxJQgQIAAAQIECBAgQIAAAQIECAhMzAECBAgQIECAAAECBAgQIECAwNMCdpiYIQQIECBAgAABAgQIECBAgACBGwGBiSlBgAABAgQIECBAgAABAgQIEBCYmAMECBAgQIAAAQIECBAgQIAAgacF7DAxQwgQIECAAAECBAgQIECAAAECNwICE1OCAAECBAgQIECAAAECBAgQICAwMQcIECBAgAABAgQIECBAgAABAk8L2GFihhAgQIAAAQIECBAgQIAAAQIEbgQEJqYEAQIECBAgQIAAAQIECBAgQEBgYg4QIECAAAECBAgQIECAAAECBJ4WsMPEDCFAgAABAgQIECBAgAABAgQI3AgITEwJAgQIECBAgAABAgQIECBAgIDAxBwgQIAAAQIECBAgQIAAAQIECDwtYIeJGUKAAAECBAgQIECAAAECBAgQuBEQmJgSBAgQIECAAAECBAgQIECAAAGBiTlAgAABAgQIECBAgAABAgQIEHhawA4TM4QAAQIECBAgQIAAAQIECBAgcCMgMDElCBAgQIAAAQIECBAgQIAAAQICE3OAAAECBAgQIECAAAECBAgQIPC0gB0mZggBAgQIECBAgAABAgQIECBA4EZAYGJKECBAgAABAgQIECBAgAABAgQEJuYAAQIECBAgQIAAAQIECBAgQOBpATtMzBACTwv82FLKjymlvM7PyrcrpfzKUsrnKqX8vUbQn7OU8ndLKd++lPKrLujjK5ZS/mgp5RuXUj7ggvZWamK/969USvljAwb+cUopf7mU8htLKT9xQP+67CfwaUsp/2D7Ofv9/brVEwECBAgQIECAwD0Cr7MIvKd95xIYJfCFSik/rJSSxe+7lFL+1RYE/KRSyl+9Y1ACkzuwFjn1e5RS/vMDAdPowORbllJ+QSnls5dS/t1m+clKKT+olPKlSilfspTyqV8RjiV0+a7bfz7fdp9/qZTy/Usp+b/7kfN+YCnlu5dSPlMp5W+VUn7yFtacLeMn2cb4dUsp715KyT18VCnll2z/+V83HfyI7f5yj5++lPLjSin5uXvo+CyllJ9ZSnlbKSX3kHAv9/Z3Kgad0PJzHM7LHMi/BT+vlPJrHrk+43lZSvmaW/D58Uop/6iU8idLKb+8lPKnDtftAemxqX+59fHTSikfeNPHzy6lfNlSyhevGLtTCBAgQIAAAQIEBggITAag67K5wDfYFn7/elvUZAdGdmJ8x1JKPtn9ZqWU31E5iiyQ8p//Wnn+8bSPW0r5+KWU/1ZKecdrXF9ziR0mNUrvfM5fKaV8dCklAcnxyAL8E5RS/nsp5X/f3+zpK/5iKeXPbmHH3the3+xGSCiQMT+1myi7jBK8JAD4M6WUT1pKeY9Syq8vpfzBwwgTjvzQUsovLaX8+VJKwo2EAt+8lPKbTt7JF952yvzhUsrbSyn/vpTy1UspX38b17e9aT8/G/9sC3Ry3mOBSYKXDyulfMpSys8opfyPLSzJ/459sS0UfWroCUz+zXZtzktQ9J1KKZ+3lPJdNovj9Qmofl8p5ZNvJnHKz3J2jH29UsoXLKV8hVLKn9gu2gOTH73t+sq4PkMpJX+eAPdrl1J+76GDL1BK+WullK9SSvkjJ81dToAAAQIECBAg0EBAYNIAVZNDBT7PtljLAvPLl1LyCe9+ZKdJPhn+bKWUL/KKT6Wz0PxPQ++krvPnHJhkp0J2AVx9PBaYXN3PPe0l1EgY8FVLKQka9uMTbrtKEii81xZuPBaYfJNSym8upSQwfCoQzC6NhIjZ8fG9to7yvwV/fAsDMqeOu0Cym+NvbI+Q3N5TQowEAXmM6PhzlqDgdifXr9jCnnfbdpzs56e/hBn5+czP62OByQ8upfzUbadNwoscn7+UknpmB8cPfwV4+si5X+tw3qfb/h34h1sAsv9VdvJk/HHJLrXc//HInyd4zc6ZfSx7YPIlSikfejg5bf3zUspv3cKsYzsfUUr58FLKt7lnsjiXAAECBAgQIECgj4DApI+zXvoJ/KLtE/qEJQlHbo/8eRaGv7iU8t22v9wfu8mnwD+ylPJ+2wIui9iHHsn5xNvC7VuUUrKgzWMBebQhW/WPi72H3mGyL9p+SinlrS24+SdbP8fHAj7NtgDMJ+75RDs7Hv70tivg+GjFPYHJ597GnU+0P9EWLP2E7VP03Wl/LCWLwS9aSvkO2yfsWcR/z1JKFpb7kYVv7uPLlFI+1bZrI48o5JGQ/ZGSnPuttp0A+UT+v2y7DvKYybGtvDMkC+bsPsgjFwkHsqB/120hm7HfHtlBkR08OTdHgoRvXUrJDofsQvjbpZSfW0r5hYcLbx/LyF9lPuS+H3skJ+9zyW6MjD8h2geVUn5IKeUfH9rNzo5vVErJYzA/fws+cq+/ejv39jGU23vJvEkfCSCyc+Kh41WByYdsC/w82pLdMpmnD4V+eSQpY8x8zw6H/cjukt9QSvlyh0dN0s5f33ZI5c+PAWR242QHxpfe5ujx7x4af4KV311K+TqllN/zwAmvCkz+3HZNdn4cjz9QSklQmrny1PFQYJLzE3gkQM3P8n7kcb48vpefg4RQNcdjgUn+d/bfllJ+5za/j23l34DM2/y8t9qFVjN25xAgQIAAAQIECDwgIDAxLZ6bQBaxeaQiIcNjRz5dz2M22WmSYw9Fsnj8yFJKFmD52cj7JB4KTLKAyqf5v7aUkkVqtuVnoZyAoSYwyeM9CRjyDoSEJQklEs7kfQ/7p/JZHOfRiHwqnfHmE/sEEVlQZ+Ge63LUBia5PkFLdm38nO3xhYQT6TML/X1Hwh4a5JPvLOASBOQ9Dt9v22GQRx8SBGSxnE/ds8iMU3ZAZOdCPr1PwPD3t/Hl/RQJZX7LFkzkE/3vXUr5j9s9ZyGZI4FJDPMYU+47OwHyqfyn2B7jyCJ5/yQ/5+ddFFkAJ3j56VsbWVDHL/f5P7edD9kdkV0UCQhy5FGKhCjpf3+xavrJ4yoPBSb7Ijh9J0yI4/fdxpaa7eOP0zfdQpo8VpPzs1vkG5ZSElAcQ5uH5mX6z6L5qfdZPBWYxCljSS3yCEyMM1cydxLExH8/8hhOwpE8anJcpCd0yI6J77MZ7efn0ZGEj9m1FaO0nzrl5yDhRx7nuX0/x0P3+J23EOx9tseFbs95KjBJcJPdRtmlEs/jkfmVoDMG/+GxH/ptvtzuMMm/Awnu8vP+GQ/XfvD2mE+Ct8cCrNuu9rmSumcOps387KQWuff3vXksKtfn8alft/0cZmwOAgQIECBAgACBiQQEJhMVw1BOC2Rxk0Xj79oWxo81mL/PQm9fYO2hSB4ryK6R43EbmLxnKeUvlFJ+1rZrYj8334aTBVNNYJLF/nEHTEKELNry8sm8iDNHgogs1I7v0kg4kpAiC/0sEnPUBibZtZHQ47h7IAvqfCtL/h3IYjl97aFBgqcslPcFaEKQLLoTFiRwSXCSRwme+jad3Gd2eeSdDvm0fj+yAyTX5tuH9j9PYJLgKbt+svtnP1Kjf3Fjk79LUJLHM3L/WcjnyI6KhDnHI7tBshMm97cfjz2ScxuYZPdKdg2l/zxmsb/HJu/6yLsofvx2D2k3gUkCqNzrXpv8eR6zieu+C+ZmeP/nv6b+CVoSXj12PBWY7I/05OXGCYsyD7PLJ/XK2L/GtjMmbWfsqe3RJH+eMC07UrJrKDssjkfayHs2cj9Z+GcO5J1AWfAfH8d5bOwJ2FLz1CjvDMkYb4+nApP9725908a+YyaP5/zNJ/wSsOXvM+YcCUjymE92JSVQ2x9Pyt/l/UcJ/eJ6PBIyHXeiZL7tu3geeulrrs17TxJ2ZrfR7ZHdOQlnErYdQ60nbsNfESBAgAABAgQI9BIQmPSS1k8Pgc+6BQ/5xDaLoMeO/H0WTTk/wcAeihxf4LhfexuY5D0JCSyy6MtulP3Yg5SawCQLrDwOcTzyiXTChbx/4vbIp/nZkZKf1zwak5d/5gWaOWoDkywUEyblcY3jkd0HeQFodpokSNhDg/zZ8Z0Q6TtWCViyYM4Onozjl22L8ofeNZJvL8nLOWO178TY+86LMtPeV9v+IIHJe28hVnYIHY/sfsnOiwQw+46IvCMi52W3wkNHwrMEHvlkP6FM/PbHhGoDk30x+9AOkTymkjruQcgemGRHwfHRlHwTSuZido88dcQvL2bNeB87ngpMEoTtLx+NY8KXHPsuk9Q/38iSI3MoLzzNTqXjkV0ceXQoY064dnvkUa48gpMdOfkmn+POnVfc3sc8XpV7S9j02NfoPhWYZDdYgrE8CpX3lRyP7NDKbq2EG3lx7mPHQ49j5dyEnXnc7Bi2JdDJI19xPR55rCY7avbjGLTsgUnayrcO5ciOpDySll0nCUV++017CXkyl3JNdgc5CBAgQIAAAQIEJhIQmExUDEM5LXDvDpOcn8cL9lAki8DjezUyoNvAJLsf8s0a+ZT5+Cl5dkJkQV4TmGSBlPekHI8EBgkD8oLJHFm8ZndAFusJJxKa7EfemfKVt/9SG5hkd0Q+wb59uWQWf1kE5lGaLIb3wCSL0Cwkj0cW5AkEssjLkTAkX7mahWYe2cj7KRJG7cFEFoB5t8tjR8KXPMaUI/efRfHtrof8Xd4jkV0MeVdKPo3fHx3Joj6L+/3I38c/QUd2SxyPhC37TpTawGTv96FvMUmIkwAiu4Ny7I/kZAfF8aj9WuoEJnnkJ3PrseOpwGT/uzyCc/u+lzzGkkV7TDJnX2eHyT6m7JLIHMpukfRZ821C2Q2UkONHlVLe/4n767HDJI+O5fGd/Dxlp1P+/8y97Ow5vmcm36aTgOV2h0nedZKfgRx5jOqhwOT2pa/5WY5X5kp+Xo+B4P5NOTWPbT1B568IECBAgAABAgRaCAhMWqhqc6RA3u2RcOChl4Tu48qiMrsPssMkx76ozYImXzd7PFoEJrfvUUh/WbTl2L/qNgu5PNqRxW4WZnlEIIvTPAp0/ErckYFJxpudKQld8q6QBBb/dNspkkdZ8gLefF1rwqGHXnqa94jkHTD7/WfBnEXs7ZFvLMpjMXn3RkKSPC6ShXcClv1dLglR8v6SPLKU94Uk+MrCNI+iZKdLQqcsgHO0Ckyy6M6Ojqfmz2M/Gxlv3sGSd548djwVmHzmbcdOPBMYHY88YpOdGfsumzjm0bOMteYdJntbCb8SgmU+ZsdEwsOnArFcl10XmcM15454h8n+eNUP2F7CvN9rdpckzHvqHSaxqwlM0mZ+bhOAZn4fvz0oO6TyMue8EynvK3IQIECAAAECBAhMJCAwmagYhnKJwL71//iujmPD+6MLD31LTk1gcsUjOTWBSR4tSEiy7yTZ7yFBRF7MuQcrtYHJY4/kZCGdBfW9j+Q8VKx98ZdHlhL47DsL8jLX/RGFx4q8f0vOQ4FJrskLRrOjIyFJ3qORR3x2g/x9gpS8p+W4kyR/nrGkZsfAJC+0zbs+jtfn3Nt3mDz1SE5eEJxdIcdHcs4EJgkhPm0pJY92PXa86ltyElblvTfZKXU88u1LeddMgqeEbnn8I+/Luf2WnIQoeSzooW+Yym6b/F12HWUXTBb/CQFuH9069psg7bdtO5gSCLxqN8qrviUnL9JNSHH7LTlv33YdPbQ76Tiex74lJ3Mv8y5zZ38fSV5WnFDuqXeL3BOYxDvux8elMjYvfb3kn32NECBAgAABAgTaCAhM2rhqdZxAXvCZ94FkF0kWflkY70feI5FHRxIyZGt93hmS454dJnmXRt6fcealrzWBSV4sm8eF9kd0Ms79xav71+Dmz2oDk/2lr8dvKMkCOo/F5JGB2pe+7o/B5BGkBAbHx5LyQswEGfmq1IQlaTNBTcKOPBJy3M2Qf3tSj70+rwpM8m6XLL7z8syEXbePMOSbSPIi0njs39CT3QF5/Cnv6zgGJtmFka9Vzotrj8djL33NYxxZpOflnTmyYybv4bh96euZwCRt5X0yMdz7uf0pelVgsu9iyG6fBDA5EkJknufrnrObIkd2VuX9MwkX9xedph6ZV9mZleDguCMou3Ty2FZelpzgZP+7PP6Ul7DmRcV5POt45Gcv3zaVnRrxeuyejte8KjDZw73jIy8J47JjI9+UFL+njscCk72e2YkUwxyZmwnFcq/5GbwN/OKVAKhmh0l2s2WMcU0om5/r/cjPSh5/+9S+VvgV1fPXBAgQIECAAIEBAgKTAei6bC6QYCGfhufRlbwMMuFJFtL5Vo8syvKVqseXL94TmGTwH7A9OnH8WuG82DQL8LSVhWSO/SWQx8X6U59y55p918O+GM27MfLejuwAyafRCSTy+Ma9O0z2rxVOUJBgIbtX8q0ueewgj4E89rXC2VGQaxOUZHdLzk9Qkq/nzafmeYwgi8l8PWtebhqDLJb3R232l8rmHrLozrfuxCMvrc2Cff9K4FcFJhl3HsvJkXdx5BGU/b/nz7JwTviTgCaBSh43yUtG89hPxnysQRa5eZQki/3s1kk7+QaYp75WOC9RzXtU9q8VzjW3Xyt8JjDZg7ivXkrJjonjkVAjj9PknjPuzN28EyNHviJ5f2dMxpY/z71nIZ4/z7cOZVdOdsskSNyPvFMkoVZqkJ0bqWcClcyxvEtlPxKmJXTKvP3aN+/fyDmZS1nwx3d/2W2CgfSVb8ZJmHIMCHJN6pT/7EfmTa5JXfO4Vd7Rk3rkyM/YHoAlTMr95f9m3mQ3Td6hk/eRZN4dX7Z7Q/gx//Wxn738XXYdJWBLyLd/jXDMfs+2MydfdR2n/F08829M3uVzfAnt/vOeeZV/c3LkfSfZuZP6PvTtQ+k3u8meekn1Q/fizwgQIECAAAECBDoICEw6IOtiiEAChiy+sghOSJKdDFmI5RtTssPjeNwbmGRhlwVngpcsCv/QtnDKYj0BQb7uNseZwCQvlc3jJFlsZbGcx1Cy+MyiK8e9gUmuye6BjC3vn0gAkUVrdjbkZa/7sYcGubfswknIlAVqFrDZ1bG/ODUL5Dx2k28W+ixbiJJFcsacb2E5Htkdkk/v9xdoJvDJOVls75/cvyowSXv7txvFe/92nWM/WdDnMYqEV9kVkneZZBGdd2gcA5MECwnSEuzk3vYdOw8FJmk/j5OkrvlWmTyy8YFbvfMtP/uRYOtMYJJ24pdF+e2LXx/7dpdcc7yvvcYJE/Ki2uxsyA6PjD3tHo8EIVnsZ8dOduDkG5/yeE2Cxtsju7Zyrw99E1L+NyQ/a8cAZHd8oKmP+aPji5Hz3/evlH7o/Ozu2N/vk7/P7pjslsoumtxD/i5zK8HXq46nApOEh6nht9/+795Wvno4oUzCpFinz1hkx06+ISo71vbjoa8VzvuU8l6dhHj5z3GX1f4NOfl5vP2ZedW9+HsCBAgQIECAAIEOAgKTDsi6eCME8gl3Pv3OoycPLTrfCAQ3eUoguwyy+yXvILn9GuZTDbt4SoE8/pPQLrtPjkHKlIM1KAIECBAgQIDAmyggMHkTq+6ezwrkq2PzVbrHI59OZ8GbR39uv5r4bH+ufzMEsnshOzXy6E926jier0Be8JtHjbJ7Ke/DcRAgQIAAAQIECEwoIDCZsCiGNL3Aj9k+Fc4jPnnpaV4amf/kfRB5xMFBgAABAgQIECBAgAABAosLCEwWL6DhDxHI+zMSmuSdFnnBZt7rkZdTZlfA8VtjhgxOpwQIECBAgAABAgQIECBwXkBgct5QCwQIECBAgAABAgQIECBAgMAzExCYPLOCuh0CBAgQIECAAAECBAgQIEDgvIDA5LyhFggQIECAAAECBAgQIECAAIFnJiAweWYFdTsECBAgQIAAAQIECBAgQIDAeQGByXnDN6WFd7wpN+o+CRAgQOBJAb87mCAECBAgQIDAGyHgl543osyX3KTA5BJGjRAgQGB5Ab87LF9CN0CAAAECBAjUCPilp0bJOREQmJgHBAgQIBABvzuYBwQIECBAgMAbIeCXnjeizJfcpMDkEkaNECBAYHkBvzssX0I3QIAAAQIECNQI+KWnRsk5ERCYmAcECBAgEAG/O5gHBAgQIECAwBsh4JeeN6LMl9ykwOQSRo0QIEBgeQG/OyxfQjdAgAABAgQI1Aj4padGyTkREJiYBwQIECAQAb87mAcECBAgQIDAGyHgl543osyX3KTA5BJGjRAgQGB5Ab87LF9CN0CAAAECBAjUCPilp0bJOREQmJgHBAgQIBABvzuYBwQIECBAgMAbIeCXnjeizJfcpMDkEkaNECBAYHkBvzssX0I3QIAAAQIECNQI+KWnRsk5ERCYmAcECBAgEAG/O5gHBAgQIECAwBsh4JeeN6LMl9ykwOQSRo0QIEBgeQG/OyxfQjdAgAABAgQI1Aj4padGyTkREJg0mgdvvfVWo5Yfb/bly5dVfRrbOzPVulXhDjhpRD1zm6u71ZaqxveZWPjdoXZSOI8AAQIECBBYWsAvPUuXr+vgBSaNuGsWWVd3XbtoMzaByRVzr3a+XdHXyDZqfl6eiYXfHUZONH0TIECAAAEC3QT80tONevmOBCaNSlizyLq669pFm7EJTK6Ye7Xz7Yq+RrZR8/PyTCz87jByoumbAAECBAgQ6Cbgl55u1Mt3JDBpVMKaRdbVXdcu2oxNYHLF3Kudb1f0NbKNmp+XZ2Lhd4eRE03fBAgQIECAQDcBv/R0o16+I4FJoxLWLLKu7rp20WZsApMr5l7tfLuir5Ft1Py8PBMLvzuMnGj6JkCAAAECBLoJ+KWnG/XyHQlMGpWwZpGVrmsXWjXtXdlW7dhqxlXbVs67sr0r22o0TS5p9k25z0uwXqORGt/an73X6L7nJX536KmtLwIECBAgQGCYgF96htEv17HApFHJahZZVwcJtYu2K8d2ZVsCk9ebjFfX4PVG8XyvqvGt/dmbXMnvDpMXyPAIECBAgACBawT80nON45vQisCkUZVrFlkCk/8X/0q3K9tqNE0uafZNuc9LsF6jkRpfgclrwLqEAAECBAgQIDBIQGAyCH7BbgUmjYpWs8gSmAhMrph+V8+1K8b0nNqo8RWYPKeKuxcCBAgQIEDguQsITJ57ha+7P4HJdZbv1FLNIktgIjC5YvpdPdeuGNNzaqPGV2DynCruXggQIECAAIHnLiAwee4Vvu7+BCbXWQpMnrCsXVDWLE5rg6Yr22o0TS5ptvY+L+ns0EhtTa/ut3d7Nb7PxMLvDr0nl/4IECBAgACBIQJ+6RnCvmSnApNGZatZZNUu/HNeTXu1i7aatmrHdmVbtfc5amyNpsrpZsT18k4AACAASURBVGtrcLqjmwZq59vV/fZur8b3mVj43aH35NIfAQIECBAgMETALz1D2JfsVGDSqGw1i6zahX9tkFC7aLtybFe2VXuftW5Xj63RVDndbO19nu5IYPIoYe3P3tU1uLg9vztcDKo5AgQIECBAYE4Bv/TMWZcZRyUwaVSV2kVs7UKrpr0r2xoVStTc56ixNZoqp5utNTvdkcBEYHL1JNIeAQIECBAgQGCAgMBkAPqiXQpMGhWudhF7ZchxZVujQokr3a5sq9E0uaTZ2vu8pLNDI7Xz7ep+e7dX4/tMLPzu0Hty6Y8AAQIECBAYIuCXniHsS3YqMGlUtppFVm0okfNq2qtdtNW0VTu2K9uqvc9RY2s0VaZr9uqaTneDdw6oxqP2Z+/Ornuf7neH3uL6I0CAAAECBIYI+KVnCPuSnQpMGpWtZpFVu/CvDRJqF21Xju3Ktmrvs9bt6rE1mirTNcvtnUtS41H7szddsd95QH53mLxAhkeAAAECBAhcI+CXnmsc34RWBCaNqlyzyKpd+NcGCbWLtivHdmVbtfdZ63b12BpNlema5SYwmW5SGhABAgQIECBA4EIBgcmFmM+8KYFJowJfveisaU9gcv9CtzZ8aTRNpmy2Zq69SW41HrU/e1MW/P8Oyu8OkxfI8AgQIECAAIFrBPzSc43jm9CKwKRRlWsWWfcsOmvaq1201bRVO7Yr20qfV7Z3ZVuNpsmUzXK7P3ir/dmbsuACk8nLYngECBAgQIDA1QICk6tFn297ApNGta1ddF7Zfe2izdjeWb3W7cpaXdnWiHrWBmpX3ueotmp8V59Dm63fHUZNMv0SIECAAAECXQX80tOVe+nOBCaNylezyLq669pFm7EJTK6Ye7Xz7Yq+RrZR8/PyTCz87jByoumbAAECBAgQ6Cbgl55u1Mt3JDBpVMKaRdbVXdcu2oxNYHLF3Kudb1f0NbKNmp+XZ2Lhd4eRE03fBAgQIECAQDcBv/R0o16+I4FJoxLWLLKu7rp20WZsApMr5l7tfLuir5Ft1Py8PBMLvzuMnGj6JkCAAAECBLoJ+KWnG/XyHQlMli+hGyBAgMAlAn53uIRRIwQIECBAgMDsAn7pmb1C84xPYDJPLYyEAAECIwX87jBSX98ECBAgQIBANwG/9HSjXr4jgcnyJXQDBAgQuETA7w6XMGpkUYFfVUr5iqWUz3nR+P9YKeVdSilf+KL2Vmom954jniOOb1JK+UWllM9eSvmPIwagz2kEvl0p5VeWUj5XKeXv3Tmqn1JK+UqllC9153VOX0TALz2LFGqCYQpMJiiCIRAgQGACAb87XFuE/Rf1vdX/Vkr5B6WUt5dSfkIp5Z9f253WTgoITO4D/IKllAQTcbtdiI4MTD5uKeWvlFJ+Synlxxxu6eOUUr5LKeW7lVLetZTyn0opH7b9LH7wza1/wlLKjy+lfOtSyqcupfzlUsqPLKX8wfuIHj37E5VSvv/WfgK6f1NKyRh+bCnlr95c9alKKT+tlPL1SymfpJTy50opP2Ab+0XDedbNnAlMPuM2tzPPf/ezVnpDb84vPW9o4V/jtgUmr4HmEgIECDxDAb87XFvU/Rf1H11K+bullCySvuy2SPr7286D/3xtl1o7ISAwuQ/vG5VSfuv2CfwekOwtfILt//nv9zV5ydlfr5Ty20spn62U8o8PLf6MUsrLUsqvK6X8yVJKgojvuu1C+TJbELGf/htLKbm/n1VK+chSSn6Wv8R2r3/qglH+tlLK1yml/NIt+PjMpZTvWUr5xKWUdy+l5N+HHAl5MtYvWkr5/0opH11K+R7bvX3xbWwXDOdZN3EmMAnMby6lfKZSypd/1kpv6M35pecNLfy9t/3WW28JTO5Fcz4BAgSeocDLly/97nBtXfdf1LPQ+tAHFm7fopSShdlDxyfdPgG/dkSPt5ZPrt/08Oa5BiYJ6hJc/O+LJ9NTgcnFXd3V3O8qpXyaUsqXO1z18Uop/76U8vtKKd/48Od5TOPvlFJ+Tinl+25//iVLKX+2lPKDSik/ffuzGGbXyr8opbzPzWji8IdKKf/2gVFmh8J7lVJ+7+HvPksp5R9tbaeP/cijH39kC3V+5vaH2dmQBXvG/AHbn326UsrfKqV8YCkl/4a0Onr/G9TqPs4GJt9wCwazKylzxfGMBPzS84yK2fJWBCYtdbVNgACBdQQEJpfX6rHA5GtuC6gfUUr5Sdun13nGPu97+KbbJ9sff3sUIIPKAiuP8OS6fCr+UaWUfFr+Kw4jzrV/tJTyzbZPo79DKeWTl1L+8PbJ9T88nLu/W+PbllKyMMuC7peUUr7fdk4+wc6n3Vkg/KtSyu8opWSstwvCPNefRx6+dCklOwr+dinll5dSfvahr89fSnn/UspX3h4nyKIzjzoct7fnXn94KeVbbZ+c51GJv15K+XGHRyCy8PzJpZSvVkrJgvFfbzsCssg9Pg7yfltb77kFBH+ilPKDH3jMIbsQMq7cYzx/VCnl697xDpMao6Pzzy2lvEcp5Z+VUn7q9n6N44T73tujIlnA59GtWL5VSvkNNwvt2nnwzbcdTN9++3Q8IcCf3+bar76Z6V+9lPJBpZSvvc3Lz1FK+SGllK+y7b5IkJaFfBb3u/Xt42bHRX/u+6FHcj79VsOvVUr5lKWUv7nd43E8eTwlu7HSVwKOjOOzbo/ExDz38NSRYOPfbT9XmT/7kZ0buY+fX0r5Xoc/TyiQfrJ744duf57HX7ITJaFL/m4/ftjWbt6Lsv885Wcsu0H+WinlbTehY35W/3gp5dOWUj73FlqlrfxMZH4fA5njn3/3w/zIY0XZ2ZAdKMfA6xdvPy8ZY+bLY0d2qGSHWx5FyngSBOVn+/dvNUodc+z1fOjfoJr5kPvLnI3bHvbsY0rA9Ke3cCcBccwyj/MzmJ0bqddf2mqdR6T241X/vnyRrb/dJ/8+5b7imn+39uOxwKT234rM1Twylcegbu/t6dnob6cXEJhMX6I5BigwmaMORkGAAIHRAgKTyyvwWGDyfbZQIe9SyMJnPy+Lrn+5fZKchVwW1p9h252S3aDZvp+/zy/62c6fdyDkkYEce2DyEaWUnJvdElmgJgTJe1O+WCnlv2znZjH7+UopedfDb9o+Oc/7VPLJfN6hkBAkn5jnv+e8LOCykMljC/9jayPBRT41/6dbXwkCvkApJe+1yN/l+ELbQimPRWRRnCAkn5jnk/98apsgJsdPLKVkMfrLthDkU2whThZRMciRBVfaS/CQRXvuLf1kvAlFcuR9E+nnD2w7CbJrJmPPQjFhxb7Yz8I2n87HO6FTFrRZROdT/09W8dLXWqM4v1spJbsbsvDNroDcfx7L+o6HwOs7b4FVdhDkHRlZ9GcxGK9918O98yD3ll0lv6aUkvdxZAdF5sbf2IK342SPQRav6SP1zY6JvK8j9Y9JQow4JjxIfRM8ZIGcsWUuJ/RLAJAj489cug1MElj8hS2g+nlbKJJdE5kLmaN7yLYHJh++Lawz5zOfE3r9163ffQ4+9AObOZpHZvLz8XtuTviQLUTKYzj7IzkJyrKzI4HSvnsg95CQMvd6PBIg5efitu0s2DPnElgmdMv4MvfyrqI8XpP2j0FAAsL0lXmRICP3mkAkQU3CmIR9eziZx4Hyn69xM5bMn/y8ZJ6kro8d+fmJXSwyxjzak38/Msey2+Y2MHno36Ca+ZD+4552E8Aej4RU+dlM6Jm58+u3OZZ5kP7y85efieykyd/lqPn3JQFGvFOv/PuTfx/iGY/33uZN2nooMKn9t2K/j9Qg/x7FwvGMBAQmz6iYLW9FYNJSV9sECBBYR0Bgcnmt9l/Uv+r2y3YWE1nQZQGRBVUW0wkT9vOy4Ejw8b8OI8miKIulLLyOn5rmk9osfPIJbYKQPTBJewku/sPWRhalWaxncZtFc44sZr/CtqMhgc1+ZOdGFsj5+7S9f6KdT6SzuMmuleyESdCSBUQ+vU4Qc9x5kt8/90d9s7hMsJFHkvZPwfP3uc/09Xm3jv/i1m92Hjx0JPDIJ7y3n8gfz03QkU/9806NLJr2IyFAdjLEYP/zLFDz53HKp9s5skDLAje7BZ76lpxao6NzFnbZLZIjO3HyKX8WyNk5kcX179yChKe+TefeeZAFedrbQ7L0nWDjB273Hs99PAk48s6PLMJzJNw4Xpc/ywL0z5RSvk0p5ddu5z31SM5tYJL5l3Avu4j2RXGCg+zAyNyOR+bsHphkrufnYx9nQooEOPsumEemysfcQ6zSZnYzHY/sJsqiPIHEfsQpP1+ZI/uR62KSgOR4JEDJC1n3oPP4dxlXDPNukuzcShiYoOR9t3u8HW8Cmuwe+jyHv0iglJ+BLP73I9/wkzHvtdn/PGNO4JH2E4Q8dGSO5+c5wWZeGLsfCUQT+iVcvA1MHvo3qHY+5Ocr/57k5yrBXI7U+J/chDP59yLvkTnu9DmOv/bfl4fGlR12+bcxIVZCsRy3gck9/1bs44px3olzG6I9Qu+PVxEQmKxSqcHjFJgMLoDuCRAgMImAwOTyQjz22EIW5fmUe1/o7OdloZUdAfuR3+Xy6EkW+3kk5nhkYZXwIp/MZvfFHpjksZU83nJsIyFKvuUji6scWcxmAZydHMeXcuYxjizishjLDoz9yCI/O1vySW4WyfkEOY9GHHe43OLlUYG8oDKPA+TrXY9HHq3I4xIJDDK2jCfb/rPzI0HM7ZEdEtndkP7zyfC+iD6elwVhFqx59Of2E/cs0LMjIgvwBExZwOXrQrOr5XhkMZydPU8FJrVGu3MCsgQ+2S2yH1lw/8LtUabseshuoOzwSGjz0CMnrzMPsijOo0/HI7sLEk59p+3Rqfxd5lF2H+SxnARGt0cWvJknGUNqk7Gm7jnuCUwy17MbIjs3jo+W7AvcPQjZA5NfsD06so8n31STn4Vj8PfAcD9mN0V2Vexz63hOAoQ8epO5lEfVsuMhj+Fk10N2umS+5sijJQlQbnd17I+dPDbv910LCe4SAGUX1WPfrJK5mJ/VmGYOJMzJfMxOncyD7KbJkfA0IUR+Zo5H5nnuIfM+gdtDR95vkrmfn6vjt/vkZzOB1EOBye2/QffMh8zz7DjLe1+yc+c4v3JPCVBzZKdX/j3J7pD8LN4eNf++3F6TMDpBSP6TR7qOu5ZuA5PafyuOfWQnXswTADuekYDA5BkVs+WtCExa6mqbAAEC6wgITC6v1f6LenZo5HGM/7l9cp3F2HHRuJ93/FQ0g8kv56/66uFvsH2avQcm+y6Q483kkZW0lXcn5EhAkU9Lj59u58+zeMwiLn9++3LD7MrI+LNbJO9ZyQLiuAi6xdtfnPkUaj7pT7u57+weyIIrn+7nfRrZxZCQZz+yAMp7W7KAzAIzn5onXNo/jd8Xyo/1l0Vy3kWw75Q4PhKzX5PAJWN6KjCpNdqd806ShEHHY1/sJnyJYz6R3x8DyftUElwkuEoQ9rrz4LgT5Nh3FuRZ0GcRnSPOCdISJKW+OfLJfRbvef9JAo7jmiIhXeZYjnsCk+w4yAtTb79pZA9xstsgO6/2wCTO++NY+/izcyk7I47vJrmt9z4PMr+zu2I/8vhL5lrmft4Xsx8JLhKU5d0UeV9KjtfZYbK3lx0zuccEJQkEHjr297ckvMmc3o/s+sr4Eo4kUMtxZofJ/s6VBD0JEY5HwqeM8XaHye2/Qbmmdj7k3IS7+fae/d+W7PZIGJVHjfZ/8/JYWsKaBLHZVZP3juRnef83p+bfl/SV4CfBYEK32yDjGBjeBia1/1YcvbLLJ//GJnRzPCMBgckzKmbLWxGYtNTVNgECBNYREJhcXqvH3mFy29Fj5+UT8Hxim+3rty/q3NtIqJCF6L2Bybtsj2wcx1IbBtQsaPZgIp82P/bIQB5N2R8dyuInC8ws5LOAz4shsxMjj1fsRxZh2YmR3RBZXGZnQMKHLIT3sedT/uMjDfu1CQP2nTV5tGSmwCRjzM6W7PbIvedxqCzMskMkC7/XmQfHb1U51jiBQ3YrJSCJfeZOFrWx3o+YJyzJIzSxymNLCSsS7iSo2hfZLQOThx6/yhgSluQeHjsee7fHHlLd7rZIO3k3RSyyWyvHve8w2ceyvy8kgVf6SWCRnUy3RwKnvBw5j6Td7qiKdXb85NGlHGfeYfI6gcntN3plDLXzIefuO5aysyo7vfKzmN1Cx28DynmZf9npEacEr3m8L+FvdrbV/PuSNrJjJS+UTfCUnVMJl9JOAtfjPLkNTGr/rTjWLTVNeJj3pDiekYDA5BkVs+WtCExa6mqbAAEC6wgITC6v1dnAJM/y5/GTLFJf9fWh9z6S81Bg8tTjJllYZ4FS+0jOvjvm9hGhGuRsq993xeTRioeO7AzIIinvisjicn9Xy2OPluxttHwk52iU/hLQ1DySc3t/+eQ9u10SnsQi7zm5dx48FphkN0tetJmAJLuX9ndt7O8cyVjyjon0v+8kyZ/lkYcsSBPe7YFJHjnJi2rzro7j9fu95/9mXuZ47JGcfXF8+0jO6wYm+0tfE74dH4fZ53bCqCyoj0c8sgMpIV+OLMDz2M3tt+TkUbe8oPj4LTl7O9mdknAkj6DlG2CyYyW7ovLoXb6B6njsQcbxXR/5+6zdEtzk5z27JnLknTzZoXH7LTlp81tuY3zsW3IeeyQnL1nN40cPPZLzUGBSOx8y3uzkyWM2GXcC0fSRXUTH3WK38z3/VuSluHlUJ6FVzSM5+yNat4+e5d+F7OZ7KjCp/bfiOE4vfb2t2jP57wKTZ1LI1rchMGktrH0CBAisISAwubxOZwOTDCiPQGThk23uty+xzAtI8y6AHK966evxmf79625vXzK6v9A0XyGb9zfsL2/NN6TkU+L9cZ98iptHR1710td8a0jeW5F+slPmeBzHngXc8YW2OS9b+7MQz3l5QW628+/vdcjfp+8szPJixyyA8p6NPGqS3Sb5xPr2m1SO/V3x0tdXGWWM+8t1H3rpaz6tzn8yzofuP9+YkuvymFIW0ffOg8cCk4wri9csmBOYZKdOQqnjI2KpRcKG7DLZjwQYGdNxkZ1AJzsCHnqPxmMvfc1czo6WHFlc57wsqG9f+vq6gcn+tcLZ7ZHwYj/y8/OhN+PP3+URrLw3JgFE5nmOfJ1tHvs6jiHv0cnPX2z2YGVve/+Wo4Qk+VrdHFmH5duH8mhUwprM5/3Yg6bb3TIJefI+kuPjSHugdKxnws4s4BNC7cHKO/1wbf9lf+lrdqxk98Z+PPXS14cCk9r5sLefbzxKnbPDJGPNvwH7kRA4IeD+suX9z//cZpb+a/59yc972rh9RCuPde3vSNp3It3uMLnn34qMb/9a4bwweX9580Pe/mxBAYHJgkUbMWSByQh1fRIgQGA+AYHJ5TW5IjDJoief1GbBn69Yzafh+eQ7C718+07+/xy3XyucBXauTVCSdzlkUZpHWHI8Fpjk7/avzM0W9Cya87XCWYDcfq1wdnJkIZbQIn0lEMk7UrJlPX+XI98okW/dyGI8Y887CjKmL70t0jOmHPvX0OZ9Bnm3Qj5hzjdu5Jt58rW1+SaevOAyi87cfx6vySI9wUh2vOSbSXJkkZZ3cuScPD6SMCm7Ab7m9j6Q/Vs5stDPN4zsXyscw7zX4t6vFX6V0fFrhfMOhHzynQVwPkXP/cUkR+47jy7knSWxyM6DjDXt59thctw7D54KTPJITh73SQCVR0NifDwSimT3wv61r6lX5lreZXF8JCePCsUsgUNe7JudDgmSstPmsa8VzmNV+1dDp3YJbB76WuHXDUxyH5mXCaHyuMbxiGfmTHbV5P/PbqPUPTt6Eqgcvykncy1zLCFIwsG8DDXv5ck35+xfY5228+hYXuKcoCOPA+0hY/4u4UB24GQ+5773Fyynv/w85ecj1vtLX1Pz7CRKwLC/gDZt5GcooWN2vuTP8/OYeZ1w4Tjmm9v9mP+aR+ISvMUkO2v2rxXea7mHYk/9W1U7H/b+93Aq/z07bxK07UcCwMyZuORRqOxaytzKe02OwWLNvy95X0xsU6O8PDqP9+SdQbnHp3aY3PNvRc7dA668mDcvBHY8IwGByTMqZstbEZi01NU2AQIE1hEQmFxeqysCkwwqW9bzaXkWz1mk5hPfvKgyi/B90b0HJvk0OwuuLN6ymMsCNgusf3C4u6cCk5yWl9Rm8ZZFXgKMPJ6RxxGOXx+c8/L4Qz6tzifu+VQ4i4mMJwvt/cgLJ3NOFjNZxGYxnR0e+baVPejIAj73lnc65JP8LEATfGSBuO/AyAIoi9W8zDOBSV4imhdmZuv/8YhDPqHPmNJWFlLZhZIxJZjYj3zi/v7bt+dk3D9ye4dKrn/qpa/79TVGu3MW2wkJ3mMLRHJf+SR8PxKeJKBI2JRP37OgjHnGl0dF9uOeefBUYJKF3/7ujP1blo6GWdTmk/Q8JpMdGwly8u002dGQe9ofyck1+cadPGKSF9tmcb8/nnMbmOzzOI+tpN18yp/FfvrJXNiP/aWvZwKTBB2ZWxlTdh3tR0KC7BLIrowsrBNgZG7kG13yeNfxyH3n0Zo87pXHP7IrJ+c99D6ehBl5me7x68D3tjIH09f+Nbv7n6fNtJcwL+PMLqI88pafs9sXtObczJm8vyf3kIAq95EdM686UpP8/GUXTHZK5J00mbsJYfKzs++qeerfqnvmwz6e7MZJ8JdgJz+D+5GwKPM6/x7k34Z9N0m+CWh/0e1+7qv+fckOrfxcZc5l3ZsQLPM0Ie6rApP0UftvRcLX9JVHoxzPTEBg8swK2up2BCatZLVLgACBtQQEJmvV62a0e2Dy1EJ56Rs0eAKVAgkJsnsou0T2r7etvPSNOC0BSHayJCTMO1laHAlFE7Ym5Fz5SECdACshW77Jy/HMBAQmz6ygrW5HYNJKVrsECBBYS0Bgsla9BCZL18vg2wrk0afsWMgOhzz28aYe2ZHyX25ufn/s7qHdRVc47S9tza6Vx77d64p+erSRHVH5hqU8juV4hgICk2dY1Ba3JDBpoapNAgQIrCcgMFmvZocR22GydPkMnkATgYQW+c/v34KjhCR5bC+Pr+zvGrqq4zyalPeX5F0kedlrHrk5vqj5qn60Q+AyAYHJZZTPuyGByfOur7sjQIBArYDApFZqyvMEJlOWxaAIDBXIy6Hz0tW8ODnvjclLhfN+lzyOc/XOm+xcybuW8m6afG11XsrqIDC1gMBk6vLMMziByTy1MBICBAiMFBCYjNTXNwECBAgQINBTQGDSU3vhvgQmCxfP0AkQIHChgMDkQkxNESBAgAABAlMLCEymLs88gxOYzFMLIyFAgMBIAYHJSH19EyBAgAABAj0FBCY9tRfuS2CycPEMnQABAhcKCEwuxNQUAQIECBAgMLWAwGTq8swzOIHJPLUwEgIECIwUEJiM1Nc3AQIECBAg0FNAYNJTe+G+BCYLF8/QCRAgcKGAwOQ+zHe84x3vuO+Kx89+8cKvbVdZaocAAQI1Ahf+E15e+Ee8hny6c/wv73QlmXNAApM562JUBAgQ6C0gMLlPXGByn5ezCRAgMJOAwGSmaowZi8BkjPtyvQpMliuZARMgQKCJgMDkPlaByX1eziZAgMBMAgKTmaoxZiwCkzHuy/UqMFmuZAZMgACBJgICk/tYBSb3eTmbAAECMwkITGaqxpixCEzGuC/Xq8BkuZIZMAECBJoICEzuYxWY3OflbAIECMwkIDCZqRpjxiIwGeO+XK8Ck+VKZsAECBBoIiAwuY9VYHKfl7MJECAwk4DAZKZqjBmLwGSM+3K9CkyWK5kBEyBAoImAwOQ+VoHJfV7OJkCAwEwCApOZqjFmLAKTMe7L9SowWa5kBkyAAIEmAgKT+1gFJvd5OZsAAQIzCQhMZqrGmLEITMa4L9erwGS5khkwAQIEmggITO5jFZjc5+VsAgQIzCQgMJmpGmPGIjAZ475crwKT5UpmwAQIEGgiIDC5j1Vgcp+XswkQIDCTgMBkpmqMGYvAZIz7cr0KTJYrmQETIECgiYDA5D5Wgcl9Xs4mQIDATAICk5mqMWYsApMx7sv1KjBZrmQGTIAAgSYCApP7WAUm93k5mwABAjMJCExmqsaYsQhMxrgv16vAZLmSGTABAgSaCAhM7mMVmNzn5WwCBAjMJCAwmakaY8YiMBnjvlyvApPlSmbABAgQaCIgMLmPVWByn5ezCRAgMJOAwGSmaowZi8BkjPtyvQpMliuZARMgQKCJgMDkPlaByX1eziZAgMBMAgKTmaoxZiwCkzHuy/UqMFmuZAZMgACBJgICk/tYBSb3eTmbAAECMwkITGaqxpixCEzGuC/Xq8BkuZIZMAECBJoICEzuYxWY3OflbAIECMwkIDCZqRpjxiIwGeO+XK9XBiYvX75c7v4NmAABAisLvPXWW5cNX2ByH6XA5D4vZxMgQGAmAYHJTNUYMxaByRj35XoVmCxXMgMmQIDA/xEQmIybDAKTcfZ6JkCAwFkBgclZwfWvF5isX8MudyAw6cKsEwIECDQREJg0Ya1qVGBSxeQkAgQITCkgMJmyLF0HJTDpyr1uZwKTdWtn5AQIEBCYjJsDApNx9nomQIDAWQGByVnB9a8XmKxfwy53IDDpwqwTAgQINBEQmDRhrWpUYFLF5CQCBAhMKSAwmbIsXQclMOnKvW5nApN1a2fkBAgQEJiMmwMCk3H2eiZAgMBZAYHJWcH1rxeYrF/DLncgMOnCrBMCBAg0ERCYNGGtalRgUsXkJAIECEwpIDCZsixdByUw6cq9bmcCk3VrZ+QECBAQmIybAwKTcfZ6JkCAwFkBgclZwfWvF5isX8MudyAw6cKsEwIECDQREJg0Ya1qVGBSxeQkAgQITCkgMJmyLF0HJTDpyr1uZwKTdWtn5AQIEBCYjJsDApNx9nomQIDAWQGByVnB9a8XmKxfwy53IDDpwqwTAgQINBEQmDRhmE0a1QAAIABJREFUrWpUYFLF5CQCBAhMKSAwmbIsXQclMOnKvW5nApN1a2fkBAgQEJiMmwMCk3H2eiZAgMBZAYHJWcH1rxeYrF/DLncgMOnCrBMCBAg0ERCYNGGtalRgUsXkJAIECEwpIDCZsixdByUw6cq9bmcCk3VrZ+QECBAQmIybAwKTcfZ6JkCAwFkBgclZwfWvF5isX8MudyAw6cKsEwIECDQREJg0Ya1qVGBSxeQkAgQITCkgMJmyLF0HJTDpyr1uZwKTdWtn5AQIEBCYjJsDApNx9nomQIDAWQGByVnB9a8XmKxfwy53IDDpwqwTAgQINBEQmDRhrWpUYFLF5CQCBAhMKSAwmbIsXQclMOnKvW5nApN1a2fkBAgQEJiMmwMCk3H2eiZAgMBZAYHJWcH1rxeYrF/DLncgMOnCrBMCBAg0ERCYNGGtalRgUsXkJAIECEwpIDCZsixdByUw6cq9bmcCk3VrZ+QECBAQmIybAwKTcfZ6JkCAwFkBgclZwfWvF5isX8MudyAw6cKsEwIECDQREJg0Ya1qVGBSxeQkAgQITCkgMJmyLF0HJTDpyr1uZwKTdWtn5AQIEBCYjJsDApNx9nomQIDAWQGByVnB9a8XmKxfwy53IDDpwqwTAgQINBEQmDRhrWpUYFLF5CQCBAhMKSAwmbIsXQclMOnKvW5nApN1a2fkBAgQEJiMmwMCk3H2eiZAgMBZAYHJWcH1rxeYrF/DLncgMOnCrBMCBAg0ERCYNGGtalRgUsXkJAIECEwpIDCZsixdByUw6cq9bmcCk3VrZ+QECBAQmIybAwKTcfZ6JkCAwFkBgclZwfWvF5isX8MudyAw6cKsEwIECDQREJg0Ya1qVGBSxeQkAgQITCkgMJmyLF0HJTDpyr1uZwKTdWtn5AQIEBCYjJsDApNx9nomQIDAWQGByVnB9a8XmKxfwy53IDDpwqwTAgQINBEQmDRhrWpUYFLF5CQCBAhMKSAwmbIsXQclMOnKvW5nApN1a2fkBAgQEJiMmwMCk3H2eiZAgMBZAYHJWcH1rxeYrF/DLncgMOnCrBMCBAg0ERCYNGGtalRgUsXkJAIECEwpIDCZsixdByUw6cq9bmcCk3VrZ+QECBAQmIybAwKTcfZ6JkCAwFkBgclZwfWvF5isX8MudyAw6cKsEwIECDQREJg0Ya1qVGBSxeQkAgQITCkgMJmyLF0HJTDpyr1uZwKTdWtn5AQIEBCYjJsDApNx9nomQIDAWQGByVnB9a8XmKxfwy53IDDpwqwTAgQINBEQmDRhrWpUYFLF5CQCBAhMKSAwmbIsXQclMOnKvW5nApN1a2fkBAgQEJiMmwMCk3H2eiZAgMBZAYHJWcH1rxeYrF/DLncgMOnCrBMCBAg0ERCYNGGtalRgUsXkJAIECEwpIDCZsixdByUw6cq9bmcCk3VrZ+QECBAQmIybAwKTcfZ6JkCAwFkBgclZwfWvF5isX8MudyAw6cKsEwIECDQREJg0Ya1qVGBSxeQkAgQITCkgMJmyLF0HJTDpyr1uZwKTdWtn5AQIEBCYjJsDApNx9nomQIDAWQGByVnB9a8XmKxfwy53IDDpwqwTAgQINBEQmDRhrWr0ysDkwz/8w6v6dBIBAgQIXCPwHu/xHtc0VEp58eKFtfdlmv0aUrR+1kv3JDBZunwGT4DAGy4gMBk3AQQm4+z1TIAAgbMCApOzgutfLzBZv4Zd7kBg0oVZJwQIEGgiIDBpwlrVqMCkislJBAgQmFJAYDJlWboOSmDSlXvdzgQm69bOyAkQICAwGTcHBCbj7PVMgACBswICk7OC618vMFm/hl3uQGDShVknBAgQaCIgMGnCWtWowKSKyUkECBCYUkBgMmVZug5KYNKVe93OBCbr1s7ICRAgIDAZNwcEJuPs9UyAAIGzAgKTs4LrXy8wWb+GXe5AYNKFWScECBBoIiAwacJa1ajApIrJSQQIEJhSQGAyZVm6Dkpg0pV73c4EJuvWzsgJECAgMBk3BwQm4+z1TIAAgbMCApOzgutfLzBZv4Zd7kBg0oVZJwQIEGgiIDBpwlrVqMCkislJBAgQmFJAYDJlWboOSmDSlXvdzgQm69bOyAkQICAwGTcHBCbj7PVMgACBswICk7OC618vMFm/hl3uQGDShVknBAgQaCIgMGnCWtWowKSKyUkECBCYUkBgMmVZug5KYNKVe93OBCbr1s7ICRAgIDAZNwcEJuPs9UyAAIGzAgKTs4LrXy8wWb+GXe5AYNKFWScECBBoIiAwacJa1ajApIrJSQQIEJhSQGAyZVm6Dkpg0pV73c4EJuvWzsgJECAgMBk3BwQm4+z1TIAAgbMCApOzgutfLzBZv4Zd7kBg0oVZJwQIEGgiIDBpwlrVqMCkislJBAgQmFJAYDJlWboOSmDSlXvdzgQm69bOyAkQICAwGTcHrgxMPvIjP3LcjeiZAAECb6DAu73bu1121y9evLD2vkyzX0OK1s966Z4EJkuXz+AJEHjDBQQm4yaAwGScvZ4JECBwVkBgclZw/esFJuvXsMsdCEy6MOuEAAECTQQEJk1YqxoVmFQxOYkAAQJTCghMpixL10EJTLpyr9uZwGTd2hk5AQIEBCbj5oDAZJy9ngkQIHBWQGByVnD96wUm69ewyx0ITLow64QAAQJNBAQmTVirGhWYVDE5iQABAlMKCEymLEvXQQlMunKv29mVgcn7vP3t60IYOQECBBYU+OC3ve2yUb98+dLvDndoCkzuwHIqAQIEJhMQmExWkAHD8UvPAPQVuxSYrFg1YyZAgMDHCghMxs0Egck4ez0TIEDgrIDA5Kzg+tcLTNavYZc7EJh0YdYJAQIEmggITJqwVjUqMKlichIBAgSmFBCYTFmWroMSmHTlXrczgcm6tTNyAgQICEzGzQGByTh7PRMgQOCsgMDkrOD61wtM1q9hlzsQmHRh1gkBAgSaCAhMmrBWNSowqWJyEgECBKYUEJhMWZaugxKYdOVetzOBybq1M3ICBAgITMbNAYHJOHs9EyBA4KyAwOSs4PrXC0zWr2GXOxCYdGHWCQECBJoICEyasFY1KjCpYnISAQIEphQQmExZlq6DEph05V63M4HJurUzcgIECAhMxs0Bgck4ez0TIEDgrIDA5Kzg+tcLTNavYZc7EJh0YdYJAQIEmggITJqwVjUqMKlichIBAgSmFBCYTFmWroMSmHTlXrczgcm6tTNyAgQICEzGzYErA5OP+IiPGHcjeiZAgMAbKPDu7/7ul931ixcvrL0v0+zXkKL1s166J4HJ0uUzeAIE3nABgcm4CSAwGWevZwIECJwVEJicFVz/eoHJ+jXscgcCky7MOiFAgEATAYFJE9aqRgUmVUxOIkCAwJQCApMpy9J1UAKTrtzrdiYwWbd2Rk6AAAGBybg5IDAZZ69nAgQInBUQmJwVXP96gcn6NexyBwKTLsw6IUCAQBMBgUkT1qpGBSZVTE4iQIDAlAICkynL0nVQApOu3Ot2JjBZt3ZGToAAAYHJuDkgMBlnr2cCBAicFRCYnBVc/3qByfo17HIHVwYm7/13PqDLmHVCgAABAh8r8CGf+xtdRvHy5Uu/O9yhKTC5A8upBAgQmExAYDJZQQYMxy89A9BX7FJgsmLVjJkAAQICk9FzQGAyugL6J0CAwOsLCExe3+65XCkweS6VbHwfApPGwJonQIBAQwE7TBrivqJpgck4ez0TIEDgrIDA5Kzg+tcLTNavYZc7EJh0YdYJAQIEmggITJqwVjUqMKlichIBAgSmFBCYTFmWroMSmHTlXrczgcm6tTNyAgQICEzGzQGByTh7PRMgQOCsgMDkrOD61wtM1q9hlzsQmHRh1gkBAgSaCAhMmrBWNSowqWJyEgECBKYUEJhMWZaugxKYdOVetzOBybq1M3ICBAgITMbNAYHJOHs9EyBA4KyAwOSs4PrXC0zWr2GXOxCYdGHWCQECBJoICEyasFY1KjCpYnISAQIEphQQmExZlq6DEph05V63M4HJurUzcgIECAhMxs0Bgck4ez0TIEDgrIDA5Kzg+tcLTNavYZc7EJh0YdYJAQIEmggITJqwVjV6ZWDyUR/1UVV9OokAAQIErhF413d912saKqW8ePHC2vsyzX4NKVo/66V7EpgsXT6DJ0DgDRcQmIybAAKTcfZ6JkCAwFkBgclZwfWvF5isX8MudyAw6cKsEwIECDQREJg0Ya1qVGBSxeQkAgQITCkgMJmyLF0HJTDpyr1uZwKTdWtn5AQIEBCYjJsDApNx9nomQIDAWQGByVnB9a8XmKxfwy53IDDpwqwTAgQINBEQmDRhrWpUYFLF5CQCBAhMKSAwmbIsXQclMOnKvW5nVwYmn/ejf826EEZOgACBBQX+1rt8m8tG/fLlS7873KEpMLkDy6kECBCYTEBgMllBBgzHLz0D0FfsUmCyYtWMmQABAh8rIDAZNxMEJuPs9UyAAIGzAgKTs4LrXy8wWb+GXe5AYNKFWScECBBoIiAwacJa1ajApIrJSQQIEJhSQGAyZVm6Dkpg0pV73c4EJuvWzsgJECAgMBk3BwQm4+z1TIAAgbMCApOzgutfLzBZv4Zd7kBg0oVZJwQIEGgiIDBpwlrVqMCkislJBAgQmFJAYDJlWboOSmDSlXvdzgQm69bOyAkQICAwGTcHBCbj7PVMgACBswICk7OC618vMFm/hl3uQGDShVknBAgQaCIgMGnCWtWowKSKyUkECBCYUkBgMmVZug5KYNKVe93OBCbr1s7ICRAgIDAZNwcEJuPs9UyAAIGzAgKTs4LrXy8wWb+GXe5AYNKFWScECBBoIiAwacJa1ajApIrJSQQIEJhSQGAyZVm6Dkpg0pV73c4EJuvWzsgJECAgMBk3B64MTD7wAz9w3I3omQABAm+gwPu93/tddtcvXryw9r5Ms19DitbPeumeBCZLl8/gCRB4wwUEJuMmgMBknL2eCRAgcFZAYHJWcP3rBSbr17DLHQhMujDrhAABAk0EBCZNWKsaFZhUMTmJAAECUwoITKYsS9dBCUy6cq/bmcBk3doZOQECBAQm4+aAwGScvZ4JECBwVkBgclZw/esFJuvXsMsdCEy6MOuEAAECTQQEJk1YqxoVmFQxOYkAAQJTCghMpixL10EJTLpyr9uZwGTd2hk5AQIEBCbj5oDAZJy9ngkQIHBWQGByVnD96wUm69ewyx1cGZi8y7v8si5j1gkBAgQIfKzAR3/0d7qM4uXLl353uENTYHIHllMJECAwmYDAZLKCDBiOX3oGoK/YpcBkxaoZMwECBAQmo+eAwGR0BfRPgACB1xcQmLy+3XO5UmDyXCrZ+D4EJo2BNU+AAIGGAnaYNMR9RdMCk3H2eiZAgMBZAYHJWcH1rxeYrF/DLncgMOnCrBMCBAg0ERCYNGGtalRgUsXkJAIECEwpIDCZsixdByUw6cq9bmcCk3VrZ+QECBAQmIybAwKTcfZ6JkCAwFkBgclZwfWvF5isX8MudyAw6cKsEwIECDQREJg0Ya1qVGBSxeQkAgQITCkgMJmyLF0HJTDpyr1uZwKTdWtn5AQIEBCYjJsDApNx9nomQIDAWQGByVnB9a8XmKxfwy53IDDpwqwTAgQINBEQmDRhrWpUYFLF5CQCBAhMKSAwmbIsXQclMOnKvW5nApN1a2fkBAgQEJiMmwMCk3H2eiZAgMBZAYHJWcH1rxeYrF/DLncgMOnCrBMCBAg0ERCYNGGtavTKwORDP/RDq/p0EgECBAhcI/Be7/Ve1zRUSnnx4oW192Wa/RpStH7WS/ckMFm6fAZPgMAbLiAwGTcBBCbj7PVMgACBswICk7OC618vMFm/hl3uQGDShVknBAgQaCIgMGnCWtWowKSKyUkECBCYUkBgMmVZug5KYNKVe93OBCbr1s7ICRAgIDAZNwcEJuPs9UyAAIGzAgKTs4LrXy8wWb+GXe5AYNKFWScECBBoIiAwacJa1ajApIrJSQQIEJhSQGAyZVm6Dkpg0pV73c6uDEze/mnevi6EkRMgQGBBgbf967ddNuqXL1/63eEOTYHJHVhOJUCAwGQCApPJCjJgOH7pGYC+YpcCkxWrZswECBD4WAGBybiZIDAZZ69nAgQInBUQmJwVXP96gcn6NexyBwKTLsw6IUCAQBMBgUkT1qpGBSZVTE4iQIDAlAICkynL0nVQApOu3Ot2JjBZt3ZGToAAAYHJuDkgMBlnr2cCBAicFRCYnBVc/3qByfo17HIHApMuzDohQIBAEwGBSRPWqkYFJlVMTiJAgMCUAgKTKcvSdVACk67c63YmMFm3dkZOgAABgcm4OSAwGWevZwIECJwVEJicFVz/eoHJ+jXscgcCky7MOiFAgEATAYFJE9aqRgUmVUxOIkCAwJQCApMpy9J1UAKTrtzrdiYwWbd2Rk6AAAGBybg5IDAZZ69nAgQInBUQmJwVXP96gcn6NexyBwKTLsw6IUCAQBMBgUkT1qpGBSZVTE4iQIDAlAICkynL0nVQApOu3Ot2JjBZt3ZGToAAAYHJuDlwZWDy4oVf28ZVUs8ECLyJAu94xzsuu+0X/hG/zLJnQ/6Xt6f2wn0JTBYunqETIPDGCwhMxk0Bgck4ez0TIEDgrIDA5Kzg+tcLTNavYZc7EJh0YdYJAQIEmggITJqwVjUqMKlichIBAgSmFBCYTFmWroMSmHTlXrczgcm6tTNyAgQICEzGzQGByTh7PRMgQOCsgMDkrOD61wtM1q9hlzsQmHRh1gkBAgSaCAhMmrBWNSowqWJyEgECBKYUEJhMWZaugxKYdOVetzOBybq1M3ICBAgITMbNAYHJOHs9EyBA4KyAwOSs4PrXC0zWr2GXO7gyMPm9b3+fLmPWCQECBAh8rMDXetsHX0bx8uVLvzvcoSkwuQPLqQQIEJhMQGAyWUEGDMcvPQPQV+xSYLJi1YyZAAECApPRc0BgMroC+idAgMDrCwhMXt/uuVwpMHkulWx8HwKTxsCaJ0CAQEMBO0wa4r6iaYHJOHs9EyBA4KyAwOSs4PrXC0zWr2GXOxCYdGHWCQECBJoICEyasFY1KjCpYnISAQIEphQQmExZlq6DEph05V63M4HJurUzcgIECAhMxs0Bgck4ez0TIEDgrIDA5Kzg+tcLTNavYZc7EJh0YdYJAQIEmggITJqwVjUqMKlichIBAgSmFBCYTFmWroMSmHTlXrczgcm6tTNyAgQICEzGzQGByTh7PRMgQOCsgMDkrOD61wtM1q9hlzsQmHRh1gkBAgSaCAhMmrBWNSowqWJyEgECBKYUEJhMWZaugxKYdOVetzOBybq1M3ICBAgITMbNAYHJOHs9EyBA4KyAwOSs4PrXC0zWr2GXOxCYdGHWCQECBJoICEyasFY1KjCpYnISAQIEphQQmExZlq6DEph05V63M4HJurUzcgIECAhMxs0Bgck4ez0TIEDgrIDA5Kzg+tcLTNavYZc7EJh0YdYJAQIEmggITJqwVjUqMKlichIBAgSmFBCYTFmWroMSmHTlXrczgcm6tTNyAgQICEzGzQGByTh7PRMgQOCsgMDkrOD61wtM1q9hlzsQmHRh1gkBAgSaCAhMmrBWNSowqWJyEgECBKYUEJhMWZaugxKYdOVet7MrA5OXL1+uC2HkBAgQWFDgrbfeumzUL1++9LvDHZoCkzuwnEqAAIHJBAQmkxVkwHD80jMAfcUuBSYrVs2YCRAg8LECApNxM0FgMs5ezwQIEDgrIDA5K7j+9QKT9WvY5Q4EJl2YdUKAAIEmAgKTJqxVjQpMqpicRIAAgSkFBCZTlqXroAQmXbnX7Uxgsm7tjJwAAQICk3FzQGAyzl7PBAgQOCsgMDkruP71ApP1a9jlDgQmXZh1QoAAgSYCApMmrFWNCkyqmJxEgACBKQUEJlOWpeugBCZdudftTGCybu2MnAABAgKTcXNAYDLOXs8ECBA4KyAwOSu4/vUCk/Vr2OUOBCZdmHVCgACBJgICkyasVY0KTKqYnESAAIEpBQQmU5al66AEJl251+1MYLJu7YycAAECApNxc0BgMs5ezwQIEDgrIDA5K7j+9QKT9WvY5Q4EJl2YdUKAAIEmAgKTJqxVjQpMqpicRIAAgSkFBCZTlqXroAQmXbnX7Uxgsm7tjJwAAQICk3FzQGAyzl7PBAgQOCsgMDkruP71ApP1a9jlDgQmXZh1QoAAgSYCApMmrFWNCkyqmJxEgACBKQUEJlOWpeugBCZdudftTGCybu2MnAABAgKTcXNAYDLOXs8ECBA4KyAwOSu4/vUCk/Vr2OUOBCZdmHVCgACBJgICkyasVY0KTKqYnESAAIEpBQQmU5al66AEJl251+1MYLJu7YycAAECApNxc0BgMs5ezwQIEDgrIDA5K7j+9QKT9WvY5Q4EJl2YdUKAAIEmAgKTJqxVjQpMqpicRIAAgSkFBCZTlqXroAQmXbnX7Uxgsm7tjJwAAQICk3FzQGAyzl7PBAgQOCsgMDkruP71ApP1a9jlDgQmXZh1QoAAgSYCApMmrFWNCkyqmJxEgACBKQUEJlOWpeugBCZdudftTGCybu2MnAABAgKTcXNAYDLOXs8ECBA4KyAwOSu4/vUCk/Vr2OUOBCZdmHVCgACBJgICkyasVY0KTKqYnESAAIEpBQQmU5al66AEJl251+1MYLJu7YycAAECApNxc0BgMs5ezwQIEDgrIDA5K7j+9QKT9WvY5Q4EJl2YdUKAAIEmAgKTJqxVjQpMqpicRIAAgSkFBCZTlqXroAQmXbnX7Uxgsm7tjJwAAQICk3FzQGAyzl7PBAgQOCsgMDkruP71ApP1a9jlDgQmXZh1QoAAgSYCApMmrFWNCkyqmJxEgACBKQUEJlOWpeugBCZdudftTGCybu2MnAABAgKTcXNAYDLOXs8ECBA4KyAwOSu4/vUCk/Vr2OUOBCZdmHVCgACBJgICkyasVY0KTKqYnESAAIEpBQQmU5al66AEJl251+1MYLJu7YycAAECApNxc0BgMs5ezwQIEDgrIDA5K7j+9QKT9WvY5Q4EJl2YdUKAAIEmAgKTJqxVjQpMqpicRIAAgSkFBCZTlqXroAQmXbnX7Uxgsm7tjJwAAQICk3FzQGAyzl7PBAgQOCsgMDkruP71ApP1a9jlDgQmXZh1QoAAgSYCApMmrFWNCkyqmJxEgACBKQUEJlOWpeugBCZdudftTGCybu2MnAABAgKTcXNAYDLOXs8ECBA4KyAwOSu4/vUCk/Vr2OUOBCZdmHVCgACBJgICkyasVY0KTKqYnESAAIEpBQQmU5al66AEJl251+1MYLJu7YycAAECApNxc0BgMs5ezwQIEDgrIDA5K7j+9QKT9WvY5Q4EJl2YdUKAAIEmAgKTJqxVjQpMqpicRIAAgSkFBCZTlqXroAQmXbnX7Uxgsm7tjJwAAQICk3FzQGAyzl7PBAgQOCsgMDkruP71ApP1a9jlDgQmXZh1QoAAgSYCApMmrFWNCkyqmJxEgACBKQUEJlOWpeugBCZdudftTGCybu2MnAABAgKTcXNAYDLOXs8ECBA4KyAwOSu4/vUCk/Vr2OUOBCZdmHVCgACBJgICkyasVY0KTKqYnESAAIEpBQQmU5al66AEJl251+1MYLJu7YycAAECApNxc0BgMs5ezwQIEDgrIDA5K7j+9QKT9WvY5Q4EJl2YdUKAAIEmAgKTJqxVjQpMqpicRIAAgSkFBCZTlqXroAQmXbnX7Uxgsm7tjJwAAQICk3FzQGAyzl7PBAgQOCsgMDkruP71ApP1a9jlDgQmXZh1QoAAgSYCApMmrFWNCkyqmJxEgACBKQUEJlOWpeugBCZdudftTGCybu2MnAABAgKTcXNAYDLOXs8ECBA4KyAwOSu4/vUCk/Vr2OUOBCZdmHVCgACBJgICkyasVY0KTKqYnESAAIEpBQQmU5al66AEJl251+1MYLJu7YycAAECApNxc0BgMs5ezwQIEDgrIDA5K7j+9QKT9WvY5Q4EJl2YdUKAAIEmAgKTJqxVjQpMqpicRIAAgSkFBCZTlqXroAQmXbnX7Uxgsm7tjJwAAQICk3FzQGAyzl7PBAgQOCsgMDkruP71ApP1a9jlDgQmXZh1QoAAgSYCApMmrFWNCkyqmJxEgACBKQUEJlOWpeugBCZdudftTGCybu2MnAABAgKTcXNAYDLOXs8ECBA4KyAwOSu4/vUCk/Vr2OUOBCZdmHVCgACBJgICkyasVY0KTKqYnESAAIEpBQQmU5al66AEJl251+1MYLJu7YycAAECApNxc0BgMs5ezwQIEDgrIDA5K7j+9QKT9WvY5Q4EJl2YdUKAAIEmAgKTJqxVjQpMqpicRIAAgSkFBCZTlqXroAQmXbnX7Uxgsm7tjJwAAQICk3FzQGAyzl7PBAgQOCsgMDkruP71ApP1a9jlDgQmXZh1QoAAgSYCApMmrFWNCkyqmJxEgACBKQUEJlOWpeugBCZdudftTGCybu2MnAABAgKTcXNAYDLOXs8ECBA4KyAwOSu4/vUCk/Vr2OUOBCZdmHVCgACBJgICkyasVY0KTKqYnESAAIEpBQQmU5al66AEJl251+1MYLJu7YycAAECApNxc0BgMs5ezwQIEDgrIDA5K7j+9QKT9WvY5Q4EJl2YdUKAAIEmAgKTJqxVjQpMqpicRIAAgSkFBCZTlqXroAQmXbnX7Uxgsm7tjJwAAQICk3FzQGAyzl7PBAgQOCsgMDkruP71ApP1a9jlDgQmXZh1QoAAgSYCApMmrFWNCkyqmJxEgACBKQUEJlOWpeugBCZdudftTGCybu2MnAABAgKTcXNAYDLOXs8ECBA4KyAwOSu4/vUCk/Vr2OUOBCZdmHVCgACBJgICkyasVY0KTKqYnESAAIEpBQQmU5al66AEJl251+1MYLJu7YycAAECApNxc0BgMs5ezwRKHZl2AAAgAElEQVQIEDgrIDA5K7j+9QKT9WvY5Q4EJl2YdUKAAIEmAgKTJqxVjQpMqpicRIAAgSkFBCZTlqXroAQmXbnX7Uxgsm7tjJwAAQICk3FzQGAyzl7PBAgQOCsgMDkruP71ApP1a9jlDgQmXZh1QoAAgSYCApMmrFWNCkyqmJxEgACBKQUEJlOWpeugBCZdudftTGCybu2MnAABAgKTcXNAYDLOXs8ECBA4KyAwOSu4/vUCk/Vr2OUOBCZdmHVCgACBJgICkyasVY0KTKqYnESAAIEpBQQmU5al66AEJl251+1MYLJu7YycAAECApNxc0BgMs5ezwQIEDgrIDA5K7j+9QKT9WvY5Q4EJl2YdUKAAIEmAgKTJqxVjQpMqpicRIAAgSkFBCZTlqXroAQmXbnX7Uxgsm7tjJwAAQICk3FzQGAyzl7PBAgQOCsgMDkruP71ApP1a9jlDgQmXZh1QoAAgSYCApMmrFWNCkyqmJxEgACBKQUEJlOWpeugBCZdudftTGCybu2MnAABAgKTcXNAYDLOXs8ECBA4KyAwOSu4/vUCk/Vr2OUOBCZdmHVCgACBJgICkyasVY0KTKqYnESAAIEpBQQmU5al66AEJl251+1MYLJu7YycAAECApNxc0BgMs5ezwQIEDgrIDA5K7j+9QKT9WvY5Q4EJl2YdUKAAIEmAgKTJqxVjQpMqpicRIAAgSkFBCZTlqXroAQmXbnX7Uxgsm7tjJwAAQICk3FzQGAyzl7PBAgQOCsgMDkruP71ApP1a9jlDgQmXZh1QoAAgSYCApMmrFWNCkyqmJxEgACBKQUEJlOWpeugBCZdudftTGCybu2MnAABAgKTcXNAYDLOXs8ECBA4KyAwOSu4/vUCk/Vr2OUOBCZdmHVCgACBJgICkyasVY0KTKqYnESAAIEpBQQmU5al66AEJl251+1MYLJu7YycAAECApNxc0BgMs5ezwQIEDgrIDA5K7j+9QKT9WvY5Q4EJl2YdUKAAIEmAgKTJqxVjQpMqpicRIAAgSkFBCZTlqXroAQmXbnX7Uxgsm7tjJwAAQICk3FzQGAyzl7PBAgQOCsgMDkruP71ApP1a9jlDgQmXZh1QoAAgSYCApMmrFWNCkyqmJxEgACBKQUEJlOWpeugBCZdudftTGCybu2MnAABAgKTcXNAYDLOXs8ECBA4KyAwOSu4/vUCk/Vr2OUOBCZdmHVCgACBJgICkyasVY0KTKqYnESAAIEpBQQmU5al66AEJl251+1MYLJu7YycAAECApNxc0BgMs5ezwQIEDgrIDA5K7j+9QKT9WvY5Q4EJl2YdUKAAIEmAgKTJqxVjQpMqpicRIAAgSkFBCZTlqXroAQmXbnX7Uxgsm7tjJwAAQICk3FzQGAyzl7PBAgQOCsgMDkruP71ApP1a9jlDgQmXZh1QoAAgSYCApMmrFWNCkyqmJxEgACBKQUEJlOWpeugBCZdudftTGCybu2MnAABAgKTcXNAYDLOXs8ECBA4KyAwOSu4/vUCk/Vr2OUOBCZdmHVCgACBJgICkyasVY0KTKqYnESAAIEpBQQmU5al66AEJl251+1MYLJu7YycAAECApNxc0BgMs5ezwQIEDgrIDA5K7j+9QKT9WvY5Q4EJl2YdUKAAIEmAgKTJqxVjQpMqpicRIAAgSkFBCZTlqXroAQmXbnX7Uxgsm7tjJwAAQICk3FzQGAyzl7PBAgQOCsgMDkruP71ApP1a9jlDgQmXZh1QoAAgSYCApMmrFWNCkyqmJxEgACBKQUEJlOWpeugBCZdudftTGCybu2MnAABAgKTcXNAYDLOXs8ECBA4KyAwOSu4/vUCk/Vr2OUOBCZdmHVCgACBJgICkyasVY0KTKqYnESAAIEpBQQmU5al66AEJl251+1MYLJu7YycAAECApNxc0BgMs5ezwQIEDgrIDA5K7j+9QKT9WvY5Q4EJl2YdUKAAIEmAgKTJqxVjQpMqpicRIAAgSkFBCZTlqXroAQmXbnX7Uxgsm7tjJwAAQICk3FzQGAyzl7PBAgQOCsgMDkruP71ApP1a9jlDgQmXZh1QoAAgSYCApMmrFWNCkyqmJxEgACBKQUEJlOWpeugBCZdudftTGCybu2MnAABAgKTcXNAYDLOXs8ECBA4KyAwOSu4/vUCk/Vr2OUOBCZdmHVCgACBJgICkyasVY0KTKqYnESAAIEpBQQmU5al66AEJl251+1MYLJu7YycAAECApNxc0BgMs5ezwQIEDgrIDA5K7j+9QKT9WvY5Q4EJl2YdUKAAIEmAgKTJqxVjQpMqpicRIAAgSkFBCZTlqXroAQmXbnX7Uxgsm7tjJwAAQICk3FzQGAyzl7PBAgQOCsgMDkruP71ApP1a9jlDgQmXZh1QoAAgSYCApMmrFWNCkyqmJxEgACBKQUEJlOWpeugBCZdudftTGCybu2MnAABAgKTcXNAYDLOXs8ECBA4KyAwOSu4/vUCk/Vr2OUOBCZdmHVCgACBJgICkyasVY0KTKqYnESAAIEpBQQmU5al66AEJl251+1MYLJu7YycAAECApNxc0BgMs5ezwQIEDgrIDA5K7j+9QKT9WvY5Q4EJl2YdUKAAIEmAgKTJqxVjQpMqpicRIAAgSkFBCZTlqXroAQmXbnX7Uxgsm7tjJwAAQICk3FzQGAyzn5kzx/2YR/2/3T/nu/5niOHpG8CBF5DQGDyGmjP7BKByTMraKvbEZi0ktUuAQIE2gsITNobP9aDwGSc/cieBSYj9fVN4DoBgcl1lqu2JDBZtXKdxy0w6QyuOwIECFwoIDC5EPPOpgQmd4I9k9MFJs+kkG7jjRcQmLzxU6AITMyBKgGBSRWTkwgQIDClgMBkXFkEJuPsR/YsMBmpr28C1wkITK6zXLUlgcmqles8boFJZ3DdESBA4EIBgcmFmHc2JTC5E+yZnC4weSaFdBtvvIDA5I2fAnaYmAJ1AgKTOidnESBAYEYBgcm4qghMxtmP7FlgMlJf3wSuExCYXGe5akt2mKxauc7jFph0BtcdAQIELhQQmFyIeWdTApM7wZ7J6QKTZ1JIt/HGCwhM3vgpYIeJKVAnIDCpc3IWAQIEZhQQmIyrisBknP3IngUmI/X1TeA6AYHJdZartmSHyaqV6zxugUlncN0RIEDgQgGByYWYdzYlMLkT7JmcLjB5JoV0G2+8gMDkjZ8CdpiYAnUCApM6J2cRIEBgRgGBybiqCEzG2Y/sWWAyUl/fBK4TEJhcZ7lqS3aYrFq5zuMWmHQG1x0BAgQuFBCYXIh5Z1MCkzvBnE6AAIGJBAQmExVj0FAEJoPgV+tWYLJaxYyXAAEC/1dAYDJuNghMxtnrmQABAmcFBCZnBde/XmCyfg273IHApAuzTggQINBEQGDShLWqUYFJFZOTCBAgMKWAwGTKsnQdlMCkK/e6nQlM1q2dkRMgQEBgMm4OCEzG2euZAAECZwUEJmcF179eYLJ+DbvcgcCkC7NOCBAg0ERAYNKEtapRgUkVk5MIECAwpYDAZMqydB2UwKQr97qdCUzWrZ2REyBAQGAybg4ITMbZ65kAAQJnBQQmZwXXv15gsn4Nu9yBwKQLs04IECDQREBg0oS1qlGBSRWTkwgQIDClgMBkyrJ0HZTApCv3up0JTNatnZETIEBAYDJuDghMxtnrmQABAmcFBCZnBde/XmCyfg273IHApAuzTggQINBEQGDShLWqUYFJFZOTCBAgMKWAwGTKsnQdlMCkK/e6nQlM1q2dkRMgQEBgMm4OCEzG2euZAAECZwUEJmcF179eYLJ+DbvcgcCkC7NOCBAg0ERAYNKEtapRgUkVk5MIECAwpYDAZMqydB2UwKQr97qdCUzWrZ2REyBAQGAybg4ITMbZ65kAAQJnBQQmZwXXv15gsn4Nu9yBwKQLs04IECDQREBg0oS1qlGBSRWTkwgQIDClgMBkyrJ0HZTApCv3up0JTNat3ZmRf8j7vu//c/l7f9AHnWnStQQIDBAQmAxA37oUmIyz1zMBAgTOCghMzgquf73AZP0adrkDgUkX5uk6EZhMVxIDIvBaAgKT12K75CKBySWMGiFAgMAQAYHJEPapOhWYTFWOeQcjMJm3Ni1HJjBpqattAv0EBCb9rG97EpiMs9czAQIEzgoITM4Krn+9wGT9Gna5A4FJF+bpOhGYTFcSAyLwWgICk9diu+QigckljBohQIDAEAGByRD2qToVmExVjnkHIzCZtzYtRyYwaamrbQL9BAQm/axvexKYjLPXMwECBM4KCEzOCq5/vcBk/Rp2uQOBSRfm6ToRmExXEgMi8FoCApPXYrvkIoHJJYwaIUCAwBABgckQ9qk6FZhMVY55ByMwmbc2LUcmMGmpq20C/QQEJv2sb3sSmIyz1zMBAgTOCghMzgquf73AZP0adrkDgUkX5uk6EZhMVxIDIvBaAgKT12K75CKBySWMGiFAgMAQAYHJEPapOhWYTFWOeQcjMJm3Ni1HJjBpqattAv0EBCb9rG97EpiMs9czAQIEzgoITM4Krn+9wGT9Gna5A4FJF2adECBAoImAwKQJa1WjApMqJicRIEBgSgGByZRl6ToogUlX7nU7E5isWzsjJ0CAgMBk3BwQmIyz1zMBAgTOCghMzgquf73AZP0adrkDgUkXZp0QIECgiYDApAlrVaMCkyomJxEgQGBKAYHJlGXpOiiBSVfudTsTmKxbOyMnQICAwGTcHBCYjLPXMwECBM4KCEzOCq5/vcBk/Rp2uQOBSRdmnRAgQKCJgMCkCWtVowKTKiYnESBAYEoBgcmUZek6KIFJV+51OxOYrFs7IydAgIDAZNwcEJiMs9czAQIEzgoITM4Krn+9wGT9Gna5A4FJF2adECBAoImAwKQJa1WjApMqJicRIEBgSgGByZRl6ToogUlX7nU7E5isWzsjJ0CAgMBk3BwQmIyz1zMBAgTOCghMzgquf73AZP0adrkDgUkXZp0QIECgiYDApAlrVaMCkyomJxEgQGBKAYHJlGXpOiiBSVfudTsTmKxbOyMnQICAwGTcHBCYjLPXMwECBM4KCEzOCq5/vcBk/Rp2uQOBSRdmnRAgQKCJgMCkCWtVowKTKiYnESBAYEoBgcmUZek6KIFJV+51OxOYrFs7IydAgIDAZNwcEJiMs9czAQIEzgoITM4Krn+9wGT9Gna5A4FJF2adECBAoImAwKQJa1WjApMqJicRIEBgSgGByZRl6ToogUlX7nU7E5isWzsjJ0CAgMBk3BwQmIyz1zMBAgTOCghMzgquf73AZP0adrkDgUkXZp0QIECgiYDApAlrVaMCkyomJxEgQGBKAYHJlGXpOiiBSVfudTsTmKxbOyMnQICAwGTcHBCYjLPXMwECBM4KCEzOCq5/vcBk/Rp2uQOBSRdmnRAgQKCJgMCkCWtVowKTKiYnESBAYEoBgcmUZek6KIFJV+51OxOYrFs7IydAgIDAZNwcEJiMs9czAQIEzgoITM4Krn+9wGT9Gna5A4FJF2adECBAoImAwKQJa1WjApMqJicRIEBgSgGByZRl6ToogUlX7nU7E5isWzsjJ0CAgMBk3BwQmIyz1zMBAgTOCghMzgquf73AZP0adrkDgUkXZp0QIECgiYDApAlrVaMCkyomJxEgQGBKAYHJlGXpOiiBSVfudTsTmKxbOyMnQICAwGTcHBCYjLPXMwECBM4KCEzOCq5/vcBk/Rp2uQOBSRdmnRAgQKCJgMCkCWtVowKTKiYnESBAYEoBgcmUZek6KIFJV+51OxOYrFs7IydAgIDAZNwcEJiMs9czAQIEzgoITM4Krn+9wGT9Gna5A4FJF2adECBAoImAwKQJa1WjApMqJicRIEBgSgGByZRl6ToogUlX7nU7E5isWzsjJ0CAgMBk3BwQmIyz1zMBAgTOCghMzgquf73AZP0adrkDgUkXZp0QIECgiYDApAlrVaMCkyomJxEgQGBKAYHJlGXpOiiBSVfudTsTmKxbOyMnQICAwGTcHBCYjLPXMwECBM4KCEzOCq5/vcBk/Rp2uQOBSRdmnRAgQKCJgMCkCWtVowKTKiYnESBAYEoBgcmUZek6KIFJV+51OxOYrFs7IydAgIDAZNwcEJiMs9czAQIEzgoITM4Krn+9wGT9Gna5A4FJF2adECBAoImAwKQJa1WjApMqJicRIEBgSgGByZRl6ToogUlX7nU7E5isWzsjJ0CAgMBk3BwQmIyz1zMBAgTOCghMzgquf73AZP0adrkDgUkXZp0QIECgiYDApAlrVaMCkyomJxEgQGBKAYHJlGXpOiiBSVfudTsTmKxbOyMnQICAwGTcHBCYjLPXMwECBM4KCEzOCq5/vcBk/Rp2uQOBSRdmnRAgQKCJgMCkCWtVowKTKiYnESBAYEoBgcmUZek6KIFJV+51OxOYrFs7IydAgIDAZNwcEJiMs9czAQIEzgoITM4Krn+9wGT9Gna5A4FJF2adECBAoImAwKQJa1WjApMqJicRIEBgSgGByZRl6ToogUlX7nU7E5isWzsjJ0CAgMBk3BwQmIyz1zMBAgTOCghMzgquf73AZP0adrkDgUkXZp0QIECgiYDApAlrVaMCkyomJxEgQGBKAYHJlGXpOiiBSVfudTsTmKxbOyMnQICAwGTcHBCYjLPXMwECBM4KCEzOCq5/vcBk/Rp2uQOBSRdmnRAgQKCJgMCkCWtVowKTKiYnESBAYEoBgcmUZek6KIFJV+51OxOYrFs7IydAgIDAZNwcEJiMs9czAQIEzgoITM4Krn+9wGT9Gna5A4FJF2adECBAoImAwKQJa1WjApMqJicRIEBgSgGByZRl6ToogUlX7nU7E5isWzsjJ0CAgMBk3BwQmIyz1zMBAgTOCghMzgquf73AZP0adrkDgUkXZp0QIECgiYDApAlrVaMCkyomJxEgQGBKAYHJlGXpOiiBSVfudTsTmKxbOyMnQICAwGTcHBCYjLPXMwECBM4KCEzOCq5/vcBk/Rp2uQOBSRdmnRAgQKCJgMCkCWtVowKTKiYnESBAYEoBgcmUZek6KIFJV+51OxOYrFs7IydAgIDAZNwcEJiMs9czAQIEzgoITM4Krn+9wGT9Gna5A4FJF2adECBAoImAwKQJa1WjApMqJicRIEBgSgGByZRl6ToogUlX7nU7E5isWzsjJ0CAgMBk3BwQmIyz1zMBAgTOCghMzgquf73AZP0adrkDgUkXZp0QIECgiYDApAlrVaMCkyomJxEgQGBKAYHJlGXpOiiBSVfudTsTmKxbOyMnQICAwGTcHBCYjLPXMwECBM4KCEzOCq5/vcBk/Rp2uQOBSRdmnRAgQKCJgMCkCWtVowKTKiYnESBAYEoBgcmUZek6KIFJV+51OxOYrFs7IydAgIDAZNwcEJiMs9czAQIEzgoITM4Krn+9wGT9Gna5A4FJF2adECBAoImAwKQJa1WjApMqJicRIEBgSgGByZRl6ToogUlX7nU7E5isWzsjJ0CAgMBk3BwQmIyz1zMBAgTOCghMzgquf73AZP0adrkDgUkXZp0QIECgiYDApAlrVaMCkyomJxEgQGBKAYHJlGXpOiiBSVfudTsTmKxbOyMnQICAwGTcHPj/27lDZD12IwrAKzK56OGgt44sJCALyTqMgoNMvCCnBhi4civVz9MjTet8XFea/o6A/lMuK0z22TuZAAECdwUUJncF5/+9wmR+hksmUJgsYXYIAQIEHhFQmDzCWtpUYVJisogAAQKvFFCYvDKWpR+lMFnKPfcwhcnc7Hw5AQIEFCb77oDCZJ+9kwkQIHBXQGFyV3D+3ytM5me4ZAKFyRJmhxAgQOARAYXJI6ylTRUmJSaLCBAg8EoBhckrY1n6UQqTpdxzD1OYzM3OlxMgQEBhsu8OKEz22TuZAAECdwUUJncF5/+9wmR+hksmUJgsYXYIAQIEHhFQmDzCWtpUYVJisogAAQKvFFCYvDKWpR+lMFnKPfcwhcnc7Hw5AQIEFCb77oDCZJ+9kwkQIHBXQGFyV3D+3ytM5me4ZAKFyRJmhxAgQOARAYXJI6ylTRUmJSaLCBAg8EoBhckrY1n6UQqTpdxzD1OYzM3OlxMgQEBhsu8OKEz22TuZAAECdwUUJncF5/+9wmR+hksmUJgsYXYIAQIEHhFQmDzCWtpUYVJisogAAQKvFFCYvDKWpR+lMFnKPfcwhcnc7Hw5AQIEFCb77oDCZJ+9kwkQIHBXQGFyV3D+3ytM5me4ZAKFyRJmhxAgQOARAYXJI6ylTRUmJSaLCBAg8EoBhckrY1n6UQqTpdxzD1OYzM3OlxMgQEBhsu8OKEz22TuZAAECdwUUJncF5/+9wmR+hksmUJgsYXYIAQIEHhFQmDzCWtr0+/fvP0oLC4u+fPlSWGUJAQLdAt++fevesm2/j4+Ptr1s9L8Cndl/fHz47T3wkgltYGg7PllhskPdmQQIEOgRUJj0OP7OLgqT31HzNwTeJdD5o7l7MoVJt+iv+3VmrzB5NqundleYPCV72L4Kk8MCNQ4BAlECCpN9cStM9tk7mUCXQOeP5q5v+rmPwqRbVGHyrOi83RUm8zLb8sUKky3sDiVAgECLgMKkhfG3NlGY/BabPyLwKgGFyaviWPoxndn7FyZLo2s7TGHSRnn2RgqTs/M1HQECZwsoTPblqzDZZ+9kAl0CnT+au77p5z7+hUm36K/7dWavMHk2q6d2V5g8JXvYvgqTwwI1DgECUQIKk31xK0z22TuZQJdA54/mrm9SmHRLfr5fZ/YKkzWZdZ+iMOkWPXQ/hcmhwRqLAIEIAYXJvpgVJvvsnUygS6DzR3PXNylMuiUVJmtE552iMJmX2ZYvVphsYXcoAQIEWgQUJi2Mv7WJwuS32PwRgVcJKExeFcfSj+nM3r8wWRpd22EKkzbKszdSmJydr+kIEDhbQGGyL1+FyT57JxPoEuj80dz1TT/38X+YdIv+ul9n9gqTZ7N6aneFyVOyh+2rMDksUOMQIBAloDDZF7fCZJ+9kwl0CXT+aO76JoVJt+Tn+3VmrzBZk1n3KQqTbtFD91OYHBqssQgQiBBQmETEbEgCBAgQIECgWUBh0gx66nYKk1OTNRcBAgkCCpOElM1IgAABAgQIdAsoTLpFD91PYXJosMYiQCBCQGESEbMhCRAgQIAAgWYBhUkz6KnbKUxOTdZcBAgkCChMElI2IwECBAgQINAtoDDpFj10P4XJocEaiwCBCAGFSUTMhiRAgAABAgSaBRQmzaCnbqcwOTVZcxEgkCCgMElI2YwECBAgQIBAt4DCpFv00P0UJocGaywCBCIEFCYRMRuSAAECBAgQaBZQmDSDnrqdwuTUZM1FgECCgMIkIWUzEiBAgAABAt0CCpNu0UP3U5gcGqyxCBCIEFCYRMRsSAIECBAgQKBZQGHSDHrqdgqTU5M1FwECCQIKk4SUzUiAAAECBAh0CyhMukUP3e9v//rnj67R/v33f3RtZR8CBAoC//nzz8KqPUv++Pp1z8Fhp3begT++fvV2CLs/xiVAgAABAqkCHj2pyf/FuRUmfxHMcgIvEuj8sdw9lsKkW/Tz/TrvgMJkTWZOIUCAAAECBPYLKEz2ZzDiCxQmI2LykQQ+Fej8sdxNrDDpFlWYrBF1CgECBAgQIJAgoDBJSLlhRoVJA6ItCGwSUJhsgn/RsZ13wL8weVGwPoUAAQIECBB4VEBh8ijvOZsrTM7J0iR5Ap0/lrv1/AuTbtHP9+u8AwqTNZk5hQABAgQIENgvoDDZn8GIL1CYjIjJRxL4VKDzx3I3scKkW1RhskbUKQQIECBAgECCgMIkIeWGGRUmDYi2ILBJQGGyCf5Fx3beAf/C5EXB+hQCBAgQIEDgUQGFyaO852yuMDknS5PkCXT+WO7W8y9MukU/36/zDihM1mTmFAIECBAgQGC/gMJkfwYjvkBhMiImH0ngU4HOH8vdxAqTblGFyRpRpxAgQIAAAQIJAgqThJQbZlSYNCDagsAmAYXJJvgXHdt5B/wLkxcF61MIECBAgACBRwUUJo/ynrO5wuScLE1CgECegMIkL3MTEyBAgAABAvcFFCb3DSN2UJhExGxIAgQOFVCYHBqssQgQIECAAIFHBRQmj/Kes7nC5JwsTUKAQJ6AwiQvcxMTIECAAAEC9wUUJvcNI3ZQmETEbEgCBA4VUJgcGqyxCBAgQIAAgUcFFCaP8p6zucLknCxNQoBAnoDCJC9zExMgQIAAAQL3BRQm9w0jdlCYRMRsSAIECFQEvB0qStYQIECAAAEC4wU8esZHuGYAhckaZ6cQIEBggIC3w4CQfCIBAgQIECBwX8Cj575hyg4/UgY1JwECBAj8XwFvBxeEAAECBAgQiBDw6ImIuWVIhUkLo00IECAwXsDbYXyEBiBAgAABAgQqAh49FSVrLgGFiXtAgAABApeAt4N7QIAAAQIECEQIePRExNwypMKkhdEmBAgQGC/g7TA+QgMQIECAAAECFQGPnoqSNZeAwsQ9IECAAIFLwNvBPSBAgAABAgQiBDx6ImJuGVJh0sJoEwIECIwX8HYYH6EBCBAgQIAAgYqAR09FyZpLQGHiHhAgQIDAJeDt4B4QIECAAAECEQIePRExtwypMGlhtAkBAgTGC3g7jI/QAAQIECBAgEBFwKOnomTNJaAwcQ8IECBA4BLwdnAPCBAgQIAAgQgBj56ImFuGVJi0MNqEAAEC4wW8HcZHaAACBAgQIECgIuDRU1Gy5hJQmLgHBAgQIHAJeDu4BwQIECBAgECEgEdPRMwtQypMWhhtQoAAgfEC3g7jIzQAAQIECBAgUBHw6KkoWXMJKEzcAwIECBC4BLwd3AMCBAgQIEAgQsCjJyLmliEVJi2MNiFAgMB4AW+H8REagAABAgQIEKgIePRUlKy5BBQm7gEBAgQIXALeDu4BAQIECBAgECHg0RMRc8uQCpMWRuQm4GcAAAp9SURBVJsQIEBgvIC3w/gIDUCAAAECBAhUBDx6KkrWXAIKE/eAAAECBC4Bbwf3gAABAgQIEIgQ8OiJiLllSIVJC6NNCBAgMF7A22F8hAYgQIAAAQIEKgIePRUlay4BhYl7QIAAAQKXgLeDe0CAAAECBAhECHj0RMTcMqTCpIXRJgQIEBgv4O0wPkIDECBAgAABAhUBj56KkjWXgMLEPSBAgACBS8DbwT0gQIAAAQIEIgQ8eiJibhlSYdLCaBMCBAiMF/B2GB+hAQgQIECAAIGKgEdPRcmaS0Bh4h4QIECAwCXg7eAeECBAgAABAhECHj0RMbcMqTBpYbQJAQIExgt4O4yP0AAECBAgQIBARcCjp6JkzSWgMHEPCBAgQOAS8HZwDwgQIECAAIEIAY+eiJhbhlSYtDDahAABAuMFvB3GR2gAAgQIECBAoCLg0VNRsuYSUJi4BwQIECBwCXg7uAcECBAgQIBAhIBHT0TMLUMqTFoYbUKAAIHxAt4O4yM0AAECBAgQIFAR8OipKFlzCShM3AMCBAgQuAS8HdwDAgQIECBAIELAoyci5pYhFSYtjDYhQIDAeAFvh/ERGoAAAQIECBCoCHj0VJSsuQQUJu4BAQIECFwC3g7uAQECBAgQIBAh4NETEXPLkAqTFkabECBAYLyAt8P4CA1AgAABAgQIVAQ8eipK1lwCChP3gAABAgQuAW8H94AAAQIECBCIEPDoiYi5ZUiFSQujTQgQIDBewNthfIQGIECAAAECBCoCHj0VJWsIECBAgAABAgQIECBAgACBKAGFSVTchiVAgAABAgQIECBAgAABAgQqAgqTipI1BAgQIECAAAECBAgQIECAQJSAwiQqbsMSIECAAAECBAgQIECAAAECFQGFSUXJGgIECBAgQIAAAQIECBAgQCBKQGESFbdhCRAgQIAAAQIECBAgQIAAgYqAwqSiZA0BAgQIECBAgAABAgQIECAQJaAwiYrbsAQIECBAgAABAgQIECBAgEBFQGFSUbKGAAECBAgQIECAAAECBAgQiBJQmETFbVgCBAgQIECAAAECBAgQIECgIqAwqShZQ4AAAQIECBAgQIAAAQIECEQJKEyi4jYsAQIECBAgQIAAAQIECBAgUBFQmFSUrCFAgAABAgQIECBAgAABAgSiBBQmUXEblgABAgQIECBAgAABAgQIEKgIKEwqStYQIECAAAECBAgQIECAAAECUQIKk6i4DUuAAAECBAgQIECAAAECBAhUBBQmFSVrCBAgQIAAAQIECBAgQIAAgSgBhUlU3IYlQIAAAQIECBAgQIAAAQIEKgIKk4qSNQQIECBAgAABAgQIECBAgECUgMIkKm7DEiBAgAABAgQIECBAgAABAhUBhUlFyRoCBAgQIECAAAECBAgQIEAgSkBhEhW3YQkQIECAAAECBAgQIECAAIGKgMKkomQNAQIECBAgQIAAAQIECBAgECWgMImK27AECBAgQIAAAQIECBAgQIBARUBhUlGyhgABAgQIECBAgAABAgQIEIgSUJhExW1YAgQIECBAgAABAgQIECBAoCKgMKkoWUOAAAECBAgQIECAAAECBAhECShMouI2LAECBAgQIECAAAECBAgQIFARUJhUlKwhQIAAAQIECBAgQIAAAQIEogQUJlFxG5YAAQIECBAgQIAAAQIECBCoCChMKkrWECBAgAABAgQIECBAgAABAlECCpOouA1LgAABAgQIECBAgAABAgQIVAQUJhUlawgQIECAAAECBAgQIECAAIEoAYVJVNyGJUCAAAECBAgQIECAAAECBCoCCpOKkjUECBAgQIAAAQIECBAgQIBAlIDCJCpuwxIgQIAAAQIECBAgQIAAAQIVAYVJRckaAgQIECBAgAABAgQIECBAIEpAYRIVt2EJECBAgAABAgQIECBAgACBioDCpKJkDQECBAgQIECAAAECBAgQIBAloDCJituwBAgQIECAAAECBAgQIECAQEVAYVJRsoYAAQIECBAgQIAAAQIECBCIElCYRMVtWAIECBAgQIAAAQIECBAgQKAioDCpKFlDgAABAgQIECBAgAABAgQIRAkoTKLiNiwBAgQIECBAgAABAgQIECBQEVCYVJSsIUCAAAECBAgQIECAAAECBKIEFCZRcRuWAAECBAgQIECAAAECBAgQqAgoTCpK1hAgQIAAAQIECBAgQIAAAQJRAgqTqLgNS4AAAQIECBAgQIAAAQIECFQEFCYVJWsIECBAgAABAgQIECBAgACBKAGFSVTchiVAgAABAgQIECBAgAABAgQqAgqTipI1BAgQIECAAAECBAgQIECAQJSAwiQqbsMSIECAAAECBAgQIECAAAECFQGFSUXJGgIECBAgQIAAAQIECBAgQCBKQGESFbdhCRAgQIAAAQIECBAgQIAAgYqAwqSiZA0BAgQIECBAgAABAgQIECAQJaAwiYrbsAQIECBAgAABAgQIECBAgEBFQGFSUbKGAAECBAgQIECAAAECBAgQiBJQmETFbVgCBAgQIECAAAECBAgQIECgIqAwqShZQ4AAAQIECBAgQIAAAQIECEQJKEyi4jYsAQIECBAgQIAAAQIECBAgUBFQmFSUrCFAgAABAgQIECBAgAABAgSiBBQmUXEblgABAgQIECBAgAABAgQIEKgIKEwqStYQIECAAAECBAgQIECAAAECUQIKk6i4DUuAAAECBAgQIECAAAECBAhUBBQmFSVrCBAgQIAAAQIECBAgQIAAgSgBhUlU3IYlQIAAAQIECBAgQIAAAQIEKgIKk4qSNQQIECBAgAABAgQIECBAgECUgMIkKm7DEiBAgAABAgQIECBAgAABAhUBhUlFyRoCBAgQIECAAAECBAgQIEAgSkBhEhW3YQkQIECAAAECBAgQIECAAIGKgMKkomQNAQIECBAgQIAAAQIECBAgECWgMImK27AECBAgQIAAAQIECBAgQIBARUBhUlGyhgABAgQIECBAgAABAgQIEIgSUJhExW1YAgQIECBAgAABAgQIECBAoCKgMKkoWUOAAAECBAgQIECAAAECBAhECShMouI2LAECBAgQIECAAAECBAgQIFARUJhUlKwhQIAAAQIECBAgQIAAAQIEogQUJlFxG5YAAQIECBAgQIAAAQIECBCoCChMKkrWECBAgAABAgQIECBAgAABAlECCpOouA1LgAABAgQIECBAgAABAgQIVAQUJhUlawgQIECAAAECBAgQIECAAIEoAYVJVNyGJUCAAAECBAgQIECAAAECBCoCCpOKkjUECBAgQIAAAQIECBAgQIBAlIDCJCpuwxIgQIAAAQIECBAgQIAAAQIVAYVJRckaAgQIECBAgAABAgQIECBAIEpAYRIVt2EJECBAgAABAgQIECBAgACBioDCpKJkDQECBAgQIECAAAECBAgQIBAloDCJituwBAgQIECAAAECBAgQIECAQEVAYVJRsoYAAQIECBAgQIAAAQIECBCIElCYRMVtWAIECBAgQIAAAQIECBAgQKAioDCpKFlDgAABAgQIECBAgAABAgQIRAkoTKLiNiwBAgQIECBAgAABAgQIECBQEfgv/vSuu5c4YqkAAAAASUVORK5CYII=\" width=\"1100\">"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(figsize=(11, 7))\n",
    "plt.subplot(121)\n",
    "plt.title(\"Original observation (160×210 RGB)\")\n",
    "plt.imshow(obs)\n",
    "plt.axis(\"off\")\n",
    "plt.subplot(122)\n",
    "plt.title(\"Preprocessed observation (80×80 grayscale)\")\n",
    "plt.imshow(img, interpolation=\"nearest\", cmap=\"gray\")\n",
    "plt.axis(\"off\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you can see, a single image does not give you the direction and speed of the ball, which are crucial informations for playing this game. For this reason, it is best to actually combine several consecutive observations to create the environment's state representation. One way to do that is to create a multi-channel image, with one channel per recent observation. Another is to merge all recent observations into a single-channel image, using `np.max()`. In this case, we need to dim the older images so that the DQN can distinguish the past from the present."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {},
   "outputs": [],
   "source": [
    "def combine_observations_multichannel(preprocessed_observations):\n",
    "    return np.array(preprocessed_observations).transpose([1, 2, 0])\n",
    "\n",
    "def combine_observations_singlechannel(preprocessed_observations, dim_factor=0.5):\n",
    "    dimmed_observations = [obs * dim_factor**index\n",
    "                           for index, obs in enumerate(reversed(preprocessed_observations))]\n",
    "    return np.max(np.array(dimmed_observations), axis=0)\n",
    "\n",
    "n_observations_per_state = 3\n",
    "preprocessed_observations = deque([], maxlen=n_observations_per_state)\n",
    "\n",
    "obs = env.reset()\n",
    "for step in range(10):\n",
    "    obs, _, _, _ = env.step(1)\n",
    "    preprocessed_observations.append(preprocess_observation(obs))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/javascript": [
       "/* Put everything inside the global mpl namespace */\n",
       "window.mpl = {};\n",
       "\n",
       "\n",
       "mpl.get_websocket_type = function() {\n",
       "    if (typeof(WebSocket) !== 'undefined') {\n",
       "        return WebSocket;\n",
       "    } else if (typeof(MozWebSocket) !== 'undefined') {\n",
       "        return MozWebSocket;\n",
       "    } else {\n",
       "        alert('Your browser does not have WebSocket support.' +\n",
       "              'Please try Chrome, Safari or Firefox ≥ 6. ' +\n",
       "              'Firefox 4 and 5 are also supported but you ' +\n",
       "              'have to enable WebSockets in about:config.');\n",
       "    };\n",
       "}\n",
       "\n",
       "mpl.figure = function(figure_id, websocket, ondownload, parent_element) {\n",
       "    this.id = figure_id;\n",
       "\n",
       "    this.ws = websocket;\n",
       "\n",
       "    this.supports_binary = (this.ws.binaryType != undefined);\n",
       "\n",
       "    if (!this.supports_binary) {\n",
       "        var warnings = document.getElementById(\"mpl-warnings\");\n",
       "        if (warnings) {\n",
       "            warnings.style.display = 'block';\n",
       "            warnings.textContent = (\n",
       "                \"This browser does not support binary websocket messages. \" +\n",
       "                    \"Performance may be slow.\");\n",
       "        }\n",
       "    }\n",
       "\n",
       "    this.imageObj = new Image();\n",
       "\n",
       "    this.context = undefined;\n",
       "    this.message = undefined;\n",
       "    this.canvas = undefined;\n",
       "    this.rubberband_canvas = undefined;\n",
       "    this.rubberband_context = undefined;\n",
       "    this.format_dropdown = undefined;\n",
       "\n",
       "    this.image_mode = 'full';\n",
       "\n",
       "    this.root = $('<div/>');\n",
       "    this._root_extra_style(this.root)\n",
       "    this.root.attr('style', 'display: inline-block');\n",
       "\n",
       "    $(parent_element).append(this.root);\n",
       "\n",
       "    this._init_header(this);\n",
       "    this._init_canvas(this);\n",
       "    this._init_toolbar(this);\n",
       "\n",
       "    var fig = this;\n",
       "\n",
       "    this.waiting = false;\n",
       "\n",
       "    this.ws.onopen =  function () {\n",
       "            fig.send_message(\"supports_binary\", {value: fig.supports_binary});\n",
       "            fig.send_message(\"send_image_mode\", {});\n",
       "            if (mpl.ratio != 1) {\n",
       "                fig.send_message(\"set_dpi_ratio\", {'dpi_ratio': mpl.ratio});\n",
       "            }\n",
       "            fig.send_message(\"refresh\", {});\n",
       "        }\n",
       "\n",
       "    this.imageObj.onload = function() {\n",
       "            if (fig.image_mode == 'full') {\n",
       "                // Full images could contain transparency (where diff images\n",
       "                // almost always do), so we need to clear the canvas so that\n",
       "                // there is no ghosting.\n",
       "                fig.context.clearRect(0, 0, fig.canvas.width, fig.canvas.height);\n",
       "            }\n",
       "            fig.context.drawImage(fig.imageObj, 0, 0);\n",
       "        };\n",
       "\n",
       "    this.imageObj.onunload = function() {\n",
       "        this.ws.close();\n",
       "    }\n",
       "\n",
       "    this.ws.onmessage = this._make_on_message_function(this);\n",
       "\n",
       "    this.ondownload = ondownload;\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._init_header = function() {\n",
       "    var titlebar = $(\n",
       "        '<div class=\"ui-dialog-titlebar ui-widget-header ui-corner-all ' +\n",
       "        'ui-helper-clearfix\"/>');\n",
       "    var titletext = $(\n",
       "        '<div class=\"ui-dialog-title\" style=\"width: 100%; ' +\n",
       "        'text-align: center; padding: 3px;\"/>');\n",
       "    titlebar.append(titletext)\n",
       "    this.root.append(titlebar);\n",
       "    this.header = titletext[0];\n",
       "}\n",
       "\n",
       "\n",
       "\n",
       "mpl.figure.prototype._canvas_extra_style = function(canvas_div) {\n",
       "\n",
       "}\n",
       "\n",
       "\n",
       "mpl.figure.prototype._root_extra_style = function(canvas_div) {\n",
       "\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._init_canvas = function() {\n",
       "    var fig = this;\n",
       "\n",
       "    var canvas_div = $('<div/>');\n",
       "\n",
       "    canvas_div.attr('style', 'position: relative; clear: both; outline: 0');\n",
       "\n",
       "    function canvas_keyboard_event(event) {\n",
       "        return fig.key_event(event, event['data']);\n",
       "    }\n",
       "\n",
       "    canvas_div.keydown('key_press', canvas_keyboard_event);\n",
       "    canvas_div.keyup('key_release', canvas_keyboard_event);\n",
       "    this.canvas_div = canvas_div\n",
       "    this._canvas_extra_style(canvas_div)\n",
       "    this.root.append(canvas_div);\n",
       "\n",
       "    var canvas = $('<canvas/>');\n",
       "    canvas.addClass('mpl-canvas');\n",
       "    canvas.attr('style', \"left: 0; top: 0; z-index: 0; outline: 0\")\n",
       "\n",
       "    this.canvas = canvas[0];\n",
       "    this.context = canvas[0].getContext(\"2d\");\n",
       "\n",
       "    var backingStore = this.context.backingStorePixelRatio ||\n",
       "\tthis.context.webkitBackingStorePixelRatio ||\n",
       "\tthis.context.mozBackingStorePixelRatio ||\n",
       "\tthis.context.msBackingStorePixelRatio ||\n",
       "\tthis.context.oBackingStorePixelRatio ||\n",
       "\tthis.context.backingStorePixelRatio || 1;\n",
       "\n",
       "    mpl.ratio = (window.devicePixelRatio || 1) / backingStore;\n",
       "\n",
       "    var rubberband = $('<canvas/>');\n",
       "    rubberband.attr('style', \"position: absolute; left: 0; top: 0; z-index: 1;\")\n",
       "\n",
       "    var pass_mouse_events = true;\n",
       "\n",
       "    canvas_div.resizable({\n",
       "        start: function(event, ui) {\n",
       "            pass_mouse_events = false;\n",
       "        },\n",
       "        resize: function(event, ui) {\n",
       "            fig.request_resize(ui.size.width, ui.size.height);\n",
       "        },\n",
       "        stop: function(event, ui) {\n",
       "            pass_mouse_events = true;\n",
       "            fig.request_resize(ui.size.width, ui.size.height);\n",
       "        },\n",
       "    });\n",
       "\n",
       "    function mouse_event_fn(event) {\n",
       "        if (pass_mouse_events)\n",
       "            return fig.mouse_event(event, event['data']);\n",
       "    }\n",
       "\n",
       "    rubberband.mousedown('button_press', mouse_event_fn);\n",
       "    rubberband.mouseup('button_release', mouse_event_fn);\n",
       "    // Throttle sequential mouse events to 1 every 20ms.\n",
       "    rubberband.mousemove('motion_notify', mouse_event_fn);\n",
       "\n",
       "    rubberband.mouseenter('figure_enter', mouse_event_fn);\n",
       "    rubberband.mouseleave('figure_leave', mouse_event_fn);\n",
       "\n",
       "    canvas_div.on(\"wheel\", function (event) {\n",
       "        event = event.originalEvent;\n",
       "        event['data'] = 'scroll'\n",
       "        if (event.deltaY < 0) {\n",
       "            event.step = 1;\n",
       "        } else {\n",
       "            event.step = -1;\n",
       "        }\n",
       "        mouse_event_fn(event);\n",
       "    });\n",
       "\n",
       "    canvas_div.append(canvas);\n",
       "    canvas_div.append(rubberband);\n",
       "\n",
       "    this.rubberband = rubberband;\n",
       "    this.rubberband_canvas = rubberband[0];\n",
       "    this.rubberband_context = rubberband[0].getContext(\"2d\");\n",
       "    this.rubberband_context.strokeStyle = \"#000000\";\n",
       "\n",
       "    this._resize_canvas = function(width, height) {\n",
       "        // Keep the size of the canvas, canvas container, and rubber band\n",
       "        // canvas in synch.\n",
       "        canvas_div.css('width', width)\n",
       "        canvas_div.css('height', height)\n",
       "\n",
       "        canvas.attr('width', width * mpl.ratio);\n",
       "        canvas.attr('height', height * mpl.ratio);\n",
       "        canvas.attr('style', 'width: ' + width + 'px; height: ' + height + 'px;');\n",
       "\n",
       "        rubberband.attr('width', width);\n",
       "        rubberband.attr('height', height);\n",
       "    }\n",
       "\n",
       "    // Set the figure to an initial 600x600px, this will subsequently be updated\n",
       "    // upon first draw.\n",
       "    this._resize_canvas(600, 600);\n",
       "\n",
       "    // Disable right mouse context menu.\n",
       "    $(this.rubberband_canvas).bind(\"contextmenu\",function(e){\n",
       "        return false;\n",
       "    });\n",
       "\n",
       "    function set_focus () {\n",
       "        canvas.focus();\n",
       "        canvas_div.focus();\n",
       "    }\n",
       "\n",
       "    window.setTimeout(set_focus, 100);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._init_toolbar = function() {\n",
       "    var fig = this;\n",
       "\n",
       "    var nav_element = $('<div/>')\n",
       "    nav_element.attr('style', 'width: 100%');\n",
       "    this.root.append(nav_element);\n",
       "\n",
       "    // Define a callback function for later on.\n",
       "    function toolbar_event(event) {\n",
       "        return fig.toolbar_button_onclick(event['data']);\n",
       "    }\n",
       "    function toolbar_mouse_event(event) {\n",
       "        return fig.toolbar_button_onmouseover(event['data']);\n",
       "    }\n",
       "\n",
       "    for(var toolbar_ind in mpl.toolbar_items) {\n",
       "        var name = mpl.toolbar_items[toolbar_ind][0];\n",
       "        var tooltip = mpl.toolbar_items[toolbar_ind][1];\n",
       "        var image = mpl.toolbar_items[toolbar_ind][2];\n",
       "        var method_name = mpl.toolbar_items[toolbar_ind][3];\n",
       "\n",
       "        if (!name) {\n",
       "            // put a spacer in here.\n",
       "            continue;\n",
       "        }\n",
       "        var button = $('<button/>');\n",
       "        button.addClass('ui-button ui-widget ui-state-default ui-corner-all ' +\n",
       "                        'ui-button-icon-only');\n",
       "        button.attr('role', 'button');\n",
       "        button.attr('aria-disabled', 'false');\n",
       "        button.click(method_name, toolbar_event);\n",
       "        button.mouseover(tooltip, toolbar_mouse_event);\n",
       "\n",
       "        var icon_img = $('<span/>');\n",
       "        icon_img.addClass('ui-button-icon-primary ui-icon');\n",
       "        icon_img.addClass(image);\n",
       "        icon_img.addClass('ui-corner-all');\n",
       "\n",
       "        var tooltip_span = $('<span/>');\n",
       "        tooltip_span.addClass('ui-button-text');\n",
       "        tooltip_span.html(tooltip);\n",
       "\n",
       "        button.append(icon_img);\n",
       "        button.append(tooltip_span);\n",
       "\n",
       "        nav_element.append(button);\n",
       "    }\n",
       "\n",
       "    var fmt_picker_span = $('<span/>');\n",
       "\n",
       "    var fmt_picker = $('<select/>');\n",
       "    fmt_picker.addClass('mpl-toolbar-option ui-widget ui-widget-content');\n",
       "    fmt_picker_span.append(fmt_picker);\n",
       "    nav_element.append(fmt_picker_span);\n",
       "    this.format_dropdown = fmt_picker[0];\n",
       "\n",
       "    for (var ind in mpl.extensions) {\n",
       "        var fmt = mpl.extensions[ind];\n",
       "        var option = $(\n",
       "            '<option/>', {selected: fmt === mpl.default_extension}).html(fmt);\n",
       "        fmt_picker.append(option)\n",
       "    }\n",
       "\n",
       "    // Add hover states to the ui-buttons\n",
       "    $( \".ui-button\" ).hover(\n",
       "        function() { $(this).addClass(\"ui-state-hover\");},\n",
       "        function() { $(this).removeClass(\"ui-state-hover\");}\n",
       "    );\n",
       "\n",
       "    var status_bar = $('<span class=\"mpl-message\"/>');\n",
       "    nav_element.append(status_bar);\n",
       "    this.message = status_bar[0];\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.request_resize = function(x_pixels, y_pixels) {\n",
       "    // Request matplotlib to resize the figure. Matplotlib will then trigger a resize in the client,\n",
       "    // which will in turn request a refresh of the image.\n",
       "    this.send_message('resize', {'width': x_pixels, 'height': y_pixels});\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.send_message = function(type, properties) {\n",
       "    properties['type'] = type;\n",
       "    properties['figure_id'] = this.id;\n",
       "    this.ws.send(JSON.stringify(properties));\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.send_draw_message = function() {\n",
       "    if (!this.waiting) {\n",
       "        this.waiting = true;\n",
       "        this.ws.send(JSON.stringify({type: \"draw\", figure_id: this.id}));\n",
       "    }\n",
       "}\n",
       "\n",
       "\n",
       "mpl.figure.prototype.handle_save = function(fig, msg) {\n",
       "    var format_dropdown = fig.format_dropdown;\n",
       "    var format = format_dropdown.options[format_dropdown.selectedIndex].value;\n",
       "    fig.ondownload(fig, format);\n",
       "}\n",
       "\n",
       "\n",
       "mpl.figure.prototype.handle_resize = function(fig, msg) {\n",
       "    var size = msg['size'];\n",
       "    if (size[0] != fig.canvas.width || size[1] != fig.canvas.height) {\n",
       "        fig._resize_canvas(size[0], size[1]);\n",
       "        fig.send_message(\"refresh\", {});\n",
       "    };\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_rubberband = function(fig, msg) {\n",
       "    var x0 = msg['x0'] / mpl.ratio;\n",
       "    var y0 = (fig.canvas.height - msg['y0']) / mpl.ratio;\n",
       "    var x1 = msg['x1'] / mpl.ratio;\n",
       "    var y1 = (fig.canvas.height - msg['y1']) / mpl.ratio;\n",
       "    x0 = Math.floor(x0) + 0.5;\n",
       "    y0 = Math.floor(y0) + 0.5;\n",
       "    x1 = Math.floor(x1) + 0.5;\n",
       "    y1 = Math.floor(y1) + 0.5;\n",
       "    var min_x = Math.min(x0, x1);\n",
       "    var min_y = Math.min(y0, y1);\n",
       "    var width = Math.abs(x1 - x0);\n",
       "    var height = Math.abs(y1 - y0);\n",
       "\n",
       "    fig.rubberband_context.clearRect(\n",
       "        0, 0, fig.canvas.width, fig.canvas.height);\n",
       "\n",
       "    fig.rubberband_context.strokeRect(min_x, min_y, width, height);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_figure_label = function(fig, msg) {\n",
       "    // Updates the figure title.\n",
       "    fig.header.textContent = msg['label'];\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_cursor = function(fig, msg) {\n",
       "    var cursor = msg['cursor'];\n",
       "    switch(cursor)\n",
       "    {\n",
       "    case 0:\n",
       "        cursor = 'pointer';\n",
       "        break;\n",
       "    case 1:\n",
       "        cursor = 'default';\n",
       "        break;\n",
       "    case 2:\n",
       "        cursor = 'crosshair';\n",
       "        break;\n",
       "    case 3:\n",
       "        cursor = 'move';\n",
       "        break;\n",
       "    }\n",
       "    fig.rubberband_canvas.style.cursor = cursor;\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_message = function(fig, msg) {\n",
       "    fig.message.textContent = msg['message'];\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_draw = function(fig, msg) {\n",
       "    // Request the server to send over a new figure.\n",
       "    fig.send_draw_message();\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_image_mode = function(fig, msg) {\n",
       "    fig.image_mode = msg['mode'];\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.updated_canvas_event = function() {\n",
       "    // Called whenever the canvas gets updated.\n",
       "    this.send_message(\"ack\", {});\n",
       "}\n",
       "\n",
       "// A function to construct a web socket function for onmessage handling.\n",
       "// Called in the figure constructor.\n",
       "mpl.figure.prototype._make_on_message_function = function(fig) {\n",
       "    return function socket_on_message(evt) {\n",
       "        if (evt.data instanceof Blob) {\n",
       "            /* FIXME: We get \"Resource interpreted as Image but\n",
       "             * transferred with MIME type text/plain:\" errors on\n",
       "             * Chrome.  But how to set the MIME type?  It doesn't seem\n",
       "             * to be part of the websocket stream */\n",
       "            evt.data.type = \"image/png\";\n",
       "\n",
       "            /* Free the memory for the previous frames */\n",
       "            if (fig.imageObj.src) {\n",
       "                (window.URL || window.webkitURL).revokeObjectURL(\n",
       "                    fig.imageObj.src);\n",
       "            }\n",
       "\n",
       "            fig.imageObj.src = (window.URL || window.webkitURL).createObjectURL(\n",
       "                evt.data);\n",
       "            fig.updated_canvas_event();\n",
       "            fig.waiting = false;\n",
       "            return;\n",
       "        }\n",
       "        else if (typeof evt.data === 'string' && evt.data.slice(0, 21) == \"data:image/png;base64\") {\n",
       "            fig.imageObj.src = evt.data;\n",
       "            fig.updated_canvas_event();\n",
       "            fig.waiting = false;\n",
       "            return;\n",
       "        }\n",
       "\n",
       "        var msg = JSON.parse(evt.data);\n",
       "        var msg_type = msg['type'];\n",
       "\n",
       "        // Call the  \"handle_{type}\" callback, which takes\n",
       "        // the figure and JSON message as its only arguments.\n",
       "        try {\n",
       "            var callback = fig[\"handle_\" + msg_type];\n",
       "        } catch (e) {\n",
       "            console.log(\"No handler for the '\" + msg_type + \"' message type: \", msg);\n",
       "            return;\n",
       "        }\n",
       "\n",
       "        if (callback) {\n",
       "            try {\n",
       "                // console.log(\"Handling '\" + msg_type + \"' message: \", msg);\n",
       "                callback(fig, msg);\n",
       "            } catch (e) {\n",
       "                console.log(\"Exception inside the 'handler_\" + msg_type + \"' callback:\", e, e.stack, msg);\n",
       "            }\n",
       "        }\n",
       "    };\n",
       "}\n",
       "\n",
       "// from http://stackoverflow.com/questions/1114465/getting-mouse-location-in-canvas\n",
       "mpl.findpos = function(e) {\n",
       "    //this section is from http://www.quirksmode.org/js/events_properties.html\n",
       "    var targ;\n",
       "    if (!e)\n",
       "        e = window.event;\n",
       "    if (e.target)\n",
       "        targ = e.target;\n",
       "    else if (e.srcElement)\n",
       "        targ = e.srcElement;\n",
       "    if (targ.nodeType == 3) // defeat Safari bug\n",
       "        targ = targ.parentNode;\n",
       "\n",
       "    // jQuery normalizes the pageX and pageY\n",
       "    // pageX,Y are the mouse positions relative to the document\n",
       "    // offset() returns the position of the element relative to the document\n",
       "    var x = e.pageX - $(targ).offset().left;\n",
       "    var y = e.pageY - $(targ).offset().top;\n",
       "\n",
       "    return {\"x\": x, \"y\": y};\n",
       "};\n",
       "\n",
       "/*\n",
       " * return a copy of an object with only non-object keys\n",
       " * we need this to avoid circular references\n",
       " * http://stackoverflow.com/a/24161582/3208463\n",
       " */\n",
       "function simpleKeys (original) {\n",
       "  return Object.keys(original).reduce(function (obj, key) {\n",
       "    if (typeof original[key] !== 'object')\n",
       "        obj[key] = original[key]\n",
       "    return obj;\n",
       "  }, {});\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.mouse_event = function(event, name) {\n",
       "    var canvas_pos = mpl.findpos(event)\n",
       "\n",
       "    if (name === 'button_press')\n",
       "    {\n",
       "        this.canvas.focus();\n",
       "        this.canvas_div.focus();\n",
       "    }\n",
       "\n",
       "    var x = canvas_pos.x * mpl.ratio;\n",
       "    var y = canvas_pos.y * mpl.ratio;\n",
       "\n",
       "    this.send_message(name, {x: x, y: y, button: event.button,\n",
       "                             step: event.step,\n",
       "                             guiEvent: simpleKeys(event)});\n",
       "\n",
       "    /* This prevents the web browser from automatically changing to\n",
       "     * the text insertion cursor when the button is pressed.  We want\n",
       "     * to control all of the cursor setting manually through the\n",
       "     * 'cursor' event from matplotlib */\n",
       "    event.preventDefault();\n",
       "    return false;\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._key_event_extra = function(event, name) {\n",
       "    // Handle any extra behaviour associated with a key event\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.key_event = function(event, name) {\n",
       "\n",
       "    // Prevent repeat events\n",
       "    if (name == 'key_press')\n",
       "    {\n",
       "        if (event.which === this._key)\n",
       "            return;\n",
       "        else\n",
       "            this._key = event.which;\n",
       "    }\n",
       "    if (name == 'key_release')\n",
       "        this._key = null;\n",
       "\n",
       "    var value = '';\n",
       "    if (event.ctrlKey && event.which != 17)\n",
       "        value += \"ctrl+\";\n",
       "    if (event.altKey && event.which != 18)\n",
       "        value += \"alt+\";\n",
       "    if (event.shiftKey && event.which != 16)\n",
       "        value += \"shift+\";\n",
       "\n",
       "    value += 'k';\n",
       "    value += event.which.toString();\n",
       "\n",
       "    this._key_event_extra(event, name);\n",
       "\n",
       "    this.send_message(name, {key: value,\n",
       "                             guiEvent: simpleKeys(event)});\n",
       "    return false;\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.toolbar_button_onclick = function(name) {\n",
       "    if (name == 'download') {\n",
       "        this.handle_save(this, null);\n",
       "    } else {\n",
       "        this.send_message(\"toolbar_button\", {name: name});\n",
       "    }\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.toolbar_button_onmouseover = function(tooltip) {\n",
       "    this.message.textContent = tooltip;\n",
       "};\n",
       "mpl.toolbar_items = [[\"Home\", \"Reset original view\", \"fa fa-home icon-home\", \"home\"], [\"Back\", \"Back to  previous view\", \"fa fa-arrow-left icon-arrow-left\", \"back\"], [\"Forward\", \"Forward to next view\", \"fa fa-arrow-right icon-arrow-right\", \"forward\"], [\"\", \"\", \"\", \"\"], [\"Pan\", \"Pan axes with left mouse, zoom with right\", \"fa fa-arrows icon-move\", \"pan\"], [\"Zoom\", \"Zoom to rectangle\", \"fa fa-square-o icon-check-empty\", \"zoom\"], [\"\", \"\", \"\", \"\"], [\"Download\", \"Download plot\", \"fa fa-floppy-o icon-save\", \"download\"]];\n",
       "\n",
       "mpl.extensions = [\"eps\", \"jpeg\", \"pdf\", \"png\", \"ps\", \"raw\", \"svg\", \"tif\"];\n",
       "\n",
       "mpl.default_extension = \"png\";var comm_websocket_adapter = function(comm) {\n",
       "    // Create a \"websocket\"-like object which calls the given IPython comm\n",
       "    // object with the appropriate methods. Currently this is a non binary\n",
       "    // socket, so there is still some room for performance tuning.\n",
       "    var ws = {};\n",
       "\n",
       "    ws.close = function() {\n",
       "        comm.close()\n",
       "    };\n",
       "    ws.send = function(m) {\n",
       "        //console.log('sending', m);\n",
       "        comm.send(m);\n",
       "    };\n",
       "    // Register the callback with on_msg.\n",
       "    comm.on_msg(function(msg) {\n",
       "        //console.log('receiving', msg['content']['data'], msg);\n",
       "        // Pass the mpl event to the overriden (by mpl) onmessage function.\n",
       "        ws.onmessage(msg['content']['data'])\n",
       "    });\n",
       "    return ws;\n",
       "}\n",
       "\n",
       "mpl.mpl_figure_comm = function(comm, msg) {\n",
       "    // This is the function which gets called when the mpl process\n",
       "    // starts-up an IPython Comm through the \"matplotlib\" channel.\n",
       "\n",
       "    var id = msg.content.data.id;\n",
       "    // Get hold of the div created by the display call when the Comm\n",
       "    // socket was opened in Python.\n",
       "    var element = $(\"#\" + id);\n",
       "    var ws_proxy = comm_websocket_adapter(comm)\n",
       "\n",
       "    function ondownload(figure, format) {\n",
       "        window.open(figure.imageObj.src);\n",
       "    }\n",
       "\n",
       "    var fig = new mpl.figure(id, ws_proxy,\n",
       "                           ondownload,\n",
       "                           element.get(0));\n",
       "\n",
       "    // Call onopen now - mpl needs it, as it is assuming we've passed it a real\n",
       "    // web socket which is closed, not our websocket->open comm proxy.\n",
       "    ws_proxy.onopen();\n",
       "\n",
       "    fig.parent_element = element.get(0);\n",
       "    fig.cell_info = mpl.find_output_cell(\"<div id='\" + id + \"'></div>\");\n",
       "    if (!fig.cell_info) {\n",
       "        console.error(\"Failed to find cell for figure\", id, fig);\n",
       "        return;\n",
       "    }\n",
       "\n",
       "    var output_index = fig.cell_info[2]\n",
       "    var cell = fig.cell_info[0];\n",
       "\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_close = function(fig, msg) {\n",
       "    var width = fig.canvas.width/mpl.ratio\n",
       "    fig.root.unbind('remove')\n",
       "\n",
       "    // Update the output cell to use the data from the current canvas.\n",
       "    fig.push_to_output();\n",
       "    var dataURL = fig.canvas.toDataURL();\n",
       "    // Re-enable the keyboard manager in IPython - without this line, in FF,\n",
       "    // the notebook keyboard shortcuts fail.\n",
       "    IPython.keyboard_manager.enable()\n",
       "    $(fig.parent_element).html('<img src=\"' + dataURL + '\" width=\"' + width + '\">');\n",
       "    fig.close_ws(fig, msg);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.close_ws = function(fig, msg){\n",
       "    fig.send_message('closing', msg);\n",
       "    // fig.ws.close()\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.push_to_output = function(remove_interactive) {\n",
       "    // Turn the data on the canvas into data in the output cell.\n",
       "    var width = this.canvas.width/mpl.ratio\n",
       "    var dataURL = this.canvas.toDataURL();\n",
       "    this.cell_info[1]['text/html'] = '<img src=\"' + dataURL + '\" width=\"' + width + '\">';\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.updated_canvas_event = function() {\n",
       "    // Tell IPython that the notebook contents must change.\n",
       "    IPython.notebook.set_dirty(true);\n",
       "    this.send_message(\"ack\", {});\n",
       "    var fig = this;\n",
       "    // Wait a second, then push the new image to the DOM so\n",
       "    // that it is saved nicely (might be nice to debounce this).\n",
       "    setTimeout(function () { fig.push_to_output() }, 1000);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._init_toolbar = function() {\n",
       "    var fig = this;\n",
       "\n",
       "    var nav_element = $('<div/>')\n",
       "    nav_element.attr('style', 'width: 100%');\n",
       "    this.root.append(nav_element);\n",
       "\n",
       "    // Define a callback function for later on.\n",
       "    function toolbar_event(event) {\n",
       "        return fig.toolbar_button_onclick(event['data']);\n",
       "    }\n",
       "    function toolbar_mouse_event(event) {\n",
       "        return fig.toolbar_button_onmouseover(event['data']);\n",
       "    }\n",
       "\n",
       "    for(var toolbar_ind in mpl.toolbar_items){\n",
       "        var name = mpl.toolbar_items[toolbar_ind][0];\n",
       "        var tooltip = mpl.toolbar_items[toolbar_ind][1];\n",
       "        var image = mpl.toolbar_items[toolbar_ind][2];\n",
       "        var method_name = mpl.toolbar_items[toolbar_ind][3];\n",
       "\n",
       "        if (!name) { continue; };\n",
       "\n",
       "        var button = $('<button class=\"btn btn-default\" href=\"#\" title=\"' + name + '\"><i class=\"fa ' + image + ' fa-lg\"></i></button>');\n",
       "        button.click(method_name, toolbar_event);\n",
       "        button.mouseover(tooltip, toolbar_mouse_event);\n",
       "        nav_element.append(button);\n",
       "    }\n",
       "\n",
       "    // Add the status bar.\n",
       "    var status_bar = $('<span class=\"mpl-message\" style=\"text-align:right; float: right;\"/>');\n",
       "    nav_element.append(status_bar);\n",
       "    this.message = status_bar[0];\n",
       "\n",
       "    // Add the close button to the window.\n",
       "    var buttongrp = $('<div class=\"btn-group inline pull-right\"></div>');\n",
       "    var button = $('<button class=\"btn btn-mini btn-primary\" href=\"#\" title=\"Stop Interaction\"><i class=\"fa fa-power-off icon-remove icon-large\"></i></button>');\n",
       "    button.click(function (evt) { fig.handle_close(fig, {}); } );\n",
       "    button.mouseover('Stop Interaction', toolbar_mouse_event);\n",
       "    buttongrp.append(button);\n",
       "    var titlebar = this.root.find($('.ui-dialog-titlebar'));\n",
       "    titlebar.prepend(buttongrp);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._root_extra_style = function(el){\n",
       "    var fig = this\n",
       "    el.on(\"remove\", function(){\n",
       "\tfig.close_ws(fig, {});\n",
       "    });\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._canvas_extra_style = function(el){\n",
       "    // this is important to make the div 'focusable\n",
       "    el.attr('tabindex', 0)\n",
       "    // reach out to IPython and tell the keyboard manager to turn it's self\n",
       "    // off when our div gets focus\n",
       "\n",
       "    // location in version 3\n",
       "    if (IPython.notebook.keyboard_manager) {\n",
       "        IPython.notebook.keyboard_manager.register_events(el);\n",
       "    }\n",
       "    else {\n",
       "        // location in version 2\n",
       "        IPython.keyboard_manager.register_events(el);\n",
       "    }\n",
       "\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._key_event_extra = function(event, name) {\n",
       "    var manager = IPython.notebook.keyboard_manager;\n",
       "    if (!manager)\n",
       "        manager = IPython.keyboard_manager;\n",
       "\n",
       "    // Check for shift+enter\n",
       "    if (event.shiftKey && event.which == 13) {\n",
       "        this.canvas_div.blur();\n",
       "        // select the cell after this one\n",
       "        var index = IPython.notebook.find_cell_index(this.cell_info[0]);\n",
       "        IPython.notebook.select(index + 1);\n",
       "    }\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_save = function(fig, msg) {\n",
       "    fig.ondownload(fig, null);\n",
       "}\n",
       "\n",
       "\n",
       "mpl.find_output_cell = function(html_output) {\n",
       "    // Return the cell and output element which can be found *uniquely* in the notebook.\n",
       "    // Note - this is a bit hacky, but it is done because the \"notebook_saving.Notebook\"\n",
       "    // IPython event is triggered only after the cells have been serialised, which for\n",
       "    // our purposes (turning an active figure into a static one), is too late.\n",
       "    var cells = IPython.notebook.get_cells();\n",
       "    var ncells = cells.length;\n",
       "    for (var i=0; i<ncells; i++) {\n",
       "        var cell = cells[i];\n",
       "        if (cell.cell_type === 'code'){\n",
       "            for (var j=0; j<cell.output_area.outputs.length; j++) {\n",
       "                var data = cell.output_area.outputs[j];\n",
       "                if (data.data) {\n",
       "                    // IPython >= 3 moved mimebundle to data attribute of output\n",
       "                    data = data.data;\n",
       "                }\n",
       "                if (data['text/html'] == html_output) {\n",
       "                    return [cell, data, j];\n",
       "                }\n",
       "            }\n",
       "        }\n",
       "    }\n",
       "}\n",
       "\n",
       "// Register the function which deals with the matplotlib target/channel.\n",
       "// The kernel may be null if the page has been refreshed.\n",
       "if (IPython.notebook.kernel != null) {\n",
       "    IPython.notebook.kernel.comm_manager.register_target('matplotlib', mpl.mpl_figure_comm);\n",
       "}\n"
      ],
      "text/plain": [
       "<IPython.core.display.Javascript object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<img src=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAABEwAAAK8CAYAAAD1QyqZAAAgAElEQVR4XuzdB7RsTXYf9HrkHCWwxSBjkGwEDKCRyGAwWcbknGERhDGLcAERFjkYEPCRg0gmmpzBIkdjkjQCBixgRBKyDEhkMGAMw9qoe9zT8+591a+qdtW+99drzZL9fefUrvrt+m6f++/T575rXgQIECBAgAABAgQIECBAgAABAl8i8I4HAQIECBAgQIAAAQIECBAgQIDAlwoITOwIAgQIECBAgAABAgQIECBAgMCdgMDEliBAgAABAgQIECBAgAABAgQICEzsAQIECBAgQIAAAQIECBAgQIDAywLuMLFDCBAgQIAAAQIECBAgQIAAAQJ3AgITW4IAAQIECBAgQIAAAQIECBAgIDCxBwgQIECAAAECBAgQIECAAAECLwu4w8QOIUCAAAECBAgQIECAAAECBAjcCQhMbAkCBAgQIECAAAECBAgQIECAgMDEHiBAgAABAgQIECBAgAABAgQIvCzgDhM7hAABAgQIECBAgAABAgQIECBwJyAwsSUIECBAgAABAgQIECBAgAABAgITe4AAAQIECBAgQIAAAQIECBAg8LKAO0zsEAIECBAgQIAAAQIECBAgQIDAnYDAxJYgQIAAAQIECBAgQIAAAQIECAhM7AECBAgQIECAAAECBAgQIECAwMsC7jCxQwgQIECAAAECBAgQIECAAAECdwICE1uCAAECBAgQIECAAAECBAgQICAwsQcIECBAgAABAgQIECBAgAABAi8LuMPEDiFAgAABAgQIECBAgAABAgQI3AkITGwJAgQIECBAgAABAgQIECBAgIDAxB4gQIAAAQIECBAgQIAAAQIECLws4A4TO4QAAQIECBAgQIAAAQIECBAgcCcgMLElCBAgQIAAAQIECBAgQIAAAQICE3uAAAECBAgQIECAAAECBAgQIPCygDtM7BACBAgQIECAAAECBAgQIECAwJ2AwMSWIECAAAECBAgQIECAAAECBAgITOwBAgQIECBAgAABAgQIECBAgMDLAu4wsUMIECBAgAABAgQIECBAgAABAncCAhNbggABAgQIECBAgAABAgQIECAgMLEHCBAgQIAAAQIECBAgQIAAAQIvC7jDxA4hQIAAAQIECBAgQIAAAQIECNwJCExsCQIECBAgQIAAAQIECBAgQICAwMQeIECAAAECBAgQIECAAAECBAi8LOAOEzuEAAECBAgQIECAAAECBAgQIHAnIDCxJQgQIECAAAECBAgQIECAAAECAhN7gAABAgQIECBAgAABAgQIECDwsoA7TOwQAgQIECBAgAABAgQIECBAgMCdgMDEliBAgAABAgQIECBAgAABAgQICEzsAQIECBAgQIAAAQIECBAgQIDAywLuMLFDCBAgQIAAAQIECBAgQIAAAQJ3AgITW4IAAQIECBAgQIAAAQIECBAgIDCxBwgQIECAAAECBAgQIECAAAECLwu4w8QOIUCAAAECBAgQIECAAAECBAjcCQhMbAkCBAgQIECAAAECBAgQIECAgMDEHiBAgAABAgQIECBAgAABAgQIvCzgDhM7hAABAgQIECBAgAABAgQIECBwJyAwsSUIECBAgAABAgQIECBAgAABAgITe4AAAQIECBAgQIAAAQIECBAg8LKAO0zsEAIECBAgQIAAAQIECBAgQIDAnYDAxJYgQIAAAQIECBAgQIAAAQIECAhM7AECBAgQIECAAAECBAgQIECAwMsC7jCxQwgQIECAAAECBAgQIECAAAECdwICE1uCAAECBAgQIECAAAECBAgQICAwsQcIECBAgAABAgQIECBAgAABAi8LuMPEDiFAgAABAgQIECBAgAABAgQI3AkITGwJAgQIECBAgAABAgQIECBAgIDAxB4gQIAAAQIECBAgQIAAAQIECLws4A4TO4QAAQIECBAgQIAAAQIECBAgcCcgMLElCBAgQIAAAQIECBAgQIAAAQICE3uAAAECBAgQIECAAAECBAgQIPCygDtM7BACBAgQIECAAAECBAgQIECAwJ2AwMSWIECAAAECBAgQIECAAAECBAgITOwBAgQIECBAgAABAgQIECBAgMDLAu4wsUMIECBAgAABAgQIECBAgAABAncCAhNbggABAgQIECBAgAABAgQIECAgMLEHCBAgQIAAAQIECBAgQIAAAQIvC7jDxA4hQIAAAQIECBAgQIAAAQIECNwJCExsCQIECBAgQIAAAQIECBAgQICAwMQeIECAAAECBAgQIECAAAECBAi8LOAOEzuEAAECBAgQIECAAAECBAgQIHAnIDCxJQgQIECAAAECBAgQIECAAAECAhN7gAABAgQIECBAgAABAgQIECDwsoA7TOwQAgQIECBAgAABAgQIECBAgMCdgMDEliBAgAABAgQIECBAgAABAgQICEzsAQIECBAgQIAAAQIECBAgQIDAywLuMLFDCBAgQIAAAQIECBAgQIAAAQJ3AgITW4IAAQIECBAgQIAAAQIECBAgIDCxBwgQIECAAAECBAgQIECAAAECLwu4w8QOIUCAAAECBAgQIECAAAECBAjcCQhMbAkCBAgQIECAAAECBAgQIECAgMDEHiBAgAABAgQIECBAgAABAgQIvCzgDhM7hAABAgQIECBAgAABAgQIECBwJyAwsSUIECBAgAABAgQIECBAgAABAgITe4AAAQIECBAgQIAAAQIECBAg8LKAO0zsEAIECBAgQIAAAQIECBAgQIDAnYDAxJYgQIAAAQIECBAgQIAAAQIECAhM7AECBAgQIECAAAECBAgQIECAwMsC7jCxQwgQIECAAAECBAgQIECAAAECdwICE1uCAAECBAgQIECAAAECBAgQICAwsQcIECBAgAABAgQIECBAgAABAi8LuMPEDiFAgAABAgQIECBAgAABAgQI3AkITGwJAgQIECBAgAABAgQIECBAgIDAxB4gQIAAAQIECBAgQIAAAQIECLws4A4TO4QAAQIECBAgQIAAAQIECBAgcCcgMLElCBAgQIAAAQIECBAgQIAAAQICE3uAAAECBAgQIECAAAECBAgQIPCygDtM7BACBAgQIECAAAECBAgQIECAwJ2AwMSWIECAAAECBAgQIECAAAECBAgITOwBAgQIECBAgAABAgQIECBAgMDLAu4wsUMIECBAgAABAgQIECBAgAABAncCAhNbggABAgQIECBAgAABAgQIECAgMLEHCBAgQIAAAQIECBAgQIAAAQIvC7jDxA4hQIAAAQIECBAgQIAAAQIECNwJCExsCQIECBAgQIAAAQIECBAgQICAwMQeIECAAAECBAgQIECAAAECBAi8LOAOEzuEAAECBAgQIECAAAECBAgQIHAnIDCxJQgQIECAAAECBAgQIECAAAECAhN7gAABAgQIECBAgAABAgQIECDwsoA7TOwQAgQIECBAgAABAgQIECBAgMCdgMDEliBAgAABAgQIECBAgAABAgQICEzsAQIECBAgQIAAAQIECBAgQIDAywLuMLFDCBAgQIAAAQIECBAgQIAAAQJ3AgITW4IAAQIECBAgQIAAAQIECBAgIDCxBwgQIECAAAECBAgQIECAAAECLwu4w8QOIUCAAAECBAgQIECAAAECBAjcCQhMbAkCBAgQIECAAAECBAgQIECAgMDEHiBAgAABAgQIECBAgAABAgQIvCzgDhM7hAABAgQIECBAgAABAgQIECBwJyAwsSUIECBAgAABAgQIECBAgAABAgITe4AAAQIECBAgQIAAAQIECBAg8LKAO0zsEAIECBAgQIAAAQIECBAgQIDAnYDAxJYgQIAAAQIECBAgQIAAAQIECAhM7AECBAgQIECAAAECBAgQIECAwMsC7jCxQwgQIECAAAECBAgQIECAAAECdwICE1uCAAECBAgQIECAAAECBAgQICAwsQcIECBAgAABAgQIECBAgAABAi8LuMPEDiFAgAABAgQIECBAgAABAgQI3AkITGwJAgQIECBAgAABAgQIECBAgIDAxB4gQIAAAQIECBAgQIAAAQIECLws4A4TO4QAAQIECBAgQIAAAQIECBAgcCcgMLElCBAgQIAAAQIECBAgQIAAAQICE3uAAAECBAgQIECAAAECBAgQIPCygDtM7BACBAgQIECAAAECBAgQIECAwJ2AwMSWIECAAAECBAgQIECAAAECBAgITOwBAgQIECBAgAABAgQIECBAgMDLAu4wsUMIEFgp8Oe01v7s1lrPz5o/pLX2k1trP7K19l8+MKl/pbX2Fa21X+uBc04+NNYTr9/i5EmaGwECBAi8VyDev+LneLynrXrF+8O/3Fr7sZdas+tc349/vdbad8wefMN4H3t9sWGqShIgcJpAzy8xp83ZfAgQmCNwvYCI0X6z1tpPuxs2fj58b2vtU621f7q19uM/ouz7ApM/vbX2M1pr/9jdeB97QSMweX9jvqq19kdenP+9j+hdnPLjWmu/fmst+uhFgACBtyzw6csHABEi/Iqttf/+8l72T7TW/pobGIHJebvkY68v3reSP7q19rNba3/7Ry5zxnvzR5Z2GgECHyMgMPkYNecQeB0C1wuI//NyZ0dcBNy+rp9g/V+ttX9hYmDyv7XW/qH3fPr287fWfsHWWtT7wgPEApP3Y31ja+3fba39oQMXdn9ta+0ndt4h9EDLHEqAAIFSAr/x5Y6O+BDh72it/TettV+5tfYbttZ+tdba19ys5hdurf2/rbX/e+EK3WHyGO7MwOQ/bK394MBdoDPemx9bvaMJEBgSEJgM8TmZQGmB6wXEP9Ja+zGttR/eWvu5Nyv6m1prn7l83SUuEGbdYfJcYPKxmAITgcnH7h3nESBAoEcg7rKMO0t+VGvtf7o74Vdorf13PYNMPEZg8himwOQxL0cTIHAjIDCxHQi8XYHrBcTv2Vr7B1prv0Nr7dsvHL/Q5RO0v6C19se21m4Dk+cu1H6V1tp/cXdHw/1Xct5350h8Whdzee6C5ptaa3/qJbyJ8/+T1tpf0Vr7KZe5XgOT36u19te11n6D1tr/2Fr7q1pr33rT3ljTn3FZZ3wa+Au01j7bWvuzLp8cXg+9ruNPbq39L621P+XytaT/oLUWd+HEXRvXV9yS+3u01n70pfZv3Vr7Py6fQMZ5/8/NsT/fxfKPuHwi+T9fvi4Ta4v5Xl+9zzD5bS63h8ezW2ItP7O19g+31uIrT9ce3e/u690m8RWs6GtYxa3lcbEfd/3EuTH/eMXa/uD3/Odxfd/oXc/b/S/MygkQeC0C/3Fr7WddnhnyoTXdfyXn+t72m7bWfvfW2h/YWvvFWmv/3OVrkz9w9z4R70nxdcpfprX2b1/u8vupd89Fee59OH6m/7mttd/ocsdmvF/Fz/V/427Sv1Jr7c9rrcX76y/fWvv+1to/01r741prP+fm/bhnzr/zZb5ffxnr+y7vHz/p7j2w9736urbfu7X2ta21n3D54CbW8M2tte+5W0vPmnsDkx/WWvuLWmvx/vqVrbX/obX271xcoq/xvx9xV/9fvbzn/nIX69/u8iy2uMso5hzv8f/+5ZwPvTfHYT3r+dAe9O8JEJgoIDCZiGkoAsUEbh/qFuHCf9Za+4Mua4gLoLjzJG45/ukTA5M/oLX2t1wuQOIOlnhF3X/zmcAk5vi3tdb+o9ba33v5ZC8uyuKW5+tc4yIsLqoinIg5R6ASIcZveXkGxzUEigfDRugR43y+tfZLttb+sNbar3p5Tsf1OR/XwOS7Lsf8zZevCH1Lay2+vhTHX2+1jlAhLupiDXFhGxenEZrERXGEK3/DzZ6Ica7m33m5oPpjLt+B/01uxuwJTH7NS9gT6/m7Ll9jihAonjfym19CkLjgjgvicP7XL/OIXv7nrbW/+hLaxHNr4nv4cV7M7R9trUWAFq+44I4L77hwjAv86+vvvvw/etdT7D8L0yVAgMCXCfyzl5+J8dWc+ADhpddzgUm8p0Q4Hj9n433mj7+E3PEecn39Ja21eK/5J1trUfPXuYQav8jlWWLXB8m+LzCJ97x4v4v3lwjA4xf2CMl/9ctzyuIX/3jFMzTivSoCmXh/iDAoApR434z1xR001/eqnjnHeiJkiTHjDtKYR7yP/GWttfjg4frqfa++ri1qxxriPeeXvriEfQQK11fvmnsDkwg44v01nkkTfYy7h+I9MD78+ddaa7/L5d/FOv/CyyT+29baP99ai6/a/H2ttX/w8uFRfBgRAc8v0Vr7NS6hVPyzl96be9fjP1ECBBIFBCaJ2EoROEzgNjCJC5D4VCXezOMOg7jjJAKGePOOi4ZZd5gEwXNfybm/oIkLpP/6EijEBVSEFddX/Oy63q0SF2EREkSAEuFBvOJukv/q8ulOXATGK56REv+LC7vrKy4Y42IxbreO8CRe18AkgoQIYq53f/xOrbV/vLX2O7bW/qnLsde7MOITwT//Zty4cyUu9OICKl7xKV2EFr//zZ0x8c/jk6j4VO/2n/cEJnGhHXfZxCdg8V3q971e+p70L3pzJ8n13PgULD4RjPXH9/Tj9dwzTB5ZzzPT848JECBQRiB+ab6G7xE8xM/zf/Fyd+L9s0qeC0ziWWC/7c171yeXO/3iDo+44zDef+PujHh/+V1vZOIvzcUv7Ne7MeNf3Qcm8Z4YHxZEIB53jVzfH+NnfXzgEHdlRO14xTjx4UW879//BZzre+v1/fhDc47x3vd+8jdegva46yKeSxav3vfq69q+u7X26968Z8ddkfHhTjx8N65JHllzT2AS1wPxfh8hT4Q9z72ee4ZJfJATeyHe+6+veD+Na4wIV67XCM+9Nz+ynjL/4ZgogdcgIDB5DV20BgIfJ3AbmES4ELfk/n6XX+DjE5O4OIm7QXYFJhF0xCc1ceF4/xd1blccF2Hf0Fr7pe4eFhvhRtwhE89huX/F10ni4ij+b4Qe8ela3LkSr2tg8tdfboW+nvvLXm7PjVuW4w6NeF0Dk/gU6va26rioi7sy4mIxXvH/j6+3RABz/7Wk+BpTfCoVX9WJV09gcu3dH355YO/tBdp1vr0PlvvFLxe88QlY3Focn6CFXbyeC0weWc/H7U5nESBA4CyBeIbJn3YJuuMrNfGKn/vxczj+Us719VxgEl8bjfe06yve2+KuyLiLJO4WjPffv+cSbMQdC9dXvI9EgP9SYBLvXxHUx/tMfH3n9hUfhly/BhT/PEKB+JPE8bP+udf1PeZDc74/P+7cjOAgPgyIO0Mi8Lh+HaX3vfoamMSdNn/pTYHrGuMO2PDuXXO8P/YEJjHv+Bpu2IfX7Vdlb9fZ89DX+HAmrjHi96wI1iLIuoZgz703P7Kes/7LMBsCr1xAYPLKG2x5BDouiOIiMD5lik/P4i6OCCfiNt34tCtuzd0VmMQzQP7iS8hw/53l22XFRVjM9evu1hphRtx58iNv/nlcTP6Jl1uU4y/yXF8RWsRXbeJ1DUzijou4Pfr2FWFHfNIXX1WJ1/UrOfEJ2+3r/tktcQEbn/o994qLv7gIjFdPYBL14qIuvsoTd5jEBVlceF9vw45xXgpMvvrydZ24ayaCoNtXGP2dl3/wXGDyyHpeWLZ/RYAAgXICcQdjhBzxC/CfcLlzMYKBn3FZyXOBSfxFnfjq5vV1DQbi/0ZYHWFM3OUX70XxnnT7imdpxPvEc1/JiWDj7/+AZAQv1+eTxR0P8Uyv517XgOFDc47z4yss8byzuCM1Pri4fcV7cHyVJV6979VXl9/nbk3X9+aYW4RHvWuO4KMnMIk5xt2bf/nlK77/1uVun3g/jL+KdH09F5jEBzDxgUp8HTeuOyI0ub4ioAqfeD333vzIej7Qav+aAIGZAgKTmZrGIlBL4PYOkwhM4hOVeC5FfG847jC5fvp0H5jEBVBc+PzYm1/uY+XxpxUj2Lj9M7b3wUEc1/uVnEcCk/j6UDz89PYVYUZceMVFVrziFuT4yk4EQhEuxINO47kncZEac78ed/vQ1/vbciMwibAk1hWv60Nf4zvKt6/7dcfXbuLTo/jqzfte8Snl7adwcUzM/aVXXJxFD+Jhvb/9JTD6ly6fTsa6nrsoi4u4uLCPi+dYX9wu/L9f7rKJ9dz277nA5JH11PqvwmwJECDQL3B9H70N0p8LTK4fTlxHv/9azUhgEuFCPJ8rvk5yfR7X/SoilImf+/HLf29g8qE5x10U8b4fd2Zcn4UWH7zEnZ3xgcPtdcJzf9Hu/r366hLPQYkPAa6v+wfL9645vibTG5hErbgeiOufuEsmrnd+9iXsiGujeD0XmEQAFV+7ieeuxQcaEXLF3S1/5d2fIX7uvfmR9fTvUEcSIDAsIDAZJjQAgbIC94FJ/NIfIULcvRAPoYvnmMTrPjD5tS+/3N9/VSY+PYk7HT4UmPyvlwfdXT8puwLeX9A88pWcnsAkgpL47nM8HPX2azHxkLf4Ss7KwCT+ek88/C1uV77+FZrnNk7PHSbvOzf+EkJcBMd37eN75/E1pQjCbvsR58UnoXHhd3snSfzzOC/+asPt8fHgu3gw7f17xSPrKfsfiIkTIEDgAwIR1H+utfZtrbU/6uY9M36OX9/j7t9rr0PeBybPfSUnnnESdxK+9JWcCDbi2SrxPnN9oPr7ph5B+yNfyflQYBLBQjz09fZOkqgbXzGNeawMTHrXHPN5JDC5dYuv0UYAFWuMD13iFf2Or0jdf6gRx0VIcr2T5DpOPJcmQqXr8c+9Nz+yHv9hEiCQKCAwScRWisBhAu+7iItfoiM4iD/He/3F/j4wiYexxsVCPMfj6WZN8UlQ/HWYDwUm8elW3Op6//3p+wuauLU3LjTiwW9xMfbSQ197ApP4k7sRFsQF0PWZH/HQu/gLPfGQ05WByfWunPgueQQbt6/4k8ARVsXXn+LVE5jEp4RxYXb7+nGXh9f++Mv/jb+MEHZxy3h8wnV9RWgU35e/3tYc/zzeC+KvMsTdKrf9i69ExZ0+8bWd6/zi+EfWc9i2Nx0CBAg8LHD9xf/+GVTxnI24kyLeC+NB3PH62DtMrg99jZ/Fv9vNDHse+hpByH96eW+LuzviTs7bVzwg/PqcrUce+vqhwCQegh5fFbp+rShqxtd+4qtH8X67MjB5ZM09gUk8lyauDW6vNaJGPN8tHvJ7/Qtycf0Sf7Uo1nf7ir9OFHfaxJqvrzgnPny6/unh+OfPvTc/sp6HN7ATCBD4eAGBycfbOZNAdYHnPvW6X9d9YBL/Pm79jTtAIjSJP6kbv6THg0/jk5MPBSbxF2niF+74yzJxIRLf1Y6Lq/dd0MRfrokHz8YtsD/l8slYfHc8Lmwi3IlX722+Ma+4VTYu7mIO8R3j+ETwZ14Ci5WBScwz/mpAfPoXz4qJOzniNuEIb+KCKr73fL31uCcwiQDkx1zWEQ/sDfv43nRczMcnnvEXF+IZLXHHUHy9Kh6cF1+7CedrCBXBV1zgxwVeBF0RioTtbf+uF3vxVab4E5fxVZ94QO0j66n+34n5EyBAIN6D4n0n7jSIrzFGKBB/gjfuxoy/5hZfubyGyh8bmIRyfE0ynrMVoUl89fH6Z4Xjzs/46znx8zle7/uzwvHP4v0lfu7/5Mt7W9w9Gb/Ax8/5CDfiFf8s7j6MDyXiLpAI1n/45b0o/gLa7Z8V/lBgEne/fP4yflwPxHtQfL03fvmPua8MTK4OPWvuCUwiAIm7ZCPgiK+t/tzLc2ri7su43okPXeIVd1j+hMs1TNw5Et7xddj4um5c18RXjH765Y7W+BpueMYeud5h8tx7c1wL9fbQf5EECCQKCEwSsZUicJjASGASd3REABAPMo0/GRgXGPH1jbio/FBg8qMvF2lxIRYXgdfbjJ+7oImLvHgAa1yQRsgQF6vxi/71F/fewCR+3sU4EVr8sMsF0Z95uUi8fdbJimeYXFsftylH/fiLNHExFhfWcbEXAcjPuhzUE5jELb/xV4zCMHoRt2vHJ1jxSWRcvF5f8VDXuKvlR7XW4k6Wa2/iAblxcRsP9ItP0+KXgHheSTxH5bZ/8byTsI7vVkedMLx93+hZz2Hb3nQIECDwsEA8JyoC5AhJPnUJTOLOxPj5HQ88jV+ar6+RwCR+5sbP8fjZGqF23AH5E1trP+3yF3biF/VrUBAPEr1/llj80h/va/GhRNy5GHd0RlAeXxmKX+qvr3jwdzxvI97DIziJDw5iLXGnzM+5+QDjQ4FJjBcm8aDUCEji6z7x13EieIiQfXVgEvV71twTmET4E6HHb3X5C3vxHh3XG7G2279uFHcC/a2XDy3ia7bXu0fir+zE12Ljq1XxbJf4q0V/0uXh9deefei9uXc9D29gJxAg8PECApOPt3MmAQIECBAgQIAAgZUC8ct3BBHxUNH4hdyLAAECBBIFBCaJ2EoRIECAAAECBAgQeEYg7rq8fzD49a+uxddl4iHlXgQIECCQKCAwScRWigABAgQIECBAgMAzAvHVkfjfT708uDVCkt/38tyr+DO3XgQIECCQLCAwSQZXjgABAgQIECBAgMB7BOIv3MRfqYvncsSzReKh3fGw0fg6zv1fvgFIgAABAgkCApMEZCUIECBAgAABAgQIECBAgACBWgICk1r9MlsCBAgQIECAAAECBAgQIEAgQUBgkoCsBAECBAgQIECAAAECBAgQIFBLQGBSq19mS4AAAQIECBAgQIAAAQIECCQICEwSkJUgQIAAAQIECBAgQIAAAQIEagkITGr1a9tsP/nkky/MKv709DRrKOMQIECAQIfAJ5980nFU3yFPT0+uHfqo/v+jvvCFL0x7/3z3Dv0D9A4lQIDAsMDEH+HtnR/iw/3YMYB33h3qBWsKTAo2zZQJECBwERCY7NsKApN99ioTIEBgVEBgMipY/3yBSf0epqxAYJLCrAgBAgSWCAhMlrB2DSow6WJyEAECBI4UEJgc2ZbUSQlMUrnrFhOY1O2dmRMgQEBgsm8PCEz22atMgACBUQGByahg/fMFJvV7mAl+QvYAACAASURBVLICgUkKsyIECBBYIiAwWcLaNajApIvJQQQIEDhSQGByZFtSJyUwSeWuW0xgUrd3Zk6AAAGByb49IDDZZ68yAQIERgUEJqOC9c8XmNTvYcoKBCYpzIoQIEBgiYDAZAlr16ACky4mBxEgQOBIAYHJkW1JnZTAJJW7bjGBSd3emTkBAgQEJvv2gMBkn73KBAgQGBUQmIwK1j9fYFK/hykrEJikMCtCgACBJQICkyWsXYMKTLqYHESAAIEjBQQmR7YldVICk1TuusUEJnV7Z+YECBAQmOzbAwKTffYqEyBAYFRAYDIqWP98gUn9HqasQGCSwqwIAQIElggITJawdg0qMOlichABAgSOFBCYHNmW1EkJTFK56xYTmNTtnZkTIEBAYLJvDwhM9tmrTIAAgVEBgcmoYP3zBSb1e5iyAoFJCrMiBAgQWCIgMFnC2jWowKSLyUEECBA4UkBgcmRbUiclMEnlrltMYFK3d2ZOgAABgcm+PSAw2WevMgECBEYFBCajgvXPF5jU72HKCgQmKcyKECBAYImAwGQJa9egApMuJgcRIEDgSAGByZFtSZ2UwCSVu24xgUnd3pk5AQIEBCb79oDAZJ+9ygQIEBgVEJiMCtY/X2BSv4cpKxCYpDArQoAAgSUCApMlrF2DCky6mBxEgACBIwUEJke2JXVSApNU7rrFBCZ1e2fmBAgQEJjs2wMCk332KhMgQGBUQGAyKlj/fIFJ/R6mrEBgksKsCAECBJYICEyWsHYNKjDpYnIQAQIEjhQQmBzZltRJCUxSuesWE5jU7Z2ZEyBAQGCybw8ITPbZq0yAAIFRAYHJqGD98wUm9XuYsgKBSQqzIgQIEFgiIDBZwto1qMCki8lBBAgQOFJAYHJkW1InJTBJ5a5bTGBSt3dmToAAAYHJvj0gMNlnrzIBAgRGBQQmo4L1zxeY1O9hygoEJinMihAgQGCJgMBkCWvXoAKTLiYHESBA4EgBgcmRbUmdlMAklbtuMYFJ3d6ZOQECBAQm+/aAwGSfvcoECBAYFRCYjArWP19gUr+HKSsQmKQwK0KAAIElAgKTJaxdgwpMupgcRIAAgSMFBCZHtiV1UgKTVO66xQQmdXtn5gQIEBCY7NsDApN99ioTIEBgVEBgMipY/3yBSf0epqxAYJLCrAgBAgSWCAhMlrB2DSow6WJyEAECBI4UEJgc2ZbUSQlMUrnrFhOY1O2dmRMgQEBgsm8PCEz22atMgACBUQGByahg/fMFJvV7mLICgUkKsyIECBBYIiAwWcLaNajApIvJQQQIEDhSQGByZFtSJyUwSeWuW0xgUrd3Zk6AAAGByb49IDDZZ68yAQIERgUEJqOC9c8XmNTvYcoKBCYpzIoQIEBgiYDAZAlr16ACky4mBxEgQOBIAYHJkW1JnZTAJJW7bjGBSd3emTkBAgQEJvv2gMBkn73KBAgQGBUQmIwK1j9fYFK/hykrEJikMCtCgACBJQICkyWsXYMKTLqYHESAAIEjBQQmR7YldVICk1TuusUEJnV7Z+YECBAQmOzbAwKTffYqEyBAYFRAYDIqWP98gUn9HqasQGCSwqwIAQIElggITJawdg0qMOlichABAgSOFBCYHNmW1EkJTFK56xYTmNTtnZkTIEBAYLJvDwhM9tmrTIAAgVEBgcmoYP3zBSb1e5iyAoFJCrMiBAgQWCIgMFnC2jWowKSLyUEECBA4UkBgcmRbUiclMEnlrltMYFK3d2ZOgAABgcm+PSAw2WevMgECBEYFBCajgvXPF5jU72HKCgQmKcyKECBAYImAwGQJa9egApMuJgcRIEDgSAGByZFtSZ2UwCSVu24xgUnd3pk5AQIEBCb79oDAZJ+9ygQIEBgVEJiMCtY/X2BSv4cpKxCYpDArQoAAgSUCApMlrF2DCky6mBxEgACBIwUEJke2JXVSApNU7rrFBCZ1e2fmBAgQEJjs2wMCk332KhMgQGBUQGAyKlj/fIFJ/R6mrEBgksKsCAECBJYICEyWsHYNKjDpYnIQAQIEjhQQmBzZltRJCUxSuesWE5jU7Z2ZEyBAQGCybw8ITPbZq0yAAIFRAYHJqGD98wUm9XuYsgKBSQqzIgQIEFgiIDBZwto1qMCki8lBBAgQOFJAYHJkW1InJTBJ5a5bTGBSt3dmToAAAYHJvj0gMNlnrzIBAgRGBQQmo4L1zxeY1O9hygoEJinMihAgQGCJgMBkCWvXoAKTLiYHESBA4EgBgcmRbUmdlMAklbtuMYFJ3d6ZOQECBAQm+/aAwGSfvcoECBAYFRCYjArWP19gUr+HKSsQmKQwK0KAAIElAgKTJaxdgwpMupgcRIAAgSMFBCZHtiV1UgKTVO66xQQmdXtn5gQIEBCY7NsDApN99ioTIEBgVEBgMipY/3yBSf0epqxAYJLCrAgBAgSWCAhMlrB2DSow6WJyEAECBI4UEJgc2ZbUSQlMUrnrFhOY1O2dmRMgQEBgsm8PCEz22atMgACBUQGByahg/fMFJvV7mLICgUkKsyIECBBYIiAwWcLaNajApIvJQQQIEDhSQGByZFtSJyUwSeWuW0xgUrd3Zk6AAAGByb49IDDZZ68yAQIERgUEJqOC9c8XmNTvYcoKBCYpzIoQIEBgiYDAZAlr16ACky4mBxEgQOBIAYHJkW1JnZTAJJW7bjGBSd3emTkBAgQEJvv2gMBkn73KBAgQGBUQmIwK1j9fYFK/hykrEJikMCtCgACBJQICkyWsXYMKTLqYHESAAIEjBQQmR7YldVICk1TuusVmBibf933fVxfCzAkQIFBQ4FOf+tS0WT89Pbl2eEBzZmDyXd/1XQ9UdigBAgQIjAp8/dd//egQXzz/3bt33j+naeYNpGl51qUrCUxKt8/kCRB44wICk30bQGCyz15lAgQIjAoITEYF658vMKnfw5QVCExSmBUhQIDAEgGByRLWrkEFJl1MDiJAgMCRAgKTI9uSOimBSSp33WICk7q9M3MCBAgITPbtAYHJPnuVCRAgMCogMBkVrH++wKR+D1NWIDBJYVaEAAECSwQEJktYuwYVmHQxOYgAAQJHCghMjmxL6qQEJqncdYsJTOr2zswJECAgMNm3BwQm++xVJkCAwKiAwGRUsP75ApP6PUxZgcAkhVkRAgQILBEQmCxh7RpUYNLF5CACBAgcKSAwObItqZMSmKRy1y0mMKnbOzMnQICAwGTfHhCY7LNXmQABAqMCApNRwfrnC0zq9zBlBQKTFGZFCBAgsERAYLKEtWtQgUkXk4MIECBwpIDA5Mi2pE5KYJLKXbeYwKRu78ycAAECApN9e0Bgss9eZQIECIwKCExGBeufLzCp38OUFQhMUpgVIUCAwBIBgckS1q5BBSZdTA4iQIDAkQICkyPbkjopgUkqd91iApO6vTNzAgQICEz27QGByT57lQkQIDAqIDAZFax/vsCkfg9TViAwSWFWhAABAksEBCZLWLsGFZh0MTmIAAECRwoITI5sS+qkBCap3HWLCUzq9s7MCRAgIDDZtwcEJvvsVSZAgMCogMBkVLD++QKT+j1MWYHAJIVZEQIECCwREJgsYe0aVGDSxeQgAgQIHCkgMDmyLamTEpikctctNjMw+YEf+IG6EGZOgACBggJf+ZVfOW3WT09Prh0e0JwZmHz+859/oLJDCRAgQGBU4Gu/9mtHh/ji+e/evfP+OU0zbyBNy7MuXUlgUrp9Jk+AwBsXEJjs2wACk332KhMgQGBUQGAyKlj/fIFJ/R6mrEBgksKsCAECBJYICEyWsHYNKjDpYnIQAQIEjhQQmBzZltRJCUxSuesWE5jU7Z2ZEyBAQGCybw8ITPbZq0yAAIFRAYHJqGD98wUm9XuYsgKBSQqzIgQIEFgiIDBZwto1qMCki8lBBAgQOFJAYHJkW1InJTBJ5a5bTGBSt3dmToAAAYHJvj0gMNlnrzIBAgRGBQQmo4L1zxeY1O9hygoEJinMihAgQGCJgMBkCWvXoAKTLiYHESBA4EgBgcmRbUmdlMAklbtuMYFJ3d6ZOQECBAQm+/aAwGSfvcoECBAYFRCYjArWP19gUr+HKSsQmKQwK0KAAIElAgKTJaxdgwpMupgcRIAAgSMFBCZHtiV1UgKTVO66xQQmdXtn5gQIEBCY7NsDApN99ioTIEBgVEBgMipY/3yBSf0epqxAYJLCrAgBAgSWCAhMlrB2DSow6WJyEAECBI4UEJgc2ZbUSQlMUrnrFhOY1O2dmRMgQEBgsm8PCEz22atMgACBUQGByahg/fMFJvV7mLICgUkKsyIECBBYIiAwWcLaNajApIvJQQQIEDhSQGByZFtSJyUwSeWuW0xgUrd3Zk6AAAGByb49IDDZZ68yAQIERgUEJqOC9c8XmNTvYcoKZgYm3//9358yZ0UIECBA4IcEvuqrvmoaxdPTk2uHBzRnBiaf+9znHqjsUAIECBAYFfj0pz89OsQXz3/37p33z2maeQNpWp516UoCk9LtM3kCBN64gMBk3wYQmOyzV5kAAQKjAgKTUcH65wtM6vcwZQUCkxRmRQgQILBEQGCyhLVrUIFJF5ODCBAgcKSAwOTItqROSmCSyl23mMCkbu/MnAABAgKTfXtAYLLPXmUCBAiMCghMRgXrny8wqd/DlBUITFKYFSFAgMASAYHJEtauQQUmXUwOIkCAwJECApMj25I6KYFJKnfdYgKTur0zcwIECAhM9u0Bgck+e5UJECAwKiAwGRWsf77ApH4PU1YgMElhVoQAAQJLBAQmS1i7BhWYdDE5iAABAkcKCEyObEvqpAQmqdx1iwlM6vbOzAkQICAw2bcHBCb77FUmQIDAqIDAZFSw/vkCk/o9TFmBwCSFWRECBAgsERCYLGHtGlRg0sXkIAIECBwpIDA5si2pkxKYpHLXLSYwqds7MydAgIDAZN8eEJjss1eZAAECowICk1HB+ucLTOr3MGUFApMUZkUIECCwREBgsoS1a1CBSReTgwgQIHCkgMDkyLakTkpgkspdt5jApG7vzJwAAQICk317QGCyz15lAgQIjAoITEYF658vMKnfw5QVCExSmBUhQIDAEgGByRLWrkEFJl1MDiJAgMCRAgKTI9uSOimBSSp33WICk7q9M3MCBAgITPbtAYHJPnuVCRAgMCogMBkVrH++wKR+D1NWIDBJYVaEAAECSwQEJktYuwYVmHQxOYgAAQJHCghMjmxL6qQEJqncdYvNDEx+8Ad/sC6EmRMgQKCgwFd8xVdMm/XT05Nrhwc0ZwYm3/M93/NAZYcSIECAwKjA13zN14wO8cXz37175/1zmmbeQJqWZ126ksCkdPtMngCBNy4gMNm3AQQm++xVJkCAwKiAwGRUsP75ApP6PUxZgcAkhVkRAgQILBEQmCxh7RpUYNLF5CACBAgcKSAwObItqZMSmKRy1y0mMKnbOzMnQICAwGTfHhCY7LNXmQABAqMCApNRwfrnC0zq9zBlBQKTFGZFCBAgsERAYLKEtWtQgUkXk4MIECBwpIDA5Mi2pE5KYJLKXbeYwKRu78ycAAECApN9e0Bgss9eZQIECIwKCExGBeufLzCp38OUFQhMUpgVIUCAwBIBgckS1q5BBSZdTA4iQIDAkQICkyPbkjopgUkqd91iApO6vTNzAgQICEz27QGByT57lQkQIDAqIDAZFax/vsCkfg9TViAwSWFWhAABAksEBCZLWLsGFZh0MTmIAAECRwoITI5sS+qkBCap3HWLCUzq9s7MCRAgIDDZtwcEJvvsVSZAgMCogMBkVLD++QKT+j1MWYHAJIVZEQIECCwREJgsYe0aVGDSxeQgAgQIHCkgMDmyLamTEpikctctJjCp2zszJ0CAgMBk3x4QmOyzV5kAAQKjAgKTUcH65wtM6vcwZQUCkxRmRQgQILBEQGCyhLVrUIFJF5ODCBAgcKSAwOTItqROSmCSyl23mMCkbu/MnAABAgKTfXtAYLLPXmUCBAiMCghMRgXrny8wqd/DlBXMDEy++7u/O2XOihAgQIDADwl83dd93TSKp6cn1w4PaM4MTL7927/9gcoOJUCAAIFRgW/6pm8aHeKL579798775zTNvIE0Lc+6dCWBSen2mTwBAm9cQGCybwMITPbZq0yAAIFRAYHJqGD98wUm9XuYsgKBSQqzIgQIEFgiIDBZwto1qMCki8lBBAgQOFJAYHJkW1InJTBJ5a5bTGBSt3dmToAAAYHJvj0gMNlnrzIBAgRGBQQmo4L1zxeY1O9hygoEJinMihAgQGCJgMBkCWvXoAKTLiYHESBA4EgBgcmRbUmdlMAklbtuMYFJ3d6ZOQECBAQm+/aAwGSfvcoECBAYFRCYjArWP19gUr+HKSsQmKQwK0KAAIElAgKTJaxdgwpMupgcRIAAgSMFBCZHtiV1UgKTVO66xQQmdXtn5gQIEBCY7NsDApN99ioTIEBgVEBgMipY/3yBSf0epqxAYJLCrAgBAgSWCAhMlrB2DSow6WJyEAECBI4UEJgc2ZbUSQlMUrnrFhOY1O2dmRMgQEBgsm8PCEz22atMgACBUQGByahg/fMFJvV7mLICgUkKsyIECBBYIiAwWcLaNajApIvJQQQIEDhSQGByZFtSJyUwSeWuW0xgUrd3Zk6AAAGByb49IDDZZ68yAQIERgUEJqOC9c8XmNTvYcoKBCYpzIoQIEBgiYDAZAlr16ACky4mBxEgQOBIAYHJkW1JnZTAJJW7bjGBSd3emTkBAgQEJvv2gMBkn73KBAgQGBUQmIwK1j9fYFK/hykrEJikMCtCgACBJQICkyWsXYMKTLqYHESAAIEjBQQmR7YldVICk1TuusVmBibf+73fWxfCzAkQIFBQ4Ku/+qunzfrp6cm1wwOaMwOT7/iO73igskMJECBAYFTgG7/xG0eH+OL579698/45TTNvIE3Lsy5dSWBSun0mT4DAGxcQmOzbAAKTffYqEyBAYFRAYDIqWP98gUn9HqasQGCSwqwIAQIElggITJawdg0qMOlichABAgSOFBCYHNmW1EkJTFK56xYTmNTtnZkTIEBAYLJvDwhM9tmrTIAAgVEBgcmoYP3zBSb1e5iyAoFJCrMiBAgQWCIgMFnC2jWowKSLyUEECBA4UkBgcmRbUiclMEnlrltMYFK3d2ZOgAABgcm+PSAw2WevMgECBEYFBCajgvXPF5jU72HKCgQmKcyKECBAYImAwGQJa9egApMuJgcRIEDgSAGByZFtSZ2UwCSVu24xgUnd3pk5AQIEBCb79oDAZJ+9ygQIEBgVEJiMCtY/X2BSv4cpKxCYpDArQoAAgSUCApMlrF2DCky6mBxEgACBIwUEJke2JXVSApNU7rrFBCZ1e2fmBAgQEJjs2wMCk332KhMgQGBUQGAyKlj/fIFJ/R6mrEBgksKsCAECBJYICEyWsHYNKjDpYnIQAQIEjhQQmBzZltRJCUxSuesWE5jU7Z2ZEyBAQGCybw8ITPbZq0yAAIFRAYHJqGD98wUm9XuYsgKBSQqzIgQIEFgiIDBZwto1qMCki8lBBAgQOFJAYHJkW1InJTBJ5a5bTGBSt3dmToAAAYHJvj0gMNlnrzIBAgRGBQQmo4L1zxeY1O9hygpmBiZPT08pc1aEAAECBH5I4JNPPplG8fT05NrhAc2Zgcm7d+gfoHcoAQIEhgW+8IUvDI9xHeCdH+LTLDMH8s6bqV24lsCkcPNMnQCBNy8gMNm3BQQm++xVJkCAwKiAwGRUsP75ApP6PUxZgcAkhVkRAgQILBEQmCxh7RpUYNLF5CACBAgcKSAwObItqZMSmKRy1y0mMKnbOzMnQICAwGTfHhCY7LNXmQABAqMCApNRwfrnC0zq9zBlBQKTFGZFCBAgsERAYLKEtWtQgUkXk4MIECBwpIDA5Mi2pE5KYJLKXbeYwKRu78ycAAECApN9e0Bgss9eZQIECIwKCExGBeufLzCp38OUFQhMUpgVIUCAwBIBgckS1q5BBSZdTA4iQIDAkQICkyPbkjopgUkqd91iApO6vTNzAgQICEz27QGByT57lQkQIDAqIDAZFax/vsCkfg9TViAwSWFWhAABAksEBCZLWLsGFZh0MTmIAAECRwoITI5sS+qkBCap3HWLCUzq9s7MCRAgIDDZtwcEJvvsVSZAgMCogMBkVLD++QKT+j1MWYHAJIVZEQIECCwREJgsYe0aVGDSxeQgAgQIHCkgMDmyLamTEpikctctJjCp2zszJ0CAgMBk3x4QmOyzV5kAAQKjAgKTUcH65wtM6vcwZQUCkxRmRQgQILBEQGCyhLVrUIFJF5ODCBAgcKSAwOTItqROSmCSyl23mMCkbu/MnAABAgKTfXtAYLLPXmUCBAiMCghMRgXrny8wqd/DlBUITFKYFSFAgMASAYHJEtauQQUmXUwOIkCAwJECApMj25I6KYFJKnfdYgKTur0zcwIECAhM9u0Bgck+e5UJECAwKiAwGRWsf77ApH4PU1YgMElhVoQAAQJLBAQmS1i7BhWYdDE5iAABAkcKCEyObEvqpAQmqdx1iwlM6vbOzAkQICAw2bcHBCb77FUmQIDAqIDAZFSw/vkCk/o9TFmBwCSFWRECBAgsERCYLGHtGlRg0sXkIAIECBwpIDA5si2pkxKYpHLXLSYwqds7MydAgIDAZN8eEJjss1eZAAECowICk1HB+ucLTOr3MGUFApMUZkUIECCwREBgsoS1a1CBSReTgwgQIHCkgMDkyLakTkpgkspdt5jApG7vzJwAAQICk317QGCyz15lAgQIjAoITEYF658vMKnfw5QVCExSmBUhQIDAEgGByRLWrkEFJl1MDiJAgMCRAgKTI9uSOimBSSp33WICk7q9M3MCBAgITPbtAYHJPnuVCRAgMCogMBkVrH++wKR+D1NWIDBJYVaEAAECSwQEJktYuwYVmHQxOYgAAQJHCghMjmxL6qQEJqncdYsJTOr2zswJECAgMNm3BwQm++xVJkCAwKiAwGRUsP75ApP6PUxZgcAkhVkRAgQILBEQmCxh7RpUYNLF5CACBAgcKSAwObItqZMSmKRy1y0mMKnbOzMnQICAwGTfHhCY7LNXmQABAqMCApNRwfrnC0zq9zBlBQKTFGZFCBAgsERAYLKEtWtQgUkXk4MIECBwpIDA5Mi2pE5KYJLKXbeYwKRu78ycAAECApN9e0Bgss9eZQIECIwKCExGBeufLzCp38OUFQhMUpgVIUCAwBIBgckS1q5BBSZdTA4iQIDAkQICkyPbkjopgUkqd91iApO6vTNzAgQICEz27QGByT57lQkQIDAqIDAZFax/vsCkfg9TViAwSWFWhAABAksEBCZLWLsGFZh0MTmIAAECRwoITI5sS+qkBCap3HWLCUzq9s7MCRAgIDDZtwcEJvvsVSZAgMCogMBkVLD++QKT+j1MWYHAJIVZEQIECCwREJgsYe0aVGDSxeQgAgQIHCkgMDmyLamTEpikctctJjCp2zszJ0CAgMBk3x4QmOyzV5kAAQKjAgKTUcH65wtM6vcwZQUCkxRmRQgQILBEQGCyhLVrUIFJF5ODCBAgcKSAwOTItqROSmCSyl23mMCkbu/MnAABAgKTfXtAYLLPXmUCBAiMCghMRgXrny8wqd/DlBUITFKYFSFAgMASAYHJEtauQQUmXUwOIkCAwJECApMj25I6KYFJKnfdYgKTur0zcwIECAhM9u0Bgck+e5UJECAwKiAwGRWsf77ApH4PU1YgMElhVoQAAQJLBAQmS1i7BhWYdDE5iAABAkcKCEyObEvqpAQmqdx1iwlM6vbOzAkQICAw2bcHBCb77FUmQIDAqIDAZFSw/vkCk/o9TFmBwCSFWRECBAgsERCYLGHtGlRg0sXkIAIECBwpIDA5si2pkxKYpHLXLSYwqds7MydAgIDAZN8eEJjss1eZAAECowICk1HB+ucLTOr3MGUFApMUZkUIECCwREBgsoS1a1CBSReTgwgQIHCkgMDkyLakTkpgkspdt5jApG7vzJwAAQICk317QGCyz15lAgQIjAoITEYF658vMKnfw5QVCExSmBUhQIDAEgGByRLWrkEFJl1MDiJAgMCRAgKTI9uSOimBSSp33WICk7q9M3MCBAgITPbtAYHJPnuVCRAgMCogMBkVrH++wKR+D1NWIDBJYVaEAAECSwQEJktYuwYVmHQxOYgAAQJHCghMjmxL6qQEJqncdYsJTOr2zswJECAgMNm3BwQm++xVJkCAwKiAwGRUsP75ApP6PUxZgcAkhVkRAgQILBEQmCxh7RpUYNLF5CACBAgcKSAwObItqZMSmKRy1y0mMKnbOzMnQICAwGTfHhCY7LNXmQABAqMCApNRwfrnC0zq9zBlBQKTFGZFCBAgsERAYLKEtWtQgUkXk4MIECBwpIDA5Mi2pE5KYJLKXbeYwKRu78ycAAECApN9e0Bgss9eZQIECIwKCExGBeufLzCp38OUFQhMUpgVIUCAwBIBgckS1q5BBSZdTA4iQIDAkQICkyPbkjopgUkqd91iApO6vTNzAgQICEz27QGByT57lQkQIDAqIDAZFax/vsCkfg9TViAwSWFWhAABAksEBCZLWLsGFZh0MTmIAAECRwoITI5sS+qkBCap3HWLCUzq9s7MCRAgIDDZtwcEJvvsVSZAgMCogMBkVLD++QKT+j1MWYHAJIVZEQIECCwREJgsYe0aVGDSxeQgAgQIHCkgMDmyLamTEpikctctJjCp2zszJ0CAgMBk3x4QmOyzV5kAAQKjAgKTUcH65wtM6vcwZQUCkxRmRQgQILBEQGCyhLVrUIFJF5ODCBAgcKSAwOTItqROSmCSyl23mMCkbu/MnAABAgKTfXtAYLLPXmUCBAiMCghMRgXrny8wqd/DlBUITFKYFSFAgMASAYHJEtauQQUmXUwOIkCAwJECApMj25I6KYFJKnfdYgKTur0z5sjCgwAAIABJREFUcwIECAhM9u0Bgck+e5UJECAwKiAwGRWsf77ApH4PU1YgMElhVoQAAQJLBAQmS1i7BhWYdDE5iAABAkcKCEyObEvqpAQmqdx1iwlM6vbOzAkQICAw2bcHBCb77FUmQIDAqIDAZFSw/vkCk/o9TFmBwCSFWRECBAgsERCYLGHtGlRg0sXkIAIECBwpIDA5si2pkxKYpHLXLSYwqds7MydAgIDAZN8eEJjss1eZAAECowICk1HB+ucLTOr3MGUFApMUZkUIECCwREBgsoS1a1CBSReTgwgQIHCkgMDkyLakTkpgkspdt5jApG7vzJwAAQICk317QGCyz15lAgQIjAoITEYF658vMKnfw5QVCExSmBUhQIDAEgGByRLWrkEFJl1MDiJAgMCRAgKTI9uSOimBSSp33WICk7q9M3MCBAgITPbtAYHJPnuVCRAgMCogMBkVrH++wKR+D1NWIDBJYVaEAAECSwQEJktYuwYVmHQxOYgAAQJHCghMjmxL6qQEJqncdYsJTOr2zswJECAgMNm3BwQm++xVJkCAwKiAwGRUsP75ApP6PUxZgcAkhVkRAgQILBEQmCxh7RpUYNLF5CACBAgcKSAwObItqZMSmKRy1y0mMKnbOzMnQICAwGTfHhCY7LNXmQABAqMCApNRwfrnC0zq9zBlBQKTFGZFCBAgsERAYLKEtWtQgUkXk4MIECBwpIDA5Mi2pE5KYJLKXbeYwKRu78ycAAECApN9e0Bgss9eZQIECIwKCExGBeufLzCp38OUFQhMUpgVIUCAwBIBgckS1q5BBSZdTA4iQIDAkQICkyPbkjopgUkqd91iApO6vTNzAgQICEz27QGByT57lQkQIDAqIDAZFax/vsCkfg9TViAwSWFWhAABAksEBCZLWLsGFZh0MTmIAAECRwoITI5sS+qkBCap3HWLCUzq9s7MCRAgIDDZtwcEJvvsVSZAgMCogMBkVLD++QKT+j1MWYHAJIVZEQIECCwREJgsYe0aVGDSxeQgAgQIHCkgMDmyLamTEpikctctJjCp2zszJ0CAgMBk3x4QmOyzV5kAAQKjAgKTUcH65wtM6vcwZQUCkxRmRQgQILBEQGCyhLVrUIFJF5ODCBAgcKSAwOTItqROSmCSyl23mMCkbu/MnAABAgKTfXtAYLLPXmUCBAiMCghMRgXrny8wqd/DlBUITFKYFSFAgMASAYHJEtauQQUmXUwOIkCAwJECApMj25I6KYFJKnfdYgKTur0zcwIECAhM9u0Bgck+e5UJECAwKiAwGRWsf77ApH4PU1YgMElhVoQAAQJLBAQmS1i7BhWYdDE5iAABAkcKCEyObEvqpAQmqdx1iwlM6vbOzAkQICAw2bcHBCb77FUmQIDAqIDAZFSw/vkCk/o9TFmBwCSFWRECBAgsERCYLGHtGlRg0sXkIAIECBwpIDA5si2pkxKYpHLXLSYwqds7MydAgIDAZN8eEJjss1eZAAECowICk1HB+ucLTOr3MGUFApMUZkUIECCwREBgsoS1a1CBSReTgwgQIHCkgMDkyLakTkpgkspdt5jApG7vzJwAAQICk317QGCyz15lAgQIjAoITEYF658vMKnfw5QVCExSmBUhQIDAEgGByRLWrkEFJl1MDiJAgMCRAgKTI9uSOimBSSp33WICk7q9M3MCBAgITPbtAYHJPnuVCRAgMCogMBkVrH++wKR+D1NWIDBJYVaEAAECSwQEJktYuwYVmHQxOYgAAQJHCghMjmxL6qQEJqncdYsJTOr2zswJECAgMNm3BwQm++xVJkCAwKiAwGRUsP75ApP6PUxZgcAkhVkRAgQILBEQmCxh7RpUYNLF5CACBAgcKSAwObItqZMSmKRy1y0mMKnbOzMnQICAwGTfHhCY7LNXmQABAqMCApNRwfrnC0zq9zBlBQKTFGZFCBAgsERAYLKEtWtQgUkXk4MIECBwpIDA5Mi2pE5KYJLKXbeYwKRu78ycAAECApN9e0Bgss9eZQIECIwKCExGBeufLzCp38OUFQhMUpgVIUCAwBIBgckS1q5BBSZdTA4iQIDAkQICkyPbkjopgUkqd91iApO6vTNzAgQICEz27QGByT57lQkQIDAqIDAZFax/vsCkfg9TViAwSWFWhAABAksEBCZLWLsGFZh0MTmIAAECRwoITI5sS+qkBCap3HWLCUzq9s7MCRAgIDDZtwcEJvvsVSZAgMCogMBkVLD++QKT+j1MWYHAJIVZEQIECCwREJgsYe0aVGDSxeQgAgQIHCkgMDmyLamTEpikctctJjCp2zszJ0CAgMBk3x4QmOyzV5kAAQKjAgKTUcH65wtM6vcwZQUCkxRmRQgQILBEQGCyhLVrUIFJF5ODCBAgcKSAwOTItqROSmCSyl23mMCkbu/MnAABAgKTfXtAYLLPXmUCBAiMCghMRgXrny8wqd/DlBUITFKYFSFAgMASAYHJEtauQQUmXUwOIkCAwJECApMj25I6KYFJKnfdYgKTur0zcwIECAhM9u0Bgck+e5UJECAwKiAwGRWsf77ApH4PU1YgMElhVoQAAQJLBAQmS1i7BhWYdDE5iAABAkcKCEyObEvqpAQmqdx1iwlM6vbOzAkQICAw2bcHBCb77FUmQIDAqIDAZFSw/vkCk/o9TFmBwCSFWRECBAgsERCYLGHtGlRg0sXkIAIECBwpIDA5si2pkxKYpHLXLSYwqds7MydAgIDAZN8eEJjss1eZAAECowICk1HB+ucLTOr3MGUFApMUZkUIECCwREBgsoS1a1CBSReTgwgQIHCkgMDkyLakTkpgkspdt5jApG7vzJwAAQICk317QGCyz15lAgQIjAoITEYF658vMKnfw5QVCExSmBUhQIDAEgGByRLWrkEFJl1MDiJAgMCRAgKTI9uSOimBSSp33WICk7q9M3MCBAgITPbtAYHJPnuVCRAgMCogMBkVrH++wKR+D1NWIDBJYVaEAAECSwQEJktYuwYVmHQxOYgAAQJHCghMjmxL6qQEJqncdYsJTOr2zswJECAgMNm3BwQm++xVJkCAwKiAwGRUsP75ApP6PUxZgcAkhblskaf3zPyTsqsxcQKvT0Bgsq+nApN99hUqf8M3fMOXTfM7v/M7K0zdHAm8CQGByZto84uLFJjYA10CApMupjd7kMDkzbbewosICEz2NUpgss++QmWBSYUumeNbFhCYvOXu/9DaBSb2QJeAwKSL6c0eJDB5s6238CICApN9jRKY7LOvUFlgUqFL5viWBQQmb7n7AhPdf0BAYPIA1hs8VGDyBptuyaUEBCb72iUw2WdfobLApEKXzPEtCwhM3nL3BSa6/4CAwOQBrDd4qMDkDTbdkksJCEz2tUtgss++QmWBSYUumeNbFhCYvOXuC0x0/wEBgckDWG/wUIHJG2y6JZcSEJjsa5fAZJ99hcoCkwpdMse3LCAwecvdF5jo/gMCApMHsN7goQKTN9h0Sy4lIDDZ1y6ByT77CpUFJhW6ZI5vWUBg8pa7LzDR/QcEBCYPYL3BQwUmb7DpllxKQGCyr10Ck332FSoLTCp0yRzfsoDA5C13X2Ci+w8ICEwewHqDhwpM3mDTLbmUgMBkX7sEJvvsK1QWmFTokjm+ZQGByVvuvsBE9x8QEJg8gPXaD5WOvPYOW98rFBCY7GuqwGSf/WmVv/mbv/nLpvRt3/Ztp03TfAgQuBEQmNgO7xAQ6BEQmPQovZFjBCZvpNGW+ZoEBCb7uikw2Wd/WmWByWkdMR8CHxYQmHzY6LUfITB57R2etD6BySTI1zCMwOQ1dNEa3piAwGRfwwUm++xPqywwOa0j5kPgwwICkw8bvfYjBCavvcOT1icwmQT5GoYRmLyGLlrDGxMQmOxruMBkn/1plQUmp3XEfAh8WEBg8mGj136EwOS1d3jS+gQmkyBfwzACk9fQRWt4YwICk30NF5jssz+tssDktI6YD4EPCwhMPmz02o8QmLz2Dk9an8BkEmS5Yd6Tjjx98uWreM8/KrdUEybwigUEJvuaKzDZZ7+z8mc/+9kvK/++B7x66OvOLqlN4MMCApMPG732IwQmr73Dk9YnMJkEWW4YgUm5lpkwgfcICEz2bQuByT77nZUFJjv11SYwT0BgMs+y6kgCk6qdS563wCQZ/JhyApNjWmEiBAYEBCYDeIOnCkwGAYueLjAp2jjTJnAnIDCxJQQm9kCXgMCki+kVHiQweYVNtaQ3KCAw2dd0gck++52VBSY79dUmME9AYDLPsupIApOqnUuet8AkGfyYcgKTY1phIgQGBAQmA3iDpwpMBgGLni4wKdo40yZwJyAwsSUEJvZAl4DApIvpFR7kT+K8wqZa0hsUEJjsa7rAZJ/9zsrvC0w+85nP7JyS2gQIfISAwOQj0F7ZKQKTV9bQVcsRmKySPX1cgcnpHTI/Aj0CApMepTXHCEzWuJ4+qsDk9A6ZH4E+AYFJn9NrPkpg8pq7O3FtApOJmKWGEpiUapfJEnhGQGCyb2sITPbZ76wsMNmprzaBeQICk3mWVUcSmFTtXPK8BSbJ4MeUE5gc0woTITAgIDAZwBs8VWAyCFj0dIFJ0caZNoE7AYGJLSEwsQe6BAQmXUyv8CCByStsqiW9QQGByb6mC0z22e+sLDDZqa82gXkCApN5llVHEphU7VzyvAUmyeDKESBAYKKAwGQi5oNDCUweBHM4AQIEDhIQmBzUjE1TEZhsgq9WVmBSrWPmS4AAgZ8nIDDZtxsEJvvsVSZAgMCogMBkVLD++QKT+j1MWYHAJIVZEQIECCwREJgsYe0aVGDSxeQgAgQIHCkgMDmyLamTEpikctctJjCp2zszJ0CAgMBk3x4QmOyzV5kAAQKjAgKTUcH65wtM6vcwZQUCkxRmRQgQILBEQGCyhLVrUIFJF5ODCBAgcKSAwOTItqROSmCSyl23mMCkbu/MnAABAgKTfXtAYLLPXmUCBAiMCghMRgXrny8wqd/DlBUITFKYFSFAgMASAYHJEtauQQUmXUwOIkCAwJECApMj25I6KYFJKnfdYgKTur0zcwIECAhM9u0Bgck+e5UJECAwKiAwGRWsf77ApH4PU1YgMElhVoQAAQJLBAQmS1i7BhWYdDE5iAABAkcKCEyObEvqpAQmqdx1iwlM6vbOzAkQICAw2bcHBCb77FUmQIDAqIDAZFSw/vkCk/o9TFmBwCSFWRECBAgsERCYLGHtGlRg0sXkIAIECBwpIDA5si2pkxKYpHLXLSYwqds7MydAgIDAZN8eEJjss1eZAAECowICk1HB+ucLTOr3MGUFApMUZkUIECCwREBgsoS1a1CBSReTgwgQIHCkgMDkyLakTkpgkspdt5jApG7vzJwAAQICk317QGCyz15lAgQIjAoITEYF658vMKnfw5QVCExSmBUhQIDAEgGByRLWrkEFJl1MDiJAgMCRAgKTI9uSOimBSSp33WICk7q9M3MCBAgITPbtAYHJPnuVCRAgMCogMBkVrH++wKR+D1NWIDBJYVaEAAECSwQEJktYuwYVmHQxOYgAAQJHCghMjmxL6qQEJqncdYsJTOr2zswJECAgMNm3BwQm++xVJkCAwKiAwGRUsP75ApP6PUxZgcAkhVkRAgQILBEQmCxh7RpUYNLF5CACBAgcKSAwObItqZMSmKRy1y0mMKnbOzMnQICAwGTfHhCY7LNXmQABAqMCApNRwfrnC0zq9zBlBQKTFGZFCBAgsERAYLKEtWtQgUkXk4MIECBwpIDA5Mi2pE5KYJLKXbeYwKRu78ycAAECApN9e0Bgss9eZQIECIwKCExGBeufLzCp38OUFQhMUpgVIUCAwBIBgckS1q5BBSZdTA4iQIDAkQICkyPbkjopgUkqd91iApO6vTNzAgQICEz27QGByT57lQkQIDAqIDAZFax/vsCkfg9TViAwSWFWhAABAksEBCZLWLsGFZh0MTmIAAECRwoITI5sS+qkBCap3HWLCUzq9s7MCRAgIDDZtwcEJvvsVSZAgMCogMBkVLD++QKT+j1MWYHAJIVZEQIECCwREJgsYe0aVGDSxeQgAgQIHCkgMDmyLamTEpikctctJjCp2zszJ0CAgMBk3x4QmOyzV5kAAQKjAgKTUcH65wtM6vcwZQUCkxRmRQgQILBEQGCyhLVrUIFJF5ODCBAgcKSAwOTItqROSmCSyl23mMCkbu/MnAABAgKTfXtAYLLPXmUCBAiMCghMRgXrny8wqd/DlBUITFKYFSFAgMASAYHJEtauQQUmXUwOIkCAwJECApMj25I6KYFJKnfdYgKTur0zcwIECAhM9u0Bgck+e5UJECAwKiAwGRWsf77ApH4PU1YgMElhVoQAAQJLBAQmS1i7BhWYdDE5iAABAkcKCEyObEvqpAQmqdx1iwlM6vbOzAkQICAw2bcHBCb77FUmQIDAqIDAZFSw/vkCk/o9TFmBwCSFWRECBAgsERCYLGHtGlRg0sXkIAIECBwpIDA5si2pkxKYpHLXLSYwqds7MydAgIDAZN8eEJjss1eZAAECowICk1HB+ucLTOr3MGUFApMUZkUIECCwREBgsoS1a1CBSReTgwgQIHCkgMDkyLakTkpgkspdt5jApG7vzJwAAQICk317QGCyz15lAgQIjAoITEYF658vMKnfw5QVCExSmBUhQIDAEgGByRLWrkEFJl1MDiJAgMCRAgKTI9uSOimBSSp33WICk7q9M3MCBAgITPbtAYHJPnuVCRAgMCogMBkVrH++wKR+D1NWIDBJYVaEAAECSwQEJktYuwYVmHQxOYgAAQJHCghMjmxL6qQEJqncdYsJTOr2zswJECAgMNm3BwQm++xVJkCAwKiAwGRUsP75ApP6PUxZgcAkhVkRAgQILBEQmCxh7RpUYNLF5CACBAgcKSAwObItqZMSmKRy1y0mMKnbOzMnQICAwGTfHhCY7LNXmQABAqMCApNRwfrnC0zq9zBlBQKTFGZFCBAgsERAYLKEtWtQgUkXk4MIECBwpIDA5Mi2pE5KYJLKXbeYwKRu78ycAAECApN9e0Bgss9eZQIECIwKCExGBeufLzCp38OUFQhMUpgVIUCAwBIBgckS1q5BBSZdTA4iQIDAkQICkyPbkjopgUkqd91iApO6vTNzAgQICEz27QGByT57lQkQIDAqIDAZFax/vsCkfg9TViAwSWFWhAABAksEBCZLWLsGFZh0MTmIAAECRwoITI5sS+qkBCap3HWLCUzq9s7MCRAgIDDZtwcEJvvsVSZAgMCogMBkVLD++QKT+j1MWYHAJIVZEQIECCwREJgsYe0aVGDSxeQgAgQIHCkgMDmyLamTEpikctctJjCp2zszJ0CAgMBk3x4QmOyzV5kAAQKjAgKTUcH65wtM6vcwZQUCkxRmRQgQILBEQGCyhLVrUIFJF5ODCBAgcKSAwOTItqROSmCSyl23mMCkbu/MnAABAgKTfXtAYLLPXmUCBAiMCghMRgXrny8wqd/DlBUITFKYFSFAgMASAYHJEtauQQUmXUwOIkCAwJECApMj25I6KYFJKnfdYgKTur0zcwIECAhM9u0Bgck+e5UJECAwKiAwGRWsf77ApH4PU1YgMElhVoQAAQJLBAQmS1i7BhWYdDE5iAABAkcKCEyObEvqpAQmqdx1iwlM6vbOzAkQICAw2bcHBCb77FUmQIDAqIDAZFSw/vkCk/o9TFmBwCSFWRECBAgsERCYLGHtGlRg0sXkIAIECBwpIDA5si2pkxKYpHLXLSYwqds7MydAgIDAZN8eEJjss1eZAAECowICk1HB+ucLTOr3MGUFApMUZkUIECCwREBgsoS1a1CBSReTgwgQIHCkgMDkyLakTkpgkspdt5jApG7vzJwAAQICk317QGCyz15lAgQIjAoITEYF658vMKnfw5QVCExSmBUhQIDAEgGByRLWrkEFJl1MDiJAgMCRAgKTI9uSOimBSSp33WICk7q9M3MCBAgITPbtAYHJPnuVCRAgMCogMBkVrH++wKR+D1NWIDBJYVaEAAECSwQEJktYuwYVmHQxOYgAAQJHCghMjmxL6qQEJqncdYsJTOr2zswJECAgMNm3BwQm++xVJkCAwKiAwGRUsP75ApP6PUxZgcAkhVkRAgQILBEQmCxh7RpUYNLF5CACBAgcKSAwObItqZMSmKRy1y0mMKnbOzMnQICAwGTfHhCY7LNXmQABAqMCApNRwfrnC0zq9zBlBQKTFGZFCBAgsERAYLKEtWtQgUkXk4MIECBwpIDA5Mi2pE5KYJLKXbeYwKRu78ycAAECApN9e0Bgss9eZQIECIwKCExGBeufLzCp38OUFQhMUpgVIUCAwBIBgckS1q5BBSZdTA4iQIDAkQICkyPbkjopgUkqd91iApO6vTNzAgQICEz27QGByT57lQkQIDAqIDAZFax/vsCkfg9TViAwSWFWhAABAksEBCZLWLsGFZh0MTmIAAECRwoITI5sS+qkBCap3HWLCUzq9s7MCRAgIDDZtwcEJvvsVSZAgMCogMBkVLD++QKT+j1MWYHAJIVZEQIECCwREJgsYe0aVGDSxeQgAgQIHCkgMDmyLamTEpikctctJjCp2zszJ0CAgMBk3x4QmOyzV5kAAQKjAgKTUcH65wtM6vcwZQUCkxRmRQgQILBEQGCyhLVrUIFJF5ODCBAgcKSAwOTItqROSmCSyl23mMCkbu/MnAABAgKTfXtAYLLPXmUCBAiMCghMRgXrny8wqd/DlBUITFKYFSFAgMASAYHJEtauQQUmXUwOIkCAwJECApMj25I6KYFJKnfdYgKTur0zcwIECAhM9u0Bgck+e5UJECAwKiAwGRWsf77ApH4PU1YgMElhVoQAAQJLBAQmS1i7BhWYdDE5iAABAkcKCEyObEvqpAQmqdx1iwlM6vbOzAkQICAw2bcHBCb77FUmQIDAqIDAZFSw/vkCk/o9TFmBwCSFWRECBAgsERCYLGHtGlRg0sXkIAIECBwpIDA5si2pkxKYpHLXLSYwqds7MydAgIDAZN8eEJjss1eZAAECowICk1HB+ucLTOr3MGUFApMUZkUIECCwREBgsoS1a1CBSReTgwgQIHCkgMDkyLakTkpgkspdt5jApG7vzJwAAQICk317QGCyz15lAgQIjAoITEYF658vMKnfw5QVCExSmBUhQIDAEgGByRLWrkEFJl1MDiJAgMCRAgKTI9uSOimBSSp33WLf8i3f8oVZs//Wb/3WWUMZhwCBBwSenp4eODr30Jm/0OfOvEa1mb3/5JNPXDs80PbPfe5z094/P/3pTz9Q2aEECMwS+OxnPztrqOnjfOYzn5k+pgF/nsDM3n/mM5/x/llwc2lawabtmLLAZIe6mgTmCsz8pXnuzFoTmMwW/dLxZvZeYPJYrwQmj3k5msCJAjN/aZ69PoHJbNEvHW9m7wUma3u1anSBySrZVzauwOSVNdRy3qTAzF+aZwMKTGaLCkzWivaPLjDpt3IkgVMFZv7SPHuNApPZogKTtaL1RheY1OvZlhkLTLawK0pgqoDAZCpnqcFm9t4dJo+1XmDymJejCZwoIDA5sSs5c5rZe3eY5PRsdhWByWzRVzqewOSVNtay3pTAzF+aZ8O5w2S26JeON7P3ApPHeiUweczL0QROFJj5S/Ps9bnDZLbol443s/cCk7W9WjW6wGSV7CsbV2DyyhpqOW9SYOYvzbMBBSazRQUma0X7RxeY9Fs5ksCpAjN/aZ69RoHJbFGByVrReqMLTOr1bMuMBSZb2BUlMFVAYDKVs9RgM3vvDpPHWi8weczL0QROFBCYnNiVnDnN7L07THJ6NruKwGS26CsdT2DyShtrWW9KYOYvzbPh3GEyW/RLx5vZe4HJY70SmDzm5WgCJwrM/KV59vrcYTJb9EvHm9l7gcnaXq0aXWCySvaVjSsweWUNtZw3KTDzl+bZgAKT2aICk7Wi/aMLTPqtHEngVIGZvzTPXqPAZLaowGStaL3RBSb1erZlxgKTLeyKEpgqIDCZyllqsJm9d4fJY60XmDzm5WgCJwoITE7sSs6cZvbeHSY5PZtdRWAyW9R4BAgQIECAAAECBAgQIECAQHkBgUn5FloAAQIECBAgQIAAAQIECBAgMFtAYDJb1HgECBAgQIAAAQIECBAgQIBAeQGBSfkWWgABAgQIECBAgAABAgQIECAwW0BgMlvUeAQIECBAgAABAgQIECBAgEB5AYFJ+RZaAAECBAgQIECAAAECBAgQIDBbQGAyW9R4BAgQIECAAAECBAgQIECAQHkBgUn5FloAAQIECBAgQIAAAQIECBAgMFtAYDJb1HgECBAgQIAAAQIECBAgQIBAeQGBSfkWWgABAgQIECBAgAABAgQIECAwW0BgMlvUeAQIECBAgAABAgQIECBAgEB5AYFJ+RZaAAECBAgQIECAAAECBAgQIDBbQGAyW9R4BAgQIECAAAECBAgQIECAQHkBgUn5FloAAQIECBAgQIAAAQIECBAgMFtAYDJb1HgECBAgQIAAAQIECBAgQIBAeQGBSfkWWgABAgQIECBAgAABAgQIECAwW0BgMlvUeAQIECBAgAABAgQIECBAgEB5AYFJ+RZaAAECBAgQIECAAAECBAgQIDBbQGAyW9R4BAgQIECAAAECBAgQIECAQHkBgUn5FloAAQIECBAgQIAAAQIECBAgMFtAYDJb1HgECBAgQIAAAQIECBAgQIBAeQGBSfkWWgABAgQIECBAgAABAgQIECAwW0BgMlvUeAQIECBAgAABAgQIECBAgEB5AYFJ+RZaAAECBAgQIECAAAECBAgQIDBbQGAyW9R4BAgQIECAAAECBAgQIECAQHkBgUn5FloAAQIECBAgQIAAAQIECBAgMFtAYDJb1HgECBAgQIAAAQIECBAgQIBAeQGBSfkWWgABAgQIECBAgAABAgQIECCUazBfAAALhklEQVQwW0BgMlvUeAQIECBAgAABAgQIECBAgEB5AYFJ+RZaAAECBAgQIECAAAECBAgQIDBbQGAyW9R4BAgQIECAAAECBAgQIECAQHkBgUn5FloAAQIECBAgQIAAAQIECBAgMFtAYDJb1HgECBAgQIAAAQIECBAgQIBAeQGBSfkWWgABAgQIECBAgAABAgQIECAwW0BgMlvUeAQIECBAgAABAgQIECBAgEB5AYFJ+RZaAAECBAgQIECAAAECBAgQIDBbQGAyW9R4BAgQIECAAAECBAgQIECAQHkBgUn5FloAAQIECBAgQIAAAQIECBAgMFtAYDJb1HgECBAgQIAAAQIECBAgQIBAeQGBSfkWWgABAgQIECBAgAABAgQIECAwW0BgMlvUeAQIECBAgAABAgQIECBAgEB5AYFJ+RZaAAECBAgQIECAAAECBAgQIDBbQGAyW9R4BAgQIECAAAECBAgQIECAQHkBgUn5FloAAQIECBAgQIAAAQIECBAgMFtAYDJb1HgECBAgQIAAAQIECBAgQIBAeQGBSfkWWgABAgQIECBAgAABAgQIECAwW0BgMlvUeAQIECBAgAABAgQIECBAgEB5AYFJ+RZaAAECBAgQIECAAAECBAgQIDBbQGAyW9R4BAgQIECAAAECBAgQIECAQHkBgUn5FloAAQIECBAgQIAAAQIECBAgMFtAYDJb1HgECBAgQIAAAQIECBAgQIBAeQGBSfkWWgABAgQIECBAgAABAgQIECAwW0BgMlvUeAQIECBAgAABAgQIECBAgEB5AYFJ+RZaAAECBAgQIECAAAECBAgQIDBbQGAyW9R4BAgQIECAAAECBAgQIECAQHkBgUn5FloAAQIECBAgQIAAAQIECBAgMFtAYDJb1HgECBAgQIAAAQIECBAgQIBAeQGBSfkWWgABAgQIECBAgAABAgQIECAwW0BgMlvUeAQIECBAgAABAgQIECBAgEB5AYFJ+RZaAAECBAgQIECAAAECBAgQIDBbQGAyW9R4BAgQIECAAAECBAgQIECAQHkBgUn5FloAAQIECBAgQIAAAQIECBAgMFtAYDJb1HgECBAgQIAAAQIECBAgQIBAeQGBSfkWWgABAgQIECBAgAABAgQIECAwW0BgMlvUeAQIECBAgAABAgQIECBAgEB5AYFJ+RZaAAECBAgQIECAAAECBAgQIDBbQGAyW9R4BAgQIECAAAECBAgQIECAQHkBgUn5FloAAQIECBAgQIAAAQIECBAgMFtAYDJb1HgECBAgQIAAAQIECBAgQIBAeQGBSfkWWgABAgQIECBAgAABAgQIECAwW0BgMlvUeAQIECBAgAABAgQIECBAgEB5AYFJ+RZaAAECBAgQIECAAAECBAgQIDBbQGAyW9R4BAgQIECAAAECBAgQIECAQHkBgUn5FloAAQIECBAgQIAAAQIECBAgMFtAYDJb1HgECBAgQIAAAQIECBAgQIBAeQGBSfkWWgABAgQIECBAgAABAgQIECAwW0BgMlvUeAQIECBAgAABAgQIECBAgEB5AYFJ+RZaAAECBAgQIECAAAECBAgQIDBbQGAyW9R4BAgQIECAAAECBAgQIECAQHkBgUn5FloAAQIECBAgQIAAAQIECBAgMFtAYDJb1HgECBAgQIAAAQIECBAgQIBAeQGBSfkWWgABAgQIECBAgAABAgQIECAwW0BgMlvUeAQIECBAgAABAgQIECBAgEB5AYFJ+RZaAAECBAgQIECAAAECBAgQIDBbQGAyW9R4BAgQIECAAAECBAgQIECAQHkBgUn5FloAAQIECBAgQIAAAQIECBAgMFtAYDJb1HgECBAgQIAAAQIECBAgQIBAeQGBSfkWWgABAgQIECBAgAABAgQIECAwW0BgMlvUeAQIECBAgAABAgQIECBAgEB5AYFJ+RZaAAECBAgQIECAAAECBAgQIDBbQGAyW9R4BAgQIECAAAECBAgQIECAQHkBgUn5FloAAQIECBAgQIAAAQIECBAgMFtAYDJb1HgECBAgQIAAAQIECBAgQIBAeQGBSfkWWgABAgQIECBAgAABAgQIECAwW0BgMlvUeAQIECBAgAABAgQIECBAgEB5AYFJ+RZaAAECBAgQIECAAAECBAgQIDBbQGAyW9R4BAgQIECAAAECBAgQIECAQHkBgUn5FloAAQIECBAgQIAAAQIECBAgMFtAYDJb1HgECBAgQIAAAQIECBAgQIBAeQGBSfkWWgABAgQIECBAgAABAgQIECAwW0BgMlvUeAQIECBAgAABAgQIECBAgEB5AYFJ+RZaAAECBAgQIECAAAECBAgQIDBbQGAyW9R4BAgQIECAAAECBAgQIECAQHkBgUn5FloAAQIECBAgQIAAAQIECBAgMFtAYDJb1HgECBAgQIAAAQIECBAgQIBAeQGBSfkWWgABAgQIECBAgAABAgQIECAwW0BgMlvUeAQIECBAgAABAgQIECBAgEB5AYFJ+RZaAAECBAgQIECAAAECBAgQIDBbQGAyW9R4BAgQIECAAAECBAgQIECAQHkBgUn5FloAAQIECBAgQIAAAQIECBAgMFtAYDJb1HgECBAgQIAAAQIECBAgQIBAeQGBSfkWWgABAgQIECBAgAABAgQIECAwW0BgMlvUeAQIECBAgAABAgQIECBAgEB5AYFJ+RZaAAECBAgQIECAAAECBAgQIDBbQGAyW9R4BAgQIECAAAECBAgQIECAQHkBgUn5FloAAQIECBAgQIAAAQIECBAgMFtAYDJb1HgECBAgQIAAAQIECBAgQIBAeQGBSfkWWgABAgQIECBAgAABAgQIECAwW0BgMlvUeAQIECBAgAABAgQIECBAgEB5AYFJ+RZaAAECBAgQIECAAAECBAgQIDBbQGAyW9R4BAgQIECAAAECBAgQIECAQHkBgUn5FloAAQIECBAgQIAAAQIECBAgMFtAYDJb1HgECBAgQIAAAQIECBAgQIBAeQGBSfkWWgABAgQIECBAgAABAgQIECAwW0BgMlvUeAQIECBAgAABAgQIECBAgEB5AYFJ+RZaAAECBAgQIECAAAECBAgQIDBbQGAyW9R4BAgQIECAAAECBAgQIECAQHkBgUn5FloAAQIECBAgQIAAAQIECBAgMFtAYDJb1HgECBAgQIAAAQIECBAgQIBAeQGBSfkWWgABAgQIECBAgAABAgQIECAwW0BgMlvUeAQIECBAgAABAgQIECBAgEB5AYFJ+RZaAAECBAgQIECAAAECBAgQIDBbQGAyW9R4BAgQIECAAAECBAgQIECAQHkBgUn5FloAAQIECBAgQIAAAQIECBAgMFtAYDJb1HgECBAgQIAAAQIECBAgQIBAeQGBSfkWWgABAgQIECBAgAABAgQIECAwW0BgMlvUeAQIECBAgAABAgQIECBAgEB5AYFJ+RZaAAECBAgQIECAAAECBAgQIDBbQGAyW9R4BAgQIECAAAECBAgQIECAQHkBgUn5FloAAQIECBAgQIAAAQIECBAgMFtAYDJb1HgECBAgQIAAAQIECBAgQIBAeQGBSfkWWgABAgQIECBAgAABAgQIECAwW0BgMlvUeAQIECBAgAABAgQIECBAgEB5AYFJ+RZaAAECBAgQIECAAAECBAgQIDBbQGAyW9R4BAgQIECAAAECBAgQIECAQHkBgUn5FloAAQIECBAgQIAAAQIECBAgMFtAYDJb1HgECBAgQIAAAQIECBAgQIBAeQGBSfkWWgABAgQIECBAgAABAgQIECAwW0BgMlvUeAQIECBAgAABAgQIECBAgEB5AYFJ+RZaAAECBAgQIECAAAECBAgQIDBbQGAyW9R4BAgQIECAAAECBAgQIECAQHkBgUn5FloAAQIECBAgQIAAAQIECBAgMFtAYDJb1HgECBAgQIAAAQIECBAgQOD/22gIDPkQAABC8KBDZvT0SgAAAABJRU5ErkJggg==\" width=\"1100\">"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "img1 = combine_observations_multichannel(preprocessed_observations)\n",
    "img2 = combine_observations_singlechannel(preprocessed_observations)\n",
    "\n",
    "plt.figure(figsize=(11, 7))\n",
    "plt.subplot(121)\n",
    "plt.title(\"Multichannel state\")\n",
    "plt.imshow(img1, interpolation=\"nearest\")\n",
    "plt.axis(\"off\")\n",
    "plt.subplot(122)\n",
    "plt.title(\"Singlechannel state\")\n",
    "plt.imshow(img2, interpolation=\"nearest\", cmap=\"gray\")\n",
    "plt.axis(\"off\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Exercise solutions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Coming soon..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  },
  "nav_menu": {},
  "toc": {
   "navigate_menu": true,
   "number_sections": true,
   "sideBar": true,
   "threshold": 6,
   "toc_cell": false,
   "toc_section_display": "block",
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}