doc/Bling Fire Tokenizer Demo.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Bling Fire Tokenizer Demo\n",
    "In this notebook we illustrate how to use Bling Fire tokenizer. We build a simple token-based classifier for Stack Overflow classification set and measure accuracy of the classifier with Bling Fire tokenizer vs the built in.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Basic Usage"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "All tokens:  ['In', 'order', 'to', 'login', 'to', 'Café', 'use', 'pi', '@', '1', '.', '2', '.', '1', '.', '2', '.', 'Split', 'the', 'data', 'into', 'train', '/', 'test', 'with', 'a', 'test', 'size', 'of', '20', '%', 'then', 'use', 'recurrent', 'model', '(', 'use', 'LSTM', 'or', 'GRU', ')', '.']\n",
      "Sentence text: In order to login to Café use pi@1.2.1.2.\n",
      "Tokenized sentence: In order to login to Café use pi @ 1 . 2 . 1 . 2 .\n",
      "Sentence text: Split the data into train/test with a test size of 20% then use recurrent model (use LSTM or GRU).\n",
      "Tokenized sentence: Split the data into train / test with a test size of 20 % then use recurrent model ( use LSTM or GRU ) .\n"
     ]
    }
   ],
   "source": [
    "from blingfire import *\n",
    "\n",
    "text = \"In order to login to Café use pi@1.2.1.2. Split the data into train/test with a test size of 20% then use recurrent model (use LSTM or GRU).\"\n",
    "\n",
    "# tokenize text without sentence boundaries\n",
    "ws = text_to_words(text).split(' ')\n",
    "print(\"All tokens: \", ws)\n",
    "\n",
    "# first break text to sentences and then break each sentence to words\n",
    "sents = text_to_sentences(text).split('\\n')\n",
    "for sent in sents:\n",
    "    print(\"Sentence text: \" + sent)\n",
    "    print(\"Tokenized sentence: \" + text_to_words(sent))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Get the Data Set Ready"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Download the Data Set, if needed"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import requests\n",
    "import os.path\n",
    "\n",
    "def download_file(filename, url):\n",
    "    \"\"\"\n",
    "    Download an URL to a file\n",
    "    \"\"\"\n",
    "    with open(filename, 'wb') as fout:\n",
    "        response = requests.get(url, stream=True)\n",
    "        response.raise_for_status()\n",
    "        # Write response data to file\n",
    "        for block in response.iter_content(4096):\n",
    "            fout.write(block)\n",
    "\n",
    "def download_if_not_exists(filename, url):\n",
    "    \"\"\"\n",
    "    Download a URL to a file if the file\n",
    "    does not exist already.\n",
    "    Returns\n",
    "    -------\n",
    "    True if the file was downloaded,\n",
    "    False if it already existed\n",
    "    \"\"\"\n",
    "    if not os.path.exists(filename):\n",
    "        download_file(filename, url)\n",
    "        return True\n",
    "    return False"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "INPUT_DATA = './stack-overflow-data.csv'\n",
    "url = 'https://storage.googleapis.com/tensorflow-workshop-examples/stack-overflow-data.csv'\n",
    "download_if_not_exists(filename, url)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "-rwxrwxrwx 1 root root 44319561 Mar 28 21:49 stack-overflow-data.csv\r\n"
     ]
    }
   ],
   "source": [
    "!ls -l stack-overflow-data.csv"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Read the data with Pandas"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 39999 entries, 0 to 39998\n",
      "Data columns (total 2 columns):\n",
      "post    39999 non-null object\n",
      "tags    39999 non-null object\n",
      "dtypes: object(2)\n",
      "memory usage: 625.1+ KB\n"
     ]
    }
   ],
   "source": [
    "import pandas as pd\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.feature_extraction.text import TfidfVectorizer\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "\n",
    "CSV_COLUMN_NAMES = [\"post\", \"tags\"]\n",
    "TEST_SIZE = 0.2\n",
    "\n",
    "# Parse the local CSV file.\n",
    "all_data = pd.read_csv(filepath_or_buffer=INPUT_DATA,\n",
    "                    names=CSV_COLUMN_NAMES,  # list of column names\n",
    "                    header=1  # ignore the first row of the CSV file.\n",
    "                    )\n",
    "all_data.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Split into the Train and Test"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train data length 31999\n",
      "Train labels length 31999\n"
     ]
    }
   ],
   "source": [
    "train, test = train_test_split(all_data, test_size=TEST_SIZE)\n",
    "\n",
    "train_x, train_y = train.pop('post'), train.pop('tags')\n",
    "test_x, test_y = test.pop('post'), test.pop('tags')\n",
    "\n",
    "all_texts = all_data.pop('post')\n",
    "\n",
    "print('Train data length ' + str(len(train_x)))\n",
    "print('Train labels length ' + str(len(train_y)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "31866    how to multiply a number after each 60 minute ...\n",
      "12131    priorityqueue returning incorrect ordering for...\n",
      "35742    how can i create a method so that the below co...\n",
      "22475    long.tryparse and negative values  i m writing...\n",
      "24315    how to add space between two divs in html   i ...\n",
      "Name: post, dtype: object\n",
      "31866    javascript\n",
      "12131          java\n",
      "35742            c#\n",
      "22475            c#\n",
      "24315          html\n",
      "Name: tags, dtype: object\n"
     ]
    }
   ],
   "source": [
    "print(train_x.head())\n",
    "print(train_y.head())\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "def texts2ids(vocab, series):\n",
    "    new_series = []\n",
    "    for line in series:\n",
    "        word = line.strip()\n",
    "        if not (word in vocab):\n",
    "            vocab[word] = len(vocab)\n",
    "        new_series.append(vocab[word])\n",
    "    return new_series"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'javascript': 0, 'objective-c': 8, 'c': 16, 'asp.net': 6, 'jquery': 19, 'ios': 14, 'java': 1, 'python': 13, 'mysql': 15, 'c++': 4, 'angularjs': 18, 'iphone': 9, 'c#': 2, 'css': 11, 'html': 3, 'sql': 12, 'android': 17, '.net': 10, 'ruby-on-rails': 7, 'php': 5}\n",
      "[0, 1, 2, 2, 3, 4, 5, 6, 5, 7, 8, 3, 5, 7, 9, 10, 9, 11, 5, 9, 12, 5, 9, 8, 3, 9, 13, 3, 10, 14, 0, 9, 15, 5, 1, 6, 9, 6, 9, 11, 9, 14, 7, 8, 4, 16, 7, 14, 2, 5, 10, 13, 17, 0, 10, 10, 18, 11, 11, 8, 5, 2, 3, 8, 16, 4, 6, 7, 13, 9, 19, 10, 1, 11, 16, 1, 4, 13, 8, 16, 2, 1, 8, 6, 7, 4, 14, 5, 14, 0, 2, 3, 5, 14, 7, 6, 16, 18, 17, 6]\n",
      "{'javascript': 0, 'objective-c': 8, 'c': 16, 'asp.net': 6, 'jquery': 19, 'ios': 14, 'java': 1, 'python': 13, 'mysql': 15, 'c++': 4, 'angularjs': 18, 'iphone': 9, 'c#': 2, 'css': 11, 'html': 3, 'sql': 12, 'android': 17, '.net': 10, 'ruby-on-rails': 7, 'php': 5}\n",
      "[0, 16, 8, 19, 18, 0, 19, 11, 18, 0, 2, 6, 4, 14, 6, 13, 6, 1, 1, 17, 7, 3, 1, 5, 0, 0, 5, 13, 10, 1, 10, 5, 17, 15, 7, 12, 9, 6, 8, 16, 17, 11, 2, 19, 0, 11, 3, 3, 16, 19, 14, 11, 17, 2, 0, 14, 18, 15, 15, 7, 5, 11, 12, 8, 2, 18, 12, 5, 17, 8, 2, 6, 9, 17, 3, 5, 2, 1, 14, 1, 0, 3, 10, 3, 1, 12, 0, 13, 17, 15, 2, 1, 5, 16, 13, 19, 13, 6, 8, 17]\n"
     ]
    }
   ],
   "source": [
    "label_vocab = {}\n",
    "\n",
    "train_y_ids = texts2ids(label_vocab, train_y)\n",
    "print(label_vocab)\n",
    "print(train_y_ids[0:100])\n",
    "\n",
    "test_y_ids = texts2ids(label_vocab, test_y)\n",
    "print(label_vocab)\n",
    "print(test_y_ids[0:100])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## First Let's use built in tokenizer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "# we will use all text to collect top 1000 most frequent words\n",
    "vectorizer_all = TfidfVectorizer(max_features=1000)\n",
    "vectorizer_all.fit_transform(all_texts)\n",
    "\n",
    "# we will use this vectorizer with precomputed vocabulary of 1000 words\n",
    "vectorizer = TfidfVectorizer(vocabulary=vectorizer_all.vocabulary_)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "20\n"
     ]
    }
   ],
   "source": [
    "train_x_tfidf = vectorizer.fit_transform(train_x).todense()\n",
    "test_x_tfidf = vectorizer.fit_transform(test_x).todense()\n",
    "\n",
    "import numpy as np\n",
    "from sklearn import preprocessing\n",
    "\n",
    "train_y_onehot = preprocessing.OneHotEncoder().fit_transform(np.array(train_y_ids).reshape(-1,1)).toarray()\n",
    "test_y_onehot = preprocessing.OneHotEncoder().fit_transform(np.array(test_y_ids).reshape(-1,1)).toarray()\n",
    "\n",
    "print(train_y_onehot.shape[1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "dense (Dense)                (None, 1000)              1001000   \n",
      "_________________________________________________________________\n",
      "batch_normalization (BatchNo (None, 1000)              4000      \n",
      "_________________________________________________________________\n",
      "activation (Activation)      (None, 1000)              0         \n",
      "_________________________________________________________________\n",
      "dropout (Dropout)            (None, 1000)              0         \n",
      "_________________________________________________________________\n",
      "dense_1 (Dense)              (None, 100)               100100    \n",
      "_________________________________________________________________\n",
      "batch_normalization_1 (Batch (None, 100)               400       \n",
      "_________________________________________________________________\n",
      "activation_1 (Activation)    (None, 100)               0         \n",
      "_________________________________________________________________\n",
      "dropout_1 (Dropout)          (None, 100)               0         \n",
      "_________________________________________________________________\n",
      "dense_2 (Dense)              (None, 20)                2020      \n",
      "=================================================================\n",
      "Total params: 1,107,520\n",
      "Trainable params: 1,105,320\n",
      "Non-trainable params: 2,200\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "import tensorflow as tf\n",
    "\n",
    "C = tf.feature_column\n",
    "E = tf.estimator\n",
    "D = tf.data\n",
    "L = tf.keras.layers\n",
    "\n",
    "model = tf.keras.models.Sequential()\n",
    "model.add(L.Dense(1000, input_shape=train_x_tfidf.shape[1:]))\n",
    "model.add(L.BatchNormalization())\n",
    "model.add(L.Activation('tanh'))\n",
    "model.add(L.Dropout(0.5))\n",
    "model.add(L.Dense(100))\n",
    "model.add(L.BatchNormalization())\n",
    "model.add(L.Activation('tanh'))\n",
    "model.add(L.Dropout(0.5))\n",
    "model.add(L.Dense(train_y_onehot.shape[1], activation='softmax'))\n",
    "model.compile(tf.keras.optimizers.Adam(lr=0.0001), 'categorical_crossentropy', metrics=['accuracy'])\n",
    "model.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 28799 samples, validate on 3200 samples\n",
      "Epoch 1/8\n",
      "28799/28799 [==============================] - 13s 439us/step - loss: 1.9058 - acc: 0.4548 - val_loss: 0.9705 - val_acc: 0.7406\n",
      "Epoch 2/8\n",
      "28799/28799 [==============================] - 4s 146us/step - loss: 1.1049 - acc: 0.6775 - val_loss: 0.8033 - val_acc: 0.7712\n",
      "Epoch 3/8\n",
      "28799/28799 [==============================] - 4s 148us/step - loss: 0.9139 - acc: 0.7304 - val_loss: 0.7512 - val_acc: 0.7834\n",
      "Epoch 4/8\n",
      "28799/28799 [==============================] - 4s 148us/step - loss: 0.8149 - acc: 0.7575 - val_loss: 0.7232 - val_acc: 0.7869\n",
      "Epoch 5/8\n",
      "28799/28799 [==============================] - 4s 147us/step - loss: 0.7544 - acc: 0.7737 - val_loss: 0.7097 - val_acc: 0.7887\n",
      "Epoch 6/8\n",
      "28799/28799 [==============================] - 4s 147us/step - loss: 0.7115 - acc: 0.7854 - val_loss: 0.7020 - val_acc: 0.7863\n",
      "Epoch 7/8\n",
      "28799/28799 [==============================] - 4s 148us/step - loss: 0.6834 - acc: 0.7904 - val_loss: 0.6946 - val_acc: 0.7878\n",
      "Epoch 8/8\n",
      "28799/28799 [==============================] - 4s 147us/step - loss: 0.6485 - acc: 0.8003 - val_loss: 0.6942 - val_acc: 0.7925\n"
     ]
    }
   ],
   "source": [
    "h = model.fit(train_x_tfidf, train_y_onehot, epochs=8, validation_split=0.1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "31999/31999 [==============================] - 1s 41us/step\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[0.46620027987434476, 0.8610269070703569]"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.evaluate(train_x_tfidf, train_y_onehot)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "8000/8000 [==============================] - 0s 41us/step\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "0.791"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "loss, acc = model.evaluate(test_x_tfidf, test_y_onehot, batch_size=32)\n",
    "acc"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x7fd8d8615550>"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD8CAYAAACMwORRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3Xl8VPW9//HXZ2ayTBZCdvZ9UwggAcVacauKS4u1KrXVimu11v1arXpve1t7b/fWWqs/alHxuqFiry0urYKlXpcSMICsshMQskLWyWzf3x9nkkwgy0AmOTOTz/PxmMecM+c7Zz7B+J6T7/ec7xFjDEoppRKLw+4ClFJKRZ+Gu1JKJSANd6WUSkAa7koplYA03JVSKgFpuCulVALScFdKqQSk4a6UUglIw10ppRKQy64PzsvLM6NGjbLr45VSKi6tXr260hiT310728J91KhRlJSU2PXxSikVl0RkdyTttFtGKaUSkIa7UkolIA13pZRKQLb1uSul+iefz0dZWRkej8fuUmJaamoqw4YNIykp6bjer+GulOpTZWVlZGZmMmrUKETE7nJikjGGqqoqysrKGD169HHtQ7tllFJ9yuPxkJubq8HeBREhNze3R3/daLgrpfqcBnv3evpvFH/hXrEV3vo++L12V6KUUjEr/sK9Zhd89AfY+qbdlSilVMyKv3Afdw4MGAarn7a7EqWUilnxF+4OJ8z4Fmxfbh3FK6XUMbrkkksoLi5m8uTJLFy4EIC33nqLGTNmMG3aNM455xwA6uvrufbaaykqKmLq1Km8+uqrdpZ9TOLzVMiTroJ//BTWLIZz/sPuapRSx+k//7KBjftro7rPE4cM4Adfntxlm0WLFpGTk0NTUxOzZs1i3rx53HjjjaxcuZLRo0dTXV0NwI9//GOysrJYv349ADU1NVGttTfF35E7QNZQGH8+fPI/EPDZXY1SKs787ne/Y9q0acyePZu9e/eycOFC5syZ03pOeU5ODgDvvPMOt956a+v7srOzban3eMTnkTtA8QJrUHXrW3DCl+2uRil1HLo7wu4N7733Hu+88w4ffvghaWlpnHnmmUyfPp3Nmzf3eS29KT6P3AHGfQkGDNWBVaXUMTl8+DDZ2dmkpaWxefNmPvroIzweDytXrmTnzp0Ard0y5557Lo899ljre7Vbpi84XdbA6rZ3oSai6Y2VUoq5c+fi9/s54YQTuP/++5k9ezb5+fksXLiQSy+9lGnTpjF//nwAHnroIWpqapgyZQrTpk1jxYoVNlcfufjtloHQwOrP4JNn4eyH7K5GKRUHUlJSePPNjq+TueCCC9qtZ2Rk8Mwzz/RFWVEXv0fuAFnDYPx5sOZZCPjtrkYppWJGfIc7WAOr9Qfgs7ftrkQppWJGt+EuIotEpFxEPu1ke5aI/EVE1orIBhG5NvpldmHcuZA5RAdWlVIqTCRH7k8Dc7vYfiuw0RgzDTgT+JWIJPe8tAg5XTDjavjs73Bob599rFJKxbJuw90YsxKo7qoJkCnW/JQZobZ92wF+0tXW8yfP9unHKqVUrIpGn/vvgROA/cB64A5jTDAK+43cwOEw/lwdWFVKqZBohPv5QCkwBJgO/F5EBnTUUERuEpESESmpqKiIwkeHKV4Adfth29+ju1+lVMLJyMiwu4ReF41wvxZYaizbgJ3ApI4aGmMWGmNmGmNm5ufnR+Gjw4w/HzIG6cCqUkoRnXDfA5wDICKFwERgRxT2e2xaB1b/BofL+vzjlVLxxxjDvffey5QpUygqKuKll14C4PPPP2fOnDlMnz6dKVOm8M9//pNAIMCCBQta2/7mN7+xufqudXuFqoi8gHUWTJ6IlAE/AJIAjDFPAD8GnhaR9YAA9xljKnut4q6cdDWs/KU1W+SZ99tSglLqGLx5PxxYH919DiqCC34aUdOlS5dSWlrK2rVrqaysZNasWcyZM4fnn3+e888/nwcffJBAIEBjYyOlpaXs27ePTz+1zgo/dOhQdOuOsm7D3RhzZTfb9wPnRa2insgead2pac1imHOvdWMPpZTqxPvvv8+VV16J0+mksLCQM844g1WrVjFr1iyuu+46fD4fl1xyCdOnT2fMmDHs2LGD2267jYsuuojzzouN2OtMfM8t05HiBfDSVbDtHZhwvt3VKKW6EuERdl+bM2cOK1euZNmyZSxYsIC7776bb33rW6xdu5a3336bJ554giVLlrBo0SK7S+1U/E8/cKQJcyGjUAdWlVLdOv3003nppZcIBAJUVFSwcuVKTj75ZHbv3k1hYSE33ngjN9xwA2vWrKGyspJgMMjXvvY1Hn74YdasWWN3+V1KvCN3Z5I1W+T7v4Ha/TBgiN0VKaVi1Fe/+lU+/PBDpk2bhojw85//nEGDBvHMM8/wi1/8gqSkJDIyMli8eDH79u3j2muvJRi0LuP57//+b5ur75oYY2z54JkzZ5qSkpLe2XnNLnhkGpz1IJzxvd75DKXUcdm0aRMnnHCC3WXEhY7+rURktTFmZnfvTbxuGYDsUTD2bGtgNRiwuxqllOpziRnuYA2sHt4L25fbXYlSSvW5xA33iRdCeoEOrCql+qXEDXdnEpz0TdjyJtR+bnc1SinVpxI33MG6gbYJQOn/2F2JUkr1qcQO95wxMOZMWL0Ygn07C7FSStkpscMdQgOre2CHDqwqpfqPxA/3iRdBWp4OrCqljku8zv2e+OHuSm4bWK07YHc1SinVJxJv+oGOzLgG/u8RKH0OTr/H7mqUUiE/+9fP2Fy9Oar7nJQziftOvq/T7ffffz/Dhw/n1ltvBeCHP/whLpeLFStWUFNTg8/n4+GHH2bevHndflZ9fT3z5s3r8H2LFy/ml7/8JSLC1KlTefbZZzl48CA333wzO3ZYt7x4/PHH+cIXvhCFn/po/SPcc8fC6Dmw+hk47S5wJP4fLEqpjs2fP58777yzNdyXLFnC22+/ze23386AAQOorKxk9uzZfOUrX0FEutxXamoqr7322lHv27hxIw8//DAffPABeXl5VFdXA3D77bdzxhln8NprrxEIBKivr++1n7N/hDtYA6uvXAc737OmJlBK2a6rI+zectJJJ1FeXs7+/fupqKggOzubQYMGcdddd7Fy5UocDgf79u3j4MGDDBo0qMt9GWN44IEHjnrf8uXLufzyy8nLywMgJycHgOXLl7N48WIAnE4nWVlZvfZz9p9wn3QxpOVCyVMa7kr1c5dffjmvvPIKBw4cYP78+Tz33HNUVFSwevVqkpKSGDVqFB6Pp9v9HO/7+kL/6Z9wpcD0b8CWN6DuoN3VKKVsNH/+fF588UVeeeUVLr/8cg4fPkxBQQFJSUmsWLGC3bt3R7Sfzt539tln8/LLL1NVVQXQ2i1zzjnn8PjjjwMQCAQ4fPhwL/x0lm7DXUQWiUi5iHzaRZszRaRURDaIyD+iW2IUzVgAQb81sKqU6rcmT55MXV0dQ4cOZfDgwXzzm9+kpKSEoqIiFi9ezKRJkyLaT2fvmzx5Mg8++CBnnHEG06ZN4+677wbgkUceYcWKFRQVFVFcXMzGjRt77Wfsdj53EZkD1AOLjTFTOtg+EPgAmGuM2SMiBcaY8u4+uFfnc+/K0xdbs0Xe9okOrCplA53PPXK9Op+7MWYlUN1Fk28AS40xe0Ltuw12WxUvsG7msTN2/8BQSqmeisaA6gQgSUTeAzKBR4wxi6Ow394x6WJw51hXrI49y+5qlFJxYP369Vx99dXtXktJSeHjjz+2qaLuRSPcXUAxcA7gBj4UkY+MMVuPbCgiNwE3AYwYMSIKH30cklKtgdWPn4D6csgosKcOpVTcKCoqorS01O4yjkk0Op3LgLeNMQ3GmEpgJTCto4bGmIXGmJnGmJn5+flR+OjjNOOa0MDq8/bVoJRSvSga4f6/wBdFxCUiacApwKYo7Lf35E+AkafBmmd0KmClVEKK5FTIF4APgYkiUiYi14vIzSJyM4AxZhPwFrAO+BfwpDGm09MmY0bxAqjeAbv+aXclSikVdd32uRtjroygzS+AX0Slor5ywlcg9V5rYHXMGXZXo5RSUdV/T/RuGVjd9BdoqLS7GqVUjOpqPvddu3YxZcpRl//EhP4b7hAaWPXpwKpSKuH0n4nDOlIwCUacanXNfOE26GZ6T6VUdB34r/+ieVN053NPOWESgx54oNPt0ZzPPZzH4+GWW26hpKQEl8vFr3/9a8466yw2bNjAtddei9frJRgM8uqrrzJkyBCuuOIKysrKCAQC/Pu//zvz58/v0c99pP595A6hgdXtsOt9uytRSvWB+fPns2TJktb1JUuWcM011/Daa6+xZs0aVqxYwT333EN3U7Mc6bHHHkNEWL9+PS+88ALXXHMNHo+HJ554gjvuuIPS0lJKSkoYNmwYb731FkOGDGHt2rV8+umnzJ07N9o/Zj8/cgc4cR68+T3r6H306XZXo1S/0tURdm+J5nzu4d5//31uu+02ACZNmsTIkSPZunUrp556Kj/5yU8oKyvj0ksvZfz48RQVFXHPPfdw3333cfHFF3P66dHPHj1yT3LDtCth0+vQUGV3NUqpPtAyn/tLL7101HzupaWlFBYWRm1e9m984xu8/vrruN1uLrzwQpYvX86ECRNYs2YNRUVFPPTQQ/zoRz+KymeF03AHa2A14IW1L9hdiVKqD0RrPvdwp59+Os89Z00nvnXrVvbs2cPEiRPZsWMHY8aM4fbbb2fevHmsW7eO/fv3k5aWxlVXXcW9997LmjVrov0jarcMAIUnwvBTrK6ZU2/VgVWlElxH87l/+ctfpqioiJkzZ0Y8n3u473znO9xyyy0UFRXhcrl4+umnSUlJYcmSJTz77LMkJSUxaNAgHnjgAVatWsW9996Lw+EgKSmp9QYe0dTtfO69xbb53DtT+jz8+RZY8AaMOs3uapRKWDqfe+R6dT73fuPESyAlyzp6V0qpOKfdMi2S02DafFj9DFzwM0jLsbsipVSM6K/zuSeO4gXwr4Ww9kU49Tt2V6NUwjLGIHE0tmXHfO497TLXbplwhZNh2Cyra8amsQilEl1qaipVVVU9Dq9EZoyhqqqK1NTU496HHrkfqXgB/O+tsOcjGHmq3dUolXCGDRtGWVkZFRUVdpcS01JTUxk2bNhxv1/D/UiTvwpvfd86etdwVyrqkpKSGD16tN1lJDztljlScjpMvQI2vAaN1XZXo5RSx0XDvSPFCyDQDOuWdNtUKaVikYZ7RwYVwdBiHVhVSsWtSO6hukhEykWky/uiisgsEfGLyGXRK89GxQugYhPs/ZfdlSil1DGL5Mj9aaDLyYZFxAn8DPhbFGqKDZMvheRMvWJVKRWXug13Y8xKoLuRxduAV4HyaBQVE1IyYOrlsGEpNNXYXY1SSh2THve5i8hQ4KtA9Kc1s1vxAvB7YN3LdleilFLHJBoDqr8F7jPGBLtrKCI3iUiJiJTExQUMg6fBkJN0YFUpFXeiEe4zgRdFZBdwGfAHEbmko4bGmIXGmJnGmJn5+flR+Og+ULwAyjdAWQxNT6yUUt3ocbgbY0YbY0YZY0YBrwDfMcb8uceVxYopX4PkDB1YVUrFlUhOhXwB+BCYKCJlInK9iNwsIjf3fnkxICUTii6DT18Fz2G7q1FKqYh0O7eMMebKSHdmjFnQo2piVfEC68h93RI4+Ua7q1FKqW7pFaqRGHKSNbiqA6tKqTih4R6p4gVw8FPYF/27lCulVLRpuEdqymWQlA6rn7K7EqWU6paGe6RSB0DR10IDq7V2V6OUUl3ScD8WxQvA1wjr9YpVpVRs03A/FkNmWNMBr35KB1aVUjFNw/1YiFhH7wfWw/5P7K5GKaU6peF+rIouh6Q0vWJVKRXTNNyPVWoWTLkU1r8CzXV2V6OUUh3ScD8exdeCr8EKeKWUikEa7sdjaDEUTtGuGaVUzNJwPx4tA6ufl+rAqlIqJmm4H6+iy8HlhtXP2F2JUkodRcP9eLkHhgZWX4bmerurUUqpdjTce6J4AXjrrSkJlFIqhmi498SwWVBwog6sKqVijoZ7T7QMrO5fA5+vtbsapZRqpeHeU1OvAFeqDqwqpWKKhntPubNh8letW/B5G+yuRimlgMhukL1IRMpF5NNOtn9TRNaJyHoR+UBEpkW/zBhXvAC8dfDpUrsrUUopILIj96eBuV1s3wmcYYwpAn4MLIxCXfFl+CmQP0kHVpVSMaPbcDfGrASqu9j+gTGmJrT6ETAsSrXFj5aB1X0l1nTASills2j3uV8PvNnZRhG5SURKRKSkoqIiyh9ts6nzwZmiA6tKqZgQtXAXkbOwwv2+ztoYYxYaY2YaY2bm5+dH66NjQ1oOTL4E1r0E3ka7q1FK9XNRCXcRmQo8CcwzxlRFY59xqXgBNNfChtfsrkQp1c/1ONxFZASwFLjaGLO15yXFsRGnQt4EHVhVStkuklMhXwA+BCaKSJmIXC8iN4vIzaEm/wHkAn8QkVIRKenFemNby8Bq2b/g4Aa7q1FK9WNijLHlg2fOnGlKShLwe6CxGn410bpb04U/t7sapVSCEZHVxpiZ3bXTK1SjLS0HTpwH617UgVWllG003HtD8QLwHIaN/2t3JUqpfkrDvTeMPA1yx+vAqlLKNhruvaFlYHXvR1C+ye5qlFL9kIZ7b5l2JTiT9YpVpZQtNNx7S3ounPAVWPsC+JrsrkYp1c9ouPem4gXgOaT3WFVK9TkN99406otQMBlevx3e+r51Bo1SSvUBDffeJAIL/grF18BHj8OjM2Hti2DThWNKqf5Dw723peXAxb+BG5fDwBHw2rfhqQt03nelVK/ScO8rQ2fA9X+HrzwKlVvh/82BN74HTYfsrkwplYA03PuSwwEzvgXfLYGZ18GqP8LvZ0Lp8xAM2l2dUiqBaLjbIS0HLvoV3PQeZI+GP98CT82Fz9fZXZlSKkFouNtp8DS47m2Y9weo2g4Lz4Bl/wZNNd2/VymluqDhbjeHA076Jty2GmbdCCV/ss6qWfOsdtUopY6bhnuscA+05n+/6R+QOw5e/y4sOg/2l9pdmVIqDmm4x5rBU+G6t+CSJ6BmNyw8E/56t3UTEKWUipCGeywSgelXwm0lcMrN1tTBjxZbk5BpV41SKgKR3EN1kYiUi8innWwXEfmdiGwTkXUiMiP6ZfZTqVlwwU/h2yshfxL85Xb405dg3xq7K1NKxbhIjtyfBuZ2sf0CYHzocRPweM/LUu0MmgLXvgGX/hEOl8Efz4a/3KFdNUqpTnUb7saYlUBXKTIPWGwsHwEDRWRwtApUISIw9QrrAqjZ37HOpnl0BpQsgmDA7uqUUjEmGn3uQ4G9YetloddUb0gdAHP/C25+HwpOhL/eBU+eA2Wr7a5MKRVD+nRAVURuEpESESmpqKjoy49OPIUnwoJlcOmTUPu5FfCv3wYNVXZXppSKAdEI933A8LD1YaHXjmKMWWiMmWmMmZmfnx+Fj+7nRGDq5fDdVXDqrdYcNY/OgFVPaleNUv1cNML9deBbobNmZgOHjTGfR2G/HTLBIM07d/bW7uNT6gA4/ydw8//BoCJYdg/88SzYu8ruypRSNonkVMgXgA+BiSJSJiLXi8jNInJzqMkbwA5gG/BH4Du9Vi1Q+8ab7LjoYvbf/328ZR3+gdB/FUyCa/4Cly2C+nLrtMk/3wr12gWmVH8jxqa7As2cOdOUlJQc8/v8NTVULfwjNc89hzGG7MsvJ++Wm3FpN097zXXwj5/DR3+A5HQ4+9+taYYdTrsrU0r1gIisNsbM7LZdvIV7C9+BA1Q+/gSHXn0VcbnIufpqcq+/DufAgVGsMgFUbIE37oWd/7C6bC78FYw4xe6qlFLHKeHDvYV3924qfv8YtX/9K46MDHKvv46cq6/GkZ4ehSoThDGw8c/w9oNQuw+mfQPO/U/IKLC7MqXUMeo34d7Cs2UrFb/7HfXvvoszJ4e8m7/NwPnzcaSkRO0z4l5zPfzzl/DB7yEpDc56AGbdAE6X3ZUppSLU78K9RVNpKeW/fYTGjz7CNXgw+d+9lax58xCXBlirys/gze/B9uVQOAUu/AWM/ILdVSmlIhBpuCfcrJDu6dMZ+fRTjHhqEa78fD5/8CF2XPxlat98E6MzKlryxsNVS+GKZ60bdD91ASz9NtQdtLsypVSUJNyRezhjDPXLl1Px29/S/Nk2Uk44gYI77yB9zhxEpFc/O254G+Cfv4IPHgVXKsy5F6ZdCRl69pFSsajfdst0xAQC1L7xBhW/exTf3r24Z8yg4K47SZs1q08+Py5UbbfOqtn+LiAw/BSYdJH1yB1rd3VKqRAN9w4Yn49Dr75K5WN/wF9RQfoXv0j+nXfinjK5T+uIWcbAgfWweRlsWWYtgzWX/MQLYdLFMOQk676vSilbaLh3IejxUPPc81QtXEjg8GEyzz+f/NtvI2WsHqG2U7MbtrwJm/8Kuz8AE4CMQTDpQph4EYw+HVx6NpJSfUnDPQKB+nqqn3qa6qeeIujxkDVvHnm33kryMJ2x+CiN1fDZ36yj+m3vgq8BkjNh/JesoB9/rnWTb6VUr9JwPwZHTWlwxRXk3fxtndKgMz6PdcXr5mXWkX1DOThcMOqLVtfNxAsga5jdVSqVkDTcj0PrlAavvIIkJ5Nz1VXk3nA9zqwsu0uLXcEg7Cuxum42vwFVn1mvD57eNiBbcKI1PbFSqsc03HvAu3s3FY/+ntply0JTGlxPztVX6ZQGkajYag3Gbl4GZaEphweOtI7oJ10Iw2frFbFK9YCGexR4tmyh4pHfUb98Oc7cXPK+/W0Gfn0+juRku0uLD3UHrG6bLW/Ajvcg4AV3DkyYawX92LOtGSuVUhHTcI+iptJSyn/zWxo//hjXkMHk36pTGhyz5jprIHbLG7D1LfActi6aGnOW1XUzYa5eOKVUBDTce0HDhx9S/pvf4lm3juTRo8m/43YyzzsP0fO+j03AZ51auXmZFfaH9wICI2aHzqfXC6eU6oyGey8xxlD/7rtUPPKINaXBiSdQcOedpJ9+uk5pcDyMgQPrrMHYzcvgYNiFU5Musk6z1AunlGql4d7LTCBA7bJl1pQGZWW4i4utKQ1mdvtvrrpSs9s6mt+8rO3CqczB1umVeuGUUtENdxGZCzwCOIEnjTE/PWL7COAZYGCozf3GmDe62me8h3sL4/VyaOnStikNTj+d/DvvwD1ZpzTosa4unJp0MYz7kl44pfqdqIW7iDiBrcC5QBmwCrjSGLMxrM1C4BNjzOMiciLwhjFmVFf7TZRwbxFsaqLm+RfaT2lwx+2kjBljd2mJofXCqb+GLpyqsC6cGjwNBk2FwVOt54ITITnN7mqV6jWRhnskp3ucDGwzxuwI7fhFYB6wMayNAQaElrOA/cdWbvxzuN3kXn8dA6+43JrS4Omnqfv738m65BIGXvY1UsaNwzlgQPc7Uh1LSoUJ51uP1gunlsG+1bBhKax+ymonDsgdHwr7IivwB02F9Fx761eqj0Vy5H4ZMNcYc0No/WrgFGPMd8PaDAb+BmQD6cCXjDGru9pvoh25H8lfXW1NafD88xivFwBXfj7J48aSMnYcKWPHkDx2LCnjxuHKybG52jhnDBzaY81ieWAdfL7OWq4ta2szYGgo6Ivagn/gSL1ytp8zxuANBGn2B2n2BWn2B9ote/2hbf7QNl/bsreT11veb+23421XnzqSW88ad1w1R/PIPRJXAk8bY34lIqcCz4rIFGNMu1sfichNwE0AI0aMiNJHxyZXTg6F999H7o030LRuHd7t22nevoPm7ds5vHQpwcbG1rbO7GxSxo61wn7sWFLGjSV57DhcBfl6Bg7W/4ABEyBoggRMoN160AQxGIKp6TBqNsGRJ2OMwWAwjdUEKzZhyjdhKrZgKjYT3PkuhiAGMMmZBPPGE8ybAHnjCOaOwwwciXE4rX2G9t2yv6AJtu3bGIKE1sO3H9k+bN0Yg0MciAgOHG3L4sBB2LI4EMKWO2jf3fZj3V94e0Ha/d4d+fNgIGiC+INB/MEA/qDBFwjiDwTwBQ2BQBBfMBDaHnoEDP5AoLV9y+u+QIBA2Lo/GCQQDOIPBEOvBwgY0/Z6MEggEMRvDMGWdRPEFwjiDfhp9vvwBvx4AwG8AR++gN96BAN4A378oWV/0HoOBAMgAcCABBEJAgEQax1aXgta62HL0vIsQZyOIA4HOBxBHBJEHAaHGMQRRFJb2hlEgggByoIXA7f16v83kYT7PmB42Pqw0GvhrgfmAhhjPhSRVCAPKA9vZIxZCCwE68j9OGuOK67cXDLPOgvOOqv1NWMM/gMHaN62nebt2/CGQr/2zTcJ1ta2tnNkZpIyZkzb0f44K/xdgwd3em590ARpDjTj8XusR6DtucnfhMfvad3e5G9qt71lWyAYCk6CBIPBdkHaErCRrvfkvS3rUZMOpA864sUDUH0AqldG73MSgREr4OKZM/ToRFLo0RnBgVOcOMR6djqcOMWFUxy4HC6cDqf13LLe2iapddklLuv9oWWnw9rfaSNGR/mHPVok4b4KGC8io7FC/evAN45oswc4B3haRE4AUoGKaBYa7wLBgBWqLWHq9tB0YjaeCUU0+yfQFGjC42vCV1mJ7CrDtecAyXvLce8tJ/3vW0h91dO6L1+yg8qCVA4WJLM/38neXMOe3CB7M714jPe46ktxppDqSiXFkYLD0fZL3fLobt3lcCEi7f5n6G69s/1F8t6W9SOPZIGO10Vaj0hbjlJbl43BUX8QDu3BUbMLqdmNo2YXNNfiAOt9mYOQ7NE4csciOaFHWk6H+wbwB6HJa/0J3uQL4PEZmnwBmr0BPAE/Hl8Aj8+Px+/H6w/QHAjQ7PNbf7oHAjT7/fgCAZp9gdCRaLD1Neso1foStI44jfVMy9GmtSztXm9bdjnB5QCXU0hygcshuJwm9Gxtc7oMTnHgcAiOsH9rp4TWHaH/do6W1xzWsyP82dHa1ikOnI627S2vORyCq+XZ4WjXtnU99B6Xw4nTIa3rDkfor4zQf2srWJ1hIWuFb8vvTGsAh20Pfw4P35bftXjWbbgbY/wi8l3gbazvwUXGmA0i8iOgxBjzOnAP8EcRuQvrN2iBsesE+j7U6GukoqmCyqZKKpq7twbYAAATjklEQVQqqGqqoqKxom05tK3eW483eIyhW2A93Ke4SXFmkNuczfAqB8OqYFC5j4JyL+N2NDF9TXPrWwJJTpoGD8QzPB/viEKCI4fAqGE4hw8j1Z1BqjOVVJf1cDvdpLhSWl+L91/knvIFgjQ0+6lv9tPQHKDe48N3aB+u8vWkVm4g49AmsndvJmvL8tb3HHbmsMs1hs8cY9jEKNb7R7DFl0e91xAIHsuvv4PUJBepSWm4k5yktj4cDHQ5cadZy62vu5y4kx2hZycpSU5SXQ7cyda21CRre0poe2rY9hSXFZAq8elFTEcImiA1nprWwK5sqmx9VDS2rVc0VdDkbzrq/S5xkevOJd+dT15aHnnuPAYkD7BCtSVcnam4Xe7Wo2W3y33U9lRXKinOlG773AN1dXh37Ah18YS6ebZtx7cvrOfM5SJ55MijuniSR4/GkRK/FwQ1eQPUenyhQG4L5rblltdDr3nbXqvz+GnwhoK82Y/XH+z+A4FsRyMnJZcx1bWHE2UX480uhvt348LqPmp2pFGRPo6azEnUZZ9AU84U/HmTSE9LIy3F2Rre7qS2wE5xOXRsRUVMr1A9gsfvaR/UHQR3VVMVVZ6qDvt5M5IyyHNbYR0e3Pnu/LYwd+eRlZIVE0fBwcZGmnfutAZyQ8Hv3b4d75491qmEAA4HScOGhQ3ijiVl9GhceXk48/L6PPi9/iDVDV4q65upavBS3dBMVb2XynovVfXN1raGtuVGb/f98SKQnuwiPcVJeoqLjBRXaN1FRvhrKe1fC29rbbde7zCI/c1QvqntbJ0D662Ht97a7nBZ0ykUToacMZA9CrJHW88ZBXrGjjom/SLcjTEcbj7ccViHrzdWUuerO+r9DnGQm5rbGtrhj/y0/NbgznPn4Xa5e1RrrAg2N+PdtRvv9m1W6O/YYS3v2g0+X7u2jvR0nHm5uHJyceXl4szJxZWbizM3B1duHq7cHJyhZ8eAAUeFnj8QpKbRR1VDM9X1bcFcVe+lqmW5wdsa6HUef4c1JzmF3PQUcjOSyUlPJi8jhdz0ZHIykslyJx0R2FYQtwS2O8mJw45uiGAQana2hf3n66wvgNp9WD2XLT9cWijsR7UFfk7oeeAInWpBHSVhw/3D/R/y6CePtga3L+g7qo3b5e7wqDo8uPPceWSnZON0dDGc3o8Ynw/v3r14d+3CX1VFoKo69FxlPVdX4a+sInDokHVe+RECTidNaQOoc2dSk5JJpSuNclc6h5IzqEnJ5FBKRuujLjWDrEx3a2DnhsI6N91atgK8bXlAqitxui18HmsWzOqdULMr9Ahb9jWGNRbr/PzsUZAzKuwLIBT+aTl61N8P9fV57n0mxZlCZnImo7NGt3WRHBHcaa60xAmDPlLdHOQzstiZMY4qhlOZ4qU6y0tVYduRdnWDF+P3k+VtIKu5noHN9WQ315PVXM+gYAMF/kZyvQ3kNtcxsvYA7obDOPwdH407s7Jw5uXhyslp/eug/V8EubgcubjcucThr2nnklIhb7z1OJIxUF9+dOBX74TP3oH6A+3bpwyA7JHtj/pbjvyzhoOzqxP9VKKLuyN3dfyMMVQ1eNl6sI5t5fVsPVjHZwfr+ay8nuqG9mfzZKS4rKPq9LAj64zktqPt1qPuZLLTkklyHj3OYIwhWF9vHf1XV+OvrCRQXW39BRD6S8BfXUWg0toefo5/OElNDXUHHd0t5MjIxJHmxuF2I243DnfaEetuJKX7gem44G2EQ7vbAr/dkf9uCLSdOYU4rZuUhwd++JeATrgWtxK2W0Z1zxhDRX0z2w6GAry8PhTiddQ0tnVjZaa4GF+YwfiCTOu5MJMxeenkZ6aQmtT33VVBr9cK/9buoGoCVZX4q6rxV1VaXUXV1QQqK/FXV0MgwgucHA4cqalIWhqOUOCHh7/D7UbSQl8MbjeONDeSmtrBF0XYeqq7bTkW7sgVDFpH9q2hf8SRf2Nl+/apA48O/JYvgQFDQbsrY5aGez9gjKGirpmtoeDeerCebeVWmB8KD/FUFxMKM5lQmMG4Aut5fEEmhQPi94jWBIMEDh8m2NBAsLER09REsKmJYGMTwaZGjMcTWg6tty43YTxNYduaMI2NrctBj+eogeXuSFJSuy8Ocae2fVG0fFm43ThS3Vbb5GTrueWRnNR+vaM23WwnKanr/5bNddbRfUvohx/5H9oD4WNXjiRrMHfAEEjPg/QCSM+3boOYnh9az7PO9NF74Pa5hO1z74+MMRysbW4X4FsP1vPZwTpqw84wyXInMaEwgwumDG4N8AmFGeRnxm+Id0YcDlzZ2ZCdHfV9G5+vLezDvziaPNYXRVNT+y+OJk/Yctu2QG0t/oMH275EmpowPh/mGL88InbUF0D4egdfFslDkKSRiMuJ4EOCHsTfgPjrkbI6xF+JBPYggXprm8OEHrQ9JycjaVlI+kAkIwfSs5HMPGRAfui50HoMHGytJycjTv2roC9ouMcQYwwHaj2twd3aL15e3+40wYFpSUwoyOTL04YwviCDCYWZjCvMID8j8ULcDpKUhDMpqdemaDbGgN/fGvTG58N4ve3XO3q0tPFG0KbDh7f1vcHGxrbXwtrg9REM7YfWwfCU0KMzAaAq9IiAgDgFcTqsL5Ykl/Xlk5yMJKcgyalIqjv0WtLRX1odfWE5ne3PHGq3TIevS4ftO9tHx/vreB/d78ddNKXX79qm4W4DYwz7D3uOCvBtB+upa24L8Zz0ZMYXZDBv+hArwENBnpuerCEex0SkNbBimTEGuvvCafmSCv/C8TZjGg5h6qsxDTXWcuNhTGMtpqkO01QPnkaMpwHT3IDxBzBBwQTB+ATTLJjDYEgiSBIGF8Y4McaBCTqsdgGDCQSt9/r94I/iBHN9IPfGGzTc411Ng5fSskN8FjozZWt5PdsO1tEQdnVlXkYy4woy+OqMoYwvsAY2xxdkkJuhF7Ao+4gIJFtH1L2qud66s1bLo778iOVKaAi91lTT8T5cbmuMYMAQTOaQ1mVal4eCO6ftRuvhY40ty2GvtRuJ7KhthMvthzTbVvrii13DvZcYY1i6Zh8/eH0D9aGj8byMFCYUZnBZ8bDWAB9fmElOei//z6NULEvJsB45EUyD6/dCY1Vb2Ne3fAkchNr9ULsf2fMh1O2H4BHXWDiTrZutDxjaFv6ty6HnjAJwOEmEv4s13HtBTYOXB15bz5ufHmDWqGzuOW8iEwszydYQV6pnXMkwYLD16EowaIV+7b7W0G+3vG81bPpL+2sDwLo+IHNwB+Eftpw5KC4uENNwj7J/bK3g3pfXUtPo5b65k7hpzhidYlWpvuZwQGah9Rg6o+M2xlh/BbSG/hFfBAc/hc/+dsSUEAACGYWdH/23PGyeF0jDPUqavAF++uYmnvlwN+MLMli0YBZThmbZXZZSqjMiofP482DwtI7bGAOeQx0c/Yeeq7bBzpXQ3MHV1Wl5nR/9503o/q+PHtJwj4L1ZYe586VP2F7RwHWnjeZ7cyfacoWnUirKRMCdbT0KJ3fezlMLdZ933A10eC/s/aj9YPAXbofzftyrpWu494A/EOTx97bzyLufkZeRwnM3nMJp4/LsLksp1ddSB1iP/Imdt/E2tn0BZBx5L9/o03A/TrurGrjrpVLW7DnEl6cN4eF5U8hKi/1BFqWUTZLTIHes9egDEd0ySETmisgWEdkmIvd30uYKEdkoIhtE5Pnolhk7jDG8+K89XPDIP/msvJ5Hvj6dR688SYNdKRVTuj1yFxEn8BhwLlAGrBKR140xG8PajAe+D5xmjKkRkYLeKthOlfXN3P/qet7ZdJAvjM3ll5dPY8jAxLhDk1IqsUTSLXMysM0YswNARF4E5gEbw9rcCDxmjKkBMMaUR7tQu7276SD3vbqOWo+fhy46getOG23P7duUUioCkYT7UGBv2HoZcMoRbSYAiMj/AU7gh8aYt6JSoc0amv08vGwTL/xrD5MGZfLcDbOZOCjT7rKUUqpL0RpQdQHjgTOBYcBKESkyxhwKbyQiNwE3AYwYMSJKH9171uyp4e6XStld3ci3zxjD3edOIMWlpzgqpWJfJOG+Dxgetj4s9Fq4MuBjY4wP2CkiW7HCflV4I2PMQmAhWDfrON6ie5svEOTR5dt4bMU2Bg1I5YUbZzN7TK7dZSmlVMQiCfdVwHgRGY0V6l8HvnFEmz8DVwJPiUgeVjfNjmgW2le2V9Rz90ulrC07zKUzhvLDr0xmQKqeCaOUii/dhrsxxi8i3wXexupPX2SM2SAiPwJKjDGvh7adJyIbsWbuv9cYE+HM/bHBGMP/fLyHnyzbSGqSkz98cwYXFvXu5cFKKdVb9B6qQHmth++9uo73tlQwZ0I+v7hsKoUDUu0uSymljqL3UI3QW58e4PtL19HoDfCjeZO5evZIvcuRUiru9dtwr/P4+NFfNvLy6jKKhmbxm/nTGVeQYXdZSikVFf0y3Fftquaul0rZf6iJ7541jtvPGU+yK6KZGJRSKi70q3D3+oP85p2tPPGP7QzPTuPlm0+leGSO3WUppVTU9Ztw33qwjjtfLGXj57V8fdZwHrr4RDJS+s2Pr5TqZxI+3YJBw9Mf7OKnb20mM8XFwquLOW9y78+lrJRSdkrocP/8cBP3vryO97dVcs6kAn76tankZ9p7X0OllOoLCRvuf1m7nwdfW48vYPivrxZx5cnD9RRHpVS/kXDhfrjJxw/+91P+XLqf6cMH8tv50xmVl253WUop1acSKtw/2F7Jvy1Zy8G6Zu4+dwLfOXMsLqee4qiU6n8SItw9vgC/+tsWnnx/J6Nz01l6yxeYNnyg3WUppZRt4j7cN31ey10vlbL5QB1XzR7BAxeeQFpy3P9YSinVI3GbgoGg4cl/7uBXf9tKVloST107i7MmJuStW5VS6pjFZbiX1TRyz5K1fLyzmvMnF/Lfl04lJz3Z7rKUUipmxF24r9hSzu3Pf4IBfnHZVC4rHqanOCql1BHiLtxH56Zz0shsfnLJFIbnpNldjlJKxaS4C/dReeksvu5ku8tQSqmYpieBK6VUAtJwV0qpBBRRuIvIXBHZIiLbROT+Ltp9TUSMiHR7fz+llFK9p9twFxEn8BhwAXAicKWInNhBu0zgDuDjaBeplFLq2ERy5H4ysM0Ys8MY4wVeBOZ10O7HwM8ATxTrU0opdRwiCfehwN6w9bLQa61EZAYw3BizrKsdichNIlIiIiUVFRXHXKxSSqnI9HhAVUQcwK+Be7pra4xZaIyZaYyZmZ+f39OPVkop1YlIwn0fMDxsfVjotRaZwBTgPRHZBcwGXtdBVaWUso8YY7puIOICtgLnYIX6KuAbxpgNnbR/D/g3Y0xJN/utAHYfR80AeUDlcb7XDvFUbzzVCvFVbzzVCvFVbzzVCj2rd6Qxptuuj26vUDXG+EXku8DbgBNYZIzZICI/AkqMMa8fT3WRFNcZESkxxsTNXwbxVG881QrxVW881QrxVW881Qp9U29E0w8YY94A3jjitf/opO2ZPS9LKaVUT+gVqkoplYDiNdwX2l3AMYqneuOpVoiveuOpVoiveuOpVuiDersdUFVKKRV/4vXIXSmlVBfiLtwjncQsFojIIhEpF5FP7a6lOyIyXERWiMhGEdkgInfYXVNnRCRVRP4lImtDtf6n3TVFQkScIvKJiPzV7lq6IiK7RGS9iJSKSJenNMcCERkoIq+IyGYR2SQip9pdU0dEZGLo37TlUSsid/ba58VTt0xoErOtwLlY0yCsAq40xmy0tbBOiMgcoB5YbIyZYnc9XRGRwcBgY8ya0CRwq4FLYvHfVqz7KqYbY+pFJAl4H7jDGPORzaV1SUTuBmYCA4wxF9tdT2dCFyPONMbExXnjIvIM8E9jzJMikgykGWMO2V1XV0JZtg84xRhzvNf7dCnejtwjncQsJhhjVgLVdtcRCWPM58aYNaHlOmATR8whFCuMpT60mhR6xPRRiogMAy4CnrS7lkQiIlnAHOBPAMYYb6wHe8g5wPbeCnaIv3DvdhIz1XMiMgo4iRievjnUxVEKlAN/N8bEbK0hvwW+BwTtLiQCBvibiKwWkZvsLqYbo4EK4KlQl9eTIpJud1ER+DrwQm9+QLyFu+plIpIBvArcaYyptbuezhhjAsaY6VhzHZ0sIjHb7SUiFwPlxpjVdtcSoS8aY2Zg3cPh1lD3YqxyATOAx40xJwENQKyPxSUDXwFe7s3Pibdw724SM9UDof7rV4HnjDFL7a4nEqE/wVcAc+2upQunAV8J9WW/CJwtIv9jb0mdM8bsCz2XA69hdYfGqjKgLOwvt1ewwj6WXQCsMcYc7M0PibdwXwWMF5HRoW+/rwPHNbeNai80SPknYJMx5td219MVEckXkYGhZTfWAPtme6vqnDHm+8aYYcaYUVi/s8uNMVfZXFaHRCQ9NKBOqHvjPCBmz/YyxhwA9orIxNBL5wAxdxLAEa6kl7tkIMK5ZWJFZ5OY2VxWp0TkBeBMIE9EyoAfGGP+ZG9VnToNuBpYH+rLBnggNK9QrBkMPBM648ABLDHGxPTphXGkEHjN+q7HBTxvjHnL3pK6dRvwXOiAbwdwrc31dCr0hXku8O1e/6x4OhVSKaVUZOKtW0YppVQENNyVUioBabgrpVQC0nBXSqkEpOGulFIJSMNdKaUSkIa7UkolIA13pZRKQP8fSOkGzCelU1QAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "pd.DataFrame(h.history).plot()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Now let's use Bling Fire"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "def my_tokenizer(s):\n",
    "    return text_to_words(s).split(' ')\n",
    "\n",
    "vectorizer_all = TfidfVectorizer(tokenizer=my_tokenizer, max_features=1000)\n",
    "vectorizer_all.fit_transform(all_texts)\n",
    "\n",
    "vectorizer = TfidfVectorizer(tokenizer=my_tokenizer, vocabulary=vectorizer_all.vocabulary_)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "20\n"
     ]
    }
   ],
   "source": [
    "train_x_tfidf = vectorizer.fit_transform(train_x).todense()\n",
    "test_x_tfidf = vectorizer.fit_transform(test_x).todense()\n",
    "\n",
    "import numpy as np\n",
    "from sklearn import preprocessing\n",
    "\n",
    "train_y_onehot = preprocessing.OneHotEncoder().fit_transform(np.array(train_y_ids).reshape(-1,1)).toarray()\n",
    "test_y_onehot = preprocessing.OneHotEncoder().fit_transform(np.array(test_y_ids).reshape(-1,1)).toarray()\n",
    "\n",
    "print(train_y_onehot.shape[1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "dense_3 (Dense)              (None, 1000)              1001000   \n",
      "_________________________________________________________________\n",
      "batch_normalization_2 (Batch (None, 1000)              4000      \n",
      "_________________________________________________________________\n",
      "activation_2 (Activation)    (None, 1000)              0         \n",
      "_________________________________________________________________\n",
      "dropout_2 (Dropout)          (None, 1000)              0         \n",
      "_________________________________________________________________\n",
      "dense_4 (Dense)              (None, 100)               100100    \n",
      "_________________________________________________________________\n",
      "batch_normalization_3 (Batch (None, 100)               400       \n",
      "_________________________________________________________________\n",
      "activation_3 (Activation)    (None, 100)               0         \n",
      "_________________________________________________________________\n",
      "dropout_3 (Dropout)          (None, 100)               0         \n",
      "_________________________________________________________________\n",
      "dense_5 (Dense)              (None, 20)                2020      \n",
      "=================================================================\n",
      "Total params: 1,107,520\n",
      "Trainable params: 1,105,320\n",
      "Non-trainable params: 2,200\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "model = tf.keras.models.Sequential()\n",
    "model.add(L.Dense(1000, input_shape=train_x_tfidf.shape[1:]))\n",
    "model.add(L.BatchNormalization())\n",
    "model.add(L.Activation('tanh'))\n",
    "model.add(L.Dropout(0.5))\n",
    "model.add(L.Dense(100))\n",
    "model.add(L.BatchNormalization())\n",
    "model.add(L.Activation('tanh'))\n",
    "model.add(L.Dropout(0.5))\n",
    "model.add(L.Dense(train_y_onehot.shape[1], activation='softmax'))\n",
    "model.compile(tf.keras.optimizers.Adam(lr=0.0001), 'categorical_crossentropy', metrics=['accuracy'])\n",
    "model.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 28799 samples, validate on 3200 samples\n",
      "Epoch 1/8\n",
      "28799/28799 [==============================] - 5s 171us/step - loss: 1.8345 - acc: 0.4695 - val_loss: 0.8957 - val_acc: 0.7606\n",
      "Epoch 2/8\n",
      "28799/28799 [==============================] - 4s 148us/step - loss: 1.0535 - acc: 0.6977 - val_loss: 0.7377 - val_acc: 0.7891\n",
      "Epoch 3/8\n",
      "28799/28799 [==============================] - 4s 147us/step - loss: 0.8592 - acc: 0.7475 - val_loss: 0.6831 - val_acc: 0.7994\n",
      "Epoch 4/8\n",
      "28799/28799 [==============================] - 4s 146us/step - loss: 0.7652 - acc: 0.7730 - val_loss: 0.6589 - val_acc: 0.8078\n",
      "Epoch 5/8\n",
      "28799/28799 [==============================] - 4s 149us/step - loss: 0.6935 - acc: 0.7915 - val_loss: 0.6430 - val_acc: 0.8128\n",
      "Epoch 6/8\n",
      "28799/28799 [==============================] - 4s 148us/step - loss: 0.6538 - acc: 0.8032 - val_loss: 0.6393 - val_acc: 0.8103\n",
      "Epoch 7/8\n",
      "28799/28799 [==============================] - 4s 147us/step - loss: 0.6226 - acc: 0.8105 - val_loss: 0.6348 - val_acc: 0.8103\n",
      "Epoch 8/8\n",
      "28799/28799 [==============================] - 4s 147us/step - loss: 0.5974 - acc: 0.8166 - val_loss: 0.6330 - val_acc: 0.8103\n"
     ]
    }
   ],
   "source": [
    "h = model.fit(train_x_tfidf, train_y_onehot, epochs=8, validation_split=0.1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "31999/31999 [==============================] - 1s 42us/step\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[0.4210543013810404, 0.8734335448075885]"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.evaluate(train_x_tfidf, train_y_onehot)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "8000/8000 [==============================] - 0s 42us/step\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "0.813125"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "loss, acc = model.evaluate(test_x_tfidf, test_y_onehot, batch_size=32)\n",
    "acc"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x7fd94b45bef0>"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD8CAYAAACMwORRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3Xl8VPW9//HXZ5ZkspKVnQCyC5FVBBfUumG1oraW4k6rVLEutdfqVdva1rba3ltb69LLVapYRHHB2quiteIPLWgJCCIIEVEgIGQFkpBllu/vj3OSDJBlgEnOzOTzfDzmMXPmnDnzCa3vc+Z7zvkcMcaglFIqsbicLkAppVT0abgrpVQC0nBXSqkEpOGulFIJSMNdKaUSkIa7UkolIA13pZRKQBruSimVgDTclVIqAXmc+uK8vDwzaNAgp75eKaXi0urVq8uNMfkdLedYuA8aNIiioiKnvl4ppeKSiGyLZDkdllFKqQSk4a6UUglIw10ppRKQY2PuSqnuye/3U1JSQn19vdOlxDSfz0f//v3xer1H9XkNd6VUlyopKSEjI4NBgwYhIk6XE5OMMVRUVFBSUsLgwYOPah06LKOU6lL19fXk5uZqsLdDRMjNzT2mXzca7kqpLqfB3rFj/TeKv3AvK4Y37oJAo9OVKKVUzIq/cK/6Ej58HD570+lKlFIqZsVfuA/5GmT0gTXPOF2JUkrFrA7DXUTmi0ipiHzSxvweIvJ3EVknIhtEZHb0ywzj9sC4y2HLP2D/V536VUqpxHTxxRczceJERo8ezbx58wBYunQpEyZMYOzYsZx11lkA1NTUMHv2bAoLCznhhBN46aWXnCz7iERyKuRTwCPAgjbm3wRsNMZ8Q0Tygc0istAY03mD4uOugPf+G9Y9C6f9qNO+RinVuX7+9w1s3LU/qus8vm8mP/vG6HaXmT9/Pjk5OdTV1XHiiScyY8YMrr/+epYvX87gwYOprKwE4Je//CU9evRg/fr1AFRVVUW11s7U4Z67MWY5UNneIkCGWId20+1lA9Eprw25Q2DgqfDRX8GYTv0qpVTiefjhhxk7dixTpkxhx44dzJs3j2nTpjWfU56TkwPA22+/zU033dT8uezsbEfqPRrRuIjpEeBVYBeQAcw0xoRaW1BE5gBzAAoKCo7tW8dfCa/cANtWwKBTjm1dSilHdLSH3Rneffdd3n77bVauXElqaipnnHEG48aNY9OmTV1eS2eKxgHV84C1QF9gHPCIiGS2tqAxZp4xZpIxZlJ+foftiNt3/EWQlGHtvSulVIT27dtHdnY2qampbNq0iQ8++ID6+nqWL1/OF198AdA8LHPOOefw6KOPNn82oYZlIjAbeNlYtgBfACOjsN72JaVB4Tdh4ytQH90xO6VU4po+fTqBQIBRo0Zx1113MWXKFPLz85k3bx6XXnopY8eOZebMmQDce++9VFVVMWbMGMaOHcuyZcscrj5y0RiW2Q6cBbwnIr2AEcDWKKy3Y+OvgtVPwYaXYeK1XfKVSqn4lpyczBtvvNHqvPPPP/+g6fT0dJ5++umuKCvqIjkVchGwEhghIiUi8j0RuUFEbrAX+SVwsoisB/4J3GmMKe+8ksP0mwj5o3RoRimlDtHhnrsxZlYH83cB50atoiMhYh1YfeseKN0EPTt/NEgppeJB/F2heqix3wGXBz7SK1aVUqpJ/Id7Wh6MOB/WPQdBv9PVKKVUTIj/cAfrwOqBcihe6nQlSikVExIj3IecZTUT0wOrSikFJEq4uz0wdhZ89pY2E1NKdSg9Pd3pEjpdYoQ7WGfNmBCsW+R0JUop5bjECffcITDwFG0mppSKmDGGO+64gzFjxlBYWMjzzz8PwFdffcW0adMYN24cY8aM4b333iMYDHLttdc2L/vQQw85XH37onGFauwYfyW8ciNsXwkDT3a6GqVUR964C3avj+46exfC+Q9EtOjLL7/M2rVrWbduHeXl5Zx44olMmzaNZ599lvPOO4977rmHYDDIgQMHWLt2LTt37uSTT6xbW+zduze6dUdZ4uy5Axw/Q5uJKaUi9v777zNr1izcbje9evXi9NNPZ9WqVZx44on85S9/4b777mP9+vVkZGRw3HHHsXXrVm6++WaWLl1KZmar/RFjRmLtuSelwZhLYf0LMP0B8MX2P75S3V6Ee9hdbdq0aSxfvpzXXnuNa6+9lttvv52rr76adevW8eabb/LnP/+ZxYsXM3/+fKdLbVNi7bkDTLga/AdgwxKnK1FKxbjTTjuN559/nmAwSFlZGcuXL2fy5Mls27aNXr16cf3113PdddexZs0aysvLCYVCfPOb3+T+++9nzZo1TpffrsTacwe7mdhIqx3BxGucrkYpFcMuueQSVq5cydixYxERfvvb39K7d2+efvppfve73+H1eklPT2fBggXs3LmT2bNnEwpZ9yL6zW9+43D17RPj0JklkyZNMkVFRZ2z8hV/grfuhbkfajMxpWLMp59+yqhRo5wuIy609m8lIquNMZM6+mziDcsAnKDNxJRS3Vtihnt6Pgyfrs3ElFLdVmKGO1gHVg+UQ/GbTleilFJdLnHDfchZkN5bh2aUUt1S4oa72wPjLtdmYkqpbilxwx20mZhSqttK7HDPHQIFJ2szMaVUt9NhuIvIfBEpFZFP2lnmDBFZKyIbROT/RbfEYzThKqj8HLZ/4HQlSqk4FK+93yPZc38KmN7WTBHJAh4DLjLGjAYui05pUdLcTEwPrCqluo8O2w8YY5aLyKB2FrkceNkYs91evjQ6pUVJeDOx8x+E5AynK1JK2R7894NsqtwU1XWOzBnJnZPvbHP+XXfdxYABA7jpppsAuO+++/B4PCxbtoyqqir8fj/3338/M2bM6PC7ampqmDFjRqufW7BgAf/1X/+FiHDCCSfwzDPPsGfPHm644Qa2bt0KwOOPP87JJ3dOe/Jo9JYZDnhF5F0gA/ijMWZBawuKyBxgDkBBQUEUvjpC46+CNU/DJy9rvxmlurmZM2dy2223NYf74sWLefPNN7nlllvIzMykvLycKVOmcNFFFyEi7a7L5/OxZMmSwz63ceNG7r//flasWEFeXh6VlZUA3HLLLZx++uksWbKEYDBITU1Np/2d0Qh3DzAROAtIAVaKyAfGmOJDFzTGzAPmgdVbJgrfHZn+kyBvhHVgVcNdqZjR3h52Zxk/fjylpaXs2rWLsrIysrOz6d27Nz/84Q9Zvnw5LpeLnTt3smfPHnr37t3uuowx3H333Yd97p133uGyyy4jLy8PgJycHADeeecdFiyw9n3dbjc9evTotL8zGuFeAlQYY2qBWhFZDowFDgt3x4hYB1bfuhfKNkP+CKcrUko56LLLLuPFF19k9+7dzJw5k4ULF1JWVsbq1avxer0MGjSI+vr6DtdztJ/rCtE4FfJvwKki4hGRVOAk4NMorDe6tJmYUso2c+ZMnnvuOV588UUuu+wy9u3bR8+ePfF6vSxbtoxt27ZFtJ62Pve1r32NF154gYqKCoDmYZmzzjqLxx9/HIBgMMi+ffs64a+zRHIq5CJgJTBCREpE5HsicoOI3ABgjPkUWAp8DPwbeMIY0+Zpk47RZmJKKdvo0aOprq6mX79+9OnThyuuuIKioiIKCwtZsGABI0dG1iq8rc+NHj2ae+65h9NPP52xY8dy++23A/DHP/6RZcuWUVhYyMSJE9m4cWOn/Y2J2c+9LZuXwqKZMHMhjLqwa79bKQVoP/cjof3cIzX0bLuZmN5AWymV2BLvNnvtcXtg3Cz418NQvRsy2j8SrpRSAOvXr+eqq6466L3k5GQ+/PBDhyrqWPcKd4BxV8L7D1nNxE79odPVKKXiQGFhIWvXrnW6jCPSvYZlAPKGajMxpVTC637hDlYr4Iot2kxMKZWwume4j74YktL1wKpSKmF1z3Bvaia2YQk0VDtdjVJKRV33DHewmon5a62AV0qpNrTXz/3LL79kzJgxXVhN5LpvuPc/0WomtkbbESilEk/3OxWyiYh1YPUfP9FmYko5ZPevf03Dp9Ht5548aiS97767zfnR7Ocerr6+nhtvvJGioiI8Hg+///3vOfPMM9mwYQOzZ8+msbGRUCjESy+9RN++ffn2t79NSUkJwWCQn/zkJ8ycOfOY/u5Ddd89d4CxTc3E9MCqUt3FzJkzWbx4cfP04sWLueaaa1iyZAlr1qxh2bJl/OhHP+JIW7M8+uijiAjr169n0aJFXHPNNdTX1/PnP/+ZW2+9lbVr11JUVET//v1ZunQpffv2Zd26dXzyySdMn97mze6OWvfdcwdI72k3E1sEZ/0U3F6nK1KqW2lvD7uzRLOfe7j333+fm2++GYCRI0cycOBAiouLmTp1Kr/61a8oKSnh0ksvZdiwYRQWFvKjH/2IO++8kwsvvJDTTjst6n9n995zB2toprYMPnvL6UqUUl2kqZ/7888/f1g/97Vr19KrV6+o9WW//PLLefXVV0lJSeHrX/8677zzDsOHD2fNmjUUFhZy77338otf/CIq3xVOw33oOZDeSw+sKtWNRKufe7jTTjuNhQsXAlBcXMz27dsZMWIEW7du5bjjjuOWW25hxowZfPzxx+zatYvU1FSuvPJK7rjjDtasWRPtP7GbD8uA1Uxs7CxY8SdtJqZUN9FaP/dvfOMbFBYWMmnSpIj7uYebO3cuN954I4WFhXg8Hp566imSk5NZvHgxzzzzDF6vl969e3P33XezatUq7rjjDlwuF16vt/kGHtHUvfq5t6V8CzwyEc6+T5uJKdXJtJ975LSf+7HKGwoFU7WZmFIqYeiwTJPxV8Hf5sKOD6FgitPVKKViiPZzj2fHz4A3fmwdWNVwV6pTGWMQEafLiJgT/dyPdchch2WaJKfD6Eu0mZhSnczn81FRUXHM4ZXIjDFUVFTg8/mOeh0d7rmLyHzgQqDUGNNmhxwRORFYCXzHGPPiUVfkpAlXw0fPWAE/4Wqnq1EqIfXv35+SkhLKysqcLiWm+Xw++vfvf9Sfj2RY5ingEWBBWwuIiBt4EIjvK4H6nwh5w60DqxruSnUKr9fL4MGDnS4j4XU4LGOMWQ5UdrDYzcBLQGk0inKMiHVgdceHUFbsdDVKKXXUjnnMXUT6AZcAHZ6FLyJzRKRIRIpi9ifZ2O+AuK3hGaWUilPROKD6B+BOY0yoowWNMfOMMZOMMZPy8/Oj8NWdoLmZ2HMQ9DtdjVJKHZVohPsk4DkR+RL4FvCYiFwchfU6Z8JVUFuqzcSUUnHrmMPdGDPYGDPIGDMIeBGYa4x55Zgrc1JTMzHt866UilMdhruILMI6xXGEiJSIyPdE5AYRuaHzy3NIUzOx4jetZmJKKRVnOjwV0hgzK9KVGWOuPaZqYsn4K+Fff7DG3k+9zelqlFLqiOgVqm3JG6bNxJRScUvDvT3jr4SKz6zz3pVSKo5ouLfn+IshKV3PeVdKxR0N9/Y0NRP7RJuJKaXii4Z7R8ZfBf5a2BDfZ3cqpboXDfeODJhsNxPToRmlVPzQcO+IiHVgVZuJKaXiiIZ7JE6wm4mt1StWlVLxQcM9Ehm9rGZiaxdpMzGlVFzQcI/U+CvtZmL/cLoSpZTqkIZ7pIadazcT0wOrSqnYp+EeKbfHupFH8ZtQvcfpapRSql0a7kdi/FVggvDxc05XopRS7dJwPxJ5w2DAFFjzjDYTU0rFNA33I9XcTOzfTleilFJt0nA/UqMvAW8afLTA6UqUUqpNGu5HKjkdxjQ1E6txuhqllGqVhvvRGH+13UxsidOVKKVUqzTcj8aAyZA7TG+grZSKWZHcIHu+iJSKyCdtzL9CRD4WkfUiskJExka/zBjT3EzsAyj/zOlqlFLqMJHsuT8FTG9n/hfA6caYQuCXwLwo1BX7xs6ymonpFatKqRjUYbgbY5YDle3MX2GMqbInPwD6R6m22JbRC4afp83ElFIxKdpj7t8D3ojyOmPX+Ku0mZhSKiZFLdxF5EyscL+znWXmiEiRiBSVlZVF66udM+wcSOupB1aVUjEnKuEuIicATwAzjDEVbS1njJlnjJlkjJmUn58fja92lttrNxNbqs3ElFIx5ZjDXUQKgJeBq4wx3e8+dNpMTCkVgyI5FXIRsBIYISIlIvI9EblBRG6wF/kpkAs8JiJrRaSoE+uNPfnDYcBJ1tCMNhNTSsUIT0cLGGNmdTD/OuC6qFUUj8ZfBa/+wGomVnCS09UopZReoRoVoy+2m4npOe9Kqdig4R4NyRlWM7EN2kxMKRUbNNyjZfxV0FgDG19xuhKllNJwj5oBJ2kzMaVUzNBwj5amZmLbV2ozMaWU4zTco6m5mZjuvSulnKXhHk1NzcTWLYJgwOlqlFLdmIZ7tI2/Emr2wBZtJqaUco6Ge7QNO9dqJrZGz3lXSjlHwz3atJmYUioGaLh3huZmYs87XYlSqpvScO8Mzc3EntFmYkopR2i4d5bxV0J5MZSscroSpVQ3pOHeWUZfYjUTW7PA6UqUUt2QhntnSc6wAl6biSmlHKDh3pnGX2k1E/vkJacrUUp1MxrunalgCvQqhNduh3fuh0CD0xUppboJDffOJALXvAqFl8Hy38GfT4XtHzhdlVKqG9Bw72ypOXDJn+GKl8BfB/Onw+s/1nF4pVSn0nDvKsPOhrkrYfIc+Pc8eGwKfPa201UppRJUh+EuIvNFpFREPmljvojIwyKyRUQ+FpEJ0S8zQSRnwNd/C999E7wpsPCbsOQGOFDpdGVKqQQTyZ77U8D0duafDwyzH3OAx4+9rARXcBJ8/z2YdgesfwEenQyfvKxXsyqloqbDcDfGLAfa27WcASwwlg+ALBHpE60CE5bXB1+7F+a8C5n94MXZ8NwVsP8rpytTSiWAaIy59wN2hE2X2O+pSPQuhOv+Cef8Ej7/Jzx6Eqx+WvfilVLHpEsPqIrIHBEpEpGisrKyrvzq2Ob2wCm3wI0roM8J8Pdb4OlvQMXnTlemlIpT0Qj3ncCAsOn+9nuHMcbMM8ZMMsZMys/Pj8JXJ5jcIXD1q/CNP8JX6+DxU2DFn/SWfUqpIxaNcH8VuNo+a2YKsM8YowPHR8vlgonXwk0fwpAz4a174cmzYXerJysppVSrIjkVchGwEhghIiUi8j0RuUFEbrAXeR3YCmwB/heY22nVdieZfeE7z8K3/gJ7d8C80+GdX2kLA6VURMQ4dOBu0qRJpqioyJHvjjsHKmHpf8LHz0HeCLjoT9bplEqpbkdEVhtjJnW0nF6hGg9Sc+DS/7FbGByA+efBG3dqCwOlVJs03ONJcwuD6+HD/4HHpsKWfzpdlVIqBmm4x5vkDPj67+C7S8GTDH+9FJbcqC0MlFIH0XCPVwVT4Ib34bT/gPWLrRYGG5boxU9KKUDDPb55fXDWT+D6ZdbZNS9cC89fqS0MlFIa7gmhzwlw3Ttwzi9gy9vawkAppeGeMNweOOVWq4VB70KrhcGCi6Byq9OVKaUcoOGeaHKHwDV/hwsfgl1r4bGTYcUjEAo6XZlSqgtpuCcilwsmfRfmfgDHnQ5v3QNPngN7NjhdmVKqi8RduPtLS/nqvvvw797tdCmxr0c/mPUcfPNJqNoG/zMNlv1aWxgo1Q3EXbjXrVnD3pde5vNzz2PPAw8SqKpyuqTYJgKF34Kb/g1jvgn/70Er5HescroypVQnirtwz5w+nSFvvEHmBRdQuWABn599DmWPPEqwRi/Fb1daLlw6D6540Wpb8OQ58MZd2sJAqQQV143DGrZsoeyPD1P9j3/gzsoi9/vfJ/vyWbiSk6NUZYJqqIa3fw6r/heyCqz+8UO+5nRVSqkIdIvGYclDh9L/Tw8z6IXF+I4/ntIHH+Tz86ZT9cILmIDe4KJNyRlwwX/B7KXgToZnLoFX5moLA6USSFzvuR+q9oMPKX3o99Sv+5ikgQPJv/UWMqZPR1xxvQ3rXP56WP5beP8PkJoL5z8Aoy4Ct9fpypRSrYh0zz2hwh3AGEPNsmWUPfQHGj77jORRo+h5262kTZuGiET9+xLGVx/Dqz+wbu/ny4Lh02HkBTD0LEhKc7o6pZSt24Z7ExMMsv+11yh7+E/4S0pImTiRnrf/kNSJEzvtO+NeMADFS2HTa1D8BtRVgccHx51pBf2I8yEtz+kqlerWun24NzGNjex96SXKH3ucQFkZaadPo+dtt+EbNarTvzuuBQOwfYUV9Jteg307QFxQMNUK+pEXQPYgp6tUqtvRcD9EqK6OqoULKf/fJwjt20fm188n/5ZbSBo0qMtqiFvGwO6PW4J+j32z7l5j7KC/0Opno8NeSnU6Dfc2BPfvp2L+fCqfXoBpbCTr0kvJu2ku3t69u7yWuFX5BWx+HT79P9jxAZgQ9Cho2aMvmGo1MlNKRV1Uw11EpgN/BNzAE8aYBw6ZXwA8DWTZy9xljHm9vXU6fYPsQHk55f8zj73PPQciZF9+Obnfn4MnO9uxmuJSTVnLOP3n70CwAVKyYfj5VtAP+RokpTpdpVKOCYYM/mCIQMgQCIZoDIZI8brJ8B3dGWlRC3cRcQPFwDlACbAKmGWM2Ri2zDzgI2PM4yJyPPC6MWZQe+t1Otyb+HfupOyRR9n3t7/hSkkhZ/Zscq69Fne6niFyxBpqrIDf9H9W4NfvA0+KdcbNyAusM3BScxwpzRhDwAQIhoIEQgGC5pDnULBlftjzYfOaPnMU6wqFQrjEhYhYz1jPbnE3vxf+fvgj/D0RwcUh88PeE5GWdYa95xIXLg6Z38F3xgJjTHMw+oOGQChkhyX2eyH8IUOweZ4hGDIEQ1agWuFqzQvanwkaQyAQImDCppuWCdrf1/zZkL0+a36gaX7QEDT28sbgD1jTgSDN3x0IhVq9rcJVk8fw8wumHtW/R6ThHslv58nAFmPMVnvFzwEzgI1hyxgg037dA9h1ZOU6x9uvH31/82tyv/ddyh7+E+WPPELVwoXkzpmjV7seAWMMjR4vjUPPpHHwKfjP/gkNO1bS+Pk7NH6xHP8Xb9H4ppuGXqNp7DcBf58TaEzpQWOwkYZgA/6gn8ZQI43BxpbnYMu0P+inIdjQ/Lrp/UAoEFHAhkzIsX8bj3hwu9y4xIUxhpAJESKEMYag0VbMccHFEV3y6QKS7EdrGlIvB44u3CMVyZ77t4Dpxpjr7OmrgJOMMT8IW6YP8BaQDaQBZxtjVre33ljZcz9U3fpPKHvoIWpXrMDTuzd5N80l65JLEE9sjyEbY/CHrABsCDZQH6i3noP1NAYbD5puCDQ0Lxe+bNPj0GA9LGDDphtDVjgHQtG7IjjJlUSyOxmv20uSO4kkV5L1HPba6/aS5ErC6/LidrmtZ3HjdrlxSxvT9muPy4PH5Wme75E2po9y+fDvatpD7uh/u/DAD5lQy4MQgWCIA34/BxoC1PkD1Pn9HGj0268D1PsDB72u9/upDwSp9wepDwRoCASo9wet50CAxmCABn+QhmCQxkCAhkCQxmCQkDEgIcAgcmwbQ4/LhdcluN2Cx+XC7bbe87gEj9uF2yV4XILbJXjdgtue1/R+07JNn7eWCZvvPnh5t9uFRwRX2HoPfXhcgruVZZz4hVKQWcCw7GFH9dlo7rlHYhbwlDHmv0VkKvCMiIwx5uDdJRGZA8wBKCgoiNJXR1dK4RgK5j9J7QcfUvbQQ+z+yU+pfOLJo7ra1RjDvoZ9VPurrYC1g7U5YEMNzUF7aMAeFs5hodxaQDcEG45p7zTZndz8CA/SppBN86SRlNxKyLqsAG76XNN0a4Hc/KguJWnbCpK2Lse7ay3JJkRSZn+SRlyAZ+SFyMCp4HIf9d/iFGMM9f4Q++v8VNc3sq8uwP56P9X1AfbX+alpCFDXaIXugcYgdX7rUR/2uml+y2trjPZIuQRSkzz4vEmkJLlI8bpJ8bpJ87rJ87lJybCmfUnu5nkpSW589muf10Wyx02SxwrWJLcLr8dlPbtdJHnEframvW5rXpLHCm7lvEj23KcC9xljzrOn/xPAGPObsGU2YO3d77CntwJTjDGlba03Vvfcwx12tevxo+h52224p55IRX0F5XXlVNRZz2V1ZYdNV9RXHPEerSBWyHqsoPW5fdZrl/Wez+07aH74Mj63jyR30kHTzaHdynSyOxmfx0eSK8m5q3drSmHzG9YB2a3LINhotUEYcb51iuVxZ4A3pUtKCYUMNY2B5jC2QtoK6P11fvbXB6iu97M/PLQPmecPdnyCgs/bErZtheth03ZA++z3O/q81y16RXaCiuYBVQ/WAdWzgJ1YB1QvN8ZsCFvmDeB5Y8xTIjIK+CfQz7Sz8lgL95AJUVVf1RzQTWFdXldOZW05ef/axNTXvyS3MsCn/eHZM9xsHtDyH48g5PhyyE/NJzcllzxfHnkp1iMjKSPi8PW6vN33P8qGatjyT/sK2TehYR94U+0DshfCsHPbPSAbCIbCArcpgFte728K7bC96ab3quv9VDcEOryneIrXTWaKhwyfl0yfh8wU7yGvPWT6vAe97pFiPaf7PPg8bly6Z6uOQbRPhfw68Aes0xznG2N+JSK/AIqMMa/aZ8j8L5COdXD1x8aYt9pbZ1eF+wH/gcPC+tBHRV0FFfUVrR7cSvOmkZeSR64vl57eHCauqmLUq5+QVFVL4+QxJM39LvljTyQrOQuPK7bH5eNJ0N9A7eZ3CX36Gqlbl5JUt4eQuCnJHM+6tFNZ4Z3CloYe7D3Qsgd9oLHjg5NNgZthh3FmWDBn+uzQTmlapuV1U1h73dqETjkroS9iCoaCVNZXthnW4Y8DgQOHfd4tbnJ9ueSm5JKfmt8c3nkpec3Teb48clNySfUefo62Xu165IwxVDcEqKhppKKmgfKaRiprrdcVtY2U1zRQ0fRebQOVtY2E7P9rCiEK5QvOdRdxnquIYa6dAHzuHUZx+mR2Zo6jMmc8vvSsVvecm/a005M9Oh6s4l7ChvvSL5dy5/I7Wz14mOHNIC8176BwPjSs81PzyUrOwiXHvgfWfLXrgmcwDQ3d7mrXusagFcq1jVTWWoHdFN6VtY2UN4W3HdptHRjM9HnITU8mNy2J3PSkltdp9uv0JPLSk8lJSyI7NQl35RZr6Gbz61BSBCZo9b3pXQgFJ8PAqdZzen4X/4so1fkSNtzJbmAsAAAScklEQVQ/3/s5S79c2jKmbYd5ri8Xn8fXCZV2LFGudjXGUFrd0LwXXVHb9NwS0uV2kFfUNLY5DOLzushLTyY3PZm8tCRy7JDOS7fCOyfNCu+89GSy07wke47hzJiGGihZBdtXwrYVVtgH6qx5uUOtVggDT7YeWQO1/42Kewkb7rHMv3MnZY8+xr5XXonpq11DIcPOvXV8VlrNZ3tqKN5Tw5bSaraU1lDbSmB73UJuWrId0lYoN+9VH7q3nZ5EapKDxx4CjVZP+u0rrLDfvtK6UhYgo6+9V28Hfv4o0Bu5qDij4e6ghs8/t+7t+tZbuLOzybzgAnwjR5A8YiTJw4bi8nXNL4ymEC/eU81npTUU77ECfEtpzUF73T0zkhnWK51hPTMYkp9GfoaPvPSWPe5Mnyd+z+AJhaDs05ag37YSqu0LqH1ZdtDbwzh9xoKnrWsKlYoNGu4xoG79J5Q/8gi1q1ZhDtgHdl0ukgYNssJ++AiSR47AN2IEnt69jzpAQyFDSVVLiH9mP28praHO3xLivTKTGdYzoznIh9vPPVK70S31jIGqL1uGcbavhIot1jxPCvSfZO3VF0yFAZP1LlQq5mi4xxATCuHfsYP6zZtp2LSZ+mLr2V9S0ryMq0cPfMOHkzxyJL4Rw1vdyw+GDDsqDxy0F/6ZPZxS7285WNk709cc4MN6pTO8VzpD87tZiB+JmtKWsN+2wupXb0Lg8lh7803DOAVTHWt8plQTDfc4EKypoaG4mIbNm6nftNl6Li5u3ss3LhcHevZld25/Nqf1Zo0nl83pfShP6QEi9OnhY2jPdIb3ymBYz3SG9cpgaM90eqRoiB+T+v2w49/2uP1K2FlkXTkLkD/SDnr7rJwe/Z2tVXU7Gu5xIBgybKuoPWgo5bPd+6n5cjv9K3cyeN8ujtv/FUOrd9Ozprz5cyYjg+ThI0gbNdIeyx9B8rBhXTaW3+3462HXmpZhnO0fQmO1Na9HwcEHafOG6xk5qlNpuMeQQDDEtsoDVoDvqWkeVtlaXktjoGU4pV9Wir0n3jKkMrRnOhk+b4d7+bhcJA0caI/hjyR5xHB8I0ce01i+akMoaA3dbAs7I6e2zJqXmgcFU1qGcXqfoHelUlGl4R4DSvfXc+dLH/OvLRUHXcDTLyvFHgvPaB5WGdoznfTkIwsBEwrhLymhftMmGjYXU7/Zevbv2NG8TPNY/ogRupffWYyBis9bhnG2/Qv2brPmJaVD/xOtsB8wGXKGQGbfuOx6qWKDhrvDVmwp55bnPqK2IcjlJxUwsncGw3tlMOQoQvxIWXv5n9GweVPzQdyG4mJCre7lW4HvGzECT58+upcfLft3HXz6ZemGlnkuL2QVQPZAyB508CNrIKRkOVOzigsa7g4JhQyPLtvCQ28Xc1x+Oo9dMYHhvTKcLqtlL7/pjJ3W9vIzM0keOhRPbi7unBzc2Vl4cnJwZ+fgzs7Gk5ONOzsbd06O3qHqSNVVWRdXVX15yGMb1FUevKwv65DQD9sI9BgAbj1g3p1puDugsraR255fy/LiMi4e15dfXVJIWifvpR+rQ/fyGz/fSqCqkmBlFcG9e62LgFrhSk1tDnp3dhYeewPgzslp2Qhkt2wgXJmZ+qugLfX7rJAPD/2921rCP+RvWVZc1hk6WYfu9Q+2NgKpuXpAN8FpuHex1dsq+cGzH1FR08jPLjqeyycXxH2YmVCI4L59BKuqCFZVEaistF5XVhGsqiTQ/LqqeYNg6utbX5nbbe39Z2cfvlE46FeCtVHwZGchSXq1KKEgVH918J5++Eag9pD74SSlHz7M0/y6ALx6rCXedfVt9rotYwxPvv8FD7yxib5ZKbw892TG9OvhdFlRIS4XHjuQIxWqqyNYWUmgai/Bqkr79eEbgYbNmwlWVhLct6/NdbnS01v5ZZBtbQSyrNfuzExc6em40tJxpaXiTk9HvAk0bOFyW3vqPfrDoFMPn99YC3u3Hz7cU7HFuvFJUxO1Jhl92x7rT++lvXYSiO65H4N9dX7ueGEdb23cw3mje/Hbb43VC4iOkAkErF8H4RuBvfavBHuDYP1K2GttDCorMX5/u+uU5GRcaWl26KfhDnvd8pyGu+l1Wrr9fmrLe/azuOP4rBZjrKtvDw3+piGf/buw7q1j8/jC9vQHWhuUjD7WI7MvZPTWdgwxQPfcO9n6kn3MfXY1X+2t594LRvG9UwfH/TCME8TjwZObiyc3l0gO0RpjCNUeaP5VEKyuIVRTQ6i2llBtDcGm1zW19rM131+656D3TENDZPWlpFgbgrRINg72+03Lhc9PTTmim6tHhQhk9LIeBScdPt9fD/t2tH6Qd9uKlgu1wiX3gMw+VtBn2IGf2TdsI9AH0nrquf0xQP8XOELGGBZ+uJ1f/H0juelJPP/9qUwcGF992+OZiOBOT7PaKA8YcNTrMX4/odpagk2BX9uykThoA3HYhuMA/p07mzcSwdpa6OCXhF04kpKCuN1WyLvd4HYhrsOfxeO2hmPamN/87HG3P9/tQtyejtfTvNwgxDUEUlxImgtMAAnUIP4aaNyPNO5DGvbDnipkexk0FCP1VQhBcFnbEnEZ629Ny4GMfCQjH8noCRm9kB69kR7WxkCy+kBaLuLxWL+OPHHceTRGabgfgdqGAHcvWc/f1u7i9OH5PDRzHDlpetAvHonXizsrC3fWsZ9THmpsbNkI2L8Ugof+eqitIXSgDhMMQjCICQUhGGr9ORDEhEJtL+f3E4pkuWAo7PtCrT63dTZU5NrbsamwH5siW5WAuK0Nn7WxcSNer/Xa3gg0bwjcLRu3pvcQELG3Mi5Xx9MusTYoze9FMO1yWYU2re+gaUFccvh02OcRQcRF6qSJpJ188jH+27dPwz1Cm3dXM3fhar4or+U/zh3O3DOG6l3sFQCupCRcOTmQE38dI40xEArbCARDEApa06EQJhCEYAATDGICAXuZYPNz83uBICZ46OtQy3sNdZjaSqitwhyogtq9mLq9mLr9ULcfU1eNqa+xvssIhKxDBsYIuJIxnlRwp2DcPowrGVwujLgw4gZXkrWcAYyxN2x2/cZgTMiaF7JWetB0KAQYTMhYX9i8TNvTh66PUMg6ctHe9CHry73+utgIdxGZDvwRcANPGGMeaGWZbwP3Yf3Z64wxl0exTke9tLqEe15ZT3qyl79edxInD8lzuiSlokJE7D3lGDhwbIx1sVf1buuGKtW7Yf9XYa93WaeF1uzhoAPBAOK22jGn5Vvn+qflQ1pe29O+LEfPDOqKE1k6DHcRcQOPAucAJcAqEXnVGLMxbJlhwH8CpxhjqkSkZ2cV3JXq/UF+9rcNPF+0g5MG5/CnWePpmannCSvVKUSsgE7NgV7Ht71cMGAFfPhGoPorqC2HAxVWE7ev1lnTDW2cauvytIR+am5Y8OfZrw+Z9vWI6sVhXXF8IZI998nAFmPMVgAReQ6YAWwMW+Z64FFjTBWAMab0sLXEmS/Ka7nxr6vZtLuam84cwg/PHo7HrecAK+U4twd69LMeTGx/2UCDHfjlVug3hX9tORwot98vh51rrHkN+1tfj8trhXxqHqTltr8hSMuD5EzHrxSOJNz7ATvCpkuAQ8+rGg4gIv/CGrq5zxizNCoVOuD19V/x4xc/xuMW/jL7RM4ckRA/RJTqfjzJ1qmamX0jW95f37IBCA//A/bGodaeV/Wl9X5jTevrcScdHPap9gagacPQZxz0OSFqf2ZronVA1QMMA84A+gPLRaTQGLM3fCERmQPMASgoKIjSV0dPYyDEr1//lKdWfMn4giweuXwC/bJSnC5LKdVVvL6wXwUR8Ncd/ivg0A3BgXLriuHaCvDXWp879faYCPedQPgJxf3t98KVAB8aY/zAFyJSjBX2q8IXMsbMA+aBdYXq0RbdGUqqDnDTsx+xbsdevnvKYO46fyRJHh2GUUq1w5sCWQOsRyQaD1hh7+n8Y3eRhPsqYJiIDMYK9e8Ah54J8wowC/iLiORhDdNsjWahnemdTXv44fPrCIUMj18xgfML+zhdklIqESWlQlLXjFp0GO7GmICI/AB4E2s8fb4xZoOI/AIoMsa8as87V0Q2AkHgDmNMRWcWHg2BYIj//kcxj7/7Ocf3yeSxKyYwKE97Zyil4l+3bRy2Z389Ny/6iH9/UcmsyQX87BvH4/PGwLm+SinVDm0c1o5/bSnnVvsWeA/NHMsl4/s7XZJSSkVVtwr3UMjwiH0LvCH56Sy6fgLDYuAWeEopFW3dJtwrahq47fm1vPdZedzcAk8ppY5Wt0i3oi+tW+BVHmjk15cUMmvyAG0vqpRKaAkd7sYYnnjvCx5Yuon+2Sm8fGPi3AJPKaXak7Dhvu+An/94cR3/2LiH6aN789vLTiDTp7fAU0p1DwkZ7uG3wPvphccz+5RBOgyjlOpWEircjTH89cPt/PLvG8lLT2LxDVOZUKC3wFNKdT8JE+41DQHufnk9r67bxRkj8nno2+PI1lvgKaW6qYQI9827q7lx4Wq+LK/ljvNGcOPpQ/QWeEqpbi3uw/3F1SXcq7fAU0qpg8RtuIffAm/KcTk8PGs8PTP0FnhKKQVxGu5by2qYu3ANm3ZX84Mzh3Lb2cP0FnhKKRUm7sL93c2l/ODZj/DqLfCUUqpNcRfuA3PTmDAwmwcuLaSv3gJPKaVaFXfhPjgvjQXfnex0GUopFdN0oFoppRKQhrtSSiUgDXellEpAGu5KKZWAIgp3EZkuIptFZIuI3NXOct8UESMiHd68VSmlVOfpMNxFxA08CpwPHA/MEpHjW1kuA7gV+DDaRSqllDoykey5Twa2GGO2GmMageeAGa0s90vgQaA+ivUppZQ6CpGEez9gR9h0if1eMxGZAAwwxrwWxdqUUkodpWO+iElEXMDvgWsjWHYOMMeerBGRzUf5tXlA+VF+1gnxVG881QrxVW881QrxVW881QrHVu/ASBaKJNx3AgPCpvvb7zXJAMYA79q3susNvCoiFxljisJXZIyZB8yLpLD2iEiRMSZuDtrGU73xVCvEV73xVCvEV73xVCt0Tb2RDMusAoaJyGARSQK+A7zaNNMYs88Yk2eMGWSMGQR8ABwW7EoppbpOh+FujAkAPwDeBD4FFhtjNojIL0Tkos4uUCml1JGLaMzdGPM68Poh7/20jWXPOPayOnTMQztdLJ7qjadaIb7qjadaIb7qjadaoQvqFWNMZ3+HUkqpLqbtB5RSKgHFXbhH2gohFojIfBEpFZFPnK6lIyIyQESWichGEdkgIrc6XVNbRMQnIv8WkXV2rT93uqZIiIhbRD4Skf9zupb2iMiXIrJeRNaKSMyfGCEiWSLyoohsEpFPRWSq0zW1RkRG2P+mTY/9InJbp31fPA3L2K0QioFzsC6mWgXMMsZsdLSwNojINKAGWGCMGeN0Pe0RkT5AH2PMGruVxGrg4lj8txXrnNs0Y0yNiHiB94FbjTEfOFxau0TkdmASkGmMudDpetoiIl8Ck4wxcXHeuIg8DbxnjHnCPqMv1Riz1+m62mNn2U7gJGPMts74jnjbc4+0FUJMMMYsByqdriMSxpivjDFr7NfVWGdG9Wv/U84wlhp70ms/YnovRUT6AxcATzhdSyIRkR7ANOBJAGNMY6wHu+0s4PPOCnaIv3DvsBWCOnYiMggYTww3gbOHONYCpcA/jDExW6vtD8CPgZDThUTAAG+JyGr7qvJYNhgoA/5iD3k9ISJpThcVge8AizrzC+It3FUnE5F04CXgNmPMfqfraYsxJmiMGYd1xfRkEYnZYS8RuRAoNcasdrqWCJ1qjJmA1Qn2Jnt4MVZ5gAnA48aY8UAtEOvH4pKAi4AXOvN74i3cO2qFoI6BPX79ErDQGPOy0/VEwv4JvgyY7nQt7TgFuMgey34O+JqI/NXZktpmjNlpP5cCS7CGQ2NVCVAS9svtRaywj2XnA2uMMXs680viLdzbbYWgjp59kPJJ4FNjzO+drqc9IpIvIln26xSsA+ybnK2qbcaY/zTG9Lfbc3wHeMcYc6XDZbVKRNLsA+rYwxvnAjF7tpcxZjewQ0RG2G+dBcTcSQCHmEUnD8lAFLpCdiVjTEBEmlohuIH5xpgNDpfVJhFZBJwB5IlICfAzY8yTzlbVplOAq4D19lg2wN321cmxpg/wtH3GgQurJUZMn14YR3oBS+wmgB7gWWPMUmdL6tDNwEJ7h28rMNvhetpkbzDPAb7f6d8VT6dCKqWUiky8DcsopZSKgIa7UkolIA13pZRKQBruSimVgDTclVIqAWm4K6VUAtJwV0qpBKThrpRSCej/A6WynEPcgxsrAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "pd.DataFrame(h.history).plot()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Conclusion\n",
    "\n",
    "Classifier shows 79.1% accuracy with built in tokenizer and 81.3% with Bling Fire. So importance of the tokenization cannot be underestimated, even more the correctness and consistency of tokenization is important if model uses pre-trained embeddings.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}