Getting_started_with_BigQuery.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/baranouskiart/jobboard_salary_pl/blob/master/Getting_started_with_BigQuery.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "KcAZ2RHCg_Ze"
      },
      "source": [
        "# Before you begin\n",
        "\n",
        "\n",
        "1.   Use the [Cloud Resource Manager](https://console.cloud.google.com/cloud-resource-manager) to Create a Cloud Platform project if you do not already have one.\n",
        "2.   [Enable billing](https://support.google.com/cloud/answer/6293499#enable-billing) for the project.\n",
        "3.   [Enable BigQuery](https://console.cloud.google.com/flows/enableapi?apiid=bigquery) APIs for the project.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qTmLBxDxBAZL"
      },
      "source": [
        "### Provide your credentials to the runtime"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "SeTJb51SKs_W"
      },
      "outputs": [],
      "source": [
        "from google.colab import auth\n",
        "auth.authenticate_user()\n",
        "print('Authenticated')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UMKGkkZEPVRu"
      },
      "source": [
        "## Optional: Enable data table display\n",
        "\n",
        "Colab includes the ``google.colab.data_table`` package that can be used to display large pandas dataframes as an interactive data table.\n",
        "It can be enabled with:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "LMNA-vBHPyHz"
      },
      "outputs": [],
      "source": [
        "%load_ext google.colab.data_table"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "y7QuudxdP0jH"
      },
      "source": [
        "If you would prefer to return to the classic Pandas dataframe display, you can disable this by running:\n",
        "```python\n",
        "%unload_ext google.colab.data_table\n",
        "```"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OHCBQv4uTCPc"
      },
      "source": [
        "# Use BigQuery via magics\n",
        "\n",
        "The `google.cloud.bigquery` library also includes a magic command which runs a query and either displays the result or saves it to a variable as a `DataFrame`."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 103
        },
        "id": "DMxOKQ3vJplj",
        "outputId": "2cdcd63b-e9a1-4f44-8b75-59be9b487a1e"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.module+javascript": "\n      import \"https://ssl.gstatic.com/colaboratory/data_table/81868506e94e6988/data_table.js\";\n\n      window.createDataTable({\n        data: [[{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 114420316,\n            'f': \"114420316\",\n        }]],\n        columns: [[\"number\", \"index\"], [\"number\", \"total_rows\"]],\n        rowsPerPage: 25,\n      });\n    ",
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>total_rows</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>114420316</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   total_rows\n",
              "0   114420316"
            ]
          },
          "execution_count": 0,
          "metadata": {
            "tags": []
          },
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# Display query output immediately\n",
        "\n",
        "%%bigquery --project yourprojectid\n",
        "SELECT\n",
        "  COUNT(*) as total_rows\n",
        "FROM `bigquery-public-data.samples.gsod`"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "cLh05lDpJ4wj"
      },
      "outputs": [],
      "source": [
        "# Save output in a variable `df`\n",
        "\n",
        "%%bigquery df --project yourprojectid\n",
        "SELECT\n",
        "  COUNT(*) as total_rows\n",
        "FROM `bigquery-public-data.samples.gsod`"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 110
        },
        "id": "xgopBClCKs_0",
        "outputId": "cd645873-f3f1-413a-95ef-2b32c4928460"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.module+javascript": "\n      import \"https://ssl.gstatic.com/colaboratory/data_table/a036b366c3cace79/data_table.js\";\n\n      window.createDataTable({\n        data: [[{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 114420316,\n            'f': \"114420316\",\n        }]],\n        columns: [[\"number\", \"index\"], [\"number\", \"total_rows\"]],\n        rowsPerPage: 25,\n        helpUrl: \"https://colab.research.google.com/notebooks/data_table.ipynb\",\n      });\n    ",
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>total_rows</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>114420316</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   total_rows\n",
              "0   114420316"
            ]
          },
          "execution_count": 0,
          "metadata": {
            "tags": []
          },
          "output_type": "execute_result"
        }
      ],
      "source": [
        "df"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fkhbyGaXKs_6"
      },
      "source": [
        "# Use BigQuery through google-cloud-bigquery\n",
        "\n",
        "See [BigQuery documentation](https://cloud.google.com/bigquery/docs) and [library reference documentation](https://googlecloudplatform.github.io/google-cloud-python/latest/bigquery/usage.html).\n",
        "\n",
        "The [GSOD sample table](https://console.cloud.google.com/bigquery?p=bigquery-public-data&d=samples&t=gsod&page=table) contains weather information collected by NOAA, such as precipitation amounts and wind speeds from late 1929 to early 2010.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "goQQ96EDKs_7"
      },
      "source": [
        "### Declare the Cloud project ID which will be used throughout this notebook"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "hWK0nJWgKs_8"
      },
      "outputs": [],
      "source": [
        "project_id = '[your project ID]'"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lK-9fSClKtAB"
      },
      "source": [
        "### Sample approximately 2000 random rows"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "id": "ONI1Xo0-KtAD",
        "outputId": "54c89381-2ead-4b2d-f234-e1e356dd5cdd"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Full dataset has 114420316 rows\n"
          ]
        }
      ],
      "source": [
        "from google.cloud import bigquery\n",
        "\n",
        "client = bigquery.Client(project=project_id)\n",
        "\n",
        "sample_count = 2000\n",
        "row_count = client.query('''\n",
        "  SELECT\n",
        "    COUNT(*) as total\n",
        "  FROM `bigquery-public-data.samples.gsod`''').to_dataframe().total[0]\n",
        "\n",
        "df = client.query('''\n",
        "  SELECT\n",
        "    *\n",
        "  FROM\n",
        "    `bigquery-public-data.samples.gsod`\n",
        "  WHERE RAND() < %d/%d\n",
        "''' % (sample_count, row_count)).to_dataframe()\n",
        "\n",
        "print('Full dataset has %d rows' % row_count)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Bu2C7iWMKtAM"
      },
      "source": [
        "### Describe the sampled data"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 259
        },
        "id": "XrFmZBr9KtAN",
        "outputId": "491c8e26-9333-4cf0-bc29-15e55cf101c7"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.module+javascript": "\n      import \"https://ssl.gstatic.com/colaboratory/data_table/a036b366c3cace79/data_table.js\";\n\n      window.createDataTable({\n        data: [[\"count\",\n{\n            'v': 1979.0,\n            'f': \"1979.0\",\n        },\n{\n            'v': 1979.0,\n            'f': \"1979.0\",\n        },\n{\n            'v': 1979.0,\n            'f': \"1979.0\",\n        },\n{\n            'v': 1979.0,\n            'f': \"1979.0\",\n        },\n{\n            'v': 1979.0,\n            'f': \"1979.0\",\n        },\n{\n            'v': 1979.0,\n            'f': \"1979.0\",\n        },\n{\n            'v': 1979.0,\n            'f': \"1979.0\",\n        },\n{\n            'v': 1883.0,\n            'f': \"1883.0\",\n        },\n{\n            'v': 1883.0,\n            'f': \"1883.0\",\n        },\n{\n            'v': 1460.0,\n            'f': \"1460.0\",\n        },\n{\n            'v': 1460.0,\n            'f': \"1460.0\",\n        },\n{\n            'v': 741.0,\n            'f': \"741.0\",\n        },\n{\n            'v': 741.0,\n            'f': \"741.0\",\n        },\n{\n            'v': 1776.0,\n            'f': \"1776.0\",\n        },\n{\n            'v': 1776.0,\n            'f': \"1776.0\",\n        },\n{\n            'v': 1950.0,\n            'f': \"1950.0\",\n        },\n{\n            'v': 1950.0,\n            'f': \"1950.0\",\n        },\n{\n            'v': 1922.0,\n            'f': \"1922.0\",\n        },\n{\n            'v': 241.0,\n            'f': \"241.0\",\n        },\n{\n            'v': 1977.0,\n            'f': \"1977.0\",\n        }],\n [\"mean\",\n{\n            'v': 505585.599292572,\n            'f': \"505585.599292572\",\n        },\n{\n            'v': 89647.44113188479,\n            'f': \"89647.44113188479\",\n        },\n{\n            'v': 1987.1814047498738,\n            'f': \"1987.1814047498738\",\n        },\n{\n            'v': 6.525517938352704,\n            'f': \"6.525517938352704\",\n        },\n{\n            'v': 15.715512885295604,\n            'f': \"15.715512885295604\",\n        },\n{\n            'v': 52.39186460128738,\n            'f': \"52.39186460128738\",\n        },\n{\n            'v': 13.018696311268318,\n            'f': \"13.018696311268318\",\n        },\n{\n            'v': 42.01869358592016,\n            'f': \"42.01869358592016\",\n        },\n{\n            'v': 12.982474774296335,\n            'f': \"12.982474774296335\",\n        },\n{\n            'v': 1015.2786300032107,\n            'f': \"1015.2786300032107\",\n        },\n{\n            'v': 11.506164383561643,\n            'f': \"11.506164383561643\",\n        },\n{\n            'v': 967.3964905526474,\n            'f': \"967.3964905526474\",\n        },\n{\n            'v': 11.979757085020243,\n            'f': \"11.979757085020243\",\n        },\n{\n            'v': 11.529279278473812,\n            'f': \"11.529279278473812\",\n        },\n{\n            'v': 12.73704954954955,\n            'f': \"12.73704954954955\",\n        },\n{\n            'v': 6.90338461350172,\n            'f': \"6.90338461350172\",\n        },\n{\n            'v': 12.96974358974359,\n            'f': \"12.96974358974359\",\n        },\n{\n            'v': 12.297658639755507,\n            'f': \"12.297658639755507\",\n        },\n{\n            'v': 24.82946051205837,\n            'f': \"24.82946051205837\",\n        },\n{\n            'v': 44.05923116046956,\n            'f': \"44.05923116046956\",\n        }],\n [\"std\",\n{\n            'v': 302491.1873178059,\n            'f': \"302491.1873178059\",\n        },\n{\n            'v': 27088.238467141528,\n            'f': \"27088.238467141528\",\n        },\n{\n            'v': 15.993487610677022,\n            'f': \"15.993487610677022\",\n        },\n{\n            'v': 3.419259475533833,\n            'f': \"3.419259475533833\",\n        },\n{\n            'v': 8.661562572969782,\n            'f': \"8.661562572969782\",\n        },\n{\n            'v': 23.32984218910621,\n            'f': \"23.32984218910621\",\n        },\n{\n            'v': 7.886976768461429,\n            'f': \"7.886976768461429\",\n        },\n{\n            'v': 21.691901647557273,\n            'f': \"21.691901647557273\",\n        },\n{\n            'v': 7.914004511688532,\n            'f': \"7.914004511688532\",\n        },\n{\n            'v': 9.325395155489899,\n            'f': \"9.325395155489899\",\n        },\n{\n            'v': 7.516349304555582,\n            'f': \"7.516349304555582\",\n        },\n{\n            'v': 71.7740001047588,\n            'f': \"71.7740001047588\",\n        },\n{\n            'v': 7.7834453876849885,\n            'f': \"7.7834453876849885\",\n        },\n{\n            'v': 8.320364935530248,\n            'f': \"8.320364935530248\",\n        },\n{\n            'v': 7.843475880085762,\n            'f': \"7.843475880085762\",\n        },\n{\n            'v': 5.039770886455209,\n            'f': \"5.039770886455209\",\n        },\n{\n            'v': 7.860416933750618,\n            'f': \"7.860416933750618\",\n        },\n{\n            'v': 6.77895967165126,\n            'f': \"6.77895967165126\",\n        },\n{\n            'v': 8.776778375546268,\n            'f': \"8.776778375546268\",\n        },\n{\n            'v': 23.344521106124706,\n            'f': \"23.344521106124706\",\n        }],\n [\"min\",\n{\n            'v': 10100.0,\n            'f': \"10100.0\",\n        },\n{\n            'v': 13.0,\n            'f': \"13.0\",\n        },\n{\n            'v': 1933.0,\n            'f': \"1933.0\",\n        },\n{\n            'v': 1.0,\n            'f': \"1.0\",\n        },\n{\n            'v': 1.0,\n            'f': \"1.0\",\n        },\n{\n            'v': -91.4000015258789,\n            'f': \"-91.4000015258789\",\n        },\n{\n            'v': 4.0,\n            'f': \"4.0\",\n        },\n{\n            'v': -63.79999923706055,\n            'f': \"-63.79999923706055\",\n        },\n{\n            'v': 4.0,\n            'f': \"4.0\",\n        },\n{\n            'v': 956.2999877929688,\n            'f': \"956.2999877929688\",\n        },\n{\n            'v': 4.0,\n            'f': \"4.0\",\n        },\n{\n            'v': 604.5,\n            'f': \"604.5\",\n        },\n{\n            'v': 4.0,\n            'f': \"4.0\",\n        },\n{\n            'v': 0.0,\n            'f': \"0.0\",\n        },\n{\n            'v': 4.0,\n            'f': \"4.0\",\n        },\n{\n            'v': 0.0,\n            'f': \"0.0\",\n        },\n{\n            'v': 4.0,\n            'f': \"4.0\",\n        },\n{\n            'v': 1.0,\n            'f': \"1.0\",\n        },\n{\n            'v': 8.0,\n            'f': \"8.0\",\n        },\n{\n            'v': -96.9000015258789,\n            'f': \"-96.9000015258789\",\n        }],\n [\"25%\",\n{\n            'v': 238255.0,\n            'f': \"238255.0\",\n        },\n{\n            'v': 99999.0,\n            'f': \"99999.0\",\n        },\n{\n            'v': 1978.0,\n            'f': \"1978.0\",\n        },\n{\n            'v': 4.0,\n            'f': \"4.0\",\n        },\n{\n            'v': 8.0,\n            'f': \"8.0\",\n        },\n{\n            'v': 39.29999923706055,\n            'f': \"39.29999923706055\",\n        },\n{\n            'v': 7.0,\n            'f': \"7.0\",\n        },\n{\n            'v': 30.34999942779541,\n            'f': \"30.34999942779541\",\n        },\n{\n            'v': 7.0,\n            'f': \"7.0\",\n        },\n{\n            'v': 1009.7000122070312,\n            'f': \"1009.7000122070312\",\n        },\n{\n            'v': 6.0,\n            'f': \"6.0\",\n        },\n{\n            'v': 952.9000244140625,\n            'f': \"952.9000244140625\",\n        },\n{\n            'v': 6.0,\n            'f': \"6.0\",\n        },\n{\n            'v': 6.300000190734863,\n            'f': \"6.300000190734863\",\n        },\n{\n            'v': 7.0,\n            'f': \"7.0\",\n        },\n{\n            'v': 3.4000000953674316,\n            'f': \"3.4000000953674316\",\n        },\n{\n            'v': 7.0,\n            'f': \"7.0\",\n        },\n{\n            'v': 7.800000190734863,\n            'f': \"7.800000190734863\",\n        },\n{\n            'v': 19.399999618530273,\n            'f': \"19.399999618530273\",\n        },\n{\n            'v': 32.0,\n            'f': \"32.0\",\n        }],\n [\"50%\",\n{\n            'v': 538980.0,\n            'f': \"538980.0\",\n        },\n{\n            'v': 99999.0,\n            'f': \"99999.0\",\n        },\n{\n            'v': 1990.0,\n            'f': \"1990.0\",\n        },\n{\n            'v': 7.0,\n            'f': \"7.0\",\n        },\n{\n            'v': 16.0,\n            'f': \"16.0\",\n        },\n{\n            'v': 55.0,\n            'f': \"55.0\",\n        },\n{\n            'v': 8.0,\n            'f': \"8.0\",\n        },\n{\n            'v': 44.0,\n            'f': \"44.0\",\n        },\n{\n            'v': 8.0,\n            'f': \"8.0\",\n        },\n{\n            'v': 1014.8500061035156,\n            'f': \"1014.8500061035156\",\n        },\n{\n            'v': 8.0,\n            'f': \"8.0\",\n        },\n{\n            'v': 995.5999755859375,\n            'f': \"995.5999755859375\",\n        },\n{\n            'v': 8.0,\n            'f': \"8.0\",\n        },\n{\n            'v': 9.300000190734863,\n            'f': \"9.300000190734863\",\n        },\n{\n            'v': 8.0,\n            'f': \"8.0\",\n        },\n{\n            'v': 5.800000190734863,\n            'f': \"5.800000190734863\",\n        },\n{\n            'v': 8.0,\n            'f': \"8.0\",\n        },\n{\n            'v': 11.100000381469727,\n            'f': \"11.100000381469727\",\n        },\n{\n            'v': 23.899999618530273,\n            'f': \"23.899999618530273\",\n        },\n{\n            'v': 46.400001525878906,\n            'f': \"46.400001525878906\",\n        }],\n [\"75%\",\n{\n            'v': 725273.5,\n            'f': \"725273.5\",\n        },\n{\n            'v': 99999.0,\n            'f': \"99999.0\",\n        },\n{\n            'v': 2000.0,\n            'f': \"2000.0\",\n        },\n{\n            'v': 10.0,\n            'f': \"10.0\",\n        },\n{\n            'v': 23.0,\n            'f': \"23.0\",\n        },\n{\n            'v': 69.80000305175781,\n            'f': \"69.80000305175781\",\n        },\n{\n            'v': 23.0,\n            'f': \"23.0\",\n        },\n{\n            'v': 56.70000076293945,\n            'f': \"56.70000076293945\",\n        },\n{\n            'v': 23.0,\n            'f': \"23.0\",\n        },\n{\n            'v': 1020.7999877929688,\n            'f': \"1020.7999877929688\",\n        },\n{\n            'v': 21.0,\n            'f': \"21.0\",\n        },\n{\n            'v': 1010.2999877929688,\n            'f': \"1010.2999877929688\",\n        },\n{\n            'v': 22.0,\n            'f': \"22.0\",\n        },\n{\n            'v': 13.825000047683716,\n            'f': \"13.825000047683716\",\n        },\n{\n            'v': 23.0,\n            'f': \"23.0\",\n        },\n{\n            'v': 9.100000381469727,\n            'f': \"9.100000381469727\",\n        },\n{\n            'v': 23.0,\n            'f': \"23.0\",\n        },\n{\n            'v': 15.899999618530273,\n            'f': \"15.899999618530273\",\n        },\n{\n            'v': 28.899999618530273,\n            'f': \"28.899999618530273\",\n        },\n{\n            'v': 60.79999923706055,\n            'f': \"60.79999923706055\",\n        }],\n [\"max\",\n{\n            'v': 999999.0,\n            'f': \"999999.0\",\n        },\n{\n            'v': 99999.0,\n            'f': \"99999.0\",\n        },\n{\n            'v': 2010.0,\n            'f': \"2010.0\",\n        },\n{\n            'v': 12.0,\n            'f': \"12.0\",\n        },\n{\n            'v': 31.0,\n            'f': \"31.0\",\n        },\n{\n            'v': 105.0999984741211,\n            'f': \"105.0999984741211\",\n        },\n{\n            'v': 24.0,\n            'f': \"24.0\",\n        },\n{\n            'v': 80.5,\n            'f': \"80.5\",\n        },\n{\n            'v': 24.0,\n            'f': \"24.0\",\n        },\n{\n            'v': 1052.9000244140625,\n            'f': \"1052.9000244140625\",\n        },\n{\n            'v': 24.0,\n            'f': \"24.0\",\n        },\n{\n            'v': 1037.0999755859375,\n            'f': \"1037.0999755859375\",\n        },\n{\n            'v': 24.0,\n            'f': \"24.0\",\n        },\n{\n            'v': 99.4000015258789,\n            'f': \"99.4000015258789\",\n        },\n{\n            'v': 24.0,\n            'f': \"24.0\",\n        },\n{\n            'v': 57.29999923706055,\n            'f': \"57.29999923706055\",\n        },\n{\n            'v': 24.0,\n            'f': \"24.0\",\n        },\n{\n            'v': 68.0,\n            'f': \"68.0\",\n        },\n{\n            'v': 66.0,\n            'f': \"66.0\",\n        },\n{\n            'v': 98.5999984741211,\n            'f': \"98.5999984741211\",\n        }]],\n        columns: [[\"string\", \"index\"], [\"number\", \"station_number\"], [\"number\", \"wban_number\"], [\"number\", \"year\"], [\"number\", \"month\"], [\"number\", \"day\"], [\"number\", \"mean_temp\"], [\"number\", \"num_mean_temp_samples\"], [\"number\", \"mean_dew_point\"], [\"number\", \"num_mean_dew_point_samples\"], [\"number\", \"mean_sealevel_pressure\"], [\"number\", \"num_mean_sealevel_pressure_samples\"], [\"number\", \"mean_station_pressure\"], [\"number\", \"num_mean_station_pressure_samples\"], [\"number\", \"mean_visibility\"], [\"number\", \"num_mean_visibility_samples\"], [\"number\", \"mean_wind_speed\"], [\"number\", \"num_mean_wind_speed_samples\"], [\"number\", \"max_sustained_wind_speed\"], [\"number\", \"max_gust_wind_speed\"], [\"number\", \"max_temperature\"]],\n        rowsPerPage: 25,\n        helpUrl: \"https://colab.research.google.com/notebooks/data_table.ipynb\",\n      });\n    ",
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>station_number</th>\n",
              "      <th>wban_number</th>\n",
              "      <th>year</th>\n",
              "      <th>month</th>\n",
              "      <th>day</th>\n",
              "      <th>mean_temp</th>\n",
              "      <th>num_mean_temp_samples</th>\n",
              "      <th>mean_dew_point</th>\n",
              "      <th>num_mean_dew_point_samples</th>\n",
              "      <th>mean_sealevel_pressure</th>\n",
              "      <th>num_mean_sealevel_pressure_samples</th>\n",
              "      <th>mean_station_pressure</th>\n",
              "      <th>num_mean_station_pressure_samples</th>\n",
              "      <th>mean_visibility</th>\n",
              "      <th>num_mean_visibility_samples</th>\n",
              "      <th>mean_wind_speed</th>\n",
              "      <th>num_mean_wind_speed_samples</th>\n",
              "      <th>max_sustained_wind_speed</th>\n",
              "      <th>max_gust_wind_speed</th>\n",
              "      <th>max_temperature</th>\n",
              "      <th>total_precipitation</th>\n",
              "      <th>snow_depth</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>count</th>\n",
              "      <td>1979.000000</td>\n",
              "      <td>1979.000000</td>\n",
              "      <td>1979.000000</td>\n",
              "      <td>1979.000000</td>\n",
              "      <td>1979.000000</td>\n",
              "      <td>1979.000000</td>\n",
              "      <td>1979.000000</td>\n",
              "      <td>1883.000000</td>\n",
              "      <td>1883.000000</td>\n",
              "      <td>1460.000000</td>\n",
              "      <td>1460.000000</td>\n",
              "      <td>741.000000</td>\n",
              "      <td>741.000000</td>\n",
              "      <td>1776.000000</td>\n",
              "      <td>1776.000000</td>\n",
              "      <td>1950.000000</td>\n",
              "      <td>1950.000000</td>\n",
              "      <td>1922.000000</td>\n",
              "      <td>241.000000</td>\n",
              "      <td>1977.000000</td>\n",
              "      <td>1793.000000</td>\n",
              "      <td>91.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>mean</th>\n",
              "      <td>505585.599293</td>\n",
              "      <td>89647.441132</td>\n",
              "      <td>1987.181405</td>\n",
              "      <td>6.525518</td>\n",
              "      <td>15.715513</td>\n",
              "      <td>52.391865</td>\n",
              "      <td>13.018696</td>\n",
              "      <td>42.018694</td>\n",
              "      <td>12.982475</td>\n",
              "      <td>1015.278630</td>\n",
              "      <td>11.506164</td>\n",
              "      <td>967.396491</td>\n",
              "      <td>11.979757</td>\n",
              "      <td>11.529279</td>\n",
              "      <td>12.737050</td>\n",
              "      <td>6.903385</td>\n",
              "      <td>12.969744</td>\n",
              "      <td>12.297659</td>\n",
              "      <td>24.829461</td>\n",
              "      <td>44.059231</td>\n",
              "      <td>0.065694</td>\n",
              "      <td>8.929670</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>std</th>\n",
              "      <td>302491.187318</td>\n",
              "      <td>27088.238467</td>\n",
              "      <td>15.993488</td>\n",
              "      <td>3.419259</td>\n",
              "      <td>8.661563</td>\n",
              "      <td>23.329842</td>\n",
              "      <td>7.886977</td>\n",
              "      <td>21.691902</td>\n",
              "      <td>7.914005</td>\n",
              "      <td>9.325395</td>\n",
              "      <td>7.516349</td>\n",
              "      <td>71.774000</td>\n",
              "      <td>7.783445</td>\n",
              "      <td>8.320365</td>\n",
              "      <td>7.843476</td>\n",
              "      <td>5.039771</td>\n",
              "      <td>7.860417</td>\n",
              "      <td>6.778960</td>\n",
              "      <td>8.776778</td>\n",
              "      <td>23.344521</td>\n",
              "      <td>0.283467</td>\n",
              "      <td>9.922707</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>min</th>\n",
              "      <td>10100.000000</td>\n",
              "      <td>13.000000</td>\n",
              "      <td>1933.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-91.400002</td>\n",
              "      <td>4.000000</td>\n",
              "      <td>-63.799999</td>\n",
              "      <td>4.000000</td>\n",
              "      <td>956.299988</td>\n",
              "      <td>4.000000</td>\n",
              "      <td>604.500000</td>\n",
              "      <td>4.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>4.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>4.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>8.000000</td>\n",
              "      <td>-96.900002</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.400000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>25%</th>\n",
              "      <td>238255.000000</td>\n",
              "      <td>99999.000000</td>\n",
              "      <td>1978.000000</td>\n",
              "      <td>4.000000</td>\n",
              "      <td>8.000000</td>\n",
              "      <td>39.299999</td>\n",
              "      <td>7.000000</td>\n",
              "      <td>30.349999</td>\n",
              "      <td>7.000000</td>\n",
              "      <td>1009.700012</td>\n",
              "      <td>6.000000</td>\n",
              "      <td>952.900024</td>\n",
              "      <td>6.000000</td>\n",
              "      <td>6.300000</td>\n",
              "      <td>7.000000</td>\n",
              "      <td>3.400000</td>\n",
              "      <td>7.000000</td>\n",
              "      <td>7.800000</td>\n",
              "      <td>19.400000</td>\n",
              "      <td>32.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>2.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>50%</th>\n",
              "      <td>538980.000000</td>\n",
              "      <td>99999.000000</td>\n",
              "      <td>1990.000000</td>\n",
              "      <td>7.000000</td>\n",
              "      <td>16.000000</td>\n",
              "      <td>55.000000</td>\n",
              "      <td>8.000000</td>\n",
              "      <td>44.000000</td>\n",
              "      <td>8.000000</td>\n",
              "      <td>1014.850006</td>\n",
              "      <td>8.000000</td>\n",
              "      <td>995.599976</td>\n",
              "      <td>8.000000</td>\n",
              "      <td>9.300000</td>\n",
              "      <td>8.000000</td>\n",
              "      <td>5.800000</td>\n",
              "      <td>8.000000</td>\n",
              "      <td>11.100000</td>\n",
              "      <td>23.900000</td>\n",
              "      <td>46.400002</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>5.900000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>75%</th>\n",
              "      <td>725273.500000</td>\n",
              "      <td>99999.000000</td>\n",
              "      <td>2000.000000</td>\n",
              "      <td>10.000000</td>\n",
              "      <td>23.000000</td>\n",
              "      <td>69.800003</td>\n",
              "      <td>23.000000</td>\n",
              "      <td>56.700001</td>\n",
              "      <td>23.000000</td>\n",
              "      <td>1020.799988</td>\n",
              "      <td>21.000000</td>\n",
              "      <td>1010.299988</td>\n",
              "      <td>22.000000</td>\n",
              "      <td>13.825000</td>\n",
              "      <td>23.000000</td>\n",
              "      <td>9.100000</td>\n",
              "      <td>23.000000</td>\n",
              "      <td>15.900000</td>\n",
              "      <td>28.900000</td>\n",
              "      <td>60.799999</td>\n",
              "      <td>0.010000</td>\n",
              "      <td>11.600000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>max</th>\n",
              "      <td>999999.000000</td>\n",
              "      <td>99999.000000</td>\n",
              "      <td>2010.000000</td>\n",
              "      <td>12.000000</td>\n",
              "      <td>31.000000</td>\n",
              "      <td>105.099998</td>\n",
              "      <td>24.000000</td>\n",
              "      <td>80.500000</td>\n",
              "      <td>24.000000</td>\n",
              "      <td>1052.900024</td>\n",
              "      <td>24.000000</td>\n",
              "      <td>1037.099976</td>\n",
              "      <td>24.000000</td>\n",
              "      <td>99.400002</td>\n",
              "      <td>24.000000</td>\n",
              "      <td>57.299999</td>\n",
              "      <td>24.000000</td>\n",
              "      <td>68.000000</td>\n",
              "      <td>66.000000</td>\n",
              "      <td>98.599998</td>\n",
              "      <td>5.910000</td>\n",
              "      <td>51.200001</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "       station_number   wban_number  ...  total_precipitation  snow_depth\n",
              "count     1979.000000   1979.000000  ...          1793.000000   91.000000\n",
              "mean    505585.599293  89647.441132  ...             0.065694    8.929670\n",
              "std     302491.187318  27088.238467  ...             0.283467    9.922707\n",
              "min      10100.000000     13.000000  ...             0.000000    0.400000\n",
              "25%     238255.000000  99999.000000  ...             0.000000    2.000000\n",
              "50%     538980.000000  99999.000000  ...             0.000000    5.900000\n",
              "75%     725273.500000  99999.000000  ...             0.010000   11.600000\n",
              "max     999999.000000  99999.000000  ...             5.910000   51.200001\n",
              "\n",
              "[8 rows x 22 columns]"
            ]
          },
          "execution_count": 0,
          "metadata": {
            "tags": []
          },
          "output_type": "execute_result"
        }
      ],
      "source": [
        "df.describe()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SVWFp6jIKtAa"
      },
      "source": [
        "### View the first 10 rows"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 302
        },
        "id": "hEYIhPvAKtAd",
        "outputId": "4f865e6f-6c36-4e14-e451-e1608b30129e"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.module+javascript": "\n      import \"https://ssl.gstatic.com/colaboratory/data_table/a036b366c3cace79/data_table.js\";\n\n      window.createDataTable({\n        data: [[{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 105780,\n            'f': \"105780\",\n        },\n{\n            'v': 99999,\n            'f': \"99999\",\n        },\n{\n            'v': 1968,\n            'f': \"1968\",\n        },\n{\n            'v': 9,\n            'f': \"9\",\n        },\n{\n            'v': 13,\n            'f': \"13\",\n        },\n{\n            'v': 46,\n            'f': \"46\",\n        },\n{\n            'v': 8,\n            'f': \"8\",\n        },\n{\n            'v': 44.20000076293945,\n            'f': \"44.20000076293945\",\n        },\n{\n            'v': 8,\n            'f': \"8\",\n        },\n{\n            'v': NaN,\n            'f': \"NaN\",\n        },\n{\n            'v': NaN,\n            'f': \"NaN\",\n        },\n{\n            'v': NaN,\n            'f': \"NaN\",\n        },\n{\n            'v': NaN,\n            'f': \"NaN\",\n        },\n{\n            'v': 4.699999809265137,\n            'f': \"4.699999809265137\",\n        },\n{\n            'v': 8,\n            'f': \"8\",\n        },\n{\n            'v': 15.300000190734863,\n            'f': \"15.300000190734863\",\n        },\n{\n            'v': 8,\n            'f': \"8\",\n        },\n{\n            'v': 21,\n            'f': \"21\",\n        },\n{\n            'v': NaN,\n            'f': \"NaN\",\n        },\n{\n            'v': 43,\n            'f': \"43\",\n        }],\n [{\n            'v': 1,\n            'f': \"1\",\n        },\n{\n            'v': 25710,\n            'f': \"25710\",\n        },\n{\n            'v': 99999,\n            'f': \"99999\",\n        },\n{\n            'v': 1989,\n            'f': \"1989\",\n        },\n{\n            'v': 5,\n            'f': \"5\",\n        },\n{\n            'v': 2,\n            'f': \"2\",\n        },\n{\n            'v': 51.29999923706055,\n            'f': \"51.29999923706055\",\n        },\n{\n            'v': 23,\n            'f': \"23\",\n        },\n{\n            'v': 44.900001525878906,\n            'f': \"44.900001525878906\",\n        },\n{\n            'v': 23,\n            'f': \"23\",\n        },\n{\n            'v': NaN,\n            'f': \"NaN\",\n        },\n{\n            'v': NaN,\n            'f': \"NaN\",\n        },\n{\n            'v': NaN,\n            'f': \"NaN\",\n        },\n{\n            'v': NaN,\n            'f': \"NaN\",\n        },\n{\n            'v': 6.599999904632568,\n            'f': \"6.599999904632568\",\n        },\n{\n            'v': 23,\n            'f': \"23\",\n        },\n{\n            'v': 7.5,\n            'f': \"7.5\",\n        },\n{\n            'v': 23,\n            'f': \"23\",\n        },\n{\n            'v': 12,\n            'f': \"12\",\n        },\n{\n            'v': NaN,\n            'f': \"NaN\",\n        },\n{\n            'v': 42.79999923706055,\n            'f': \"42.79999923706055\",\n        }],\n [{\n            'v': 2,\n            'f': \"2\",\n        },\n{\n            'v': 475160,\n            'f': \"475160\",\n        },\n{\n            'v': 99999,\n            'f': \"99999\",\n        },\n{\n            'v': 2003,\n            'f': \"2003\",\n        },\n{\n            'v': 4,\n            'f': \"4\",\n        },\n{\n            'v': 26,\n            'f': \"26\",\n        },\n{\n            'v': 45.20000076293945,\n            'f': \"45.20000076293945\",\n        },\n{\n            'v': 16,\n            'f': \"16\",\n        },\n{\n            'v': 44.5,\n            'f': \"44.5\",\n        },\n{\n            'v': 16,\n            'f': \"16\",\n        },\n{\n            'v': NaN,\n            'f': \"NaN\",\n        },\n{\n            'v': NaN,\n            'f': \"NaN\",\n        },\n{\n            'v': NaN,\n            'f': \"NaN\",\n        },\n{\n            'v': NaN,\n            'f': \"NaN\",\n        },\n{\n            'v': 4.5,\n            'f': \"4.5\",\n        },\n{\n            'v': 16,\n            'f': \"16\",\n        },\n{\n            'v': 7.400000095367432,\n            'f': \"7.400000095367432\",\n        },\n{\n            'v': 16,\n            'f': \"16\",\n        },\n{\n            'v': 13,\n            'f': \"13\",\n        },\n{\n            'v': NaN,\n            'f': \"NaN\",\n        },\n{\n            'v': 39.20000076293945,\n            'f': \"39.20000076293945\",\n        }],\n [{\n            'v': 3,\n            'f': \"3\",\n        },\n{\n            'v': 476720,\n            'f': \"476720\",\n        },\n{\n            'v': 99999,\n            'f': \"99999\",\n        },\n{\n            'v': 1989,\n            'f': \"1989\",\n        },\n{\n            'v': 12,\n            'f': \"12\",\n        },\n{\n            'v': 8,\n            'f': \"8\",\n        },\n{\n            'v': 51.599998474121094,\n            'f': \"51.599998474121094\",\n        },\n{\n            'v': 4,\n            'f': \"4\",\n        },\n{\n            'v': 34,\n            'f': \"34\",\n        },\n{\n            'v': 4,\n            'f': \"4\",\n        },\n{\n            'v': 1005.4000244140625,\n            'f': \"1005.4000244140625\",\n        },\n{\n            'v': 4,\n            'f': \"4\",\n        },\n{\n            'v': NaN,\n            'f': \"NaN\",\n        },\n{\n            'v': NaN,\n            'f': \"NaN\",\n        },\n{\n            'v': 9,\n            'f': \"9\",\n        },\n{\n            'v': 4,\n            'f': \"4\",\n        },\n{\n            'v': 5,\n            'f': \"5\",\n        },\n{\n            'v': 4,\n            'f': \"4\",\n        },\n{\n            'v': 8,\n            'f': \"8\",\n        },\n{\n            'v': NaN,\n            'f': \"NaN\",\n        },\n{\n            'v': 41.900001525878906,\n            'f': \"41.900001525878906\",\n        }],\n [{\n            'v': 4,\n            'f': \"4\",\n        },\n{\n            'v': 940040,\n            'f': \"940040\",\n        },\n{\n            'v': 99999,\n            'f': \"99999\",\n        },\n{\n            'v': 1991,\n            'f': \"1991\",\n        },\n{\n            'v': 6,\n            'f': \"6\",\n        },\n{\n            'v': 9,\n            'f': \"9\",\n        },\n{\n            'v': 84.30000305175781,\n            'f': \"84.30000305175781\",\n        },\n{\n            'v': 4,\n            'f': \"4\",\n        },\n{\n            'v': 75.19999694824219,\n            'f': \"75.19999694824219\",\n        },\n{\n            'v': 4,\n            'f': \"4\",\n        },\n{\n            'v': 1009.9000244140625,\n            'f': \"1009.9000244140625\",\n        },\n{\n            'v': 4,\n            'f': \"4\",\n        },\n{\n            'v': 1009.0999755859375,\n            'f': \"1009.0999755859375\",\n        },\n{\n            'v': 4,\n            'f': \"4\",\n        },\n{\n            'v': 24.899999618530273,\n            'f': \"24.899999618530273\",\n        },\n{\n            'v': 4,\n            'f': \"4\",\n        },\n{\n            'v': 6,\n            'f': \"6\",\n        },\n{\n            'v': 4,\n            'f': \"4\",\n        },\n{\n            'v': 9.899999618530273,\n            'f': \"9.899999618530273\",\n        },\n{\n            'v': NaN,\n            'f': \"NaN\",\n        },\n{\n            'v': 79.19999694824219,\n            'f': \"79.19999694824219\",\n        }],\n [{\n            'v': 5,\n            'f': \"5\",\n        },\n{\n            'v': 103250,\n            'f': \"103250\",\n        },\n{\n            'v': 99999,\n            'f': \"99999\",\n        },\n{\n            'v': 1976,\n            'f': \"1976\",\n        },\n{\n            'v': 7,\n            'f': \"7\",\n        },\n{\n            'v': 23,\n            'f': \"23\",\n        },\n{\n            'v': 63.599998474121094,\n            'f': \"63.599998474121094\",\n        },\n{\n            'v': 13,\n            'f': \"13\",\n        },\n{\n            'v': 48.599998474121094,\n            'f': \"48.599998474121094\",\n        },\n{\n            'v': 13,\n            'f': \"13\",\n        },\n{\n            'v': 1022.0999755859375,\n            'f': \"1022.0999755859375\",\n        },\n{\n            'v': 5,\n            'f': \"5\",\n        },\n{\n            'v': NaN,\n            'f': \"NaN\",\n        },\n{\n            'v': NaN,\n            'f': \"NaN\",\n        },\n{\n            'v': 24.600000381469727,\n            'f': \"24.600000381469727\",\n        },\n{\n            'v': 13,\n            'f': \"13\",\n        },\n{\n            'v': 3,\n            'f': \"3\",\n        },\n{\n            'v': 12,\n            'f': \"12\",\n        },\n{\n            'v': 6,\n            'f': \"6\",\n        },\n{\n            'v': NaN,\n            'f': \"NaN\",\n        },\n{\n            'v': 48.20000076293945,\n            'f': \"48.20000076293945\",\n        }],\n [{\n            'v': 6,\n            'f': \"6\",\n        },\n{\n            'v': 919280,\n            'f': \"919280\",\n        },\n{\n            'v': 99999,\n            'f': \"99999\",\n        },\n{\n            'v': 1981,\n            'f': \"1981\",\n        },\n{\n            'v': 2,\n            'f': \"2\",\n        },\n{\n            'v': 21,\n            'f': \"21\",\n        },\n{\n            'v': 83.80000305175781,\n            'f': \"83.80000305175781\",\n        },\n{\n            'v': 5,\n            'f': \"5\",\n        },\n{\n            'v': 75.9000015258789,\n            'f': \"75.9000015258789\",\n        },\n{\n            'v': 5,\n            'f': \"5\",\n        },\n{\n            'v': 1007.9000244140625,\n            'f': \"1007.9000244140625\",\n        },\n{\n            'v': 5,\n            'f': \"5\",\n        },\n{\n            'v': NaN,\n            'f': \"NaN\",\n        },\n{\n            'v': NaN,\n            'f': \"NaN\",\n        },\n{\n            'v': 17.100000381469727,\n            'f': \"17.100000381469727\",\n        },\n{\n            'v': 4,\n            'f': \"4\",\n        },\n{\n            'v': 8.5,\n            'f': \"8.5\",\n        },\n{\n            'v': 5,\n            'f': \"5\",\n        },\n{\n            'v': 8.899999618530273,\n            'f': \"8.899999618530273\",\n        },\n{\n            'v': NaN,\n            'f': \"NaN\",\n        },\n{\n            'v': 77,\n            'f': \"77\",\n        }],\n [{\n            'v': 7,\n            'f': \"7\",\n        },\n{\n            'v': 961710,\n            'f': \"961710\",\n        },\n{\n            'v': 99999,\n            'f': \"99999\",\n        },\n{\n            'v': 2004,\n            'f': \"2004\",\n        },\n{\n            'v': 6,\n            'f': \"6\",\n        },\n{\n            'v': 23,\n            'f': \"23\",\n        },\n{\n            'v': 81.9000015258789,\n            'f': \"81.9000015258789\",\n        },\n{\n            'v': 6,\n            'f': \"6\",\n        },\n{\n            'v': 74.9000015258789,\n            'f': \"74.9000015258789\",\n        },\n{\n            'v': 6,\n            'f': \"6\",\n        },\n{\n            'v': 1010.5,\n            'f': \"1010.5\",\n        },\n{\n            'v': 6,\n            'f': \"6\",\n        },\n{\n            'v': 1008,\n            'f': \"1008\",\n        },\n{\n            'v': 6,\n            'f': \"6\",\n        },\n{\n            'v': 4.800000190734863,\n            'f': \"4.800000190734863\",\n        },\n{\n            'v': 6,\n            'f': \"6\",\n        },\n{\n            'v': 1.5,\n            'f': \"1.5\",\n        },\n{\n            'v': 6,\n            'f': \"6\",\n        },\n{\n            'v': 5.099999904632568,\n            'f': \"5.099999904632568\",\n        },\n{\n            'v': NaN,\n            'f': \"NaN\",\n        },\n{\n            'v': 74.80000305175781,\n            'f': \"74.80000305175781\",\n        }],\n [{\n            'v': 8,\n            'f': \"8\",\n        },\n{\n            'v': 172400,\n            'f': \"172400\",\n        },\n{\n            'v': 99999,\n            'f': \"99999\",\n        },\n{\n            'v': 1990,\n            'f': \"1990\",\n        },\n{\n            'v': 6,\n            'f': \"6\",\n        },\n{\n            'v': 6,\n            'f': \"6\",\n        },\n{\n            'v': 65.4000015258789,\n            'f': \"65.4000015258789\",\n        },\n{\n            'v': 7,\n            'f': \"7\",\n        },\n{\n            'v': 43.900001525878906,\n            'f': \"43.900001525878906\",\n        },\n{\n            'v': 7,\n            'f': \"7\",\n        },\n{\n            'v': 1018.2000122070312,\n            'f': \"1018.2000122070312\",\n        },\n{\n            'v': 7,\n            'f': \"7\",\n        },\n{\n            'v': 906.0999755859375,\n            'f': \"906.0999755859375\",\n        },\n{\n            'v': 7,\n            'f': \"7\",\n        },\n{\n            'v': 18.600000381469727,\n            'f': \"18.600000381469727\",\n        },\n{\n            'v': 7,\n            'f': \"7\",\n        },\n{\n            'v': 2,\n            'f': \"2\",\n        },\n{\n            'v': 7,\n            'f': \"7\",\n        },\n{\n            'v': 4.099999904632568,\n            'f': \"4.099999904632568\",\n        },\n{\n            'v': NaN,\n            'f': \"NaN\",\n        },\n{\n            'v': 45,\n            'f': \"45\",\n        }],\n [{\n            'v': 9,\n            'f': \"9\",\n        },\n{\n            'v': 38790,\n            'f': \"38790\",\n        },\n{\n            'v': 99999,\n            'f': \"99999\",\n        },\n{\n            'v': 1973,\n            'f': \"1973\",\n        },\n{\n            'v': 4,\n            'f': \"4\",\n        },\n{\n            'v': 7,\n            'f': \"7\",\n        },\n{\n            'v': 44.79999923706055,\n            'f': \"44.79999923706055\",\n        },\n{\n            'v': 8,\n            'f': \"8\",\n        },\n{\n            'v': 32.900001525878906,\n            'f': \"32.900001525878906\",\n        },\n{\n            'v': 8,\n            'f': \"8\",\n        },\n{\n            'v': 1018.2999877929688,\n            'f': \"1018.2999877929688\",\n        },\n{\n            'v': 8,\n            'f': \"8\",\n        },\n{\n            'v': NaN,\n            'f': \"NaN\",\n        },\n{\n            'v': NaN,\n            'f': \"NaN\",\n        },\n{\n            'v': 17.200000762939453,\n            'f': \"17.200000762939453\",\n        },\n{\n            'v': 8,\n            'f': \"8\",\n        },\n{\n            'v': 9.100000381469727,\n            'f': \"9.100000381469727\",\n        },\n{\n            'v': 8,\n            'f': \"8\",\n        },\n{\n            'v': 15,\n            'f': \"15\",\n        },\n{\n            'v': NaN,\n            'f': \"NaN\",\n        },\n{\n            'v': 35.599998474121094,\n            'f': \"35.599998474121094\",\n        }]],\n        columns: [[\"number\", \"index\"], [\"number\", \"station_number\"], [\"number\", \"wban_number\"], [\"number\", \"year\"], [\"number\", \"month\"], [\"number\", \"day\"], [\"number\", \"mean_temp\"], [\"number\", \"num_mean_temp_samples\"], [\"number\", \"mean_dew_point\"], [\"number\", \"num_mean_dew_point_samples\"], [\"number\", \"mean_sealevel_pressure\"], [\"number\", \"num_mean_sealevel_pressure_samples\"], [\"number\", \"mean_station_pressure\"], [\"number\", \"num_mean_station_pressure_samples\"], [\"number\", \"mean_visibility\"], [\"number\", \"num_mean_visibility_samples\"], [\"number\", \"mean_wind_speed\"], [\"number\", \"num_mean_wind_speed_samples\"], [\"number\", \"max_sustained_wind_speed\"], [\"number\", \"max_gust_wind_speed\"], [\"number\", \"max_temperature\"]],\n        rowsPerPage: 25,\n        helpUrl: \"https://colab.research.google.com/notebooks/data_table.ipynb\",\n      });\n    ",
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>station_number</th>\n",
              "      <th>wban_number</th>\n",
              "      <th>year</th>\n",
              "      <th>month</th>\n",
              "      <th>day</th>\n",
              "      <th>mean_temp</th>\n",
              "      <th>num_mean_temp_samples</th>\n",
              "      <th>mean_dew_point</th>\n",
              "      <th>num_mean_dew_point_samples</th>\n",
              "      <th>mean_sealevel_pressure</th>\n",
              "      <th>num_mean_sealevel_pressure_samples</th>\n",
              "      <th>mean_station_pressure</th>\n",
              "      <th>num_mean_station_pressure_samples</th>\n",
              "      <th>mean_visibility</th>\n",
              "      <th>num_mean_visibility_samples</th>\n",
              "      <th>mean_wind_speed</th>\n",
              "      <th>num_mean_wind_speed_samples</th>\n",
              "      <th>max_sustained_wind_speed</th>\n",
              "      <th>max_gust_wind_speed</th>\n",
              "      <th>max_temperature</th>\n",
              "      <th>max_temperature_explicit</th>\n",
              "      <th>min_temperature</th>\n",
              "      <th>min_temperature_explicit</th>\n",
              "      <th>total_precipitation</th>\n",
              "      <th>snow_depth</th>\n",
              "      <th>fog</th>\n",
              "      <th>rain</th>\n",
              "      <th>snow</th>\n",
              "      <th>hail</th>\n",
              "      <th>thunder</th>\n",
              "      <th>tornado</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>105780</td>\n",
              "      <td>99999</td>\n",
              "      <td>1968</td>\n",
              "      <td>9</td>\n",
              "      <td>13</td>\n",
              "      <td>46.000000</td>\n",
              "      <td>8</td>\n",
              "      <td>44.200001</td>\n",
              "      <td>8.0</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>4.700000</td>\n",
              "      <td>8.0</td>\n",
              "      <td>15.3</td>\n",
              "      <td>8.0</td>\n",
              "      <td>21.0</td>\n",
              "      <td>NaN</td>\n",
              "      <td>43.000000</td>\n",
              "      <td>False</td>\n",
              "      <td>None</td>\n",
              "      <td>None</td>\n",
              "      <td>0.02</td>\n",
              "      <td>NaN</td>\n",
              "      <td>True</td>\n",
              "      <td>True</td>\n",
              "      <td>True</td>\n",
              "      <td>True</td>\n",
              "      <td>True</td>\n",
              "      <td>True</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>25710</td>\n",
              "      <td>99999</td>\n",
              "      <td>1989</td>\n",
              "      <td>5</td>\n",
              "      <td>2</td>\n",
              "      <td>51.299999</td>\n",
              "      <td>23</td>\n",
              "      <td>44.900002</td>\n",
              "      <td>23.0</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>6.600000</td>\n",
              "      <td>23.0</td>\n",
              "      <td>7.5</td>\n",
              "      <td>23.0</td>\n",
              "      <td>12.0</td>\n",
              "      <td>NaN</td>\n",
              "      <td>42.799999</td>\n",
              "      <td>True</td>\n",
              "      <td>None</td>\n",
              "      <td>None</td>\n",
              "      <td>0.00</td>\n",
              "      <td>NaN</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>475160</td>\n",
              "      <td>99999</td>\n",
              "      <td>2003</td>\n",
              "      <td>4</td>\n",
              "      <td>26</td>\n",
              "      <td>45.200001</td>\n",
              "      <td>16</td>\n",
              "      <td>44.500000</td>\n",
              "      <td>16.0</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>4.500000</td>\n",
              "      <td>16.0</td>\n",
              "      <td>7.4</td>\n",
              "      <td>16.0</td>\n",
              "      <td>13.0</td>\n",
              "      <td>NaN</td>\n",
              "      <td>39.200001</td>\n",
              "      <td>True</td>\n",
              "      <td>None</td>\n",
              "      <td>None</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>476720</td>\n",
              "      <td>99999</td>\n",
              "      <td>1989</td>\n",
              "      <td>12</td>\n",
              "      <td>8</td>\n",
              "      <td>51.599998</td>\n",
              "      <td>4</td>\n",
              "      <td>34.000000</td>\n",
              "      <td>4.0</td>\n",
              "      <td>1005.400024</td>\n",
              "      <td>4.0</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>9.000000</td>\n",
              "      <td>4.0</td>\n",
              "      <td>5.0</td>\n",
              "      <td>4.0</td>\n",
              "      <td>8.0</td>\n",
              "      <td>NaN</td>\n",
              "      <td>41.900002</td>\n",
              "      <td>True</td>\n",
              "      <td>None</td>\n",
              "      <td>None</td>\n",
              "      <td>0.00</td>\n",
              "      <td>NaN</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>940040</td>\n",
              "      <td>99999</td>\n",
              "      <td>1991</td>\n",
              "      <td>6</td>\n",
              "      <td>9</td>\n",
              "      <td>84.300003</td>\n",
              "      <td>4</td>\n",
              "      <td>75.199997</td>\n",
              "      <td>4.0</td>\n",
              "      <td>1009.900024</td>\n",
              "      <td>4.0</td>\n",
              "      <td>1009.099976</td>\n",
              "      <td>4.0</td>\n",
              "      <td>24.900000</td>\n",
              "      <td>4.0</td>\n",
              "      <td>6.0</td>\n",
              "      <td>4.0</td>\n",
              "      <td>9.9</td>\n",
              "      <td>NaN</td>\n",
              "      <td>79.199997</td>\n",
              "      <td>True</td>\n",
              "      <td>None</td>\n",
              "      <td>None</td>\n",
              "      <td>0.00</td>\n",
              "      <td>NaN</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5</th>\n",
              "      <td>103250</td>\n",
              "      <td>99999</td>\n",
              "      <td>1976</td>\n",
              "      <td>7</td>\n",
              "      <td>23</td>\n",
              "      <td>63.599998</td>\n",
              "      <td>13</td>\n",
              "      <td>48.599998</td>\n",
              "      <td>13.0</td>\n",
              "      <td>1022.099976</td>\n",
              "      <td>5.0</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>24.600000</td>\n",
              "      <td>13.0</td>\n",
              "      <td>3.0</td>\n",
              "      <td>12.0</td>\n",
              "      <td>6.0</td>\n",
              "      <td>NaN</td>\n",
              "      <td>48.200001</td>\n",
              "      <td>False</td>\n",
              "      <td>None</td>\n",
              "      <td>None</td>\n",
              "      <td>0.00</td>\n",
              "      <td>NaN</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>6</th>\n",
              "      <td>919280</td>\n",
              "      <td>99999</td>\n",
              "      <td>1981</td>\n",
              "      <td>2</td>\n",
              "      <td>21</td>\n",
              "      <td>83.800003</td>\n",
              "      <td>5</td>\n",
              "      <td>75.900002</td>\n",
              "      <td>5.0</td>\n",
              "      <td>1007.900024</td>\n",
              "      <td>5.0</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>17.100000</td>\n",
              "      <td>4.0</td>\n",
              "      <td>8.5</td>\n",
              "      <td>5.0</td>\n",
              "      <td>8.9</td>\n",
              "      <td>NaN</td>\n",
              "      <td>77.000000</td>\n",
              "      <td>False</td>\n",
              "      <td>None</td>\n",
              "      <td>None</td>\n",
              "      <td>0.00</td>\n",
              "      <td>NaN</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>7</th>\n",
              "      <td>961710</td>\n",
              "      <td>99999</td>\n",
              "      <td>2004</td>\n",
              "      <td>6</td>\n",
              "      <td>23</td>\n",
              "      <td>81.900002</td>\n",
              "      <td>6</td>\n",
              "      <td>74.900002</td>\n",
              "      <td>6.0</td>\n",
              "      <td>1010.500000</td>\n",
              "      <td>6.0</td>\n",
              "      <td>1008.000000</td>\n",
              "      <td>6.0</td>\n",
              "      <td>4.800000</td>\n",
              "      <td>6.0</td>\n",
              "      <td>1.5</td>\n",
              "      <td>6.0</td>\n",
              "      <td>5.1</td>\n",
              "      <td>NaN</td>\n",
              "      <td>74.800003</td>\n",
              "      <td>False</td>\n",
              "      <td>None</td>\n",
              "      <td>None</td>\n",
              "      <td>0.00</td>\n",
              "      <td>NaN</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>8</th>\n",
              "      <td>172400</td>\n",
              "      <td>99999</td>\n",
              "      <td>1990</td>\n",
              "      <td>6</td>\n",
              "      <td>6</td>\n",
              "      <td>65.400002</td>\n",
              "      <td>7</td>\n",
              "      <td>43.900002</td>\n",
              "      <td>7.0</td>\n",
              "      <td>1018.200012</td>\n",
              "      <td>7.0</td>\n",
              "      <td>906.099976</td>\n",
              "      <td>7.0</td>\n",
              "      <td>18.600000</td>\n",
              "      <td>7.0</td>\n",
              "      <td>2.0</td>\n",
              "      <td>7.0</td>\n",
              "      <td>4.1</td>\n",
              "      <td>NaN</td>\n",
              "      <td>45.000000</td>\n",
              "      <td>False</td>\n",
              "      <td>None</td>\n",
              "      <td>None</td>\n",
              "      <td>0.00</td>\n",
              "      <td>NaN</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>9</th>\n",
              "      <td>38790</td>\n",
              "      <td>99999</td>\n",
              "      <td>1973</td>\n",
              "      <td>4</td>\n",
              "      <td>7</td>\n",
              "      <td>44.799999</td>\n",
              "      <td>8</td>\n",
              "      <td>32.900002</td>\n",
              "      <td>8.0</td>\n",
              "      <td>1018.299988</td>\n",
              "      <td>8.0</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>17.200001</td>\n",
              "      <td>8.0</td>\n",
              "      <td>9.1</td>\n",
              "      <td>8.0</td>\n",
              "      <td>15.0</td>\n",
              "      <td>NaN</td>\n",
              "      <td>35.599998</td>\n",
              "      <td>True</td>\n",
              "      <td>None</td>\n",
              "      <td>None</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   station_number  wban_number  year  month  ...   snow   hail  thunder  tornado\n",
              "0          105780        99999  1968      9  ...   True   True     True     True\n",
              "1           25710        99999  1989      5  ...  False  False    False    False\n",
              "2          475160        99999  2003      4  ...  False  False    False    False\n",
              "3          476720        99999  1989     12  ...  False  False    False    False\n",
              "4          940040        99999  1991      6  ...  False  False    False    False\n",
              "5          103250        99999  1976      7  ...  False  False    False    False\n",
              "6          919280        99999  1981      2  ...  False  False    False    False\n",
              "7          961710        99999  2004      6  ...  False  False    False    False\n",
              "8          172400        99999  1990      6  ...  False  False    False    False\n",
              "9           38790        99999  1973      4  ...  False  False    False    False\n",
              "\n",
              "[10 rows x 31 columns]"
            ]
          },
          "execution_count": 0,
          "metadata": {
            "tags": []
          },
          "output_type": "execute_result"
        }
      ],
      "source": [
        "df.head(10)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 302
        },
        "id": "1K7weAv6KtAq",
        "outputId": "28df8986-a57c-4eb8-862a-d0dd9f23c6a1"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.module+javascript": "\n      import \"https://ssl.gstatic.com/colaboratory/data_table/a036b366c3cace79/data_table.js\";\n\n      window.createDataTable({\n        data: [[{\n            'v': 644,\n            'f': \"644\",\n        },\n{\n            'v': 230220,\n            'f': \"230220\",\n        },\n{\n            'v': 1964,\n            'f': \"1964\",\n        },\n{\n            'v': 7,\n            'f': \"7\",\n        },\n{\n            'v': 15,\n            'f': \"15\",\n        },\n{\n            'v': 5.909999847412109,\n            'f': \"5.909999847412109\",\n        }],\n [{\n            'v': 1155,\n            'f': \"1155\",\n        },\n{\n            'v': 985430,\n            'f': \"985430\",\n        },\n{\n            'v': 2008,\n            'f': \"2008\",\n        },\n{\n            'v': 12,\n            'f': \"12\",\n        },\n{\n            'v': 8,\n            'f': \"8\",\n        },\n{\n            'v': 3.4600000381469727,\n            'f': \"3.4600000381469727\",\n        }],\n [{\n            'v': 1196,\n            'f': \"1196\",\n        },\n{\n            'v': 248260,\n            'f': \"248260\",\n        },\n{\n            'v': 1961,\n            'f': \"1961\",\n        },\n{\n            'v': 11,\n            'f': \"11\",\n        },\n{\n            'v': 1,\n            'f': \"1\",\n        },\n{\n            'v': 2.950000047683716,\n            'f': \"2.950000047683716\",\n        }],\n [{\n            'v': 1588,\n            'f': \"1588\",\n        },\n{\n            'v': 257670,\n            'f': \"257670\",\n        },\n{\n            'v': 1959,\n            'f': \"1959\",\n        },\n{\n            'v': 8,\n            'f': \"8\",\n        },\n{\n            'v': 9,\n            'f': \"9\",\n        },\n{\n            'v': 2.950000047683716,\n            'f': \"2.950000047683716\",\n        }],\n [{\n            'v': 980,\n            'f': \"980\",\n        },\n{\n            'v': 299150,\n            'f': \"299150\",\n        },\n{\n            'v': 1962,\n            'f': \"1962\",\n        },\n{\n            'v': 3,\n            'f': \"3\",\n        },\n{\n            'v': 1,\n            'f': \"1\",\n        },\n{\n            'v': 2.950000047683716,\n            'f': \"2.950000047683716\",\n        }],\n [{\n            'v': 1325,\n            'f': \"1325\",\n        },\n{\n            'v': 470250,\n            'f': \"470250\",\n        },\n{\n            'v': 1965,\n            'f': \"1965\",\n        },\n{\n            'v': 11,\n            'f': \"11\",\n        },\n{\n            'v': 25,\n            'f': \"25\",\n        },\n{\n            'v': 2.950000047683716,\n            'f': \"2.950000047683716\",\n        }],\n [{\n            'v': 1917,\n            'f': \"1917\",\n        },\n{\n            'v': 288380,\n            'f': \"288380\",\n        },\n{\n            'v': 1994,\n            'f': \"1994\",\n        },\n{\n            'v': 8,\n            'f': \"8\",\n        },\n{\n            'v': 6,\n            'f': \"6\",\n        },\n{\n            'v': 2.319999933242798,\n            'f': \"2.319999933242798\",\n        }],\n [{\n            'v': 1211,\n            'f': \"1211\",\n        },\n{\n            'v': 585190,\n            'f': \"585190\",\n        },\n{\n            'v': 1995,\n            'f': \"1995\",\n        },\n{\n            'v': 4,\n            'f': \"4\",\n        },\n{\n            'v': 14,\n            'f': \"14\",\n        },\n{\n            'v': 2.319999933242798,\n            'f': \"2.319999933242798\",\n        }],\n [{\n            'v': 250,\n            'f': \"250\",\n        },\n{\n            'v': 647000,\n            'f': \"647000\",\n        },\n{\n            'v': 2005,\n            'f': \"2005\",\n        },\n{\n            'v': 8,\n            'f': \"8\",\n        },\n{\n            'v': 19,\n            'f': \"19\",\n        },\n{\n            'v': 2.200000047683716,\n            'f': \"2.200000047683716\",\n        }],\n [{\n            'v': 1418,\n            'f': \"1418\",\n        },\n{\n            'v': 964710,\n            'f': \"964710\",\n        },\n{\n            'v': 1975,\n            'f': \"1975\",\n        },\n{\n            'v': 9,\n            'f': \"9\",\n        },\n{\n            'v': 8,\n            'f': \"8\",\n        },\n{\n            'v': 1.9700000286102295,\n            'f': \"1.9700000286102295\",\n        }]],\n        columns: [[\"number\", \"index\"], [\"number\", \"station_number\"], [\"number\", \"year\"], [\"number\", \"month\"], [\"number\", \"day\"], [\"number\", \"total_precipitation\"]],\n        rowsPerPage: 25,\n        helpUrl: \"https://colab.research.google.com/notebooks/data_table.ipynb\",\n      });\n    ",
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>station_number</th>\n",
              "      <th>year</th>\n",
              "      <th>month</th>\n",
              "      <th>day</th>\n",
              "      <th>total_precipitation</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>644</th>\n",
              "      <td>230220</td>\n",
              "      <td>1964</td>\n",
              "      <td>7</td>\n",
              "      <td>15</td>\n",
              "      <td>5.91</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1155</th>\n",
              "      <td>985430</td>\n",
              "      <td>2008</td>\n",
              "      <td>12</td>\n",
              "      <td>8</td>\n",
              "      <td>3.46</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1196</th>\n",
              "      <td>248260</td>\n",
              "      <td>1961</td>\n",
              "      <td>11</td>\n",
              "      <td>1</td>\n",
              "      <td>2.95</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1588</th>\n",
              "      <td>257670</td>\n",
              "      <td>1959</td>\n",
              "      <td>8</td>\n",
              "      <td>9</td>\n",
              "      <td>2.95</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>980</th>\n",
              "      <td>299150</td>\n",
              "      <td>1962</td>\n",
              "      <td>3</td>\n",
              "      <td>1</td>\n",
              "      <td>2.95</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1325</th>\n",
              "      <td>470250</td>\n",
              "      <td>1965</td>\n",
              "      <td>11</td>\n",
              "      <td>25</td>\n",
              "      <td>2.95</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1917</th>\n",
              "      <td>288380</td>\n",
              "      <td>1994</td>\n",
              "      <td>8</td>\n",
              "      <td>6</td>\n",
              "      <td>2.32</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1211</th>\n",
              "      <td>585190</td>\n",
              "      <td>1995</td>\n",
              "      <td>4</td>\n",
              "      <td>14</td>\n",
              "      <td>2.32</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>250</th>\n",
              "      <td>647000</td>\n",
              "      <td>2005</td>\n",
              "      <td>8</td>\n",
              "      <td>19</td>\n",
              "      <td>2.20</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1418</th>\n",
              "      <td>964710</td>\n",
              "      <td>1975</td>\n",
              "      <td>9</td>\n",
              "      <td>8</td>\n",
              "      <td>1.97</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "      station_number  year  month  day  total_precipitation\n",
              "644           230220  1964      7   15                 5.91\n",
              "1155          985430  2008     12    8                 3.46\n",
              "1196          248260  1961     11    1                 2.95\n",
              "1588          257670  1959      8    9                 2.95\n",
              "980           299150  1962      3    1                 2.95\n",
              "1325          470250  1965     11   25                 2.95\n",
              "1917          288380  1994      8    6                 2.32\n",
              "1211          585190  1995      4   14                 2.32\n",
              "250           647000  2005      8   19                 2.20\n",
              "1418          964710  1975      9    8                 1.97"
            ]
          },
          "execution_count": 0,
          "metadata": {
            "tags": []
          },
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# 10 highest total_precipitation samples\n",
        "df.sort_values('total_precipitation', ascending=False).head(10)[['station_number', 'year', 'month', 'day', 'total_precipitation']]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "oKNxsRvuKtAz"
      },
      "source": [
        "# Use BigQuery through pandas-gbq\n",
        "\n",
        "The `pandas-gbq` library is a community led project by the pandas community. It covers basic functionality, such as writing a DataFrame to BigQuery and running a query, but as a third-party library it may not handle all BigQuery features or use cases.\n",
        "\n",
        "[Pandas GBQ Documentation](https://pandas.pydata.org/pandas-docs/stable/generated/pandas.read_gbq.html)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 195
        },
        "id": "dWf7KIxjKtA0",
        "outputId": "5cac675d-ba4c-4fbe-d771-776691fcf4d4"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.module+javascript": "\n      import \"https://ssl.gstatic.com/colaboratory/data_table/a036b366c3cace79/data_table.js\";\n\n      window.createDataTable({\n        data: [[{\n            'v': 0,\n            'f': \"0\",\n        },\n\"James\",\n{\n            'v': 272793,\n            'f': \"272793\",\n        }],\n [{\n            'v': 1,\n            'f': \"1\",\n        },\n\"John\",\n{\n            'v': 235139,\n            'f': \"235139\",\n        }],\n [{\n            'v': 2,\n            'f': \"2\",\n        },\n\"Michael\",\n{\n            'v': 225320,\n            'f': \"225320\",\n        }],\n [{\n            'v': 3,\n            'f': \"3\",\n        },\n\"Robert\",\n{\n            'v': 220399,\n            'f': \"220399\",\n        }],\n [{\n            'v': 4,\n            'f': \"4\",\n        },\n\"David\",\n{\n            'v': 219028,\n            'f': \"219028\",\n        }]],\n        columns: [[\"number\", \"index\"], [\"string\", \"name\"], [\"number\", \"count\"]],\n        rowsPerPage: 25,\n        helpUrl: \"https://colab.research.google.com/notebooks/data_table.ipynb\",\n      });\n    ",
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>name</th>\n",
              "      <th>count</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>James</td>\n",
              "      <td>272793</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>John</td>\n",
              "      <td>235139</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>Michael</td>\n",
              "      <td>225320</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>Robert</td>\n",
              "      <td>220399</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>David</td>\n",
              "      <td>219028</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "      name   count\n",
              "0    James  272793\n",
              "1     John  235139\n",
              "2  Michael  225320\n",
              "3   Robert  220399\n",
              "4    David  219028"
            ]
          },
          "execution_count": 0,
          "metadata": {
            "tags": []
          },
          "output_type": "execute_result"
        }
      ],
      "source": [
        "import pandas as pd\n",
        "\n",
        "sample_count = 2000\n",
        "df = pd.io.gbq.read_gbq('''\n",
        "  SELECT name, SUM(number) as count\n",
        "  FROM `bigquery-public-data.usa_names.usa_1910_2013`\n",
        "  WHERE state = 'TX'\n",
        "  GROUP BY name\n",
        "  ORDER BY count DESC\n",
        "  LIMIT 100\n",
        "''', project_id=project_id, dialect='standard')\n",
        "\n",
        "df.head()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "vU5JcPLjcj4o"
      },
      "source": [
        "# Syntax highlighting\n",
        "`google.colab.syntax` can be used to add syntax highlighting to any Python string literals which are used in a query later."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "q8kDqSYCczeM",
        "outputId": "78f1a780-a522-4769-bae9-5e982e7ee02f"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.module+javascript": "\n      import \"https://ssl.gstatic.com/colaboratory/data_table/a6224c040fa35dcf/data_table.js\";\n\n      window.createDataTable({\n        data: [[{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 114420316,\n            'f': \"114420316\",\n        }]],\n        columns: [[\"number\", \"index\"], [\"number\", \"total_rows\"]],\n        columnOptions: [{\"width\": \"1px\", \"className\": \"index_column\"}],\n        rowsPerPage: 25,\n        helpUrl: \"https://colab.research.google.com/notebooks/data_table.ipynb\",\n        suppressOutputScrolling: true,\n        minimumWidth: undefined,\n      });\n    ",
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>total_rows</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>114420316</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   total_rows\n",
              "0   114420316"
            ]
          },
          "execution_count": 0,
          "metadata": {
            "tags": []
          },
          "output_type": "execute_result"
        }
      ],
      "source": [
        "from google.colab import syntax\n",
        "query = syntax.sql('''\n",
        "SELECT\n",
        "  COUNT(*) as total_rows\n",
        "FROM\n",
        "  `bigquery-public-data.samples.gsod`\n",
        "''')\n",
        "\n",
        "pd.io.gbq.read_gbq(query, project_id=project_id, dialect='standard')"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "name": "Getting started with BigQuery",
      "provenance": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}