lectures/ci_docs/CIandDocs.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "98902889",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Continuous integration, documentation, package versioning, and releases\n",
    "\n",
    "![devops](figures/ci_devops.png)\n",
    "\n",
    "Timothy E. Holy\n",
    "\n",
    "Washington University in St. Louis"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9ee1d7eb",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Course map: where are we?\n",
    "\n",
    "- Open source, git, and GitHub\n",
    "- \"why Julia\": an introduction to the features of Julia\n",
    "- Testing & principles of design\n",
    "- **Continuous integration, documentation, versioning, and releases**\n",
    "- High performance computing I\n",
    "- High performance computing II\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6f59df04",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# The old days: break stuff & release rarely\n",
    "\n",
    "1. Release version 1.0 (abbreviated `v1.0`) of your software"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8368618e",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "2. One team (*Operations*) fixes bugs. Release `v1.1` ..."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "90c075c0",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "3. Another team (*Developers*) create a list of changes for `v2` and start it almost from scratch"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ad6e7577",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "4. One team supports `v1` while the other team develops `v2`"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a302f67d",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "5. Eventually release `v2.0`. Return to step 2."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4968434a",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "The gap between releases might be years, and the next branch *might be broken* for much of that time."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ae42f0b1",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Challenges with this model\n",
    "\n",
    "- divergence between v1 and v2 can be hard to reconcile"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "434abee2",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- if `v2` is broken for a long time, there may be design problems that remain hidden until you actually start using it"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b2310fd9",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- a large human cost:\n",
    "  + gap between design choices and feedback about how they work out slows developer growth & potential\n",
    "  + \"silos\" between people working on different things\n",
    "  + rarity of releases (the reward of seeing your software being used) can sap enthusiasm"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b47ba4b8",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Result: a big pile-up when transitioning into release mode that can (dangerously) delay releases"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ae9be168",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Modern days: change the time scale\n",
    "\n",
    "Instead of *big, rare* releases, make *small, frequent* \"releases.\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0b82ad85",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "*Continuous integration* (CI):\n",
    "- Keep things working at all times\n",
    "- Features get developed on their own (in branches), and released when ready"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "20ab795f",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Developers typically interact with all stages of the pipeline: \"DevOps\" (Developers + Operations)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "20cc2d6d",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Requirements for DevOps\n",
    "\n",
    "- Good, automated tests that ensure quality\n",
    "- Clear status updates and communication (if a project has many contributors)\n",
    "- Easy mechanisms for making a release"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b2433cba",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Example of DevOps/CI: interpreting CI tests & logs\n",
    "\n",
    "https://github.com/BioJulia/BioSequences.jl/pull/267"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f88fff87",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Impacts of continuous integration\n",
    "\n",
    "- [Zhao, Vasilescu, et al 2017](https://ieeexplore.ieee.org/abstract/document/8115619/): several hundred (depending on the particular analysis) highly-active projects 12 months prior to & after adopting CI, omitting the month surrounding the switch\n",
    "- [Bernardo, Alencar da Costa, & Kulesza 2018](https://ieeexplore.ieee.org/abstract/document/8595196/): 87 projects each over a 5-year span\n",
    "- [Raman & Roy, 2017](https://ieeexplore.ieee.org/abstract/document/7962406/): >1000 projects, analyze review participation vs CI test outcomes\n",
    "\n",
    "Fairly good consensus that CI does *not* reduce how long PRs stay open.\n",
    "\n",
    "Fairly good consensus that CI is correlated with a ~3x increase in the number of pull requests. Causality is undetermined.\n",
    "\n",
    "Pull requests that pass their tests get ~2x the code-reviews of ones that fail the tests"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "49ddd99b",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## A major effect: reduced workload for merges\n",
    "\n",
    "- Measure: the time delay between PR submission & delivery in a release\n",
    "- Determine the explanatory power of different variables\n",
    "\n",
    "![CI mechanism](figures/ci_mechanism.png)\n",
    "\n",
    "Bernardo et al 2018"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d96bded3",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "- Merge workload: a measure of \"review backlog\"\n",
    "- Queue rank: a measure of \"when in the release cycle was the PR submitted?\"\n",
    "\n",
    "Conclusion: CI reduces reviewer workload to make it less of a bottleneck."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f08bd775",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Too much of a burden for small projects?\n",
    "\n",
    "Analysis of 1086 papers published in the *Journal of Open Source Software*:\n",
    "\n",
    "![CI prevalence](figures/ci_prevalence.png)\n",
    "\n",
    "Jamie Quinn (Florida State University), [Analysing the prevalence of continuous integration in JOSS ](http://blog.jamiejquinn.com/analysing-ci-in-joss)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e860daf3",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# ...but not in the Julia community!\n",
    "\n",
    "![CI prevalence in Julia](figures/ci_prevalence_julia.png)\n",
    "\n",
    "Eric P. Hanson (Beacon Biosignals) & Mosè Giordano (UCL), [Code, docs, and tests: what's in the General registry?](https://julialang.org/blog/2021/08/general-survey/)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1fbd5872",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Overall statistics among Julia packages\n",
    "\n",
    "In the Julia community:\n",
    "- 95% of all packages use a CI provider\n",
    "- 84% of all packages have at least 20 lines of tests; 57% have at least 20% of their code as tests\n",
    "- 88% have some kind of documentation (README or `docs/`)\n",
    "- 96% have an OSI-approved license (now a requirement for new releases)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2865a051",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "This is true despite the fact that most packages are still small efforts (median number of contributors: 2)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a4f40ac7",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "The Julia community has figured this out!"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fbe0c801",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# What's needed to make this \"easy\"?\n",
    "\n",
    "- Adopt CI/DevOps from the beginning of the project, not a transition made later\n",
    "- Good tools for setting it up and maintaining it, taking burden off the developer(s)\n",
    "- Training in the essential practices (today's lecture!)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "efcbe2bb",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Julia tooling:\n",
    "- [PkgTemplates](https://github.com/invenia/PkgTemplates.jl) to set up all phases of DevOps when you create your package\n",
    "- [JuliaActions](https://github.com/julia-actions), an organization to maintain & improve DevOps\n",
    "- [CompatHelper](https://github.com/JuliaRegistries/CompatHelper.jl) to stay up-to-date with your package's dependencies\n",
    "- [Registrator](https://github.com/JuliaRegistries/Registrator.jl) for making releases and [TagBot](https://github.com/JuliaRegistries/TagBot) for tagging & writing release notes\n",
    "- [Documenter](https://github.com/JuliaDocs/Documenter.jl) for documentation (integrated with testing)\n",
    "- Stuff we won't cover: [GitHub.jl](https://github.com/JuliaWeb/GitHub.jl) (a package for \"manipulating\" GitHub via software), [MassInstallAction](https://github.com/julia-actions/MassInstallAction.jl), [RetroCap](https://github.com/JuliaRegistries/RetroCap.jl), etc., for rare changes needed across many packages\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "29f7fec6",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# PkgTemplates and its plugins\n",
    "\n",
    "```julia\n",
    "using PkgTemplates\n",
    "tpl = Template(plugins=[GitHubActions(), Codecov(), Documenter{GitHubActions}()])\n",
    "```\n",
    "\n",
    "- `GitHubActions`: the CI provider (run your tests automatically in each PR)\n",
    "- `Codecov`: analysis of test coverage (discussed in last session)\n",
    "- `Documenter`: add documentation skeleton and configure it to build automatically via `GitHubActions`\n",
    "- `TagBot` (automatic): make nice release tags"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ba35583d",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Example of DevOps/CI: releases & TagBot\n",
    "\n",
    "https://github.com/JuliaDebug/JuliaInterpreter.jl"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ad0bd167",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# \"Languages\" used for documentation, CI, & release management\n",
    "\n",
    "- [Markdown](https://www.markdownguide.org/): Julia documentation (see also [Julia's extensions](https://docs.julialang.org/en/v1/stdlib/Markdown/))\n",
    "- [Tom's Obvious Minimal Language (TOML)](https://toml.io/en/): `Project.toml`, used for release management\n",
    "- [YAML Ain't Markup Language (YAML)](https://en.wikipedia.org/wiki/YAML): used by GitHub Actions\n",
    "\n",
    "All of these are used widely outside of Julia."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f7be82e0",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Markdown example\n",
    "\n",
    "```\n",
    "An h1 header\n",
    "============\n",
    "\n",
    "Paragraphs are separated by a blank line.\n",
    "\n",
    "2nd paragraph. *Italic*, **bold**, and `monospace`. Itemized lists\n",
    "look like:\n",
    "\n",
    "  * this one\n",
    "  * that one\n",
    "  * the other one\n",
    "\n",
    "Note that --- not considering the asterisk --- the actual text\n",
    "content starts at 4-columns in.\n",
    "\n",
    "> Block quotes are\n",
    "> written like so.\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bcacc462",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Documenter.jl: documentation structure\n",
    "\n",
    "```\n",
    "Example/\n",
    "├── docs/\n",
    "│   └── ...\n",
    "├── src/\n",
    "│   └── Example.jl\n",
    "...\n",
    "```\n",
    "\n",
    "with\n",
    "\n",
    "```\n",
    "docs/\n",
    "├── src/\n",
    "└── make.jl\n",
    "```\n",
    "\n",
    "Create Markdown files in `src/`\n",
    "\n",
    "Build the documentation by running `make.jl`."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "84ffc44a",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Writing documentation: the four categories\n",
    "\n",
    "![Categories of documentation](figures/documentation_categories.png)\n",
    "\n",
    "David Liang, [The documentation system](https://documentation.divio.com/)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "624c1abe",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    " \n",
    "| | Tutorials | How-to guides | Reference | Explanation |\n",
    "|:--- |:--- |:--- |:--- |:--- |\n",
    "| oriented to | learning | | | |\n",
    "| must | allow the newcomer to get started | | | |\n",
    "| its form | a lesson | | | |\n",
    "| analogy | teaching a small child how to cook | | | |\n",
    "\n",
    "David Liang, [The documentation system](https://documentation.divio.com/)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "16bea091",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    " | | Tutorials | How-to guides | Reference | Explanation |\n",
    "|:--- |:--- |:--- |:--- |:--- |\n",
    "| oriented to | learning | a goal | | |\n",
    "| must | allow the newcomer to get started | show how to solve a specific problem | | |\n",
    "| its form | a lesson | a series of steps | | |\n",
    "| analogy | teaching a small child how to cook | a recipe in a cook book | |\n",
    "\n",
    "David Liang, [The documentation system](https://documentation.divio.com/)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "05984eb5",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    " | | Tutorials | How-to guides | Reference | Explanation |\n",
    "|:--- |:--- |:--- |:--- |:--- |\n",
    "| oriented to | learning | a goal | information | |\n",
    "| must | allow the newcomer to get started | show how to solve a specific problem | describe the machinery | |\n",
    "| its form | a lesson | a series of steps | dry description | |\n",
    "| analogy | teaching a small child how to cook | a recipe in a cook book | a reference encyclopedia article | |\n",
    "\n",
    "David Liang, [The documentation system](https://documentation.divio.com/)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9210bd26",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    " | | Tutorials | How-to guides | Reference | Explanation |\n",
    "|:--- |:--- |:--- |:--- |:--- |\n",
    "| oriented to | learning | a goal | information | understanding |\n",
    "| must | allow the newcomer to get started | show how to solve a specific problem | describe the machinery | explain |\n",
    "| its form | a lesson | a series of steps | dry description | discursive explanation |\n",
    "| analogy | teaching a small child how to cook | a recipe in a cook book | a reference encyclopedia article | an article on culinary social history |\n",
    "\n",
    "David Liang, [The documentation system](https://documentation.divio.com/)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "40e2c78a",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# \"Helper\" action languages: TOML & YAML\n",
    "\n",
    "Both TOML and YAML have a key-value structure:\n",
    "\n",
    "- TOML: `key = value`. Document `[sections]` are essentially \"nested `Dict`s\"\n",
    "- YAML: `key: value`. Nesting is provided by increasing indentation"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a6d18dc0",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Both support arrays:\n",
    "\n",
    "- TOML: `x = [1, 2, 3]`\n",
    "- YAML: both \"inline\" `x: [1, 2, 3]` and markdown-list style\n",
    "\n",
    "```yaml\n",
    "x:\n",
    "  - 1\n",
    "  - 2\n",
    "  - 3\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1f7979d4",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# TOML example\n",
    "\n",
    "```toml\n",
    "# This is a TOML document\n",
    "\n",
    "title = \"TOML Example\"\n",
    "\n",
    "[owner]\n",
    "name = \"Tom Preston-Werner\"\n",
    "dob = 1979-05-27T07:32:00-08:00\n",
    "\n",
    "[database]\n",
    "enabled = true\n",
    "ports = [ 8000, 8001, 8002 ]\n",
    "data = [ [\"delta\", \"phi\"], [3.14] ]\n",
    "temp_targets = { cpu = 79.5, case = 72.0 }\n",
    "\n",
    "[servers]\n",
    "\n",
    "[servers.alpha]\n",
    "ip = \"10.0.0.1\"\n",
    "role = \"frontend\"\n",
    "\n",
    "[servers.beta]\n",
    "ip = \"10.0.0.2\"\n",
    "role = \"backend\"\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ee0b8b7a",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# YAML example\n",
    "\n",
    "```yaml\n",
    "---\n",
    "- hosts: webservers\n",
    "\n",
    "  vars:\n",
    "    http_port: 80\n",
    "    max_clients: 200\n",
    "\n",
    "  remote_user: root\n",
    "\n",
    "  tasks:\n",
    "  - name: ensure apache is at the latest version\n",
    "    yum:\n",
    "      name: httpd\n",
    "      state: latest\n",
    "\n",
    "  - name: write the apache config file\n",
    "    template:\n",
    "      src: /srv/httpd.j2\n",
    "      dest: /etc/httpd.conf\n",
    "    notify:\n",
    "    - restart apache\n",
    "\n",
    "  - name: ensure apache is running\n",
    "    service:\n",
    "      name: httpd\n",
    "      state: started\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "edb92047",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# A typical `Project.toml`\n",
    "\n",
    "From [JuliaInterpreter](https://github.com/JuliaDebug/JuliaInterpreter.jl) (foundation of the Debugger & Revise):\n",
    "\n",
    "```toml\n",
    "name = \"JuliaInterpreter\"\n",
    "uuid = \"aa1ae85d-cabe-5617-a682-6adf51b2e16a\"\n",
    "version = \"0.8.20\"\n",
    "\n",
    "[deps]\n",
    "CodeTracking = \"da1fd8a2-8d9e-5ec2-8556-3022fb5608a2\"\n",
    "InteractiveUtils = \"b77e0a4c-d291-57a0-90e8-8db25a27a240\"\n",
    "Random = \"9a3f8284-a2c9-5f02-9a11-845980a1fd5c\"\n",
    "UUIDs = \"cf7118a7-6976-5b1a-9a39-7adc72f591a4\"\n",
    "\n",
    "[compat]\n",
    "CodeTracking = \"0.5.9, 1\"\n",
    "julia = \"1\"\n",
    "\n",
    "[extras]\n",
    "DataFrames = \"a93c6f00-e57d-5684-b7b6-d8193f3e46c0\"\n",
    "Dates = \"ade2ca70-3891-5945-98fb-dc099432e06a\"\n",
    "Distributed = \"8ba89e20-285c-5b6f-9357-94700520ee1b\"\n",
    "HTTP = \"cd3eb016-35fb-5094-929b-558a96fad6f3\"\n",
    "LinearAlgebra = \"37e2e46d-f89d-539d-b4ee-838fcccc9c8e\"\n",
    "Mmap = \"a63ad114-7e13-5084-954f-fe012c677804\"\n",
    "PyCall = \"438e738f-606a-5dbb-bf0a-cddfbfd45ab0\"\n",
    "SHA = \"ea8e919c-243c-51af-8825-aaa63cd721ce\"\n",
    "SparseArrays = \"2f01184e-e22b-5df5-ae63-d93ebab69eaf\"\n",
    "TableReader = \"70df011a-6618-58d7-8e16-3cf9e384cb47\"\n",
    "Tensors = \"48a634ad-e948-5137-8d70-aa71f2a747f4\"\n",
    "Test = \"8dfed614-e22c-5e08-85e1-65c5234f0b40\"\n",
    "\n",
    "[targets]\n",
    "test = [\"Test\", \"Dates\", \"Distributed\", \"HTTP\", \"LinearAlgebra\", \"Mmap\", \"PyCall\", \"SHA\", \"SparseArrays\", \"Tensors\", \"TableReader\", \"DataFrames\"]\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fa22c694",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# A typical GitHub Actions file\n",
    "\n",
    "Also for JuliaInterpreter\n",
    "\n",
    "```yaml\n",
    "name: CI\n",
    "on:\n",
    "  pull_request:\n",
    "  push:\n",
    "    branches:\n",
    "      - master\n",
    "    tags: ['*']\n",
    "jobs:\n",
    "  test:\n",
    "    name: Julia ${{ matrix.version }} - ${{ matrix.os }} - ${{ matrix.arch }} - ${{ github.event_name }}\n",
    "    runs-on: ${{ matrix.os }}\n",
    "    strategy:\n",
    "      fail-fast: false\n",
    "      matrix:\n",
    "        version:\n",
    "          - '1.0'\n",
    "          - '1'\n",
    "          - 'nightly'\n",
    "        os:\n",
    "          - ubuntu-latest\n",
    "          - macOS-latest\n",
    "          - windows-latest\n",
    "        arch:\n",
    "          - x64\n",
    "    env:\n",
    "      PYTHON: \"\"\n",
    "    steps:\n",
    "      - uses: actions/checkout@v2\n",
    "      - uses: julia-actions/setup-julia@v1\n",
    "        with:\n",
    "          version: ${{ matrix.version }}\n",
    "          arch: ${{ matrix.arch }}\n",
    "      - uses: actions/cache@v1\n",
    "        env:\n",
    "          cache-name: cache-artifacts\n",
    "        with:\n",
    "          path: ~/.julia/artifacts\n",
    "          key: ${{ runner.os }}-test-${{ env.cache-name }}-${{ hashFiles('**/Project.toml') }}\n",
    "          restore-keys: |\n",
    "            ${{ runner.os }}-test-${{ env.cache-name }}-\n",
    "            ${{ runner.os }}-test-\n",
    "            ${{ runner.os }}-\n",
    "      - uses: julia-actions/julia-buildpkg@v1\n",
    "      - uses: julia-actions/julia-runtest@v1\n",
    "      - uses: julia-actions/julia-processcoverage@v1\n",
    "      - uses: codecov/codecov-action@v1\n",
    "        with:\n",
    "          file: lcov.info          \n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1092a6c5",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Understanding version numbers: semantic versioning\n",
    "\n",
    "![semver](figures/semver.png)\n",
    "\n",
    "Adrian Mejia, [Node Package Manager (NPM) Tutorial](https://adrianmejia.com/node-package-manager-npm-tutorial/)\n",
    "\n",
    "Also allowed:\n",
    "\n",
    "- pre-releases: `1.0.0-alpha.1`\n",
    "- build metadata: `1.0.0+20130313144700`"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "836eaba8",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Tips\n",
    "\n",
    "- learn as little about TOML and YAML as you need to get by & consult documentation when needed\n",
    "- rely on [PkgTemplates](https://github.com/invenia/PkgTemplates.jl) and continuous improvements in [JuliaActions](https://github.com/julia-actions): give them GitHub stars, as they do a lot to make your life easy!"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7ceb45e6",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Summary\n",
    "\n",
    "*Learning* DevOps will cost some time, but the Julia community has gone to great lengths to make it as easy as possible. Similar efforts are surely underway in other programming languages."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "001f1441",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "*Maintaining* DevOps also costs some time, but it also gives you time: easier review of pull requests, near-certainty that things won't break, the freedom to release frequently, *automation of many otherwise painful steps*."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bc2ceebf",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "It is becoming standard in many ecosystems and seems likely to become so in most. Prepare yourself to remain relevant!"
   ]
  }
 ],
 "metadata": {
  "@webio": {
   "lastCommId": null,
   "lastKernelId": null
  },
  "celltoolbar": "Slideshow",
  "kernelspec": {
   "display_name": "Julia 1.8.5",
   "language": "julia",
   "name": "julia-1.8"
  },
  "language_info": {
   "file_extension": ".jl",
   "mimetype": "application/julia",
   "name": "julia",
   "version": "1.8.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}