Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add reusable workflow for creation of Solidity API docs #43

Merged
merged 3 commits into from
May 15, 2023

Commits on Apr 18, 2023

  1. Add reusable workflow for creation of Solidity API docs

    In this commit we create a reusable workflow which automatically generates the
    Markdown contracts documentation based on the functions and the NatSpec-format
    comments in the Solidity files of the specified project. Depending on the value
    of the workflow's inputs, the workflow may sync the generated files with the
    specified directory in the external repository (we will use this to push the
    documentation to a directory that's synchronized with GitBook).
    
    ---How Markdown documentation gets created and pushed---
    1. The documentation gets created based on the content of the Solidity files in
       `contracts` folder of the specified project (`projectSubfolder`). Before we
       run the Docgen tool generating Markdown files, we may need to perform some
       slight changes to the input files, as some of the formatting we use in the
       `.sol` files of the projects where we want to use the action is interpreted
       by Docgen not the way we would like or is not completely in line with the
       NatSpec format:
       - In many `@dev` clauses in the Solidity files we have the lists of
         requirements or other items which are constructed like this:
         ```
          /// @dev Requirements:
          /// - Item one. Lorem ipsum dolor sit amet, consectetur adipiscing elit.
          ///   Nulla sed porttitor enim, sit amet venenatis enim. Donec tincidunt
          ///   auctor velit non eleifend. Nunc sit amet est non ligula condimentum
          ///   mattis.
          /// - Item two. Quisque purus massa, pellentesque in viverra tempus,
          ///   aliquet nec urna.
         ```
         This doesn't get recognized by Docgen as a list and is translated to
         regular text, which is displayed as one continuous line. But when the space
         characters between `///` and the text get removed from he `.sol` files, the
         lists are recognized correctly (without breaking interpretation of other
         features). That's why (unless `removeTrailingSpacesInComments` is set to
         `false`) we run `sed -i 's_///[[:blank:]]*_///_' command on all Solidity
         files.
       - There may be other problems with the formatting of the input files that may
         need correction. To handle that we allow for specifying
         `preProcessingCommand`, which can modify the input files according to need.
         An example of such command:
         `sed -i ':a;N;$!ba;s_///\n//\n_///\n_g' ./contracts/bridge/BitcoinTx.sol`.
    2. Once the files are ready, we install dependencies. It is assumed that the
       project in which action will be run has the `solidity-docgen` package added
       to the list of dev dependencies in `package.json`.
    3. After project is installed we run the Docgen tool. In order for the tool to
       work, the Hardhat config needs to be updated (see
       https://github.com/OpenZeppelin/solidity-docgen#usage for more details). The
       workflow expects `outputDir` to be set to `generated-docs`. You may also need
       to specify other configs, like `pages`, `templates` or `exclude`, according
       to the needs of particular project.
       The tool will generate to `generated-docs` either one `index.md` file (if
       `pages` is set to `single`) or a set of files (if `pages` is set to `items`
       or `files`), with names reflecting their content.
    4. If docs are generated to one file, it may e useful to add a Table Of Contents
       to that file, to ease the navigation. This will be done if `addTOC` is set
       to `true`. We use `markdown-toc` tool for TOC generation. The specific
       options that should be used during generation can be specified in the
       `tocOptions` input.
    5. Then (if `exportAsGHArtifacts` is set to true) we export the generated
       file(s) as GH Actions artifacts. They will be available for download in the
       details of the workflow run.
    6. If artifacts were exported, workflow was triggered by a PR and the
       `commentPR` was set to true, a comment with information where the artifacts
       can be found will be posted in the PR.
    7. If `publish` input was set, we then proceed to the steps pushing the
       generated file(s) to a repository/path/branch specified in the workflow's
       inputs. To do that we first need to make sure we use correct config to be
       able to publish to the destination repository. We need to provide email and
       a username of the user who will be author of the commit. If repository allows
       only for commits from verified users, the `verifyCommits` input should be
       set to `true` and the user have the GPG key configured (see
       https://docs.github.com/en/authentication/managing-commit-signature-verification/generating-a-new-gpg-key).
       The GPG private key and a passphrase to the key should be stored as GH
       secrets in the repositories where the workflow will be used. The name of the
       secrets should be passed to the workflow as `git_user_signingkey` and
       `git_commit_gpgsign`.
       Apart from configuring aforementioned commiter-related params, a GH secret
       storing GitHub API token (personal access token) for the destination
       repository needs to be provided in the `githubToken` secret.
       If everything is configured, the workflow syncs the content of the folder
       storing generated docs with the content of the destination directory (using
       `rsync`). If `rsyncDelete` input is set to true, the files which exist in the
       destination repository, but don't exist in the `generated-docs` folder will
       be removed from the destination repo. One may want to set this option if
       Docgen's `pages` attribute is set to `pages` or `items` (so that when a
       contract or its function gets removed, we would remove the corresponding doc
       from the destination repo). The setting should be used with caution!
    michalinacienciala committed Apr 18, 2023
    Configuration menu
    Copy the full SHA
    b5c7e63 View commit details
    Browse the repository at this point in the history

Commits on Apr 25, 2023

  1. Configuration menu
    Copy the full SHA
    283d7a7 View commit details
    Browse the repository at this point in the history

Commits on May 3, 2023

  1. Create PR in destination repo instead of pushing directly to main

    Previous version of the workflow assumed the workflow will be used to push the
    updated docs directly to a branch that's synchronized with GitBook. But we
    decided it will be safer to create a PR in the destination repo instead, that
    will have the GitBook-synchronized branch set as a PR base branch.
    
    Workflows that will use our reusable workflow will need to specify
    `destinationBaseBranch` input (if not, `main` will be set as a default).
    
    Objectives:
    The destination repo may be in a bunch of different states when workflow is
    executed, we wanted to handle those states. It is assumed that the reusable
    worklow will be triggered if there's a `release` event in the project using this
    workflow.
    
    The release may not change anything in the contracts functions and comments
    resulting in no changes between docs which are already in the destination repo
    and those which got generated by the workflow. In that case we don't want to
    push anything and create/update any PRs.
    
    Probably the most common situation will be a case where a release modifies
    something in contracts' functions/comments, resulting in a need for the docs
    update. In such case we want to commit the changes and create a PR in the
    destination repo. If there is already an earlier open PR updating the docs, we
    want to update it instead of creating a new one.
    
    To handle above objectives:
    1. We configure `base_branch` and `head_branch` environment variables. The
       `base_branch` variable is set to value passed from `destinationBaseBranch`
       input. The `head_branch` will be set to `auto-update-solidity-api-docs`.
    2. Next, we clone the `destinationRepo` and name it `dest-repo-clone` (we change
       the name because we want to avoid a situation where there's already a folder
       with the name of the destination repo present in repo where we run the
       workflow). All branches are cloned, and `head_branch` gets checked out.
    3. We enter the cloned repo and checkout the `head_branch` (if it does not
       exist, we create a new branch under this name - based on `head_branch`- and
       checkout it).
    4. If destination directory for docs does not exist, we create it. Then we
       synchronize the content of the `../generated-docs` folder with the content of
       destination folder.
    5. Then we stage all the changes made in the `destinationRepo` and check if
       anything was staged. If nothing was staged, this means there were no changes
       to the dosc and we can finish the workflow without doing any changes to the
       destination repo (we end at step 5). If something was staged, we proceed to
       step 6.
    6. We configure commiter in Git and commit the changes.
    7. We push the commit to the `head_branch`.
    8. Then we check if there's already an open PR existing for the `head_branch`. We
       do that by executing a curl command that lists all the open PRs in the
       `destinationRepo` that have the head branch set to `head_branch`. If no PRs
       match the criteria, the curl returns `[\n\n]`. If there's already a PR
       existing, then we're ok and we don't need to do anything else (PR got updated
       in step 7). If not, we create a PR using `hub` CLI tool
       (https://github.com/github/hub).
    michalinacienciala committed May 3, 2023
    Configuration menu
    Copy the full SHA
    4d230f2 View commit details
    Browse the repository at this point in the history