diff --git a/README.md b/README.md index beafd650b..cbdbba806 100644 --- a/README.md +++ b/README.md @@ -81,14 +81,14 @@ jobs: run: runs-on: ubuntu-latest # optionally use a convenient Ubuntu LTS + DVC + CML image - # container: docker://ghcr.io/iterative/cml:0-dvc2-base1 + # container: ghcr.io/iterative/cml:0-dvc2-base1 steps: - - uses: actions/checkout@v2 + - uses: actions/checkout@v3 # may need to setup NodeJS & Python3 on e.g. self-hosted - # - uses: actions/setup-node@v2 + # - uses: actions/setup-node@v3 # with: # node-version: '16' - # - uses: actions/setup-python@v2 + # - uses: actions/setup-python@v4 # with: # python-version: '3.x' - uses: iterative/setup-cml@v1 @@ -103,7 +103,7 @@ jobs: run: | # Post reports as comments in GitHub PRs cat results.txt >> report.md - cml send-comment report.md + cml comment create report.md ``` ## Usage @@ -111,9 +111,9 @@ jobs: We helpfully provide CML and other useful libraries pre-installed on our [custom Docker images](https://github.com/iterative/cml/blob/master/Dockerfile). In the above example, uncommenting the field -`container: docker://ghcr.io/iterative/cml:0-dvc2-base1`) will make the runner -pull the CML Docker image. The image already has NodeJS, Python 3, DVC and CML -set up on an Ubuntu LTS base for convenience. +`container: ghcr.io/iterative/cml:0-dvc2-base1`) will make the runner pull the +CML Docker image. The image already has NodeJS, Python 3, DVC and CML set up on +an Ubuntu LTS base for convenience. ### CML Functions @@ -124,18 +124,17 @@ report. Below is a table of CML functions for writing markdown reports and delivering those reports to your CI system. -| Function | Description | Example Inputs | -| ----------------------- | ---------------------------------------------------------------- | ----------------------------------------------------------- | -| `cml runner` | Launch a runner locally or hosted by a cloud provider | See [Arguments](https://github.com/iterative/cml#arguments) | -| `cml publish` | Publicly host an image for displaying in a CML report | ` --title --md` | -| `cml send-comment` | Return CML report as a comment in your GitLab/GitHub workflow | ` --head-sha ` | -| `cml send-github-check` | Return CML report as a check in GitHub | ` --head-sha ` | -| `cml pr` | Commit the given files to a new branch and create a pull request | `...` | -| `cml tensorboard-dev` | Return a link to a Tensorboard.dev page | `--logdir --title --md` | +| Function | Description | Example Inputs | +| ------------------------- | ---------------------------------------------------------------- | ----------------------------------------------------------- | +| `cml runner launch` | Launch a runner locally or hosted by a cloud provider | See [Arguments](https://github.com/iterative/cml#arguments) | +| `cml comment create` | Return CML report as a comment in your GitLab/GitHub workflow | ` --head-sha ` | +| `cml check create` | Return CML report as a check in GitHub | ` --head-sha ` | +| `cml pr create` | Commit the given files to a new branch and create a pull request | `...` | +| `cml tensorboard connect` | Return a link to a Tensorboard.dev page | `--logdir --title --md` | #### CML Reports -The `cml send-comment` command can be used to post reports. CML reports are +The `cml comment create` command can be used to post reports. CML reports are written in markdown ([GitHub](https://github.github.com/gfm), [GitLab](https://docs.gitlab.com/ee/user/markdown.html), or [Bitbucket](https://confluence.atlassian.com/bitbucketserver/markdown-syntax-guide-776639995.html) @@ -153,11 +152,12 @@ cat results.txt >> report.md :framed_picture: **Images** Display images using the markdown or HTML. Note that if an image is an output of your ML workflow (i.e., it is produced by your -workflow), you will need to use the `cml publish` function to include it a CML -report. For example, if `graph.png` is output by `python train.py`, run: +workflow), it can be uploaded and included automaticlly to your CML report. For +example, if `graph.png` is output by `python train.py`, run: ```bash -cml publish graph.png --md >> report.md +echo "![](./graph.png)" >> report.md +cml comment create report.md ``` ### Getting Started @@ -189,8 +189,8 @@ jobs: run: runs-on: ubuntu-latest steps: - - uses: actions/checkout@v2 - - uses: actions/setup-python@v2 + - uses: actions/checkout@v3 + - uses: actions/setup-python@v4 - uses: iterative/setup-cml@v1 - name: Train model env: @@ -200,8 +200,8 @@ jobs: python train.py cat metrics.txt >> report.md - cml publish plot.png --md >> report.md - cml send-comment report.md + echo "![](./plot.png)" >> report.md + cml comment create report.md ``` 3. In your text editor of choice, edit line 16 of `train.py` to `depth = 5`. @@ -253,9 +253,9 @@ on: [push] jobs: run: runs-on: ubuntu-latest - container: docker://ghcr.io/iterative/cml:0-dvc2-base1 + container: ghcr.io/iterative/cml:0-dvc2-base1 steps: - - uses: actions/checkout@v2 + - uses: actions/checkout@v3 - name: Train model env: REPO_TOKEN: ${{ secrets.GITHUB_TOKEN }} @@ -278,16 +278,16 @@ jobs: echo "## Plots" >> report.md echo "### Class confusions" >> report.md dvc plots diff --target classes.csv --template confusion -x actual -y predicted --show-vega master > vega.json - vl2png vega.json -s 1.5 > plot.png - cml publish --md plot.png >> report.md + vl2png vega.json -s 1.5 > confusion_plot.png + echo "![](./confusion_plot.png)" >> report.md # Publish regularization function diff echo "### Effects of regularization" >> report.md dvc plots diff --target estimators.csv -x Regularization --show-vega master > vega.json vl2png vega.json -s 1.5 > plot.png - cml publish --md plot.png >> report.md + echo "![](./plot.png)" >> report.md - cml send-comment report.md + cml comment create report.md ``` > :warning: If you're using DVC with cloud storage, take note of environment @@ -423,14 +423,14 @@ jobs: runs-on: ubuntu-latest steps: - uses: iterative/setup-cml@v1 - - uses: actions/checkout@v2 + - uses: actions/checkout@v3 - name: Deploy runner on EC2 env: REPO_TOKEN: ${{ secrets.PERSONAL_ACCESS_TOKEN }} AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }} AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }} run: | - cml runner \ + cml runner launch \ --cloud=aws \ --cloud-region=us-west \ --cloud-type=g4dn.xlarge \ @@ -440,10 +440,10 @@ jobs: runs-on: [self-hosted, cml-gpu] timeout-minutes: 50400 # 35 days container: - image: docker://iterativeai/cml:0-dvc2-base1-gpu + image: ghcr.io/iterative/cml:0-dvc2-base1-gpu options: --gpus all steps: - - uses: actions/checkout@v2 + - uses: actions/checkout@v3 - name: Train model env: REPO_TOKEN: ${{ secrets.PERSONAL_ACCESS_TOKEN }} @@ -452,7 +452,7 @@ jobs: python train.py cat metrics.txt > report.md - cml send-comment report.md + cml comment create report.md ``` In the workflow above, the `deploy-runner` step launches an EC2 `g4dn.xlarge` @@ -466,9 +466,9 @@ newly-launched instance. See [Environment Variables] below for details on the #### Docker Images -The CML Docker image (`docker://iterativeai/cml`) comes loaded with Python, -CUDA, `git`, `node` and other essentials for full-stack data science. Different -versions of these essentials are available from different `iterativeai/cml` +The CML Docker image (`ghcr.io/iterative/cml` or `iterativeai/cml`) comes loaded +with Python, CUDA, `git`, `node` and other essentials for full-stack data +science. Different versions of these essentials are available from different image tags. The tag convention is `{CML_VER}-dvc{DVC_VER}-base{BASE_VER}{-gpu}`: | `{BASE_VER}` | Software included (`-gpu`) | @@ -476,62 +476,73 @@ image tags. The tag convention is `{CML_VER}-dvc{DVC_VER}-base{BASE_VER}{-gpu}`: | 0 | Ubuntu 18.04, Python 2.7 (CUDA 10.1, CuDNN 7) | | 1 | Ubuntu 20.04, Python 3.8 (CUDA 11.2, CuDNN 8) | -For example, `docker://iterativeai/cml:0-dvc2-base1-gpu`, or -`docker://ghcr.io/iterative/cml:0-dvc2-base1`. +For example, `iterativeai/cml:0-dvc2-base1-gpu`, or +`ghcr.io/iterative/cml:0-dvc2-base1`. #### Arguments -The `cml runner` function accepts the following arguments: +The `cml runner launch` function accepts the following arguments: ``` ---help Show help [boolean] ---version Show version number [boolean] ---log Maximum log level - [choices: "error", "warn", "info", "debug"] [default: "info"] ---labels One or more user-defined labels for this runner - (delimited with commas) [default: "cml"] ---idle-timeout Seconds to wait for jobs before shutting down. Set - to -1 to disable timeout [default: 300] ---name Name displayed in the repository once registered - cml-{ID} ---no-retry Do not restart workflow terminated due to instance - disposal or GitHub Actions timeout - [boolean] [default: false] ---single Exit after running a single job - [boolean] [default: false] ---reuse Don't launch a new runner if an existing one has - the same name or overlapping labels - [boolean] [default: false] ---driver Platform where the repository is hosted. If not - specified, it will be inferred from the - environment [choices: "github", "gitlab"] ---repo Repository to be used for registering the runner. - If not specified, it will be inferred from the - environment ---token Personal access token to register a self-hosted - runner on the repository. If not specified, it - will be inferred from the environment - [default: "infer"] ---cloud Cloud to deploy the runner - [choices: "aws", "azure", "gcp", "kubernetes"] ---cloud-region Region where the instance is deployed. Choices: - [us-east, us-west, eu-west, eu-north]. Also - accepts native cloud regions [default: "us-west"] ---cloud-type Instance type. Choices: [m, l, xl]. Also supports - native types like i.e. t2.micro ---cloud-gpu GPU type. - [choices: "nogpu", "k80", "v100", "tesla"] ---cloud-hdd-size HDD size in GB ---cloud-ssh-private Custom private RSA SSH key. If not provided an - automatically generated throwaway key will be used - [default: ""] ---cloud-spot Request a spot instance [boolean] ---cloud-spot-price Maximum spot instance bidding price in USD. - Defaults to the current spot bidding price - [default: "-1"] ---cloud-startup-script Run the provided Base64-encoded Linux shell script - during the instance initialization [default: ""] ---cloud-aws-security-group Specifies the security group in AWS [default: ""] + --labels One or more user-defined labels for + this runner (delimited with commas) + [string] [default: "cml"] + --idle-timeout Time to wait for jobs before + shutting down (e.g. "5min"). Use + "never" to disable + [string] [default: "5 minutes"] + --name Name displayed in the repository + once registered + [string] [default: cml-{ID}] + --no-retry Do not restart workflow terminated + due to instance disposal or GitHub + Actions timeout [boolean] + --single Exit after running a single job + [boolean] + --reuse Don't launch a new runner if an + existing one has the same name or + overlapping labels [boolean] + --reuse-idle Creates a new runner only if the + matching labels don't exist or are + already busy [boolean] + --docker-volumes Docker volumes, only supported in + GitLab [array] [default: []] + --cloud Cloud to deploy the runner + [string] [choices: "aws", "azure", "gcp", "kubernetes"] + --cloud-region Region where the instance is + deployed. Choices: [us-east, + us-west, eu-west, eu-north]. Also + accepts native cloud regions + [string] [default: "us-west"] + --cloud-type Instance type. Choices: [m, l, xl]. + Also supports native types like i.e. + t2.micro [string] + --cloud-permission-set Specifies the instance profile in + AWS or instance service account in + GCP [string] [default: ""] + --cloud-metadata Key Value pairs to associate + cml-runner instance on the provider + i.e. tags/labels "key=value" + [array] [default: []] + --cloud-gpu GPU type. Choices: k80, v100, or + native types e.g. nvidia-tesla-t4 + [string] + --cloud-hdd-size HDD size in GB [number] + --cloud-ssh-private Custom private RSA SSH key. If not + provided an automatically generated + throwaway key will be used [string] + --cloud-spot Request a spot instance [boolean] + --cloud-spot-price Maximum spot instance bidding price + in USD. Defaults to the current spot + bidding price [number] [default: -1] + --cloud-startup-script Run the provided Base64-encoded + Linux shell script during the + instance initialization [string] + --cloud-aws-security-group Specifies the security group in AWS + [string] [default: ""] + --cloud-aws-subnet, Specifies the subnet to use within + --cloud-aws-subnet-id AWS [string] [default: ""] + ``` #### Environment Variables @@ -556,12 +567,13 @@ CML support proxy via known environment variables `http_proxy` and #### On-premise (Local) Runners -This means using on-premise machines as self-hosted runners. The `cml runner` -function is used to set up a local self-hosted runner. On a local machine or -on-premise GPU cluster, [install CML as a package](#local-package) and then run: +This means using on-premise machines as self-hosted runners. The +`cml runner launch` function is used to set up a local self-hosted runner. On a +local machine or on-premise GPU cluster, +[install CML as a package](#local-package) and then run: ```bash -cml runner \ +cml runner launch \ --repo=$your_project_repository_url \ --token=$PERSONAL_ACCESS_TOKEN \ --labels="local,runner" \ @@ -577,9 +589,13 @@ pre-installed in a custom Docker image pulled by a CI runner. You can also install CML as a package: ```bash -npm i -g @dvcorg/cml +npm install --location=global @dvcorg/cml ``` +You can use `cml` without node by downloading the correct standalone binary for +your system from the asset section of the +[releases](https://github.com/iterative/cml/releases). + You may need to install additional dependencies to use DVC plots and Vega-Lite CLI commands: @@ -599,7 +615,7 @@ CML and Vega-Lite package installation require the NodeJS package manager use a set up action to install NodeJS: ```bash -uses: actions/setup-node@v2 +uses: actions/setup-node@v3 with: node-version: '16' ``` @@ -607,7 +623,7 @@ uses: actions/setup-node@v2 - **GitLab**: Requires direct installation. ```bash -curl -sL https://deb.nodesource.com/setup_12.x | bash +curl -sL https://deb.nodesource.com/setup_16.x | bash apt-get update apt-get install -y nodejs ```