-
Notifications
You must be signed in to change notification settings - Fork 386
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Move GTO docs here from mlem.ai (#4822)
* Move GTO docs here from mlem.ai * one more * fix missing links
- Loading branch information
Showing
31 changed files
with
1,286 additions
and
63 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,61 @@ | ||
# assign | ||
|
||
Assign stage to specific artifact version. | ||
|
||
## Synopsis | ||
|
||
```usage | ||
usage: gto assign [-r <text>] [--version <text>] | ||
[--stage <text>] [-m <text>] | ||
[--simple <text>] [--force] [--push] [--sr] | ||
[-h] | ||
name [ref] | ||
arguments: | ||
name Artifact name | ||
[ref] Git reference to use | ||
``` | ||
|
||
## Description | ||
|
||
To assign an actionable stage for a specific artifact version use the same | ||
`gto assign` command. Stages can mark the artifact readiness for a specific | ||
consumer. You can plug in a real downsteam system via CI/CD or web hooks, e.g. | ||
to redeploy an ML model. | ||
|
||
```cli | ||
$ gto assign awesome-model --version v0.0.1 --stage prod | ||
Created git tag 'awesome-model#prod#1' that assigns stage to 'v0.0.1' | ||
``` | ||
|
||
GTO creates a special Git tag in | ||
[the standard format](/doc/gto/user-guide#git-tags-format). | ||
|
||
## Options | ||
|
||
- `-r <text>`, `--repo <text>` - Local or remote repository [default: .] | ||
- `--version <text>` - If you provide REF, this will be used to name new version | ||
- `--stage <text>` - Stage to assign | ||
- `-m <text>`, `--message <text>` - Message to annotate the Git tag with | ||
- `--simple <text>` - Use simple notation, e.g. `rf#prod` instead of `rf#prod-5` | ||
[supported values: auto, true, false] [default: auto] | ||
- `--force` - Create the Git tag even if it already exists and is in effect | ||
- `--push` - Push created tag automatically (experimental) | ||
- `--sr`, `--skip-registration` - Don't register a version at specified commit | ||
- `-h`, `--help` - Show this message and exit. | ||
|
||
## Examples | ||
|
||
Assign artifact "nn" to "prod" at specific Git ref instead of supplying artifact | ||
version (note that this will also register a version if it doesn't exist): | ||
|
||
```cli | ||
$ gto assign nn abcd123 --stage prod | ||
``` | ||
|
||
Assign stage at specific Git ref and name the version explicitly (this assumes | ||
that version was not registered yet): | ||
|
||
```cli | ||
$ gto assign nn abcd123 --version v1.0.0 --stage prod | ||
``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,36 @@ | ||
# check-ref | ||
|
||
Find out the artifact version registered/assigned with ref. | ||
|
||
## Synopsis | ||
|
||
```usage | ||
usage: gto check-ref [-r <text>] [--json] [--name] [--version] | ||
[--event] [--stage] [-h] | ||
ref | ||
arguments: | ||
ref Git reference to analyze | ||
``` | ||
|
||
## Description | ||
|
||
You can use `gto check-ref` to interpret a Git tag: | ||
|
||
```cli | ||
$ gto check-ref -r build/example-gto churn#prod#3 | ||
β Stage "prod" was assigned to version "v3.0.0" of artifact "churn" | ||
``` | ||
|
||
For machine-consumable format, use `--json` flag or output specific pieces of | ||
information with `--name`, `--version`, `--stage` or `--event`. | ||
|
||
## Options | ||
|
||
- `-r <text>`, `--repo <text>` - Local or remote repository [default: .] | ||
- `--json` - Print output in json format | ||
- `--name` - Show artifact name | ||
- `--version` - Output artifact version | ||
- `--event` - Show event | ||
- `--stage` - Show artifact stage | ||
- `-h`, `--help` - Show this message and exit. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,141 @@ | ||
# deprecate | ||
|
||
Deprecate artifact, deregister a version, or unassign a stage. | ||
|
||
## Synopsis | ||
|
||
```usage | ||
usage: gto deprecate [-r <text>] [-m <text>] [--simple <text>] | ||
[--force] [-d] [--push] [-h] | ||
name [version] [stage] | ||
arguments: | ||
name Artifact name | ||
[version] Artifact version | ||
[stage] Stage to unassign | ||
``` | ||
|
||
## Description | ||
|
||
The command supports three use cases: | ||
|
||
```cli | ||
# deprecate an artifact: | ||
$ gto deprecate nn | ||
# deprecate a version: | ||
$ gto deprecate nn v0.0.1 | ||
# unassign a stage: | ||
$ gto deprecate nn v0.0.1 prod | ||
``` | ||
|
||
### Unassigning a stage | ||
|
||
Sometimes you need to mark an artifact version no longer ready for a specific | ||
consumer, and maybe signal a downstream system about this. You can use | ||
`gto deprecate` for that: | ||
|
||
```cli | ||
$ gto deprecate awesome-model v0.0.1 prod | ||
Created git tag 'awesome-model#prod!#2' that unassigns a stage from 'v0.0.1' | ||
``` | ||
|
||
<details> | ||
|
||
### Unassigning a stage: some details and options | ||
|
||
GTO creates a special Git tag in | ||
[the standard format](/doc/gto/user-guide#git-tags-format). | ||
|
||
Note, that later you can create this stage again, if you need to, by calling | ||
`$ gto assign` again. | ||
|
||
You also may want to delete the git tag instead of creating a new one. This is | ||
useful if you don't want to keep extra tags in you Git repo, don't need history | ||
and don't want to trigger a CI/CD or another downstream system. For that, you | ||
can use: | ||
|
||
```cli | ||
$ gto deprecate awesome-model v0.0.1 prod --delete | ||
Deleted git tag 'awesome-model#prod#1' that assigned a stage to 'v0.0.1' | ||
To push the changes upstream, run: | ||
git push origin awesome-model#prod#1 --delete | ||
``` | ||
|
||
</details> | ||
|
||
### Deregister a version | ||
|
||
Sometimes you need mark a specific artifact version as a no longer ready for | ||
usage. You could just delete a git tag, but if you want to preserve a history of | ||
the actions, you may again use `gto deprecate`. | ||
|
||
```cli | ||
$ gto deprecate awesome-model v0.0.1 | ||
Created git tag '[email protected]!' that deregistered a version. | ||
``` | ||
|
||
<details> | ||
|
||
### Deregister a version: some details and options | ||
|
||
If you want to deregister the version by deleting the Git tags itself, you could | ||
use | ||
|
||
```cli | ||
$ gto deprecate awesome-model v0.0.1 --delete | ||
Deleted git tag '[email protected]' that registered a version. | ||
Deleted git tag 'awesome-model#prod#1' that assigned a stage to 'v0.0.1'. | ||
Deleted git tag 'awesome-model#prod!#2' that unassigned a stage to 'v0.0.1'. | ||
To push the changes upstream, run: | ||
git push origin [email protected] awesome-model#prod#1 awesome-model#prod!#2 --delete | ||
``` | ||
|
||
This includes all Git tags related to the version: a tag that registered it and | ||
all tags that assigned stages to it. | ||
|
||
</details> | ||
|
||
### Deprecating an artifact | ||
|
||
Sometimes you need to need to mark the artifact as "deprecated", usually meaning | ||
it's outdated and will no longer be developed. To do this, you could run: | ||
|
||
```cli | ||
$ gto deprecate awesome-model | ||
Created Git tag 'awesome-model@deprecated' that deprecates an artifact. | ||
``` | ||
|
||
<details> | ||
|
||
### Deprecating an artifact: some details and options | ||
|
||
With `awesome-model@deprecated` Git tag the artifact will be considered | ||
deprecated until you register a new version or assign a new stage to it after | ||
the deprecation. | ||
|
||
If you want to deprecate an artifact by deleting git tags, you'll need to delete | ||
all of them for the artifact. You could do that with | ||
|
||
```cli | ||
$ gto deprecate awesome-model --delete | ||
Deleted git tag '[email protected]' that registered a version. | ||
Deleted git tag 'awesome-model#prod#1' that assigned a stage to 'v0.0.1'. | ||
Deleted git tag 'awesome-model#prod!#2' that unassigned a stage to 'v0.0.1'. | ||
To push the changes upstream, run: | ||
git push origin [email protected] awesome-model#prod#1 awesome-model#prod!#2 --delete | ||
``` | ||
|
||
</details> | ||
|
||
## Options | ||
|
||
- `-r <text>`, `--repo <text>` - Local or remote repository [default: .] | ||
- `-m <text>`, `--message <text>` - Message to annotate the Git tag with | ||
- `--simple <text>` - Use simple notation, e.g. `rf#prod` instead of `rf#prod-5` | ||
[supported values: auto, true, false] [default: auto] | ||
- `--force` - Create the Git tag even if it already exists and is in effect | ||
- `-d`, `--delete` - Delete the git tag(s) instead of creating the new one | ||
- `--push` - Push created tag automatically (experimental) | ||
- `-h`, `--help` - Show this message and exit. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,52 @@ | ||
# describe | ||
|
||
Display enrichments for an artifact. | ||
|
||
## Synopsis | ||
|
||
```usage | ||
usage: gto describe [-r <text>] [--rev <text>] [--type] [--path] | ||
[--description] [-h] | ||
name | ||
arguments: | ||
name Artifact name | ||
``` | ||
|
||
## Description | ||
|
||
To get details about an artifact (from `artifacts.yaml`) use `gto describe`: | ||
|
||
```cli | ||
$ gto describe churn -r https://github.com/iterative/example-gto | ||
{ | ||
"type": "model", | ||
"path": "models/churn.pkl", | ||
"virtual": false | ||
} | ||
``` | ||
|
||
The output is in JSON format for ease of parsing programmatically. | ||
|
||
Note, that for local repos the `artifacts.yaml` is read from the workspace | ||
without Git, so if you have uncommitted changes, they will be reflected in the | ||
output. If you want to read from specific commit, you need to specify `--rev` | ||
option. | ||
|
||
You can also get annotation for specific versions (these are the same shortcuts | ||
as in `gto show`): | ||
|
||
```cli | ||
$ gto describe churn@latest # highest version by SemVer | ||
$ gto describe churn#dev # version in stage `dev` | ||
$ gto describe [email protected] # version `v3.0.0` | ||
``` | ||
|
||
## Options | ||
|
||
- `-r <text>`, `--repo <text>` - Local or remote repository [default: .] | ||
- `--rev <text>` - Repo revision to use | ||
- `--type` - Show type | ||
- `--path` - Show path | ||
- `--description` - Show description | ||
- `-h`, `--help` - Show this message and exit. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,36 @@ | ||
# describe | ||
|
||
Display GTO version and check the registry for problems. | ||
|
||
## Synopsis | ||
|
||
```usage | ||
usage: gto doctor [-r <text>] [-A] [-h] | ||
``` | ||
|
||
## Description | ||
|
||
This will check the registry and print all the issues if found: | ||
|
||
```cli | ||
$ gto doctor | ||
πͺ΄ GTO Version: 0.2.5 | ||
--------------------------------- | ||
INDEX='artifacts.yaml' | ||
TYPES=None | ||
STAGES=None | ||
LOG_LEVEL='INFO' | ||
DEBUG=False | ||
ENRICHMENTS=[] | ||
AUTOLOAD_ENRICHMENTS=True | ||
CONFIG_FILE_NAME='.gto' | ||
EMOJIS=True | ||
--------------------------------- | ||
β No issues found | ||
``` | ||
|
||
## Options | ||
|
||
- `-r <text>`, `--repo <text>` - Local or remote repository [default: .] | ||
- `-A`, `--all-commits` - Read all commits | ||
- `-h`, `--help` - Show this message and exit. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,50 @@ | ||
# history | ||
|
||
Show a journal of registry operations. | ||
|
||
## Synopsis | ||
|
||
```usage | ||
usage: gto history [-r <text>] [-a] [-A] [--json] [--plain] | ||
[--asc] [-h] | ||
[name] | ||
arguments: | ||
[name] Artifact name to show. If empty, show all. | ||
``` | ||
|
||
## Description | ||
|
||
This command prints a journal of the events that happened to an artifact. This | ||
allows you to audit the changes. | ||
|
||
```cli | ||
$ gto history churn -r https://github.com/iterative/example-gto | ||
βββββββββββββββββββββββ€βββββββββββββ€βββββββββββββββ€ββββββββββββ€ββββββββββ€βββββββββββ€ββββββββββββββββββ | ||
β timestamp β artifact β event β version β stage β commit β ref β | ||
βββββββββββββββββββββββͺβββββββββββββͺβββββββββββββββͺββββββββββββͺββββββββββͺβββββββββββͺββββββββββββββββββ‘ | ||
β 2022-11-09 13:40:33 β churn β assignment β v3.1.1 β dev β 2f2a8de β churn#dev#5 β | ||
β 2022-11-09 13:40:33 β churn β registration β v3.1.1 β - β 2f2a8de β [email protected] β | ||
β 2022-11-08 09:53:53 β churn β commit β v3.1.1 β - β 2f2a8de β 2f2a8de β | ||
β 2022-11-07 06:07:13 β churn β assignment β v3.1.0 β dev β 064f173 β churn#dev#4 β | ||
β 2022-11-06 02:20:33 β churn β assignment β v3.0.0 β prod β ddae695 β churn#prod#3 β | ||
β 2022-11-04 22:33:53 β churn β assignment β v3.1.0 β staging β 064f173 β churn#staging#2 β | ||
β 2022-11-03 18:47:13 β churn β assignment β v3.0.0 β dev β ddae695 β churn#dev#1 β | ||
β 2022-11-02 15:00:33 β churn β registration β v3.1.0 β - β 064f173 β [email protected] β | ||
β 2022-11-01 11:13:53 β churn β commit β v3.1.0 β - β 064f173 β 064f173 β | ||
β 2022-10-28 23:53:53 β churn β registration β v3.0.0 β - β ddae695 β [email protected] β | ||
β 2022-10-27 20:07:13 β churn β commit β v3.0.0 β - β ddae695 β ddae695 β | ||
βββββββββββββββββββββββ§βββββββββββββ§βββββββββββββββ§ββββββββββββ§ββββββββββ§βββββββββββ§ββββββββββββββββββ | ||
``` | ||
|
||
Use `--all-branches` and `--all-commits` to read more than just HEAD. | ||
|
||
## Options | ||
|
||
- `-r <text>`, `--repo <text>` - Local or remote repository [default: .] | ||
- `-a`, `--all-branches` - Read heads from all branches | ||
- `-A`, `--all-commits` - Read all commits | ||
- `--json` - Print output in json format | ||
- `--plain` - Print table in grep-able format | ||
- `--ascending`, `--asc` - Show new first | ||
- `-h`, `--help` - Show this message and exit. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,11 @@ | ||
# GTO Command Reference | ||
|
||
GTO is a command line tool. It works on top of Git. For a list of all commands, | ||
type `gto -h`. | ||
|
||
## Typical GTO workflow | ||
|
||
- Register artifact versions with `gto register` | ||
- Assign stages to them with `gto assign` | ||
- Read the registry with `gto show` and `gto history` | ||
- Read and interpret Git tag with `gto check-ref` |
Oops, something went wrong.