clear¶
Once you have completed your analysis, or no longer have need of a dataset, you
+can remove it with datatrail clear. This will remove the common path of all
+of the downloaded files, typically "astro_xxxxxxxx" for baseband datasets.
+However, during the download, parent directories would've also been created,
+to remove these you can use the --clear-parents flag. Like pull, clear
+will ask for confirmation before removing the dataset.
$> datatrail clear kko.event.baseband.raw 308892599
+
+Searching for files for 308892599 kko.event.baseband.raw...
+
+Directory: ./data/kko/baseband/raw/2023/08/07/astro_308892599
+ - Found 4 files.
+ - Total size: 0.02 GB.
+
+⚠️ Delete files? [y/n]: y
+datatrail clear kko.event.baseband.raw 308892599 --clear-parents
+
+Searching for files for 308892599 kko.event.baseband.raw...
+
+Directory: ./data/kko/baseband/raw/2023/08/07/astro_308892599
+ - Found 9 files.
+ - Total size: 0.04 GB.
+
+⚠️ Delete files and empty parent directories? [y/n]: y
+Warning: Modified download paths (--directory)
If you'd downloaded a dataset to a different directory than the root
+mount for your site, you'll need to use the --directory flag with
+same value.
Verbosity: More insight
+$> datatrail clear kko.event.baseband.raw 308892599 --clear-parents -vv
+[17:13:28] DEBUG `clear` called with:
+ DEBUG scope: kko.event.baseband.raw [<class 'str'>]
+ DEBUG dataset: 308892599 [<class 'str'>]
+ DEBUG directory: None [<class 'NoneType'>]
+ DEBUG clear_parents: True [<class 'bool'>]
+ DEBUG verbose: 2 [<class 'int'>]
+ DEBUG quiet: False [<class 'bool'>]
+ DEBUG Loading configuration.
+ DEBUG Site set to: local
+ INFO No directory, setting to: ./
+
+Searching for files for 308892599 kko.event.baseband.raw...
+
+ DEBUG Loading configuration.
+ DEBUG Server: https://frb.chimenet.ca/datatrail
+ DEBUG Configuration loaded successfully.
+ INFO Querying Datatrail for 308892599 kko.event.baseband.raw.
+ DEBUG URL: https://frb.chimenet.ca/datatrail/query/dataset/find
+Directory: ./data/kko/baseband/raw/2023/08/07/astro_308892599
+ - Found 2 files.
+ - Total size: 0.01 GB.
+
+⚠️ Delete files and empty parent directories? [y/n]: y
+[17:13:34] DEBUG clear_parents: True
+ DEBUG Checking if path ./data/kko/baseband/raw/2023/08/07/astro_308892599
+ exists.
+ INFO Path successfully removed.
+ DEBUG Clearing parent directories of data/kko/baseband/raw/2023/08/07.
+ DEBUG files: []
+ DEBUG data/kko/baseband/raw/2023/08/07: ✔ # (1)!
+ DEBUG files: []
+ DEBUG data/kko/baseband/raw/2023/08: ✔
+[17:13:35] DEBUG files: []
+ DEBUG data/kko/baseband/raw/2023: ✔
+ DEBUG files: []
+ DEBUG data/kko/baseband/raw: ✔
+ DEBUG files: []
+ DEBUG data/kko/baseband: ✔
+ DEBUG files: []
+ DEBUG data/kko: ✔
+ DEBUG files: []
+ DEBUG data: ✔
+ DEBUG files: [PosixPath('.DS_Store'), PosixPath('mkdocs.yml'),
+ PosixPath('LICENSE'), PosixPath('.pytest_cache'),
+ PosixPath('CHANGELOG.md'), PosixPath('.pre-commit-config.yaml'),
+ PosixPath('dtcli'), PosixPath('pyproject.toml'), PosixPath('tests'),
+ PosixPath('docs'), PosixPath('README.md'), PosixPath('.gitignore'),
+ PosixPath('.github'), PosixPath('tox.ini'),
+ PosixPath('poetry.lock'), PosixPath('.git')]
+ DEBUG .: ✗ # (2)!
+--clear-parents checks if each successive parent directory is empty before deleting it.This page provides documentation for the Datatrail CLI.
+Datatrail Command Line Interface.
+Usage:
+datatrail [OPTIONS] COMMAND [ARGS]...
+Options:
+| Name | +Type | +Description | +Default | +
|---|---|---|---|
--help |
+boolean | +Show this message and exit. | +False |
+
Clear a dataset.
+Usage:
+datatrail clear [OPTIONS] SCOPE DATASET
+Options:
+| Name | +Type | +Description | +Default | +
|---|---|---|---|
--directory, -d |
+directory | +Root directory to use. Default: None, will use the value set in the config. | +None | +
--clear-parents |
+boolean | +Clear all empty parent directories of dataset. | +False |
+
-v, --verbose |
+integer range (0 and above) |
+Verbosity: v=INFO, vv=DEBUG. | +0 |
+
-q, --quiet |
+boolean | +Set log level to ERROR. | +False |
+
--force, -f |
+boolean | +Will not prompt for confirmation. | +False |
+
--help |
+boolean | +Show this message and exit. | +False |
+
Datatrail CLI Configuration.
+For initialising and modifying the Datatrail CLI configuration file.
+Usage:
+datatrail config [OPTIONS] COMMAND [ARGS]...
+Options:
+| Name | +Type | +Description | +Default | +
|---|---|---|---|
--help |
+boolean | +Show this message and exit. | +False |
+
Get a configuration value.
+Usage:
+datatrail config get [OPTIONS] KEY
+Options:
+| Name | +Type | +Description | +Default | +
|---|---|---|---|
--help |
+boolean | +Show this message and exit. | +False |
+
Initialise configuration.
+This will create a configuration file in the home directory under .datatrail.
+Datatrail MUST be initialised before is can be used.
Usage:
+datatrail config init [OPTIONS]
+Options:
+| Name | +Type | +Description | +Default | +
|---|---|---|---|
--site, -s |
+text | +Site to initialise Datatrail CLI for. | +_required | +
--help |
+boolean | +Show this message and exit. | +False |
+
List all configuration values.
+Usage:
+datatrail config list [OPTIONS]
+Options:
+| Name | +Type | +Description | +Default | +
|---|---|---|---|
--help |
+boolean | +Show this message and exit. | +False |
+
Set a configuration value.
+Usage:
+datatrail config set [OPTIONS] KEY VALUE
+Options:
+| Name | +Type | +Description | +Default | +
|---|---|---|---|
--help |
+boolean | +Show this message and exit. | +False |
+
List scopes & datasets
+Usage:
+datatrail list [OPTIONS] [SCOPE] [DATASETS]
+Options:
+| Name | +Type | +Description | +Default | +
|---|---|---|---|
-v, --verbose |
+integer range (0 and above) |
+Verbosity: v=INFO, vv=DEBUG. | +0 |
+
-q, --quiet |
+boolean | +Only errors shown in logs. | +False |
+
--write |
+boolean | +Write the events to file. | +False |
+
--help |
+boolean | +Show this message and exit. | +False |
+
Details of a dataset.
+Usage:
+datatrail ps [OPTIONS] SCOPE DATASET
+Options:
+| Name | +Type | +Description | +Default | +
|---|---|---|---|
-s, --show-files |
+boolean | +Show file names. | +False |
+
-v, --verbose |
+integer range (0 and above) |
+Verbosity: v=INFO, vv=DEBUG. | +0 |
+
-q, --quiet |
+boolean | +Set log level to ERROR. | +False |
+
--help |
+boolean | +Show this message and exit. | +False |
+
Download a dataset.
+Usage:
+datatrail pull [OPTIONS] SCOPE DATASET
+Options:
+| Name | +Type | +Description | +Default | +
|---|---|---|---|
--directory, -d |
+directory | +Directory to pull data to. | +None | +
--cores, -c |
+integer range (between 1 and 2) |
+Number of parallel fetch processes to use. | +1 |
+
-v, --verbose |
+integer range (0 and above) |
+Verbosity: v=INFO, vv=DEBUG. | +0 |
+
-q, --quiet |
+boolean | +Set log level to ERROR. | +False |
+
--force, -f |
+boolean | +Do not prompt for confirmation. | +False |
+
--help |
+boolean | +Show this message and exit. | +False |
+
Show versions.
+Usage:
+datatrail version [OPTIONS]
+Options:
+| Name | +Type | +Description | +Default | +
|---|---|---|---|
--help |
+boolean | +Show this message and exit. | +False |
+
+The commands available to you are:
+list: This list either the 'scopes' available or all of the datasets
+ belonging to the given dataset.ps: This provides detailed information for the given 'scope' and 'dataset'
+ combination.pull: This allows you to download all files belonging to the 'scope' and
+'dataset' provided.clear: This removes all files belonging to the 'scope' and 'dataset', only
+available for the local and canfar sites.config: Edit the .datatrail/config.yaml configuration file.version: List the CLI and server version.Detailed information on all of the CLI commands can be found on the +Reference page.
+ + + + + + +$4pHI|1rfuz9bGL&Ium2Y3vfc|lBLRvC~
zwWnE)+cV{m0+MNF!H0ziVpyj<`S{;$?!UDrX|><0s^Bc5H>xmo$SEGan7H?R97;#Z
z#Z^_Xn|poCrIm+`0_zVvO^RP1DGkJo)#atgM{QOe{Y