diff --git a/docs/deployment.md b/docs/deployment.md index 1f3e2d6a..53f682f9 100644 --- a/docs/deployment.md +++ b/docs/deployment.md @@ -99,3 +99,11 @@ ls certs/gateway/letsencrypt/live/ ``` If all went well, the `old-bento.example.com` domain should be redirected to `bento.example.com` in a browser. + +## Discovery configuration + +Bento can serve censored data publicly if configured to do so. This allows anonymous users to take a glimpse into the +data hosted by a Bento node. + +When deploying a Bento instance, make sure that the discovery settings are configured properly at the necessary levels. +Consult the [public discovery](./public_discovery.md) documentation for more details. diff --git a/docs/img/discovery_proj_creation.png b/docs/img/discovery_proj_creation.png new file mode 100644 index 00000000..f2508f9f Binary files /dev/null and b/docs/img/discovery_proj_creation.png differ diff --git a/docs/img/discovery_proj_edit.png b/docs/img/discovery_proj_edit.png new file mode 100644 index 00000000..bebc9be5 Binary files /dev/null and b/docs/img/discovery_proj_edit.png differ diff --git a/docs/public_discovery.md b/docs/public_discovery.md new file mode 100644 index 00000000..562261e1 --- /dev/null +++ b/docs/public_discovery.md @@ -0,0 +1,80 @@ +# Public data discovery configuration + +New in Bento v17. + +Previously, the public data configuration given to Katsu (`lib/katsu/config.json`) was applied on all the metadata +contained in the service. This configuration declares which fields can be queried publicly for discovery purposes, +which charts to display and which censorship rules to apply on the results. + +Katsu can hold multiple projects/datasets that may use different fields, require specific charts or custom +`extra_properties` schemas at the project level. +Therefore, there is a need to tailor the discovery configuration at different levels to properly represent the +particularities in the information of a project or dataset. + +Bento v17 gives the ability to specify a scoped Discovery configuration at the following levels: +- Dataset + - Optional at dataset creation + - For scoped queries on public endpoints targeting a project and dataset: + - Katsu will use the dataset's discovery configuration, if one exists. + - If no configuration is found, falls back to the parent project's discovery. +- Project + - Optional at project creation + - For scoped queries on public endpoints targeting a project only: + - Katsu will use the project's discovery configuration, if one exists. + - If no configuration is found, falls back to the node's config. +- Node + - Optional during Katsu deployment + - Uses the legacy `lib/katsu/config.json` file mount + - For non-scoped queries on public endpoints: + - Katsu will use the node's discovery, if one exists. + - If no node configuration is found, Katsu will respond with a 404 status. + - For scoped queries on public endpoints: + - Katsu will fallback on the node's discovery if the project and/or dataset in the scope don't have one + - If no node configuration is found, Katsu will respond with a 404 status. + + +## Creating a discovery configuration file. + +Follow these steps in order to add public discovery for a given scope. + +1. Decide at which level the discovery configuration will be applied. +2. Create a copy of `etc/katsu.config.example.json`, use it as a base template +3. Modify the `fields` section so that it includes the fields of interest. +4. Modify the `search` section, include the fields from the previous section to make them searchable +5. Modify the `overview` section in order to specify which fields should be rendered as charts and how. +6. Set the desired censorship rules in the `rules` section. + +The resulting JSON file needs to respect the JSON-schema defined in Katsu [here](https://github.com/bento-platform/katsu/blob/develop/chord_metadata_service/discovery/schemas.py). + +## Using a discovery configuration file. + +With the valid discovery configuration file in hand, you can now add it to Katsu. + +### Apply the discovery configuration at the node level: + +This operation must be done by a Bento node administrator. +```bash +# Move the file to Katsu's config volume +cp ./lib/katsu/config.json + +# restart Katsu to load the config. +./bentoctl.bash restart katsu +``` + +### Apply the discovery configuration at the project or dataset level + +These operations must be performed in `bento_web` by an authenticated user +with the `edit:dataset`, `create:dataset`, `edit:project`, `create:project` permissions. + +Before creating/editing a project/dataset to specify a discovery configuration, +make sure that the JSON file is in Bento's dropbox. + +At project/dataset creation, leaving the discovery field empty is permitted. +Specify a discovery config by selecting the desired file from the drop-down options. + +![Specifying a discovery at project creation.](./img/discovery_proj_creation.png) + +During project/dataset editing, three radio buttons are shown, allowing the user to pick the existing value, +a new value from file, or none. The 'none' option is equivalent to leaving the field empty at creation. + +![Modifying the discovery when editing a project](./img/discovery_proj_edit.png) diff --git a/etc/bento.env b/etc/bento.env index d0c69824..112ddfb8 100644 --- a/etc/bento.env +++ b/etc/bento.env @@ -97,7 +97,7 @@ BENTO_AUTHZ_DB_MEM_LIM=1G # Web BENTO_WEB_CUSTOM_HEADER= BENTOV2_WEB_IMAGE=ghcr.io/bento-platform/bento_web -BENTOV2_WEB_VERSION=5.0.1 +BENTOV2_WEB_VERSION=edge BENTOV2_WEB_VERSION_DEV=${BENTOV2_WEB_VERSION}-dev BENTOV2_WEB_CONTAINER_NAME=${BENTOV2_PREFIX}-web BENTO_WEB_NETWORK=${BENTOV2_PREFIX}-web-net @@ -371,7 +371,7 @@ BENTOV2_GOHAN_PRIVATE_AUTHZ_URL=http://${BENTOV2_GOHAN_AUTHZ_OPA_CONTAINER_NAME} # Bento-Public BENTO_PUBLIC_IMAGE=ghcr.io/bento-platform/bento_public -BENTO_PUBLIC_VERSION=0.19.1 +BENTO_PUBLIC_VERSION=edge BENTO_PUBLIC_VERSION_DEV=${BENTO_PUBLIC_VERSION}-dev BENTO_PUBLIC_CONTAINER_NAME=${BENTOV2_PREFIX}-public BENTO_PUBLIC_NETWORK=${BENTOV2_PREFIX}-public-net diff --git a/lib/katsu/docker-compose.katsu.yaml b/lib/katsu/docker-compose.katsu.yaml index e8da7e3f..181c965a 100644 --- a/lib/katsu/docker-compose.katsu.yaml +++ b/lib/katsu/docker-compose.katsu.yaml @@ -25,6 +25,7 @@ services: - DRS_URL=http://${BENTOV2_DRS_CONTAINER_NAME}:${BENTOV2_DRS_INTERNAL_PORT} - SERVICE_TEMP=/app/tmp - SERVICE_SECRET_KEY=${BENTOV2_KATSU_APP_SECRET} + - SERVICE_URL_BASE_PATH=${BENTOV2_PUBLIC_URL}/api/metadata - DJANGO_SETTINGS_MODULE=chord_metadata_service.metadata.settings - BENTOV2_PORTAL_DOMAIN # Allow access by container name or localhost for healthchecks: