Skip to content
This repository has been archived by the owner on Aug 29, 2023. It is now read-only.

Workspace Concept

Jump to bottom
Krzysztof (Chris) Bernat edited this page Feb 23, 2017 · 2 revisions

Purpose and Scope

A Workspace is a collection of named operation executions and related data(set) files. A workspace preserves the order of operations a user has executed, where the first operations are usually loading of datasets from given data sources and the last ones are storing data to files using a given format or mime-type.

A workspace can also be seen as a namespace for variables whose values are the results of operation executions. However, actual data processing is usually deferred until a workspace variable's data is really required by some operation, e.g. visualising or writing data.

Users can work with workspaces using both the GUI and the CLI. For example, all the actions that a user subsequently performs will be recorded in the workspace. This ranges from loading datasets, to applying data conversion, regridding, etc. to storing final netCDF files. Users can create new workspaces and delete them. They can open, save, rename, and close them.

Because a workspace solely comprises operation calls it could be converted into a Workflow JSON file or a Python script. The other way round, workspaces could be imported from Workflow JSON files. (TBD: could a workflow be a workspace or at least be a part of a workspace representation?)

Implementation

In the following some random notes regarding the implementation of workspaces in Cate are given.

Involved Components

  • Cate Workspace API (public Python API, for manipulating workspaces directly from Python)
  • Cate WebAPI (= webservice) which uses the Workspace API and holds references to the workspace variables
  • Cate CLI which uses the Cate WebAPI
  • Cate GUI which uses the Cate WebAPI

Cate WebAPI (Webservice)

Although the WebAPI webservice is intended to maintain state (workspace variables, dataset cache files), it may be implemented using a RESTful style. To make this possible, workspaces, workspace variables, datasets, data sources would be modeled as service resources and each REST call would contain all identifiers of all participating resources in that call. We may not need to address full REST-enabled scalability now, which would require location-independent resources shared over multiple WS instances. (See demo server in https://github.com/CCI-Tools/demo/blob/master/backend/ccitbxws/main.py)

The Cate WebAPI code should be in package cate.webapi and included in the cate-core repository to minimise deployment problems and configuration control issues.

Must be possible for clients to subscribe for state changes or at least to retrieve a delta change given the timestamp of a preceding state poll.

Cate GUI

Startup Phase

  • Either connect to an existing local, "headless" Cate WebAPI or start a new local one (with a "head" = GUI). TBD: possibility to connect to a remote Cate WebAPI.
  • Start with an empty default workspace or the last one opened before (that will be a user preference setting). Subscribe to Cate WebAPI state changes (e.g. indirectly by starting state polls)

Up-and-Running Phase

  • Create or open workspace
  • Manipulate opened workspace
  • React to opened workspace state changes
  • Close, save workspace

The WebAPI should set a lock on a webspace when the user opens it and remove the lock when user closes it. This way, we avoid that a second WebAPI process (or any other client using the Webspaces API) can unintentionally modify workspace resources whose new values wouldn't be reflected in the WebAPI that opened the workspace.

Shutdown Phase

  • Unsubscribe from Cate WebAPI state changes (e.g. indirectly by stopping state polls)
  • Cate WebAPI will be shutdown only if it has exclusively been created for the GUI.

Cate CLI

Some Cate CLI commands require or imply state and therefore rely on a running Cate WebAPI. Any command that requires a running Cate WebAPI will either reuse an existing one (headless or not) or start a new one before it executes. It will shutdown the Cate WebAPI if has created it. As this is very inefficient, there could be commands that actually start and stop a Cate WebAPI instance so that it can be reused for a session. Even a GUI can launched on an Cate WebAPI that has been created from the CLI:

  • cate webapi start
  • cate webapi stop
  • cate webapi list

These commands may have options that allow to start/stop the WS on a given port, e.g. to create multiple WebAPI service instances. Commands that require the WebAPI may then also have options to use a specific WebAPI service instance at a given port.

See CLI command overview.

Cate Workspace API (Python)

WorkspaceManager

  • new_workspace(name): Workspace
  • open_workspace(name): Workspace
  • get_workspace_names(): List[string]
  • get_workspace(name): Workspace

Workspace

  • name(): string
  • delete()
  • rename(new_name)
  • save()
  • get_var(name): WorkspaceVar
  • set_var(name, op, *args): WorkspaceVar
  • get_var_names(): List[string]
Clone this wiki locally