[5pt;8pt] graph-prototype: Implement Non-locking Runtime Graph Modifications

[ivan-cukic]

In the current design of the graph-prototype, modifications to the graph structure require a stop-modify-restart sequence for consistency, ensuring data isn't processed during these changes. This tightly couples the graph (directed graph with all processing nodes, functions, settings, and connections) and its operation to the view. The issue is particularly problematic when attempting to adjust the graph's topology or perform non-UI-triggered settings updates.

The ideal system would allow for asynchronous notifications of changes within the graph to any associated views. Correspondingly, views should be able to request modifications to the graph asynchronously rather than executing them directly. This approach would facilitate non-locking runtime changes to the graph without necessitating a complete halt of the system.

While comprehensive modifications will require a pause or complete stop to ensure the system's integrity, we aim to allow for on-the-fly changes, especially in simple sub-graphs with one input-output relationship. More complex graph topology changes would remain an exception that still likely always require a full stop and restart of the processing engine.

We should investigate a message-passing system for this feature, where the graph is owned and managed by a scheduler. The scheduler would receive and execute modification requests when most appropriate, minimizing the need for mutexes in the processing hot path and delegating some of the stop-modify-resume logic to itself.

This feature could potentially become very complex, and we should initially aim for a MVP solution that introduces non-locking runtime changes to the system.

To note: this would be a new feature that even the present GR 3.X and 4.0 does not support yet.

[ivan-cukic]

Notes from meeting 31.5.

• Introduce separate compile-time options for building without plugin loader and plugin registry
• Add API to Graph to instantiate blocks from block name, type parameter name and a property map which uses the plugin loader / block registry
• Add message-based modifications of the graph:
  • adding blocks, removing blocks, replacing blocks
  • adding connections/edges, removing connections/edges
• Add message-based graph queries - getting blocks, edges
• All changes should use the messaging system to notify the listeners of graph topology changes

[ivan-cukic]

First part #357