Initial version (#1)

Signed-off-by: gabrik <[email protected]>

Co-authored-by: Julien Loudet <[email protected]>

.gitignore

@@ -0,0 +1,22 @@
+# Generated by Cargo
+# will have compiled files and executables
+**/target
+
+# Ignore all Cargo.lock but one at top-level, since it's committed in git.
+*/**/Cargo.lock
+
+# These are backup files generated by rustfmt
+**/*.rs.bk
+
+# CLion project directory
+.idea
+
+# Emacs temps
+*~
+
+# MacOS Related
+.DS_Store
+
+# Output files
+outfile.png
+output.dot

Cargo.toml

@@ -0,0 +1,32 @@
+#
+# Copyright (c) 2017, 2021 ADLINK Technology Inc.
+#
+# This program and the accompanying materials are made available under the
+# terms of the Eclipse Public License 2.0 which is available at
+# http://www.eclipse.org/legal/epl-2.0, or the Apache License, Version 2.0
+# which is available at https://www.apache.org/licenses/LICENSE-2.0.
+#
+# SPDX-License-Identifier: EPL-2.0 OR Apache-2.0
+#
+# Contributors:
+#   ADLINK zenoh team, <[email protected]>
+#
+[workspace]
+
+members = [
+  "zenoh-flow",
+  "zenoh-flow-derive",
+  "zenoh-flow-examples",
+]
+
+[profile.dev]
+debug=true
+opt-level = 0
+
+
+[profile.release]
+debug=true
+lto="fat"
+codegen-units=1
+opt-level=3
+panic="abort"

README.md

@@ -1,3 +1,184 @@
-# Eclipse zenoh flow
+# Eclipse Zenoh Flow
 
-zenoh-flow aims at providing a zenoh-based data-flow programming framework for computations that span from the cloud to the device.
+Zenoh Flow aims at providing a Zenoh-based dataflow programming framework for computations that span from the cloud to the device.
+
+:warning: **This software is still in alpha status and should _not_ be used in production. Breaking changes are likely to happen and the API is not stable.**
+
+-----------
+## Description
+
+Users can describe a dataflow "pipeline" that should run on one or multiple Zenoh Flow instances via a `yaml` file. This file constitutes the entry point of Zenoh Flow.
+
+A pipeline is composed of set of _sources_ — producing data, _operators_ — computing over the data, and _sinks_ — consuming the resulting data. These components are _dynamically_ loaded at runtime.
+
+The different instances leverage Zenoh’s publish-subscribe model to communicate in a transparent manner for the user, i.e. without any configuration or intervention as Zenoh is built-in.
+
+We provide several working examples that illustrate how to author components and the yaml file describing a dataflow pipeline.
+
+-----------
+## How to build it
+
+Install [Cargo and Rust](https://doc.rust-lang.org/cargo/getting-started/installation.html). Zenoh Flow can be successfully compiled with Rust stable (>= 1.5.1), so no special configuration is required — except for certain examples.
+
+To build Zenoh Flow, just type the following command after having followed the previous instructions:
+
+```bash
+$ cargo build --release
+```
+
+-----------
+## How to run
+
+If you launched the previous command, the Zenoh Flow runtime is located in `target/release/runtime`. This executable expects the following arguments:
+
+- the path of the dataflow graph to execute: `--graph-file zenoh-flow-examples/graphs/fizz_buzz_pipeline.yaml`,
+- a name for the runtime: `--runtime foo`.
+
+The graph describes the different components composing the dataflow. Although mandatory, the name of the runtime is used to "deploy" the graph on different "runtime instances" (see the related examples).
+
+-----------
+## Examples
+
+### FizzBuzz
+
+First, compile the relevant examples:
+
+```bash
+cargo build --example manual-source --example example-fizz --example example-buzz --example generic-sink
+```
+
+This will create, depending on your OS, the libraries that the pipeline will fetch.
+
+#### Single runtime
+
+To run all components on the same Zenoh Flow runtime:
+
+```bash
+./target/release/runtime --graph-file zenoh-flow-examples/graphs/fizz_buzz_pipeline.yaml --runtime foo
+```
+
+_Note: in that particular case the `--runtime foo` is discarded._
+
+#### Multiple runtimes
+
+In a first machine, run:
+
+```bash
+./target/release/runtime --graph-file zenoh-flow-examples/graphs/fizz-buzz-multiple-runtimes.yaml --runtime foo
+```
+
+In a second machine, run:
+
+```bash
+./target/release/runtime --graph-file zenoh-flow-examples/graphs/fizz-buzz-multiple-runtimes.yaml --runtime bar
+```
+
+:warning: If you change the name of the runtime in the yaml file, the name(s) passed as argument of the previous commands must be changed accordingly.
+
+:warning: Without configuration, the different machines need to be on the _same local network_ for this example to work. See how to add a [Zenoh router](https://zenoh.io/docs/getting-started/key-concepts/#zenoh-router) if you want to connect them through the internet.
+
+---
+
+### OpenCV FaceDetection - Haarcascades
+
+:warning: This example works only on Linux and it require OpenCV to be installed, please follow the instruction on the [OpenCV documentation](https://docs.opencv.org/4.5.2/d7/d9f/tutorial_linux_install.html) to install it.
+
+:warning: You need a machine equipped of a webcam in order to run this example.
+
+First, compile the relevant examples:
+
+```bash
+cargo build --example camera-source --example face-detection --example video-sink
+```
+
+This will create, depending on your OS, the libraries that the pipeline will fetch.
+
+#### Single runtime
+
+To run all components on the same Zenoh Flow runtime:
+
+```bash
+./target/release/runtime --graph-file zenoh-flow-examples/graphs/face_detection.yaml --runtime foo
+```
+
+_Note: in that particular case the `--runtime foo` is discarded._
+
+#### Multiple runtimes
+
+In a first machine, run:
+
+```bash
+./target/release/runtime --graph-file zenoh-flow-examples/graphs/face-detection-multi-runtime.yaml --runtime gigot
+```
+
+In a second machine, run:
+
+```bash
+./target/release/runtime --graph-file zenoh-flow-examples/graphs/face-detection-multi-runtime.yaml --runtime nuc
+```
+
+In a third machine, run:
+
+```bash
+./target/release/runtime --graph-file zenoh-flow-examples/graphs/face-detection-multi-runtime.yaml --runtime leia
+```
+
+:warning: If you change the name of the runtime in the yaml file, the name(s) passed as argument of the previous commands must be changed accordingly.
+
+:warning: Without configuration, the different machines need to be on the _same local network_ for this example to work. See how to add a [Zenoh router](https://zenoh.io/docs/getting-started/key-concepts/#zenoh-router) if you want to connect them through the internet.
+
+---
+
+### OpenCV Object Detection - Deep Neural Network - CUDA powered
+
+:warning: This example works only on Linux and it require OpenCV with CUDA enabled to be installed, please follow the instruction on [this gits](https://gist.github.com/raulqf/f42c718a658cddc16f9df07ecc627be7) to install it.
+
+:warning: This example works only on Linux and it require a **CUDA** capable **NVIDIA GPU**, as well as NVIDIA CUDA and CuDNN to be installed, please follow [CUDA instructions](https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html) and [CuDNN instructions](https://docs.nvidia.com/deeplearning/cudnn/install-guide/index.html).
+
+:warning: You need a machine equipped of a webcam in order to run this example.
+
+:warning: You need to download a YOLOv3 configuration, weights and classes, you can use the ones from [this GitHub repository](https://github.com/sthanhng/yoloface).
+
+First, compile the relevant examples:
+
+```bash
+cargo build --example camera-source --example object-detection-dnn --example video-sink
+```
+
+This will create, depending on your OS, the libraries that the pipeline will fetch.
+
+Then please update the files `zenoh-flow-examples/graphs/dnn-object-detection.yaml` and `zenoh-flow-examples/graphs/dnn-object-detection-multi-runtime.yaml` by changing the `neural-network`, `network-weights`, and `network-classes` to match the absolute path of your *Neural Network* configuration
+
+#### Single runtime
+
+To run all components on the same Zenoh Flow runtime:
+
+```bash
+./target/release/runtime --graph-file zenoh-flow-examples/graphs/dnn-object-detection.yaml --runtime foo
+```
+
+_Note: in that particular case the `--runtime foo` is discarded._
+
+#### Multiple runtimes
+
+In a first machine, run:
+
+```bash
+./target/release/runtime --graph-file zenoh-flow-examples/graphs/dnn-object-detection-multi-runtime.yaml --runtime foo
+```
+
+In a second machine, run:
+
+```bash
+./target/release/runtime --graph-file zenoh-flow-examples/graphs/dnn-object-detection-multi-runtime.yaml --runtime cuda
+```
+
+In a third machine, run:
+
+```bash
+./target/release/runtime --graph-file zenoh-flow-examples/graphs/dnn-object-detection-multi-runtime.yaml --runtime bar
+```
+
+:warning: If you change the name of the runtime in the yaml file, the name(s) passed as argument of the previous commands must be changed accordingly.
+
+:warning: Without configuration, the different machines need to be on the _same local network_ for this example to work. See how to add a [Zenoh router](https://zenoh.io/docs/getting-started/key-concepts/#zenoh-router) if you want to connect them through the internet.

zenoh-flow-derive/Cargo.toml

@@ -0,0 +1,39 @@
+[package]
+name = "zenoh-flow-derive"
+version = "0.0.1"
+repository = "https://github.com/eclipse-zenoh/zenoh-flow"
+homepage = "http://zenoh.io"
+authors = ["kydos <[email protected]>",
+           "gabrik <[email protected]>",
+           "Julien Loudet <[email protected]>",]
+edition = "2018"
+license = " EPL-2.0 OR Apache-2.0"
+categories = ["network-programming"]
+description = "Zenoh-Flow: zenoh-based data-flow programming framework for computations that span from the cloud to the device."
+readme = "README.md"
+
+# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
+# To build with debug on macros: RUSTFLAGS="-Z macro-backtrace"
+
+[dependencies]
+async-std = { version = "=1.9.0", features = ["attributes"] }
+futures = "0.3.5"
+syn-serde = { version = "0.2", features = ["json"] }
+syn = { version = "1.0.11", features = ["full"] }
+quote = "1.0.2"
+proc-macro2 = "1.0.6"
+serde_derive = "1.0.55"
+serde = { version = "1.0.55", features = ["derive"] }
+darling = "0.13.0"
+Inflector = "0.11.4"
+proc-macro-error = "1.0.4"
+
+
+[dev-dependencies]
+env_logger = "0.9"
+
+
+[lib]
+proc-macro = true
+
+

zenoh-flow-derive/src/lib.rs

@@ -0,0 +1,57 @@
+//
+// Copyright (c) 2017, 2021 ADLINK Technology Inc.
+//
+// This program and the accompanying materials are made available under the
+// terms of the Eclipse Public License 2.0 which is available at
+// http://www.eclipse.org/legal/epl-2.0, or the Apache License, Version 2.0
+// which is available at https://www.apache.org/licenses/LICENSE-2.0.
+//
+// SPDX-License-Identifier: EPL-2.0 OR Apache-2.0
+//
+// Contributors:
+//   ADLINK zenoh team, <[email protected]>
+//
+
+use proc_macro::TokenStream;
+use quote::quote;
+use syn::{parse_macro_input, DeriveInput};
+
+#[proc_macro_derive(ZFData)]
+pub fn zf_data_derive(input: TokenStream) -> TokenStream {
+    let ast = parse_macro_input!(input as DeriveInput);
+    let ident = &ast.ident;
+    let gen = quote! {
+
+        #[typetag::serde]
+        impl zenoh_flow::DataTrait for #ident {
+            fn as_any(&self) -> &dyn std::any::Any {
+                self
+            }
+
+            fn as_mut_any(&mut self) -> &mut dyn std::any::Any {
+                self
+            }
+        }
+    };
+    gen.into()
+}
+
+#[proc_macro_derive(ZFState)]
+pub fn zf_state_derive(input: TokenStream) -> TokenStream {
+    let ast = parse_macro_input!(input as DeriveInput);
+    let ident = &ast.ident;
+    let gen = quote! {
+
+        #[typetag::serde]
+        impl zenoh_flow::StateTrait for #ident {
+            fn as_any(&self) -> &dyn std::any::Any {
+                self
+            }
+
+            fn as_mut_any(&mut self) -> &mut dyn std::any::Any {
+                self
+            }
+        }
+    };
+    gen.into()
+}