Release v0.91

This release brings integration of Mizar pod networking solution with Arktos project (A fork of Kubernetes project that adds capabilities for true multi-tenancy orchestration of pods and virtual machines). There is no existing pod networking solution that supports mult-tenancy networking. This release of Mizar project fills the gap.

Arktos scale out architecture allows for deployment of multiple cluster-master instances (called tenant partitions) where each master instance has its own set of etcd, apiserver, scheduler, and controllers (except node controller). Each tenant partition can support multiple tenant users. And each tenant partition master has a global view worker nodes across multiple resource partitions where each resource partition is its own cluster with a master that runs etcd, apiserver, and node controller designed to focus mainly on worker node resource management in that cluster. In this way, Arktos scale out deployments allow for scaling the number of nodes available for pod deployment beyond known limits in Kubernetes.

The following key features were implemented to facilitate the integration:

1. Support Mizar multi-tenancy networking deployment for the following Arktos deployment modes:
   • Scale out architecture (multiple tenant partitions and multiple resource partitions with multiple tenant-users).
   • Scale up architecture (single cluster deployment with multiple tenant-users)
   • Local cluster deployment with multiple tenant-users
   • Mizar networking support for both pods and virtual machines with with full network isolation for multiple tenant users.
2. Implement gRPC server mode in Mizar node agent (transitd daemonset) to enable multiple Mizar operators from two or more tenant partitions program eBPF maps in any given worker node that is shared across multiple tenant clusters.
3. Implement gRPC server mode of operation in Mizar operator that enables Mizar network controller in each tenant partition invoke Mizar APIs for multiple tenant users.
4. Support the creation of multiple VPC objects where each VPC represents a single tenant user which has full network isolation from other tenant users.
5. Co-locate Mizar operator with tenant apiserver for simplicity, performance, and efficiency.

Release v0.9

This release brings the following:

1. Allow users to define the network priority and traffic classification for their Kubernetes and Arktos workloads, and Mizar manages network QoS for Pod traffic.
2. Ability to create VPCs and subnets within Kubernetes cluster to provide network isolation among application containers, an ability required for multi tenancy support in Kubernetes.
3. Mizar CNI implementation in golang and changes that enable Mizar to act as container networking solution for Kubernetes as well as Arktos scale-out architecture (in next release)
4. Various stability and bug fixes:
   • Fix Kubernetes service connectivity failure. (Issue #506)
   • Fix instability in scaled-endpoint UDP checksum computation. (Issue #541)
   • Fix UDP checksum initialization issue in encapsulating IP packet.
   • Fix XDP load issue when performance / non-debug mode is enabled. (Issue #485)
   • Fix failures in Mizar bootstrap script. (Issue #538)
   • Fix containerd restart requirement issue when deploying Mizar with Arktos.
   • Fix python versioning to allow Mizar deployment on Ubuntu 20.04 systems. (Issue #509)

Release v0.8

This release brings the following:

1. Mizar deployment for Kubernetes via single yaml.
2. Major stability improvements/upgrades and bug fixes.
3. Label-based Network Policy for K8s for efficient Network Policy enforcement.
4. Bandwidth & QoS for Pod network traffic using Earliest Departure Time (EDT) algorithm.

• Mizar can now be deployed in a K8s cluster via deploy.mizar.yaml

• Mizar stabilization fixes include:

  • Add CLI support for XDP program offload to capable NICs.
  • Bootstrap install latest kind and kubectl.
  • Fix daemon gRPC issue.
  • Fix config map creation issue.
  • Fix for non-main interface droplets.
  • Fix CNI "node not ready" error.
  • Fix “no interface found” CNI error.
  • Fix endpoint droplet mismatch.
  • Extend delay for retries.
  • Fix for bootstrap of nodes.
  • Initialize RPC class with correct interface.
  • Add additional XDP tools.
  • Fix multiple clusters bring-up issues with NoneType Error.
  • Fix for load transit CLI unit tests.
  • Add CI via GitHub Actions and remove Travis CI.
  • Add starter code for additional validation of operator errors in E2E tests.

• Label-based Network Policy feature:

  • Add new BPF map to store packet metadata that holds Pod & Namespace labels for outgoing packets.
  • Translation of Pod & Namespace string labels into integer value labels.
  • CLI mechanism to configure integer label values for outgoing packets.
  • Definition of Pod & Namespace label GENEVE option types.
  • Update transit agent XDP packet encapsulation code to carry integer labels in GENEVE options.
  • CLI mechanism to configure Pod & Namespace Network Policy labels list for policy enforcement at ingress.
  • Update ingress packet processing transit XDP code to read labels in GENEVE options and enforce Network Policy via labels.

• Pod network traffic bandwidth rate-limiting and QoS feature:

  • New Linux Traffic Control (TC) eBPF program that implements EDT (Earliest Departure Time) rate-limiting.
  • New BPF map entry to hold config-specified egress bandwidth value used by TC eBPF program.
  • Create Mizar linux bridge and use it for routing low-priority egress traffic from Pod veth pairs to TC eBFP hook.
  • CLI changes to read kubernetes.io/egress-bandwidth annotation and plumb it to BPF map.
  • Transit & agent XDP program modifications to classify Pod network traffic and route via TC eBPF program.

• Bug fixes:

  • EDT eBPF Bandwidth QoS feature breaks SSH connectivity (issue #501)
  • Remove existing host_ep in baremetal & VM deployments (issue #475)
  • Fix pod creation failure in droplets without main interface (issue #474)
  • Deployment failure when operator comes up before daemon (issue #473)
  • Kind deployment issue with mizar daemon stuck due to image pull failure (issue #439)
  • Pod stuck in ContainerCreating state (issue #436)

Release v0.7

This release adds the support for Kubernetes network policy and fixes some bugs.

Feature enhancements to support Kubernetes network policy:

• Added operators to list/watch network policy objects, pod objects and namespace objects.
• Updated operator logics to calculate eBPF map entries for the CRUD events of above objects.
• Updated Mizar daemon to receive eBPF map entry changes and update maps.
• Added eBPF maps to support O(1) policy enforcement, including maps for policies, CIDRs, excepted CIDRs, etc.
• Updated Mizar transit XDP to enforce ingress rules.
• Updated Mizar transit agent to enforce egress rules.
• Added connection tracking module on XDP layer.

Bug fixes:

• Support configurable network interface names (issue #251)
• Fix Mizar CNI support for VERSION cmd (issue #403)
• Fix Mizar CNI bug in handling network namespace path (issue #404)

Release v0.6

This release is mainly for the integration with Arktos.

Feature enhancements and bug fixes include:

• Implemented Mizar gRPC server interface that exposes the following Mizar built-in workflows.
  • Pod Create, Update, Resume, and Delete.
  • Service Create, Update, Resume, and Delete.
  • Service Endpoint, Create, Update.
  • Node Create, Update, Resume, and Delete.
• Fixed an issue where rx and tx checksumming was not disabled on interfaces where XDP programs were loaded.
• Fixed and issue where inner tcp packet checksum was not computed after port translation.
• Added a Mizar deploy script for vanilla Kubernetes and Arktos.

Release v0.5

Summary

This release introduces new enhancements and bug fixes to Mizar:

• Enhancements on core features include Scaled Endpoint, Phantom Gateway and Host Endpoint.
• Integration with Kubernetes and Arktos.
• Finished the design of Zeta, a distributed elastic middlebox platform.
• Build and test infra improvements and code refactoring.

This release does not involve breaking changes to existing integrations.

Data Plane

• Scaled Endpoint Port Translation (fb35376, a26cfc3, 43289e3, 1808a30)

• Phantom Gateway Fast Path Fix d019484

Management plane

• Grpc based cni and droplet service (f4cb03d, a767723, 7c4343a)

• Scaled Endpoint Port Translation d391fb4

• Host Endpoint 284050d

• Phantom Gateway b930de6

• Fix TLS Handshake bug 711c99c

• Maglev Hashing for upcoming DFT and scaled endpoint design f21a7b5

• Improve Kopf and Luigi error handling with retries 7a11c35

Other Improvements

• Fix travis with new grpc components 3b0eebb

• Fix e2e-test post port translation 09ea1de

• Improved test environment setup with docker file refactor b5bf8d8

• Test environment fix for Kind v0.8 f3d931e

• Code clean-up and restructure ab3e764

v0.3.1

Summary

This is a minor release to include quick fixes and a basic end to end tests:

Tests

• Basic end-to-end tests #147, #142, #134

Fixes

• Fix scaled endpoint bug when the bouncer is on the same client host #146, #150
• Fix for intermittent Luigi scheduler disconnect #143
• Fix scaled endpoint delete workflow #138

Release v0.3

Summary

This release introduces a new management plane for Mizar, designed and
developed from the ground up. The design relies on extensibility
features in Kubernetes including Custom Resource Definitions and
Operators. Mizar Management Plane has several objectives:

1. Replace Kubeproxy with the scaled endpoint
2. Improve Mizar usability and deployability
3. Improve Mizar control-plane and data-plane Interfacing and workflow
4. Facilitate end-to-end testing, validation, and performance
   benchmarks
5. Extensibility to support other data-plane technologies, including ebpf, OVS, and host-networking.

Breaking Changes

This release does not involve breaking changes to existing
integrations.

Data Plane

• Update kernel requirement to 5.6-rc2
• Minor Unit test fix bdfc1f9
• Direct path for cross network traffic b5283a1

Mizar Management plane

• Mizar objects with Custom Resource Defintions

• VPC, Net, Endpoint, Bouncer, Divider and Operators, Main
  Commits:
  644d0fa,
  21a43c3,
  8934e26

• Built-in Operators for K8s Pods and Services, Main
  Commits: 42abe77

• Basic Bouncer and Divider Placement, Main
  Commits:
  f12eb3b

• Simple manual scaling workflows for bouncers and dividers, Main
  Commits:
  5b7c325,
  354dda1

• Replaces KubeProxy for Loadbalancer type with scaled-endpoint, Main
  Commits:
  42abe77

• Generic CNI RPC Interface for transit daemon, Main
  Commits: ffa7e6a

• Endpoint host management with Netlink, Main
  Commits: 5b46462

Documentation

• New documentation readthedocs page
  • Detailed data-plane design
  • Detailed management-plane design
  • Improvement in getting started guide to using the new management plane

Initial PoC release

Summary

This release primarily introduces the beta implementation of direct-path (fast-path),
and scaled endpoints. The releases also provide hooks within the data-plane to
support extensibility at various stages of the packet processing pipeline.

Breaking Changes

This release does not involve breaking changes to existing
integrations.

Data Plane

1. Fast path (beta)
2. Scaled endpoint (beta)
3. Performance improvements
4. Extensible packet processing pipeline support
5. A basic implementation of the data plane and a mini controller;
6. Create and deletion of VPC/subnet/ports objects.
7. Initial automated test and deployment scripts.

Documentation

1. Components name changes proposal

Continuous Integration

• Codedeploy support