Skip to content

Releases: openebs/mayastor

v2.7.1

20 Sep 14:44
Compare
Choose a tag to compare

Release v2.7.1

Release Date: 20th September, 2024

Summary

OpenEBS Replicated PV Mayastor version 2.7.1 provides some important stability bugfixes for the data-plane.
It's recommended that you upgrade to 2.7.1 from any previous version.

Fixes

Full Change Log:

Mayastor

Mayastor Control Plane

Mayastor Extensions

v2.7.0

05 Jul 15:43
Compare
Choose a tag to compare

Release v2.7.0

Release Date: 5th July, 2024

Summary

OpenEBS Replicated PV Mayastor version 2.7 provides placement of replica volumes across different nodes/pools. Version 2.7 has enhanced snapshot capabilities to ensure file-system consistency across multiple replicas before taking snapshots. This release provides the capability to restore from snapshot across multiple and supports volume expansion even in the presence of snapshots. In addition, this release has fixes related to snapshots, upgrades and supportability.

Features

Placement of Replica Volumes Across Different Nodes/Pools

Mayastor now uses topology parameters defined in the storage class to determine the placement of volume replicas. This allows replicas can be controlled via labels from the storage class.

Snapshot Across Multiple Replicas

Mayastor has enhanced its snapshot capabilities to ensure file-system consistency across multiple replicas before taking snapshots. This ensures that snapshots are consistent and reliable across multiple replicas

Restore Across Multiple Replicas

The capability to restore from snapshots across multiple replicas has been introduced in recent releases, enhancing data recovery options​.

Expansion of volumes with snapshots

The latest release, Mayastor v2.7, includes support for volume expansion even when snapshots are present

Snapshot Fixes

feat(snapshot): plugin changes for snapshot operation by hrudaya21 · Pull Request #500 · openebs/mayastor-extensions

fix(snapshot): scale volume with snapshot can go for replica rebuild by hrudaya21 · Pull Request #826 · openebs/mayastor-control-plane

Upgrade Fixes

fix(upgrade-job): fix deserialize failures with helm v3.13+ installations by niladrih · Pull Request #512 · openebs/mayastor-extensions

fix: don't add replica to nexus which can't rebuild properly · openebs/mayastor-control-plane@53500c0

Enhancements:

feat: expose io-engine version via rest · openebs/mayastor-control-plane@9dfb153

feat(rebuild): prefer a local replica as a rebuild source · openebs/mayastor@c3816b9

feat(config): extends config yaml with core config and delay mode · openebs/mayastor@6d6fc65

fix: crash when there are no bdevs · openebs/mayastor@5575858

fix(ha/node): don't replace path if nqn is not found · openebs/mayastor-control-plane@07ca344

fix: crash when there are no bdevs · openebs/mayastor@5575858

feat(lvs): special error code for out-of-metadata condition · openebs/mayastor@349a07c

fix: update blob metadata during properties update · openebs/mayastor@7ad7a33

Solutions:

  1. Mayastor support in GKE with local SSD’s
  2. Mayastor support in Talos
  3. Provisioning RWX PVC’s
  4. Velero DR for LocalPV ZFS

Testing

Mayastor is subject to extensive unit, component and system-level testing throughout the development and release cycle. Resources for system-level (E2E) testing are currently provided by DataCore Software.

At this time, personnel and hardware resource limitations constrain testing by the maintainers to linux builds on x86. This reflects the primary use-case which the maintainers are currently targeting with the OpenEBS Mayastor project. Therefore, the use of Mayastor with other operating systems and/or architectures, if even possible, should be considered serendipitous and wholly experimental.

This release has been subject to End-to-End testing under Ubuntu 20.04.5_LTS (kernel: ubuntu-5.15.0-50-generic)

  • Tested k8s versions
    • 1.23.7
    • 1.24.14
    • 1.25.10

Known Behavioural Limitations

As with the previous versions, the Mayastor IO engine makes full utilisation of the allocated CPU cores regardless of I/O load. This is the poller operating at full speed, waiting for I/O.

As with the previous versions, a Mayastor DiskPool is limited to a single block device and cannot span across more than one block device.

Known Issues

Mayastor does not support the capacity expansion of DiskPools as of v2.6.0
Under heavy IO and constant scaling up-down of volume replicas, the io-engine pod has been observed to restart occasionally.

v2.6.1

19 Apr 15:13
Compare
Choose a tag to compare

Release v2.6.1

Release Date: 19th April, 2024

Summary

OpenEBS Replicated PV Mayastor version 2.6 provides volume expansion support for a Mayastor volume with automatic resizing of file-system by the CSI-resizer. Mayastor v2.6 enhances snapshot capabilities by ensuring file-system consistency before taking a snapshot. Mayastor v2.6 also enhances monitoring by exporting performance metrics like IOPs, throughput, and latency for Mayastor pools, volumes, and replicas. In this release, the event generation capabilities have been expanded to include more events. In addition, this release has fixes related to snapshots, upgrades, availability, stability, and supportability.

What's Changed

This patch fixes an issue for older Linux kernels (<5.8) where /proc/mounts iteration would deadlock causing CSI volumes to get stuck during the staging process.

Fixes

Testing

Mayastor is subject to extensive unit, component and system-level testing throughout the development and release cycle. Resources for system-level (E2E) testing are currently provided by DataCore Software.

At this time, personnel and hardware resource limitations constrain testing by the maintainers to linux builds on x86. This reflects the primary use-case which the maintainers are currently targeting with the OpenEBS Mayastor project. Therefore, the use of Mayastor with other operating systems and/or architectures, if even possible, should be considered serendipitous and wholly experimental.

This release has been subject to End-to-End testing under Ubuntu 20.04.5_LTS (kernel: ubuntu-5.15.0-50-generic)

  • Tested k8s versions
    • 1.23.7
    • 1.24.14
    • 1.25.10

Known Behavioural Limitations

As with the previous versions, the Mayastor IO engine makes full utilisation of the allocated CPU cores regardless of I/O load. This is the poller operating at full speed, waiting for I/O.

As with the previous versions, a Mayastor DiskPool is limited to a single block device and cannot span across more than one block device.

Known Issues

Mayastor does not support the capacity expansion of DiskPools as of v2.6.1.

v2.6.0

16 Apr 19:05
fcfb6ad
Compare
Choose a tag to compare

Release v2.6.0

Release Date: 16th April, 2024

Summary

OpenEBS Replicated PV Mayastor version 2.6 provides volume expansion support for a Mayastor volume with automatic resizing of file-system by the CSI-resizer. Mayastor v2.6 enhances snapshot capabilities by ensuring file-system consistency before taking a snapshot. Mayastor v2.6 also enhances monitoring by exporting performance metrics like IOPs, throughput, and latency for Mayastor pools, volumes, and replicas. In this release, the event generation capabilities have been expanded to include more events. In addition, this release has fixes related to snapshots, upgrades, availability, stability, and supportability.

Features

Volume Expansion

Mayastor 2.6 adds support for allowVolumeExpansion CSI capability to resize the volumes and the mounted file-systems. Mayastor CSI plugin provides support for the expansion of volume in both online and offline states.

Refer to the documentation for more details.

Filesystem Consistent Snapshot

Mayastor 2.6 makes a best effort to ensure the mounted filesystem is consistent while taking a snapshot of the underlying volume. This allows the file-system to be recovered in a good state when the volume is restored from the snapshot.

Refer to the documentation for more details.

Performance Metrics

Mayastor 2.6 enhances product monitoring by exposing performance metrics like in the Prometheus format in addition to the existing metrics for capacity and state. IO, Throughput and Latency counters for Read and Write operations of DiskPools, Volumes and Replicas are added to the list of metrics exported from the IO Engine exporter.

Refer to the documentation for more details.

Events

The eventing framework was introduced in the Mayastor 2.4. The generated events are published onto a NATS message bus.

In 2.6, new events get generated during the provisioning of snapshots and clones, and state changes for nexus, sub-system, reactor, replica, and node.

Refer to the documentation for the list of available events.

Improvements and Stability Fixes

Rebuild refactoring by tiagolobocastro · Pull Request #1581 · openebs/mayastor

fix(nexus): fixing missing I/Os during nexus rebuild by dsavitskiy · Pull Request #1583 · openebs/mayastor

refactor(io-engine): use SafeMountIter instead of proc_mounts::MountIter by niladrih · Pull Request #1591 · openebs/mayastor

Rebuild Rangers by tiagolobocastro · Pull Request #1594 · openebs/mayastor

fix(rebuild/partial): set bits for written blks only by tiagolobocastro · Pull Request #1597 · openebs/mayastor

refactor(rebuild/bdev): use builder pattern by tiagolobocastro · Pull Request #1604 · openebs/mayastor

fix(stats/lock): sync calls with pool service by tiagolobocastro · Pull Request #1607 · openebs/mayastor

fix(lock): make import pool, create replica and share replica operations mutually exclusive by hrudaya21 · Pull Request #1611 · openebs/mayastor

fix(nvmf): fixing duplicate QID error by dsavitskiy · Pull Request #1616 · openebs/mayastor

Reconnect IO log on Detach by tiagolobocastro · Pull Request #1619 · openebs/mayastor

fix(rebuild): reconnect log on own channel when faulting by tiagolobocastro · Pull Request #1622 · openebs/mayastor

feat: capacity limit for volumes by chriswldenyer · Pull Request #710 · openebs/mayastor-control-plane

Repo chores by tiagolobocastro · Pull Request #717 · openebs/mayastor-control-plane

refactor(rest-plugin): make it more modular by tiagolobocastro · Pull Request #720 · openebs/mayastor-control-plane

refactor(rest-plugin): make executor more generic by tiagolobocastro · Pull Request #721 · openebs/mayastor-control-plane

Delete Orphaned Volumes on a timer by tiagolobocastro · Pull Request #724 · openebs/mayastor-control-plane

Node labelling UX by tiagolobocastro · Pull Request #726 · openebs/mayastor-control-plane

fix(core/ha): re-share the nexus on republish by tiagolobocastro · Pull Request #728 · openebs/mayastor-control-plane

feat(app node): register csi nodes to control plane by Abhinandan-Purkait · Pull Request #730 · openebs/mayastor-control-plane

fix(pstor): when lease is lost exit instead of panic by tiagolobocastro · Pull Request #732 · openebs/mayastor-control-plane

check cluster capacity limit during volume resize, and also allow volume resize via rest plugin by dsharma-dc · Pull Request #747 · openebs/mayastor-control-plane

fix(legacy prefix detection): calculate range end for legacy prefix instead of empty by Abhinandan-Purkait · Pull Request #761 · openebs/mayastor-control-plane

fix(csi-driver): support getting device size from devpath by niladrih · Pull Request #765 · openebs/mayastor-control-plane

fix(csi-controller/pv-watcher): race between creation and gc by tiagolobocastro · Pull Request #768 · openebs/mayastor-control-plane

Blocking nvme disconnect fixes by tiagolobocastro · Pull Request #776 · openebs/mayastor-control-plane

Tracing and Platform fixes by tiagolobocastro · Pull Request #782 · openebs/mayastor-control-plane

fix(volume/resize): handle overflow during volume shrink attempt by dsharma-dc · Pull Request #784 · openebs/mayastor-control-plane

feat(eventing): add target volumeid to nvme path events by datacore-vvarakantham · Pull Request #791 · openebs/mayastor-control-plane

fix(agent/core): rebuild space required by tiagolobocastro · Pull Request #792 · openebs/mayastor-control-plane

XFS compat with older kernels when formatting by tiagolobocastro · Pull Request #799 · openebs/mayastor-control-plane

CI fixes and disable partial rebuild with cli arg by tiagolobocastro · Pull Request #804 · openebs/mayastor-control-plane

chore(deps): address CVE-2023-39325, CVE-2023-47108 and GHSA-m425-mq9… by cmontemuino · Pull Request #402 · openebs/mayastor-extensions

refactor(kubectl-plugin): tidy up error handling and command execution by tiagolobocastro · Pull Request #407 · openebs/mayastor-extensions

fix(uncordon): Rectify uncordon resource by sinhaashish · Pull Request #409 · openebs/mayastor-extensions

feat(volume/resize): don't allow volume resize for snapshot restore vols by dsharma-dc · Pull Request #777 · openebs/mayastor-control-plane

Performance and Scalability Fixes

fix(lvs): fixing issues with large number of volumes by dsavitskiy · Pull Request #1589 · openebs/mayastor

perf: use spdk mempool per-core cache for io objects pool by dsharma-dc · Pull Request #1612 · openebs/mayastor

refactor: use pagination to load values from pstor by Abhinandan-Purkait · Pull Request #737 · openebs/mayastor-control-plane

Supportability Fixes

add json formatting support by abhilashshetty04 · Pull Request #745 · openebs/mayastor-control-plane

feat(supportability): update etcd pagination fetch method by Abhinandan-Purkait · Pull Request #426 · openebs/mayastor-extensions

[fix(kubectl/dump/etcd): use correct namespace by tiagolobocastro · Pull Request #437 · openebs/mayastor-extensions](https://github.com/openebs/mayastor-extensions/pull...

Read more

v2.5.0

18 Dec 17:03
Compare
Choose a tag to compare

Release v2.5.0

Release Date: 18th Dec., 2023

Summary

The version 2.5 adds support for btrfs file-system as a supported file-system on top of a Mayastor volume. Mayastor 2.5 provides for event generation for event categories like volume, nexus, replica, host and switch-over. This release updates SPDK to version 23.05. In addition, this release has fixes related to snapshots, upgrade, availability, stability and supportability.

Features

Support for Btrfs file-system
This release supports creation of Btrfs file-system through the Mayastor CSI provisioner, in addition to the current support for Ext4 and XFS. Kubernetes storage administrators need to specify fsType as “btrfs” in the Mayastor Storage class, and create volumes using PVC from this SC.

Eventing

The eventing framework was introduced in the Mayastor 2.4. The generated events are published onto a NATS message bus.

In the current release, events are generated for creation and deletion of volume nexus and replicas, HA switch-over and state change for node, pool, volume, nexus and replica.

Please refer the documentation for more details. Eventing can be disabled as per the steps here.

Snapshot Fixes

Performance tuning for snapshot operations feat(pool): blobstore cluster size to be set during pool creation through grpc parameter value by hrudaya21 · Pull Request #1512 · openebs/mayastor

feat(snapshot, restore): prevent volume mode conversion for restore as default behaviour by Abhinandan-Purkait · Pull Request #340 · openebs/mayastor-extensions

Security Fixes

Security updates by tiagolobocastro · Pull Request #1543 · openebs/mayastor

Stability Fixes

Updating SPDK library feat(spdk): updating to SPDK 23.05 by dsavitskiy · Pull Request #1508 · openebs/mayastor

Command Retry Delay fixes feat(nvmf): selection different CRD for replicas and certain nexus error by dsavitskiy · Pull Request #1524 · openebs/mayastor

Pull rpc from dependencies submodule and various fixes by tiagolobocastro · Pull Request #1534 · openebs/mayastor

fix(spdk): backport of SPDK fix for zero copy by dsavitskiy · Pull Request #1539 · openebs/mayastor

Fixing I/O engine crashes by dsavitskiy · Pull Request #1549 · openebs/mayastor

Various fixes for idempotency, error messages and concurrency by tiagolobocastro · Pull Request #1552 · openebs/mayastor

nvme/opts: parse timeouts in humantime by tiagolobocastro · Pull Request #1548 · openebs/mayastor

fix(pstor): increase persistence timeouts by tiagolobocastro · Pull Request #1564 · openebs/mayastor

feat(drain): disallow node draining if ha is disabled by Abhinandan-Purkait · Pull Request #692 · openebs/mayastor-control-plane

Nvme addr matches host:port only by tiagolobocastro · Pull Request #687 · openebs/mayastor-control-plane

feat(ha): csi node should label nodes with nvme ana capability, node agent should not report failed paths if ana is not supported by Abhinandan-Purkait · Pull Request #693 · openebs/mayastor-control-plane

Fixup correct nvme controller and make child add v1 idempotent by tiagolobocastro · Pull Request #695 · openebs/mayastor-control-plane

feat(snapshot, restore): prevent volume mode conversion for restore as default behaviour by Abhinandan-Purkait · Pull Request #340 · openebs/mayastor-extensions

refactoring current metrics exporter implementation by abhilashshetty04 · Pull Request #355 · openebs/mayastor-extensions

feat(nvmf/crdt): add crdt to the io-engine by tiagolobocastro · Pull Request #326 · openebs/mayastor-extensions

Supportability Fixes

fix(kubectl-plugin): non-zero exit code on error by datacore-vvarakantham · Pull Request #694 · openebs/mayastor-control-plane

feat(loki): adding multiline support by abhilashshetty04 · Pull Request #344 · openebs/mayastor-extensions

Upgrade Fixes

fix(upgrade-job): remove short args to resolve conflict over '-h' option by niladrih · Pull Request #353 · openebs/mayastor-extensions

feat: upgrade changes by niladrih · Pull Request #358 · openebs/mayastor-extensions

fix(upgrade-job): wait for event worker to exit for errors by niladrih · Pull Request #319 · openebs/mayastor-extensions

fix(upgrade-job): avoid quotes when setting yaml bools, null, int by niladrih · Pull Request #334 · openebs/mayastor-extensions

chore(upgrade) : add volume node validation by sinhaashish · Pull Request #336 · openebs/mayastor-extensions

fix(upgrade-job): set defaults for absent helm values by niladrih · Pull Request #337 · openebs/mayastor-extensions

Testing

Mayastor is subject to extensive unit, component and system-level testing throughout the development and release cycle. Resources for system-level (E2E) testing are currently provided by DataCore Software.

At this time, personnel and hardware resource limitations constrain testing by the maintainers to linux builds on x86. This reflects the primary use-case which the maintainers are currently targeting with the OpenEBS Mayastor project. Therefore, the use of Mayastor with other operating systems and/or architectures, if even possible, should be considered serendipitous and wholly experimental.

This release has been subject to End-to-End testing under Ubuntu 20.04.5_LTS (kernel: ubuntu-5.15.0-50-generic)

  • Tested k8s versions
    • 1.23.7
    • 1.24.14
    • 1.25.10

Known behavioural limitations

As with the previous versions, the Mayastor IO engine makes full utilisation of the allocated CPU cores regardless of I/O load. This is the poller operating at full speed, waiting for I/O.

As with the previous versions, a Mayastor DiskPool is limited to a single block device and cannot span across more than one block device.

Known issues

Mayastor does not support capacity expansion for volumes as of v2.5.0.

Mayastor does not support capacity expansion of DiskPools as of v2.5.0.

Under heavy IO and constant scaling up-down of volume replicas, the io-engine pod has been observed to restart occassionally.

Getting Started

Mayastor user documentation, including a quick deployment guide, can be found here

Upgrade

Upgrades from versions of Mayastor prior to v1.0.0 are not supported. Any release earlier than v1.0.0 should be removed prior to installing this version.
Users get the support for upgrading the software from version 2.0.0 | 2.0.1 | 2.1.0 | v2.2.0 | v2.3.0 | v2.4.0 to v2.5.0

Support For Migration From Legacy Versions

Mayastor versions 1.0.5 and prior, are being considered as legacy versions. Due to several breaking changes in the 2.0 codebase of the software, it is not possible to support seamless upgrades from the legacy versions to the current version. Mayastor 2.2 provides a documented migration path for users to move their legacy installations to the latest version.

Please refer the documentation for more information.

Support

If you are having issues during installation, configuration or upgrade, you can contact us via:

"Unsupported" Architectures and Operating Systems (inc. ARM, Raspberry Pi, MacOS)

As described in the section on software testing above, the maintainers build and test Mayastor only on linux, on x86-64. The use of Mayastor in other environments is therefore not necessarily possible, at least without modification. Where possible, this is currently largely coincidental - it is not "fully" tested and therefore this should be considered an entirely experimental use-case.

The maintainers will be pleased to receive contributions in this area, with the following understanding:

  • Such PR's will be reviewed for correctness, good practice, licensing compliance and general quality
  • PR's will be accepted on the basis that testing by the maintainers is restricted to demonstrating no negative affect on the stability of x86-64 builds
  • The maintainers will not perform acceptance tes...
Read more

v1.0.9

03 Nov 13:08
570c103
Compare
Choose a tag to compare

Release v1.0.9

Release Date: 3rd Nov 2023

Summary

This patch release contains an important fixes for issues identified post the release of v1.0.8
Users of prior 1.0.x versions are advised to upgrade to this version.

Fixes

Testing

Mayastor is subject to extensive unit, component and system-level testing throughout the development and release cycle. Resources for system-level (E2E) testing are currently provided by DataCore Software.

At this time, personnel and hardware resource limitations constrain testing by the maintainers to linux builds on x86. This reflects the primary use-case which the maintainers are currently targeting with the OpenEBS Mayastor project. Therefore, the use of Mayastor with other operating systems and/or architectures, if even possible, should be considered serendipitous and wholly experimental.

This release has been subject to End-to-End testing under Ubuntu 20.04.4_LTS (kernel: ubuntu-5.15.0-50-generic)

  • Tested k8s versions
    • 1.23.7

Known Issues

  • The Pool Operator is unable to provision pools directly using a file as the backing device. The operator attempts to validate any device path supplied in the pool specification as an accessible block device attached to the corresponding Mayastor node. In the case of a file store, there is no block device and the validation fails, causing provisioning of the pool to be aborted. This will be addressed in a future release

    • workaround: Mount the file/image as a loopback device (losetup) and use the device path of the loopback device in the pool spec
  • Deploying an application pod on a worker node which hosts both Mayastor and Prometheus exporter causes that node to restart.

    • workaround: Use kernel version 5.13 or later

Getting Started

Mayastor user documentation, including a quick deployment guide, can be found here

Upgrade

Upgrades from versions of Mayastor prior to v1.0.0 are not supported. Any release earlier than v1.0.0 should be removed prior to installing this version.

Upgrades are disruptive - data plane and control plane pods must be unscheduled, and hence all applications using Mayastor volumes must be stopped prior to performing the update. The upgrade process is manual. The K8s definition files corresponding to the deployment of Mayastor components should be patched to reflect the versions of images from this release (see the /deploy) folder in the appropriate repositories and/or the GitBook Quickstart instruction for details of which definition files are affected, and check the v1.0.8 versions to find the appropriate image tag values.

Support

If you are having issues during installation, configuration or upgrade, you can contact us via:

"Unsupported" Architectures and Operating Systems (inc. ARM, Raspberry Pi, MacOS)

As described in the section on software testing above, the maintainers build and test Mayastor only on linux, on x86-64. The use of Mayastor in other environments is therefore not necessarily possible, at least without modification. Where possible, this is currently largely coincidental - it is not "fully" tested and therefore this should be considered an entirely experimental use-case.

The maintainers will be pleased to receive contributions in this area, with the following understanding:

  • Such PR's will be reviewed for correctness, good practice, licensing compliance and general quality
  • PR's will be accepted on the basis that testing by the maintainers is restricted to demonstrating no negative affect on the stability of x86-64 builds
  • The maintainers will not perform acceptance testing or "positive release" of such functionality on any other OS or architecture, which is in accordance with their designation of these environments as experimental use cases at this time.
  • The maintainers will not provide build artifacts or container images for these environments

v2.4.0

05 Sep 18:34
Compare
Choose a tag to compare

Release v2.4.0

Release Date: 5th Sept., 2023

Summary

The version 2.4 introduces the standard storage cloning capability to the Mayastor storage engine by implementing Kubernetes volume restore functionality. Mayastor 2.4 also introduces a framework for generating and publishing events. The release enables provisioning persistent storage for Etcd and Loki containers with OpenEBS dynamic localpv hostpath provisioner. In addition, this release has DiskPool CRD updates and fixes related to snapshots, upgrade, availability and supportability.

Features

Volume Restore from Snapshots

A snapshot of a Mayastor volume is a point-in-time read-only copy of the volume. Snapshot support was added in the previous release (v2.3). The current release adds support to create a new volume from an existing snapshot. In Kubernetes semantics, this is the equivalent of restoring a volume from a snapshot by specifying the source of the volume as a snapshot in the PVC manifest.

It is important to note that this feature is being released with the below limitation:

The current release supports creating a restore volume with only one replica. Support for restoring to multi-replica volumes will be added in the future.

It is also important to note that the StorageClass specified in the PVC for a restore volume, must have ‘thin' parameter set to 'true’.

Please refer the documentation for more details.

Eventing

Eventing is an important part of monitoring of any system. Mayastor 2.4 introduces an eventing framework which enables important events to be generated from the core components. The generated events are published onto a NATS message bus. As a part of this framework, we have introduced NATS into the system along-with a statistics module which subscribes for the events on the NATS, aggregates them and exports useful metrics in the Prometheus format.

In the current release, events are generated for creation and deletion of Mayastor pools and volumes.

Please refer the documentation for more details. Eventing can be disabled as per the steps here.

OpenEBS LocalPV Storage for Etcd and Loki

Etcd and Loki are installed as a part of the regular deployment of Mayastor. This is because, Mayastor uses etcd to persist the configuration and state settings, and Loki for storing the system logs. Both etcd and Loki are installed as Kubernetes StatefulSets and the storage for them is provisioned using either the cluster default storage class or static host path persistent volumes. There are limitations with this configuration:

A cluster default StorageClass exists only in managed Kubernetes environments (eg. Google Kubernetes Engine). The on-premise and self-managed clusters may not have a default StorageClass.

In the static host path approach, manual intervention is needed to pre-create persistent volumes on all nodes where the stateful pods of etcd and Loki could be possibly scheduled.

Mayastor 2.4 addresses this limitation using the OpenEBS dynamic localpv provisioner for creation of hostpath PVs. This provisioner is installed as a part of Mayastor 2.4 helm installation and custom storage classes are created, one each for Etcd and Loki. Using these classes, PVs are created dynamically for each statefulset pod of Etcd and Loki. The dynamic provisioning also enables easy scaling up of etcd and Loki replicas.

DiskPool CRD Update

In release 2.4, Mayastor DiskPool CustomResource version has been updated to v1beta1 (previously v1alpha1), where the “state“ field is deprecated and replaced by “cr_state“ and “pool_status“.

Related issues encountered when upgrading to version 2.4 can be resolved by following the steps documented here.

Snapshot Fixes

Fix/SPDK API to set attributes for snapshot by mtzaurus · Pull Request #1471 · openebs/mayastor

feat(snapshot/clone): handle different query options for listsnapshot and listreplica by hrudaya21 · Pull Request #1479 · openebs/mayastor

fix(snapshot): use customize function to judge snapshot inplace of spdk native function by hrudaya21 · Pull Request #1489 · openebs/mayastor

fix(snapshot): reset usage cache for the parent lvol when snapshot is destroyed by hrudaya21 · Pull Request #1493 · openebs/mayastor

feat(clone): enable option for fs id change to clone uuid by Abhinandan-Purkait · Pull Request #650 · openebs/mayastor-control-plane

feat(snapshots): add garbage collector to cleanup leaked snapshots unknown to control-plane by Abhinandan-Purkait · Pull Request #655 · openebs/mayastor-control-plane

feat(snapshots, restores): store restores as a part of snap meta, expose counts on rest by Abhinandan-Purkait · Pull Request #656 · openebs/mayastor-control-plane

Handle discarded snapshots by tiagolobocastro · Pull Request #660 · openebs/mayastor-control-plane

fix: error for multi-replica volume from snapshot by Abhinandan-Purkait · Pull Request #669 · openebs/mayastor-control-plane

Availability Fixes

Adding Command Retry Delay option feat(nvmf): adding support for configuring target CRD by dsavitskiy · Pull Request #1491 · openebs/mayastor

Pin volume target to replicas for 1 replica volumes by tiagolobocastro · Pull Request #662 · openebs/mayastor-control-plane

Supportability Fixes

refactor: replica listing based on query by Abhinandan-Purkait · Pull Request #666 · openebs/mayastor-control-plane

refactor: snapshot listing based on query by Abhinandan-Purkait · Pull Request #664 · openebs/mayastor-control-plane

fix(loki/enabled): loki PV being added even when disabled by tiagolobocastro · Pull Request #313 · openebs/mayastor-extensions

Upgrade Enhancements and Fixes

chore(upgrade): modify check for rebuilding by sinhaashish · Pull Request #299 · openebs/mayastor-extensions

feat(upgrade): add set flag in upgrade by sinhaashish · Pull Request #307 · openebs/mayastor-extensions

fix(upgrade): re-order upgrade sequence by sinhaashish · Pull Request #309 · openebs/mayastor-extensions

chore(upgrade): fix rebuild validation by sinhaashish · Pull Request #314 · openebs/mayastor-extensions

fix(upgrade-job): wait for event worker to exit for errors by niladrih · Pull Request #319 · openebs/mayastor-extensions

refactor(upgrade-job): change dry-run helm upgrade implementation by niladrih · Pull Request #328 · openebs/mayastor-extensions

fix(upgrade-job): avoid quotes when setting yaml bools, null, int by niladrih · Pull Request #334 · openebs/mayastor-extensions

fix(upgrade-job): set defaults for absent helm values by niladrih · Pull Request #337 · openebs/mayastor-extensions

Testing

Mayastor is subject to extensive unit, component and system-level testing throughout the development and release cycle. Resources for system-level (E2E) testing are currently provided by DataCore Software.

At this time, personnel and hardware resource limitations constrain testing by the maintainers to linux builds on x86. This reflects the primary use-case which the maintainers are currently targeting with the OpenEBS Mayastor project. Therefore, the use of Mayastor with other operating systems and/or architectures, if even possible, should be considered serendipitous and wholly experimental.

This release has been subject to End-to-End testing under Ubuntu 20.04.5_LTS (kernel: ubuntu-5.15.0-50-generic)

  • Tested k8s versions
    • 1.23.7
    • 1.24.14
    • 1.25.10

Known behavioural limitations

  • As with the previous versions, the Mayastor IO engine makes full utilisation of the allocated CPU cores regardless of I/O load. This is the poller operating at full speed, waiting for I/O.

  • As with the previous versions, a Mayastor DiskPool is limited to a single block device and cannot span across more than one block device.

Known issues

  • Mayastor does not support capacity expansion for volumes as of v2.4.0.

  • Mayastor does not support capacity expansion of DiskPools as of v2.4.0.

  • Under heavy IO and constant scaling up-down of volume replicas, the io-engine pod has been observed to restart occassionally.

Getting Started

Mayastor user documentation, including a quick deployment guide, can be found here

Upgrade

Upgrad...

Read more

v2.3.0

22 Jul 12:07
Compare
Choose a tag to compare

Release v2.3.0

Release Date: 22nd July, 2023

Summary

Mayastor 2.3 introduces the industry standard storage snapshotting capabilities in the Mayastor storage engine. Kubernetes application users and administrators will now be able to create, list and delete instantaneous snapshots of Mayastor Persistent Volumes based on standard CSI specifications. Mayastor 2.3 also provides APIs to view the history of volume and replica rebuilds. In addition, this release has fixes related to HA, thin provisioning, rebuilding, stability and node drain operations.

Features

Volume Snapshots

A snapshot of a Mayastor volume is a point-in-time read-only copy of the volume. In this release, Mayastor provides support for industry standard copy-on-write (COW) snapshots, which is a popular methodology for taking snapshots by keeping track of only those blocks that have changed. Mayastor supports snapshots in a Kubernetes-native way. A snapshot controller and side-car are deployed alongside the CSI driver and the Kubernetes CRDs namely VolumeSnapshotClass, VolumeSnapshotContent and VolumeSnapshot are also deployed during the installation. By creating a custom resource of kind VolumeSnapshot, a user can easily take a snapshot of the Mayastor volume.

Please refer the documentation for more details.

It is important to note that this feature is being released with the below limitations:

The current release supports snapshots only for single-replica volumes. Support for snapshotting multi-replica volumes will be provided in the future.

The current release does not support creation of a clone or a new volume from an existing snapshot.

Rebuild History

The Mayastor volume replicas are usually distributed across storage nodes to ensure availability and guard against failures. Whenever a replica becomes inaccessible, the volume target (Mayastor nexus) marks the replica as faulted and a rebuilding process is spawned to recover the faulted replica. If the replica is rebuilt from scratch, it is known as a full rebuild process. If a replica is rebuilt from an intermediate point using a recovery log, it Is known as a partial rebuild process. Mayastor 2.3 provides API to view the history of rebuilds of all replicas attached to the current volume target (nexus). Users can invoke the Mayastor kubectl plugin with a get rebuild-history option to retrieve the rebuild records for a specific volume target.

Please refer the documentation for more details.

HA and Drain Fixes

fix(cordon-drain): return correct updated spec by tiagolobocastro · Pull Request #621 · openebs/mayastor-control-plane

fix: ensure the updated node spec is used in drain by chriswldenyer · Pull Request #632 · openebs/mayastor-control-plane

Several Snapshtot and Drain Fixes by tiagolobocastro · Pull Request #591 · openebs/mayastor-control-plane

Thin-Provisioning Fixes

fix(lvol): properly report provisioning mod for volume with snapshots by mtzaurus · Pull Request #1430 · openebs/mayastor

fix(snapshots/thin): a thick volume becomes thin after snapshots are created by tiagolobocastro · Pull Request #599 · openebs/mayastor-control-plane

fix(enospc): allow enospc nexus to fail IOs by tiagolobocastro · Pull Request #1422 · openebs/mayastor

Stability Fixes

Add concurrency limiter for create volume operations upto 10.

fix(nexus): fixing persistent store update issue during device retire by dsavitskiy · Pull Request #1398 · openebs/mayastor

fix(nexus/replica): check with replica owner before destroying it by tiagolobocastro · Pull Request #641 · openebs/mayastor-control-plane

fix(pool_controller): mark cr state as created when pool spec already present by abhilashshetty04 · Pull Request #634 · openebs/mayastor-control-plane

fix: when replica state is missing fill what we can by tiagolobocastro · Pull Request #633 · openebs/mayastor-control-plane

chore: add garbage collector for cleaning in creating snapshots with no transactions by Abhinandan-Purkait · Pull Request #608 · openebs/mayastor-control-plane

fix(snapshot): take into account snapshots for replica allocation by tiagolobocastro · Pull Request #607 · openebs/mayastor-control-plane

Volume Snapshot Commitment by tiagolobocastro · Pull Request #619 · openebs/mayastor-control-plane

chore(spdk): spdk revision changed to properly deallocate blob clusters by mtzaurus · Pull Request #1460 · openebs/mayastor

refactor: csi-driver, use filesystem enum by Abhinandan-Purkait · Pull Request #643 · openebs/mayastor-control-plane

fix(core): fixing passing read mode flags to block devices by mtzaurus · Pull Request #1454 · openebs/mayastor

fix(rebuild): fixing child retirement during rebuild running by dsavitskiy · Pull Request #1445 · openebs/mayastor

refactor(list-rebuild): add since timestamp and name by tiagolobocastro · Pull Request #1423 · openebs/mayastor

chore(core): errno mapping for core errors by mtzaurus · Pull Request #1418 · openebs/mayastor

fix(nexus/io): set nvmf subsystem as frozen by tiagolobocastro · Pull Request #1415 · openebs/mayastor

fix(fs/shutdown): freeze nexus when faulted by tiagolobocastro · Pull Request #1409 · openebs/mayastor

feat(rebuild_history): add list api with record limit per nexus by abhilashshetty04 · Pull Request #1408 · openebs/mayastor

fix(fs/shutdown): pause nexus when faulted by tiagolobocastro · Pull Request #1407 · openebs/mayastor

fix(nvmx): fixing async qpair connection bugs by dsavitskiy · Pull Request #1405 · openebs/mayastor

chore(lvs/lvol): add trace message before zeroing super by tiagolobocastro · Pull Request #1399 · openebs/mayastor

chore(csi): set provisioner to timeout after 36s and set 10 worker threads by tiagolobocastro · Pull Request #251 · openebs/mayastor-extensions

chore: add tolerations to per component by Abhinandan-Purkait · Pull Request #226 · openebs/mayastor-extensions

Supportability Fixes

feat(supportability): add supportability for call home by sinhaashish · Pull Request #261 · openebs/mayastor-extensions

fix(supportability/loki): add missing newlines by tiagolobocastro · Pull Request #265 · openebs/mayastor-extensions

chore: add snasphot and rebuild history to bundle, disable resouce level dump by Abhinandan-Purkait · Pull Request #296 · openebs/mayastor-extensions

chore: add snasphot and rebuild history to bundle, disable resouce level dump by Abhinandan-Purkait · Pull Request #296 · openebs/mayastor-extensions

Upgrade Fixes

feat(upgrade-job): dynamically generate new values options by niladrih · Pull Request #260 · openebs/mayastor-extensions

fix(upgrade-job): use '--install' flag with helm upgrade by niladrih · Pull Request #266 · openebs/mayastor-extensions

feat(upgrade): allow upgrade to develop by sinhaashish · Pull Request #224 · openebs/mayastor-extensions

feat(upgrade): add unsupported upgrade version file upgrade by sinhaashish · Pull Request #229 · openebs/mayastor-extensions

feat(upgrade-job): support for upgrade to openebs-3.7.0 by niladrih · Pull Request #231 · openebs/mayastor-extensions

fix(upgrade-job): upgrade path validation and set flags by niladrih · Pull Request #242 · openebs/mayastor-extensions

fix(upgrade-job): refactor version check by niladrih · Pull Request #238 · openebs/mayastor-extensions

feat(upgrade): add force flag for deleting upgrade resources by sinhaashish · Pull Request #245 · openebs/mayastor-extensions

[feat(upgrade-job)...

Read more

v1.0.8

20 Jul 08:51
Compare
Choose a tag to compare

Release v1.0.8

Release Date: 19th July 2023

Summary

This patch release contains an important fixes for issues identified post the release of v1.0.7
Users of prior 1.0.x versions are advised to upgrade to this version.

Fixes

Testing

Mayastor is subject to extensive unit, component and system-level testing throughout the development and release cycle. Resources for system-level (E2E) testing are currently provided by DataCore Software.

At this time, personnel and hardware resource limitations constrain testing by the maintainers to linux builds on x86. This reflects the primary use-case which the maintainers are currently targeting with the OpenEBS Mayastor project. Therefore, the use of Mayastor with other operating systems and/or architectures, if even possible, should be considered serendipitous and wholly experimental.

This release has been subject to End-to-End testing under Ubuntu 20.04.4_LTS (kernel: ubuntu-5.15.0-50-generic)

  • Tested k8s versions
    • 1.23.7

Known Issues

  • The Pool Operator is unable to provision pools directly using a file as the backing device. The operator attempts to validate any device path supplied in the pool specification as an accessible block device attached to the corresponding Mayastor node. In the case of a file store, there is no block device and the validation fails, causing provisioning of the pool to be aborted. This will be addressed in a future release

    • workaround: Mount the file/image as a loopback device (losetup) and use the device path of the loopback device in the pool spec
  • Deploying an application pod on a worker node which hosts both Mayastor and Prometheus exporter causes that node to restart.

    • workaround: Use kernel version 5.13 or later

Getting Started

Mayastor user documentation, including a quick deployment guide, can be found here

Upgrade

Upgrades from versions of Mayastor prior to v1.0.0 are not supported. Any release earlier than v1.0.0 should be removed prior to installing this version.

Upgrades are disruptive - data plane and control plane pods must be unscheduled, and hence all applications using Mayastor volumes must be stopped prior to performing the update. The upgrade process is manual. The K8s definition files corresponding to the deployment of Mayastor components should be patched to reflect the versions of images from this release (see the /deploy) folder in the appropriate repositories and/or the GitBook Quickstart instruction for details of which definition files are affected, and check the v1.0.8 versions to find the appropriate image tag values.

Support

If you are having issues during installation, configuration or upgrade, you can contact us via:

"Unsupported" Architectures and Operating Systems (inc. ARM, Raspberry Pi, MacOS)

As described in the section on software testing above, the maintainers build and test Mayastor only on linux, on x86-64. The use of Mayastor in other environments is therefore not necessarily possible, at least without modification. Where possible, this is currently largely coincidental - it is not "fully" tested and therefore this should be considered an entirely experimental use-case.

The maintainers will be pleased to receive contributions in this area, with the following understanding:

  • Such PR's will be reviewed for correctness, good practice, licensing compliance and general quality
  • PR's will be accepted on the basis that testing by the maintainers is restricted to demonstrating no negative affect on the stability of x86-64 builds
  • The maintainers will not perform acceptance testing or "positive release" of such functionality on any other OS or architecture, which is in accordance with their designation of these environments as experimental use cases at this time.
  • The maintainers will not provide build artifacts or container images for these environments

v1.0.7

16 Jun 13:24
Compare
Choose a tag to compare

Release v1.0.7

Release Date: 15th June 2023

Summary

This patch release contains an important fix for an issue identified after the release of v1.0.6
Users of prior 1.0.x versions are advised to upgrade to this version.

Fixes

Testing

Mayastor is subject to extensive unit, component and system-level testing throughout the development and release cycle. Resources for system-level (E2E) testing are currently provided by DataCore Software.

At this time, personnel and hardware resource limitations constrain testing by the maintainers to linux builds on x86. This reflects the primary use-case which the maintainers are currently targeting with the OpenEBS Mayastor project. Therefore, the use of Mayastor with other operating systems and/or architectures, if even possible, should be considered serendipitous and wholly experimental.

This release has been subject to End-to-End testing under Ubuntu 20.04.4_LTS (kernel: ubuntu-5.15.0-50-generic)

  • Tested k8s versions
    • 1.23.7

Known Issues

  • The Pool Operator is unable to provision pools directly using a file as the backing device. The operator attempts to validate any device path supplied in the pool specification as an accessible block device attached to the corresponding Mayastor node. In the case of a file store, there is no block device and the validation fails, causing provisioning of the pool to be aborted. This will be addressed in a future release

    • workaround: Mount the file/image as a loopback device (losetup) and use the device path of the loopback device in the pool spec
  • Deploying an application pod on a worker node which hosts both Mayastor and Prometheus exporter causes that node to restart.

    • workaround: Use kernel version 5.13 or later

Getting Started

Mayastor user documentation, including a quick deployment guide, can be found here

Upgrade

Upgrades from versions of Mayastor prior to v1.0.0 are not supported. Any release earlier than v1.0.0 should be removed prior to installing this version.

Upgrades are disruptive - data plane and control plane pods must be unscheduled, and hence all applications using Mayastor volumes must be stopped prior to performing the update. The upgrade process is manual. The K8s definition files corresponding to the deployment of Mayastor components should be patched to reflect the versions of images from this release (see the /deploy) folder in the appropriate repositories and/or the GitBook Quickstart instruction for details of which definition files are affected, and check the v1.0.7 versions to find the appropriate image tag values.

Support

If you are having issues during installation, configuration or upgrade, you can contact us via:

"Unsupported" Architectures and Operating Systems (inc. ARM, Raspberry Pi, MacOS)

As described in the section on software testing above, the maintainers build and test Mayastor only on linux, on x86-64. The use of Mayastor in other environments is therefore not necessarily possible, at least without modification. Where possible, this is currently largely coincidental - it is not "fully" tested and therefore this should be considered an entirely experimental use-case.

The maintainers will be pleased to receive contributions in this area, with the following understanding:

  • Such PR's will be reviewed for correctness, good practice, licensing compliance and general quality
  • PR's will be accepted on the basis that testing by the maintainers is restricted to demonstrating no negative affect on the stability of x86-64 builds
  • The maintainers will not perform acceptance testing or "positive release" of such functionality on any other OS or architecture, which is in accordance with their designation of these environments as experimental use cases at this time.
  • The maintainers will not provide build artifacts or container images for these environments