Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add capability to restrict async durability mode for remote indexes #10189

Merged
merged 3 commits into from
Sep 26, 2023

Conversation

ashking94
Copy link
Member

@ashking94 ashking94 commented Sep 22, 2023

Description

This PR brings in the code to restrict indexes with async translog durability mode. This is achieved by enabling a cluster setting that by default allows creating indexes with any translog durability mode. However, if the cluster setting is true, it would restrict creation or updation of index which tries to set the durability to async mode.

Related Issues

Resolves #10186

Check List

  • New functionality includes testing.
    • All tests pass
  • New functionality has been documented.
    • New functionality has javadoc added
  • Commits are signed per the DCO using --signoff
  • Commit changes are listed out in CHANGELOG.md file (See: Changelog)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@github-actions github-actions bot added enhancement Enhancement or improvement to existing feature or request Storage:Durability Issues and PRs related to the durability framework v2.11.0 Issues and PRs related to version 2.11.0 labels Sep 22, 2023
@github-actions
Copy link
Contributor

Gradle Check (Jenkins) Run Completed with:

@github-actions
Copy link
Contributor

github-actions bot commented Sep 22, 2023

Compatibility status:

Checks if related components are compatible with change bc860b9

Incompatible components

Skipped components

Compatible components

Compatible components: [https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/custom-codecs.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/performance-analyzer.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/reporting.git]

@github-actions
Copy link
Contributor

Gradle Check (Jenkins) Run Completed with:

  • RESULT: UNSTABLE ❕
  • TEST FAILURES:
      1 org.opensearch.snapshots.CloneSnapshotIT.testCloneAfterRepoShallowSettingDisabled
      1 org.opensearch.remotestore.RemoteStoreStatsIT.testNonZeroPrimaryStatsOnNewlyCreatedIndexWithZeroDocs

@codecov
Copy link

codecov bot commented Sep 23, 2023

Codecov Report

Merging #10189 (9dbe0a4) into main (a8969cb) will increase coverage by 0.06%.
The diff coverage is 64.28%.

❗ Current head 9dbe0a4 differs from pull request most recent head 4b7e9d2. Consider uploading reports for the commit 4b7e9d2 to get more accurate results

@@             Coverage Diff              @@
##               main   #10189      +/-   ##
============================================
+ Coverage     71.11%   71.18%   +0.06%     
- Complexity    58188    58231      +43     
============================================
  Files          4830     4828       -2     
  Lines        274440   274371      -69     
  Branches      39993    39991       -2     
============================================
+ Hits         195175   195302     +127     
+ Misses        62897    62669     -228     
- Partials      16368    16400      +32     
Files Coverage Δ
...rg/opensearch/common/settings/ClusterSettings.java 92.85% <ø> (ø)
...in/java/org/opensearch/indices/IndicesService.java 70.56% <100.00%> (-0.11%) ⬇️
...luster/metadata/MetadataUpdateSettingsService.java 69.47% <33.33%> (-1.28%) ⬇️
...h/cluster/metadata/MetadataCreateIndexService.java 79.58% <70.00%> (-0.53%) ⬇️

... and 468 files with indirect coverage changes

@github-actions
Copy link
Contributor

Gradle Check (Jenkins) Run Completed with:

@ashking94
Copy link
Member Author

Gradle Check (Jenkins) Run Completed with:

Flaky tests - #9208, #10154, #6090

@github-actions
Copy link
Contributor

Gradle Check (Jenkins) Run Completed with:

Copy link
Collaborator

@gbbafna gbbafna left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. 1 nit comment on test.

@github-actions
Copy link
Contributor

Gradle Check (Jenkins) Run Completed with:

@ashking94
Copy link
Member Author

Gradle Check (Jenkins) Run Completed with:

Flaky tests - #10231 and #10152 and

@ashking94
Copy link
Member Author

@gbbafna I have incorporated the comments. Please take a look at this.

@github-actions
Copy link
Contributor

Gradle Check (Jenkins) Run Completed with:

@github-actions
Copy link
Contributor

Gradle Check (Jenkins) Run Completed with:

  • RESULT: UNSTABLE ❕
  • TEST FAILURES:
      1 org.opensearch.smoketest.SmokeTestMultiNodeClientYamlTestSuiteIT.test {yaml=search.aggregation/20_terms/string profiler via global ordinals}

@gbbafna gbbafna merged commit 53cffef into opensearch-project:main Sep 26, 2023
14 checks passed
@gbbafna gbbafna added the backport 2.x Backport to 2.x branch label Sep 26, 2023
@opensearch-trigger-bot
Copy link
Contributor

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/OpenSearch/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/OpenSearch/backport-2.x
# Create a new branch
git switch --create backport/backport-10189-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 53cffefd22954276a9dc1cdea16ccd5ac2e2bc4c
# Push it to GitHub
git push --set-upstream origin backport/backport-10189-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/OpenSearch/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-10189-to-2.x.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport 2.x Backport to 2.x branch backport-failed enhancement Enhancement or improvement to existing feature or request Storage:Durability Issues and PRs related to the durability framework v2.11.0 Issues and PRs related to version 2.11.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Restrict async translog durability mode for remote indexes
4 participants