Skip to content

Commit

Permalink
Startree file formats codec merge (#29)
Browse files Browse the repository at this point in the history
* Fix flaky test in range aggregation yaml test (opensearch-project#14486)

Signed-off-by: bowenlan-amzn <[email protected]>

* Use CODECOV_TOKEN (opensearch-project#14536)

Signed-off-by: Prudhvi Godithi <[email protected]>

* [Tiered Caching]  Moving query recomputation logic outside of write lock (opensearch-project#14187)

* Moving query recompute out of write lock

Signed-off-by: Sagar Upadhyaya <[email protected]>

* [Tiered Caching] Moving query recomputation logic outside of write lock

Signed-off-by: Sagar Upadhyaya <[email protected]>

* Adding java doc for the completable map

Signed-off-by: Sagar Upadhyaya <[email protected]>

* Changes to call future handler only once per key

Signed-off-by: Sagar Upadhyaya <[email protected]>

* Fixing spotless check

Signed-off-by: Sagar Upadhyaya <[email protected]>

* Added changelog

Signed-off-by: Sagar Upadhyaya <[email protected]>

* Addressing comments

Signed-off-by: Sagar Upadhyaya <[email protected]>

* Fixing gradle fail

Signed-off-by: Sagar Upadhyaya <[email protected]>

* Addressing comments to refactor unit test

Signed-off-by: Sagar Upadhyaya <[email protected]>

* minor UT refactor

Signed-off-by: Sagar Upadhyaya <[email protected]>

---------

Signed-off-by: Sagar Upadhyaya <[email protected]>
Signed-off-by: Sagar <[email protected]>
Co-authored-by: Sagar Upadhyaya <[email protected]>

* Fix Flaky Test ClusterRerouteIT.testDelayWithALargeAmountOfShards (opensearch-project#14510)

Signed-off-by: kkewwei [email protected]

Signed-off-by: kkewwei [email protected]
Signed-off-by: kkewwei <[email protected]>

* Add doc for debugging rest tests (opensearch-project#14491)

* add doc for debugging rest tests

Signed-off-by: bowenlan-amzn <[email protected]>

* Update TESTING.md

Co-authored-by: Marc Handalian <[email protected]>
Signed-off-by: bowenlan-amzn <[email protected]>

* Address comment

Signed-off-by: bowenlan-amzn <[email protected]>

---------

Signed-off-by: bowenlan-amzn <[email protected]>
Co-authored-by: Marc Handalian <[email protected]>

* Fix flaky DefaultCacheStatsHolderTests (opensearch-project#14462)

Signed-off-by: Peter Alfonsi <[email protected]>
Co-authored-by: Peter Alfonsi <[email protected]>

* [AUTO] [main] Add bwc version 2.15.1. (opensearch-project#14549)

* Add bwc version 2.15.1

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

* Fix auto-generated version

Signed-off-by: Andrew Ross <[email protected]>

---------

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Signed-off-by: Andrew Ross <[email protected]>
Co-authored-by: opensearch-ci-bot <[email protected]>
Co-authored-by: Andrew Ross <[email protected]>

* Fix flaky test TieredSpilloverCacheTests.testComputeIfAbsentConcurrently (opensearch-project#14550)

* Fix flaky test TieredSpilloverCacheTests.testComputeIfAbsentConcurrently

Signed-off-by: Sagar Upadhyaya <[email protected]>

* Addressing comment

Signed-off-by: Sagar Upadhyaya <[email protected]>

---------

Signed-off-by: Sagar Upadhyaya <[email protected]>
Signed-off-by: Sagar Upadhyaya <[email protected]>
Co-authored-by: Sagar Upadhyaya <[email protected]>

* Add allowlist setting for ingest-common processors (opensearch-project#14479)

Add a new static setting that lets an operator choose specific ingest
processors to enable by name. The behavior is as follows:

- If the allowlist setting is not defined, all installed processors are
  enabled. This is the status quo.
- If the allowlist setting is defined as the empty set, then all processors
  are disabled.
- If the allowlist setting contains the names of valid processors, only those
  processors are enabled.
- If the allowlist setting contains a name of a processor that does not exist,
  then the server will fail to start with an IllegalStateException
  listing which processors were defined in the allowlist but are not
  installed.
- If the allowlist setting is changed between server restarts then any
  ingest pipeline using a now-disabled processor will fail. This is the
  same experience if a pipeline used a processor defined by a plugin but
  then that plugin were to be uninstalled across restarts.

Related to opensearch-project#14439

Signed-off-by: Andrew Ross <[email protected]>

* Fix file cache initialization (opensearch-project#14004)

* fix file cache initialization

Signed-off-by: panguixin <[email protected]>

* changelog

Signed-off-by: panguixin <[email protected]>

* add test

Signed-off-by: panguixin <[email protected]>

---------

Signed-off-by: panguixin <[email protected]>

* Add Ashish Singh as maintainer (opensearch-project#14567)

Signed-off-by: Bukhtawar Khan <[email protected]>

* Allow @internalapi annotation on classes not meant to be constructed outside of the OpenSearch core (opensearch-project#14575)

Signed-off-by: Andriy Redko <[email protected]>

* Bump com.azure:azure-storage-common from 12.21.2 to 12.25.1 in /plugins/repository-azure (opensearch-project#14517)

* Bump com.azure:azure-storage-common in /plugins/repository-azure

Bumps [com.azure:azure-storage-common](https://github.com/Azure/azure-sdk-for-java) from 12.21.2 to 12.25.1.
- [Release notes](https://github.com/Azure/azure-sdk-for-java/releases)
- [Commits](Azure/azure-sdk-for-java@azure-storage-common_12.21.2...azure-storage-blob_12.25.1)

---
updated-dependencies:
- dependency-name: com.azure:azure-storage-common
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

* Updating SHAs

Signed-off-by: dependabot[bot] <[email protected]>

* Update changelog

Signed-off-by: dependabot[bot] <[email protected]>
Signed-off-by: Andriy Redko <[email protected]>

---------

Signed-off-by: dependabot[bot] <[email protected]>
Signed-off-by: Andriy Redko <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>

* Add allowlist setting for search-pipeline-common processors (opensearch-project#14562)

Add a new static setting that lets an operator choose specific search
pipeline processors to enable by name. The behavior is as follows:

- If the allowlist setting is not defined, all installed processors are
  enabled. This is the status quo.
- If the allowlist setting is defined as the empty set, then all processors
  are disabled.
- If the allowlist setting contains the names of valid processors, only those
  processors are enabled.
- If the allowlist setting contains a name of a processor that does not exist,
  then the server will fail to start with an IllegalStateException
  listing which processors were defined in the allowlist but are not
  installed.
- If the allowlist setting is changed between server restarts then any
  ingest pipeline using a now-disabled processor will fail. This is the
  same experience if a pipeline used a processor defined by a plugin but
  then that plugin were to be uninstalled across restarts.

A distinct setting exists for each of request, response, and search phase
results processors.

Related to opensearch-project#14439

Signed-off-by: Andrew Ross <[email protected]>

* Bump Apache Lucene to 9.11.1 (opensearch-project#14576) (opensearch-project#14581)

(cherry picked from commit 0095fd1)

Signed-off-by: Andriy Redko <[email protected]>

* Add unittests for RemoteClusterStateAttributesManager (opensearch-project#14427)

* Add unittests for RemoteClusterStateAttributesManager

Signed-off-by: Shivansh Arora <[email protected]>

* Add Ashish Singh to codeowners (opensearch-project#14592)

Signed-off-by: Ashish Singh <[email protected]>

* Add batching processor base type AbstractBatchingProcessor (opensearch-project#14554)

Signed-off-by: Liyun Xiu <[email protected]>

* Add @internalapi annotation to japicmp exclusions (opensearch-project#14597)

Signed-off-by: Andriy Redko <[email protected]>

* Fix issue 14519:Parsing a GetResult returns NPE if found field is mis… (opensearch-project#14552)

* Fix issue 14519:Parsing a GetResult returns NPE if found field is missing. Signed-off-by: Vatsal <[email protected]>

Signed-off-by: vatsal <[email protected]>

* Fix issue 14519:Parsing a GetResult returns NPE if found field is missing. Signed-off-by: Vatsal <[email protected]>

Signed-off-by: vatsal <[email protected]>

* Fix issue 14519:Fix wildcart import. Signed-off-by: Vatsal <[email protected]>

Signed-off-by: vatsal <[email protected]>

* Fix issue 14519:Fix wildcart import. Signed-off-by: Vatsal <[email protected]>

Signed-off-by: vatsal <[email protected]>

* Fix issue 14519:Fix spotless issues. Signed-off-by: Vatsal <[email protected]>

Signed-off-by: vatsal <[email protected]>

* Fix issue 14519:update changelog

Signed-off-by: vatsal <[email protected]>

---------

Signed-off-by: vatsal <[email protected]>
Signed-off-by: Daniel Widdis <[email protected]>
Co-authored-by: Daniel Widdis <[email protected]>

* Star tree mapping changes (opensearch-project#14605)

* Star tree mapping changes with feature flag
---------
Signed-off-by: Bharathwaj G <[email protected]>

* Bump com.microsoft.azure:msal4j from 1.15.1 to 1.16.0 in /plugins/repository-azure (opensearch-project#14610)

* Bump com.microsoft.azure:msal4j in /plugins/repository-azure

Bumps [com.microsoft.azure:msal4j](https://github.com/AzureAD/microsoft-authentication-library-for-java) from 1.15.1 to 1.16.0.
- [Release notes](https://github.com/AzureAD/microsoft-authentication-library-for-java/releases)
- [Changelog](https://github.com/AzureAD/microsoft-authentication-library-for-java/blob/dev/changelog.txt)
- [Commits](AzureAD/microsoft-authentication-library-for-java@v1.15.1...v1.16.0)

---
updated-dependencies:
- dependency-name: com.microsoft.azure:msal4j
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

* Updating SHAs

Signed-off-by: dependabot[bot] <[email protected]>

* Update changelog

Signed-off-by: dependabot[bot] <[email protected]>

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>

* [Bugfix] Fix ICacheKeySerializerTests flakiness (opensearch-project#14564)

* Fix testInvalidInput flakiness

Signed-off-by: Peter Alfonsi <[email protected]>

* Addressed andrross's comment

Signed-off-by: Peter Alfonsi <[email protected]>

* rerun security check

Signed-off-by: Peter Alfonsi <[email protected]>

---------

Signed-off-by: Peter Alfonsi <[email protected]>
Co-authored-by: Peter Alfonsi <[email protected]>

* Correct typo in method name (opensearch-project#14621)

Signed-off-by: vatsal <[email protected]>

* Refactoring FilterPath.parse by using an iterative approach instead of recursion. (opensearch-project#14200)

* Refactor FilterPath parse function (opensearch-project#12067)
Signed-off-by: Robin Friedmann <[email protected]>

* Implement unit tests for FilterPathTests (opensearch-project#12067)
Signed-off-by: Robin Friedmann <[email protected]>

* Write warn log if Filter is empty; Add comments (opensearch-project#12067)
Signed-off-by: Robin Friedmann <[email protected]>

* Add changelog

Signed-off-by: Siddhant Deshmukh <[email protected]>

* Remove unnecessary log statement

Signed-off-by: Siddhant Deshmukh <[email protected]>

* Remove unused logger

Signed-off-by: Siddhant Deshmukh <[email protected]>

* Spotless apply

Signed-off-by: Siddhant Deshmukh <[email protected]>

* Remove incorrect changelog

Signed-off-by: Siddhant Deshmukh <[email protected]>

---------

Signed-off-by: Siddhant Deshmukh <[email protected]>
Co-authored-by: Robin Friedmann <[email protected]>

* Removing String format in RemoteStoreMigrationAllocationDecider to optimise performance(opensearch-project#14612)

Signed-off-by: RS146BIJAY <[email protected]>

* Clear templates before Adding; Use NamedWriteableAwareStreamInput for RemoteCustomMetadata; Correct the check for deciding upload of HashesOfConsistentSettings (opensearch-project#14513)

* Clear templates before Adding; Use NamedWriteableAwareStreamInput for RemoteCustomMetadata
* Correct the check for deciding upload of hashes of consistent settings

Signed-off-by: Sooraj Sinha <[email protected]>

* Improve reroute performance by optimising List.removeAll in LocalShardsBalancer to filter remote search shard from relocation decision (opensearch-project#14613)

Signed-off-by: RS146BIJAY <[email protected]>

* Fix assertion failure while deleting remote backed index (opensearch-project#14601)

Signed-off-by: Sachin Kale <[email protected]>

* OnHeap Star Tree Implementation

Signed-off-by: Sarthak Aggarwal <[email protected]>

* addressed nits

Signed-off-by: Sarthak Aggarwal <[email protected]>

* addressed major nits

Signed-off-by: Sarthak Aggarwal <[email protected]>

* includes Count Aggregator

Signed-off-by: Sarthak Aggarwal <[email protected]>

* handling for missing doc values

Signed-off-by: Sarthak Aggarwal <[email protected]>

* addressing review comments

Signed-off-by: Sarthak Aggarwal <[email protected]>

* rebasing with main

Signed-off-by: Sarthak Aggarwal <[email protected]>

* support for empty sequential doc values iterator

Signed-off-by: Sarthak Aggarwal <[email protected]>

* nits

Signed-off-by: Sarthak Aggarwal <[email protected]>

* min and max star tree aggregators

Signed-off-by: Sarthak Aggarwal <[email protected]>

* star tree file formats

* Star tree codec changes

Signed-off-by: Bharathwaj G <[email protected]>

* Adding tests

Signed-off-by: Bharathwaj G <[email protected]>

* Addressing comments

Signed-off-by: Bharathwaj G <[email protected]>

* addressing review comments

Signed-off-by: Bharathwaj G <[email protected]>

* Star tree merge changes

Signed-off-by: Bharathwaj G <[email protected]>

* fix annotations

* star-tree file formats reader and javadoc fixes

* read for composite index values

Signed-off-by: Sarthak Aggarwal <[email protected]>

* doc values file format

Signed-off-by: Sarthak Aggarwal <[email protected]>

---------

Signed-off-by: bowenlan-amzn <[email protected]>
Signed-off-by: Prudhvi Godithi <[email protected]>
Signed-off-by: Sagar Upadhyaya <[email protected]>
Signed-off-by: Sagar <[email protected]>
Signed-off-by: kkewwei [email protected]
Signed-off-by: kkewwei <[email protected]>
Signed-off-by: Peter Alfonsi <[email protected]>
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Signed-off-by: Andrew Ross <[email protected]>
Signed-off-by: Sagar Upadhyaya <[email protected]>
Signed-off-by: panguixin <[email protected]>
Signed-off-by: Bukhtawar Khan <[email protected]>
Signed-off-by: Andriy Redko <[email protected]>
Signed-off-by: dependabot[bot] <[email protected]>
Signed-off-by: Shivansh Arora <[email protected]>
Signed-off-by: Ashish Singh <[email protected]>
Signed-off-by: Liyun Xiu <[email protected]>
Signed-off-by: vatsal <[email protected]>
Signed-off-by: Daniel Widdis <[email protected]>
Signed-off-by: Siddhant Deshmukh <[email protected]>
Signed-off-by: RS146BIJAY <[email protected]>
Signed-off-by: Sooraj Sinha <[email protected]>
Signed-off-by: Sachin Kale <[email protected]>
Signed-off-by: Sarthak Aggarwal <[email protected]>
Signed-off-by: Bharathwaj G <[email protected]>
Co-authored-by: bowenlan-amzn <[email protected]>
Co-authored-by: Prudhvi Godithi <[email protected]>
Co-authored-by: Sagar <[email protected]>
Co-authored-by: Sagar Upadhyaya <[email protected]>
Co-authored-by: kkewwei <[email protected]>
Co-authored-by: Marc Handalian <[email protected]>
Co-authored-by: Peter Alfonsi <[email protected]>
Co-authored-by: Peter Alfonsi <[email protected]>
Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com>
Co-authored-by: opensearch-ci-bot <[email protected]>
Co-authored-by: Andrew Ross <[email protected]>
Co-authored-by: panguixin <[email protected]>
Co-authored-by: Bukhtawar Khan <[email protected]>
Co-authored-by: Andriy Redko <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Co-authored-by: Shivansh Arora <[email protected]>
Co-authored-by: Ashish Singh <[email protected]>
Co-authored-by: Liyun Xiu <[email protected]>
Co-authored-by: Vatsal <[email protected]>
Co-authored-by: Daniel Widdis <[email protected]>
Co-authored-by: Siddhant Deshmukh <[email protected]>
Co-authored-by: Robin Friedmann <[email protected]>
Co-authored-by: rishavz_sagar <[email protected]>
Co-authored-by: Sooraj Sinha <[email protected]>
Co-authored-by: Sachin Kale <[email protected]>
Co-authored-by: Sarthak Aggarwal <[email protected]>
  • Loading branch information
1 parent afad5eb commit 7c6b6f9
Show file tree
Hide file tree
Showing 204 changed files with 11,598 additions and 281 deletions.
1 change: 1 addition & 0 deletions .ci/bwcVersions
Original file line number Diff line number Diff line change
Expand Up @@ -34,4 +34,5 @@ BWC_VERSION:
- "2.14.0"
- "2.14.1"
- "2.15.0"
- "2.15.1"
- "2.16.0"
4 changes: 2 additions & 2 deletions .github/CODEOWNERS
Validating CODEOWNERS rules …
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
# 3. Use the command palette to run the CODEOWNERS: Show owners of current file command, which will display all code owners for the current file.

# Default ownership for all repo files
* @anasalkouz @andrross @Bukhtawar @CEHENKLE @dblock @dbwiddis @gbbafna @kotwanikunal @mch2 @msfroh @nknize @owaiskazi19 @reta @Rishikesh1159 @sachinpkale @saratvemulapalli @shwetathareja @sohami @VachaShah
* @anasalkouz @andrross @ashking94 @Bukhtawar @CEHENKLE @dblock @dbwiddis @gbbafna @kotwanikunal @mch2 @msfroh @nknize @owaiskazi19 @reta @Rishikesh1159 @sachinpkale @saratvemulapalli @shwetathareja @sohami @VachaShah

/modules/transport-netty4/ @peternied

Expand All @@ -24,4 +24,4 @@

/.github/ @peternied

/MAINTAINERS.md @anasalkouz @andrross @Bukhtawar @CEHENKLE @dblock @dbwiddis @gbbafna @jed326 @kotwanikunal @mch2 @msfroh @nknize @owaiskazi19 @peternied @reta @Rishikesh1159 @sachinpkale @saratvemulapalli @shwetathareja @sohami @VachaShah
/MAINTAINERS.md @anasalkouz @andrross @ashking94 @Bukhtawar @CEHENKLE @dblock @dbwiddis @gbbafna @jed326 @kotwanikunal @mch2 @msfroh @nknize @owaiskazi19 @peternied @reta @Rishikesh1159 @sachinpkale @saratvemulapalli @shwetathareja @sohami @VachaShah
1 change: 1 addition & 0 deletions .github/workflows/gradle-check.yml
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,7 @@ jobs:
if: success()
uses: codecov/codecov-action@v4
with:
token: ${{ secrets.CODECOV_TOKEN }}
files: ./codeCoverage.xml

- name: Create Comment Success
Expand Down
10 changes: 10 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,9 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
- [Remote Store] Rate limiter for remote store low priority uploads ([#14374](https://github.com/opensearch-project/OpenSearch/pull/14374/))
- Apply the date histogram rewrite optimization to range aggregation ([#13865](https://github.com/opensearch-project/OpenSearch/pull/13865))
- [Writable Warm] Add composite directory implementation and integrate it with FileCache ([12782](https://github.com/opensearch-project/OpenSearch/pull/12782))
- Add batching supported processor base type AbstractBatchingProcessor ([#14554](https://github.com/opensearch-project/OpenSearch/pull/14554))
- Fix race condition while parsing derived fields from search definition ([14445](https://github.com/opensearch-project/OpenSearch/pull/14445))
- Add allowlist setting for ingest-common and search-pipeline-common processors ([#14439](https://github.com/opensearch-project/OpenSearch/issues/14439))

### Dependencies
- Bump `org.gradle.test-retry` from 1.5.8 to 1.5.9 ([#13442](https://github.com/opensearch-project/OpenSearch/pull/13442))
Expand All @@ -24,11 +26,16 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
- Bump `com.gradle.develocity` from 3.17.4 to 3.17.5 ([#14397](https://github.com/opensearch-project/OpenSearch/pull/14397))
- Bump `opentelemetry` from 1.36.0 to 1.39.0 ([#14457](https://github.com/opensearch-project/OpenSearch/pull/14457))
- Bump `azure-identity` from 1.11.4 to 1.13.0, Bump `msal4j` from 1.14.3 to 1.15.1, Bump `msal4j-persistence-extension` from 1.2.0 to 1.3.0 ([#14506](https://github.com/opensearch-project/OpenSearch/pull/14506))
- Bump `com.azure:azure-storage-common` from 12.21.2 to 12.25.1 ([#14517](https://github.com/opensearch-project/OpenSearch/pull/14517))
- Bump `com.microsoft.azure:msal4j` from 1.15.1 to 1.16.0 ([#14610](https://github.com/opensearch-project/OpenSearch/pull/14610))

### Changed
- [Tiered Caching] Move query recomputation logic outside write lock ([#14187](https://github.com/opensearch-project/OpenSearch/pull/14187))
- unsignedLongRangeQuery now returns MatchNoDocsQuery if the lower bounds are greater than the upper bounds ([#14416](https://github.com/opensearch-project/OpenSearch/pull/14416))
- Updated the `indices.query.bool.max_clause_count` setting from being static to dynamically updateable ([#13568](https://github.com/opensearch-project/OpenSearch/pull/13568))
- Make the class CommunityIdProcessor final ([#14448](https://github.com/opensearch-project/OpenSearch/pull/14448))
- Allow @InternalApi annotation on classes not meant to be constructed outside of the OpenSearch core ([#14575](https://github.com/opensearch-project/OpenSearch/pull/14575))
- Add @InternalApi annotation to japicmp exclusions ([#14597](https://github.com/opensearch-project/OpenSearch/pull/14597))

### Deprecated

Expand All @@ -45,6 +52,9 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
- Fix fs info reporting negative available size ([#11573](https://github.com/opensearch-project/OpenSearch/pull/11573))
- Add ListPitInfo::getKeepAlive() getter ([#14495](https://github.com/opensearch-project/OpenSearch/pull/14495))
- Fix FuzzyQuery in keyword field will use IndexOrDocValuesQuery when both of index and doc_value are true ([#14378](https://github.com/opensearch-project/OpenSearch/pull/14378))
- Fix file cache initialization ([#14004](https://github.com/opensearch-project/OpenSearch/pull/14004))
- Handle NPE in GetResult if "found" field is missing ([#14552](https://github.com/opensearch-project/OpenSearch/pull/14552))
- Refactoring FilterPath.parse by using an iterative approach ([#14200](https://github.com/opensearch-project/OpenSearch/pull/14200))

### Security

Expand Down
1 change: 1 addition & 0 deletions MAINTAINERS.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ This document contains a list of maintainers in this repo. See [opensearch-proje
| Anas Alkouz | [anasalkouz](https://github.com/anasalkouz) | Amazon |
| Andrew Ross | [andrross](https://github.com/andrross) | Amazon |
| Andriy Redko | [reta](https://github.com/reta) | Aiven |
| Ashish Singh | [ashking94](https://github.com/ashking94) | Amazon |
| Bukhtawar Khan | [Bukhtawar](https://github.com/Bukhtawar) | Amazon |
| Charlotte Henkle | [CEHENKLE](https://github.com/CEHENKLE) | Amazon |
| Dan Widdis | [dbwiddis](https://github.com/dbwiddis) | Amazon |
Expand Down
15 changes: 14 additions & 1 deletion TESTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,8 @@ OpenSearch uses [jUnit](https://junit.org/junit5/) for testing, it also uses ran
- [Miscellaneous](#miscellaneous)
- [Running verification tasks](#running-verification-tasks)
- [Testing the REST layer](#testing-the-rest-layer)
- [Running REST Tests Against An External Cluster](#running-rest-tests-against-an-external-cluster)
- [Debugging REST Tests](#debugging-rest-tests)
- [Testing packaging](#testing-packaging)
- [Testing packaging on Windows](#testing-packaging-on-windows)
- [Testing VMs are disposable](#testing-vms-are-disposable)
Expand Down Expand Up @@ -272,7 +274,18 @@ yamlRestTest’s and javaRestTest’s are easy to identify, since they are found

If in doubt about which command to use, simply run &lt;gradle path&gt;:check

Note that the REST tests, like all the integration tests, can be run against an external cluster by specifying the `tests.cluster` property, which if present needs to contain a comma separated list of nodes to connect to (e.g. localhost:9300).
## Running REST Tests Against An External Cluster

Note that the REST tests, like all the integration tests, can be run against an external cluster by specifying the following properties `tests.cluster`, `tests.rest.cluster`, `tests.clustername`. Use a comma separated list of node properties for the multi-node cluster.

For example :

./gradlew :rest-api-spec:yamlRestTest \
-Dtests.cluster=localhost:9200 -Dtests.rest.cluster=localhost:9200 -Dtests.clustername=opensearch

## Debugging REST Tests

You can launch a local OpenSearch cluster in debug mode following [Launching and debugging from an IDE](#launching-and-debugging-from-an-ide), and run your REST tests against that following [Running REST Tests Against An External Cluster](#running-rest-tests-against-an-external-cluster).

# Testing packaging

Expand Down
2 changes: 1 addition & 1 deletion buildSrc/version.properties
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
opensearch = 3.0.0
lucene = 9.12.0-snapshot-c896995
lucene = 9.12.0-snapshot-847316d

bundled_jdk_vendor = adoptium
bundled_jdk = 21.0.3+9
Expand Down
4 changes: 4 additions & 0 deletions distribution/src/config/opensearch.yml
Original file line number Diff line number Diff line change
Expand Up @@ -125,3 +125,7 @@ ${path.logs}
# Gates the functionality of enabling Opensearch to use pluggable caches with respective store names via setting.
#
#opensearch.experimental.feature.pluggable.caching.enabled: false
#
# Gates the functionality of star tree index, which improves the performance of search aggregations.
#
#opensearch.experimental.feature.composite_index.star_tree.enabled: true
Original file line number Diff line number Diff line change
Expand Up @@ -473,4 +473,17 @@ public void testPublicApiWithProtectedInterface() {

assertThat(failure.diagnotics(), not(hasItem(matching(Diagnostic.Kind.ERROR))));
}

/**
* The constructor arguments have relaxed semantics at the moment: those could be not annotated or be annotated as {@link InternalApi}
*/
public void testPublicApiConstructorAnnotatedInternalApi() {
final CompilerResult result = compile("PublicApiConstructorAnnotatedInternalApi.java", "NotAnnotated.java");
assertThat(result, instanceOf(Failure.class));

final Failure failure = (Failure) result;
assertThat(failure.diagnotics(), hasSize(2));

assertThat(failure.diagnotics(), not(hasItem(matching(Diagnostic.Kind.ERROR))));
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,9 @@

package org.opensearch.common.annotation.processor;

import org.opensearch.common.annotation.PublicApi;
import org.opensearch.common.annotation.InternalApi;

@PublicApi(since = "1.0.0")
@InternalApi
public class InternalApiAnnotated {

}
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
/*
* SPDX-License-Identifier: Apache-2.0
*
* The OpenSearch Contributors require contributions made to
* this file be licensed under the Apache-2.0 license or a
* compatible open source license.
*/

package org.opensearch.common.annotation.processor;

import org.opensearch.common.annotation.InternalApi;
import org.opensearch.common.annotation.PublicApi;

@PublicApi(since = "1.0.0")
public class PublicApiConstructorAnnotatedInternalApi {
/**
* The constructors have relaxed semantics at the moment: those could be not annotated or be annotated as {@link InternalApi}
*/
@InternalApi
public PublicApiConstructorAnnotatedInternalApi(NotAnnotated arg) {}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
51ff4940eb1024184bbaa5dae39695d2392c5bab

This file was deleted.

3 changes: 2 additions & 1 deletion libs/core/src/main/java/org/opensearch/Version.java
Original file line number Diff line number Diff line change
Expand Up @@ -105,7 +105,8 @@ public class Version implements Comparable<Version>, ToXContentFragment {
public static final Version V_2_14_0 = new Version(2140099, org.apache.lucene.util.Version.LUCENE_9_10_0);
public static final Version V_2_14_1 = new Version(2140199, org.apache.lucene.util.Version.LUCENE_9_10_0);
public static final Version V_2_15_0 = new Version(2150099, org.apache.lucene.util.Version.LUCENE_9_10_0);
public static final Version V_2_16_0 = new Version(2160099, org.apache.lucene.util.Version.LUCENE_9_11_0);
public static final Version V_2_15_1 = new Version(2150199, org.apache.lucene.util.Version.LUCENE_9_10_0);
public static final Version V_2_16_0 = new Version(2160099, org.apache.lucene.util.Version.LUCENE_9_11_1);
public static final Version V_3_0_0 = new Version(3000099, org.apache.lucene.util.Version.LUCENE_9_12_0);
public static final Version CURRENT = V_3_0_0;

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,6 @@
public class FilterPath {

static final FilterPath EMPTY = new FilterPath();

private final String filter;
private final String segment;
private final FilterPath next;
Expand Down Expand Up @@ -99,32 +98,29 @@ public static FilterPath[] compile(Set<String> filters) {

List<FilterPath> paths = new ArrayList<>();
for (String filter : filters) {
if (filter != null) {
if (filter != null && !filter.isEmpty()) {
filter = filter.trim();
if (filter.length() > 0) {
paths.add(parse(filter, filter));
paths.add(parse(filter));
}
}
}
return paths.toArray(new FilterPath[0]);
}

private static FilterPath parse(final String filter, final String segment) {
int end = segment.length();

for (int i = 0; i < end;) {
char c = segment.charAt(i);
private static FilterPath parse(final String filter) {
// Split the filter into segments using a regex
// that avoids splitting escaped dots.
String[] segments = filter.split("(?<!\\\\)\\.");
FilterPath next = EMPTY;

if (c == '.') {
String current = segment.substring(0, i).replaceAll("\\\\.", ".");
return new FilterPath(filter, current, parse(filter, segment.substring(i + 1)));
}
++i;
if ((c == '\\') && (i < end) && (segment.charAt(i) == '.')) {
++i;
}
for (int i = segments.length - 1; i >= 0; i--) {
// Replace escaped dots with actual dots in the current segment.
String segment = segments[i].replaceAll("\\\\.", ".");
next = new FilterPath(filter, segment, next);
}
return new FilterPath(filter, segment.replaceAll("\\\\.", "."), EMPTY);

return next;
}

@Override
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@
import org.opensearch.common.util.set.Sets;
import org.opensearch.test.OpenSearchTestCase;

import java.util.HashSet;
import java.util.Set;

import static java.util.Collections.singleton;
Expand Down Expand Up @@ -369,4 +370,20 @@ public void testMultipleFilterPaths() {
assertThat(filterPath.getSegment(), is(emptyString()));
assertSame(filterPath, FilterPath.EMPTY);
}

public void testCompileWithEmptyString() {
Set<String> filters = new HashSet<>();
filters.add("");
FilterPath[] filterPaths = FilterPath.compile(filters);
assertNotNull(filterPaths);
assertEquals(0, filterPaths.length);
}

public void testCompileWithNull() {
Set<String> filters = new HashSet<>();
filters.add(null);
FilterPath[] filterPaths = FilterPath.compile(filters);
assertNotNull(filterPaths);
assertEquals(0, filterPaths.length);
}
}
Loading

0 comments on commit 7c6b6f9

Please sign in to comment.