Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Star tree] Adding date field rounding support in star tree #15249

Open
wants to merge 16 commits into
base: main
Choose a base branch
from

Conversation

bharath-techie
Copy link
Contributor

@bharath-techie bharath-techie commented Aug 14, 2024

Description

User can give up to three calendar_intervals as part of Date_Dimension as part of star tree mapping. We will round off the value in date field as per the interval. Both nano and millis resolution are rounded off.

Anything greater than or equal to Hour interval will only work for UTC timezone and 00:00 offset.

We will add support for timezones and offsets later. #15266

        "composite": {
            "startree1: {
                "type": "star_tree",
                "config": {
                    "ordered_dimensions": [
                        {
                            "name": "size"
                        }
                    ],
                    "date_dimension": {
                        "name": "date",
                        "calendar_intervals": [
                            "minute",
                            "hour",
                             "day"
                        ]
                    },
                    "metrics": [
                        {
                            "name": "size",
                            "stats": [
                                "sum",
                                "count"
                            ]
                        }
                    ]
                }
            }
        }

User can specify date dimension as above with defaults being 'minute' and 'half an hour'. The date field ( both milliseconds and nanoseconds resolution ) will get rounded off to the associated calendar interval.

Apart from date histogram calendar intervals,
User can provide quarter of hour / half an hour as part of mapping .

15m (or) quarter-hour
30m (or) half-hour

This is done to handle all timezones.
15m interval will handle all timezones and 30m interval will handle 99% of timezones bar the ones which has :45 minute part as part of timezone.

Related Issues

Resolves #15265

Check List

  • Functionality includes testing.
  • API changes companion pull request created, if applicable.
  • Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Copy link
Contributor

❌ Gradle check result for 5fdce71: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

❌ Gradle check result for 23e600a: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

❌ Gradle check result for 6fbaf1b: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@github-actions github-actions bot added the enhancement Enhancement or improvement to existing feature or request label Aug 15, 2024
Copy link
Contributor

github-actions bot commented Sep 3, 2024

❕ Gradle check result for 4387345: UNSTABLE

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

Copy link

codecov bot commented Sep 3, 2024

Codecov Report

Attention: Patch coverage is 92.68293% with 15 lines in your changes missing coverage. Please review.

Project coverage is 71.86%. Comparing base (d1fd47c) to head (8215865).
Report is 3 commits behind head on main.

Files with missing lines Patch % Lines
...va/org/opensearch/index/mapper/StarTreeMapper.java 84.84% 2 Missing and 3 partials ⚠️
...h/index/compositeindex/datacube/ReadDimension.java 50.00% 2 Missing ⚠️
...index/datacube/startree/StarTreeIndexSettings.java 71.42% 1 Missing and 1 partial ⚠️
...datacube/startree/builder/BaseStarTreeBuilder.java 95.55% 1 Missing and 1 partial ⚠️
...acube/startree/utils/date/DateTimeUnitAdapter.java 80.00% 0 Missing and 2 partials ⚠️
...ndex/compositeindex/datacube/DimensionFactory.java 83.33% 0 Missing and 1 partial ⚠️
...ompositeindex/datacube/startree/StarTreeField.java 94.44% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##               main   #15249      +/-   ##
============================================
- Coverage     71.87%   71.86%   -0.01%     
- Complexity    64528    64542      +14     
============================================
  Files          5296     5298       +2     
  Lines        301744   301867     +123     
  Branches      43585    43603      +18     
============================================
+ Hits         216864   216926      +62     
- Misses        67014    67164     +150     
+ Partials      17866    17777      -89     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Contributor

github-actions bot commented Sep 9, 2024

✅ Gradle check result for 9a4e7f3: SUCCESS

Copy link
Contributor

❌ Gradle check result for eb43e37: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

❌ Gradle check result for 93760bc: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Signed-off-by: Bharathwaj G <[email protected]>
Copy link
Contributor

github-actions bot commented Oct 1, 2024

❕ Gradle check result for d34a36f: UNSTABLE

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

Signed-off-by: Bharathwaj G <[email protected]>
Copy link
Contributor

github-actions bot commented Oct 4, 2024

❌ Gradle check result for 309f539: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

github-actions bot commented Oct 4, 2024

❌ Gradle check result for d9e7852: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

github-actions bot commented Oct 4, 2024

✅ Gradle check result for 8215865: SUCCESS

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Enhancement or improvement to existing feature or request Indexing:Performance skip-changelog
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Star tree] Rounding off date field in Star tree
4 participants