[Star Tree] [Search] Resolving Range aggregations with Star-tree

[sandeshkr419]

Description

Resolving numeric range aggregation queries via star-tree.
Supporting numeric range aggregations with/without metric sub-aggregations.

Temp Notes

This PR contains unmerged changes keywords aggregation PR from as well. Kindly only look into range aggs last commit changes commit while reviewing the changes for now.

(Depending changes merged)

Related Issues

Resolves #16553

Check List

• Functionality includes testing.
• API changes companion pull request created, if applicable.
• Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[github-actions]

❌ Gradle check result for 4e15156: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[sandeshkr419]

Sharing some benchmarking results with http_logs data:

Range Aggregation with a sub-metric aggregation:

Request:

{ "size":0,
    "aggs": {
        "low_value": {
            "range": {
                "field": "status",
                "ranges": [
                    {
                        "from": 200,
                        "to": 201
                    },
                    {
                        "from": 202,
                        "to": 203
                    },
                    {
                        "from": 204,
                        "to": 400
                    }
                ]
            }
        }
    }
}

Default (un-optimized code flow):

|                                        50th percentile latency | multi_term_agg |     538.971 |     ms |
|                                        90th percentile latency | multi_term_agg |     555.397 |     ms |
|                                        99th percentile latency | multi_term_agg |      591.43 |     ms |
|                                       100th percentile latency | multi_term_agg |     651.087 |     ms |
|                                   50th percentile service time | multi_term_agg |     535.287 |     ms |
|                                   90th percentile service time | multi_term_agg |     552.762 |     ms |
|                                   99th percentile service time | multi_term_agg |     589.559 |     ms |
|                                  100th percentile service time | multi_term_agg |     649.834 |     ms |
|                                                     error rate | multi_term_agg |           0 |      % |

Star Tree Optimized:

|                                        50th percentile latency | multi_term_agg |     32.1939 |     ms |
|                                        90th percentile latency | multi_term_agg |     50.7508 |     ms |
|                                        99th percentile latency | multi_term_agg |     70.5129 |     ms |
|                                       100th percentile latency | multi_term_agg |     127.205 |     ms |
|                                   50th percentile service time | multi_term_agg |     29.0528 |     ms |
|                                   90th percentile service time | multi_term_agg |     47.8311 |     ms |
|                                   99th percentile service time | multi_term_agg |     69.6054 |     ms |
|                                  100th percentile service time | multi_term_agg |     125.448 |     ms |
|                                                     error rate | multi_term_agg |           0 |      % |

Since there is a pre-compute optimization in place already, so benchmarking its performance over star-tree as well:

Request:

{
    "size": 0,
    "aggs": {
      "status_ranges": {
        "range": {
          "field": "status",
          "ranges": [
            { "from": 200, "to": 201 },
            { "from": 201, "to": 205 },
            { "from": 205, "to": 206 },
            { "from": 206, "to": 500 }
          ]
        }
      }
    }

Non-star Tree Optimization (filter rewrite):
(match all, no sub-aggs)

|                                        50th percentile latency | multi_term_agg |     19.9869 |     ms |
|                                        90th percentile latency | multi_term_agg |     28.2602 |     ms |
|                                        99th percentile latency | multi_term_agg |     36.7925 |     ms |
|                                       100th percentile latency | multi_term_agg |     39.9778 |     ms |
|                                   50th percentile service time | multi_term_agg |     18.0554 |     ms |
|                                   90th percentile service time | multi_term_agg |     25.5458 |     ms |
|                                   99th percentile service time | multi_term_agg |     33.7608 |     ms |
|                                  100th percentile service time | multi_term_agg |      37.165 |     ms |
|                                                     error rate | multi_term_agg |           0 |      % |

Star Tree Optimization:

|                                        50th percentile latency | multi_term_agg |     29.7188 |     ms |
|                                        90th percentile latency | multi_term_agg |     56.7995 |     ms |
|                                        99th percentile latency | multi_term_agg |     72.3995 |     ms |
|                                       100th percentile latency | multi_term_agg |     76.3429 |     ms |
|                                   50th percentile service time | multi_term_agg |     27.1909 |     ms |
|                                   90th percentile service time | multi_term_agg |     53.3197 |     ms |
|                                   99th percentile service time | multi_term_agg |     69.6937 |     ms |
|                                  100th percentile service time | multi_term_agg |     73.5412 |     ms |
|                                                     error rate | multi_term_agg |           0 |      % |

Based on the pereformance numbers, keeping the already non-star-tree pre-computation on higher priority over star-tree pre-computation.

[github-actions]

❌ Gradle check result for 2b4bfab: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for 549d737: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for 2dda9b3: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for 6b38984: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for 6b38984: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for e2ef393: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

✅ Gradle check result for e2ef393: SUCCESS

[codecov]

Codecov Report

Attention: Patch coverage is 88.34951% with 12 lines in your changes missing coverage. Please review.

Project coverage is 72.46%. Comparing base (7388205) to head (5e92828).

Files with missing lines	Patch %	Lines
...rch/aggregations/bucket/range/RangeAggregator.java	88.88%	3 Missing and 7 partials ⚠️
...ensearch/search/startree/StarTreeQueryContext.java	84.61%	1 Missing and 1 partial ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main   #17273      +/-   ##
============================================
+ Coverage     71.54%   72.46%   +0.91%     
- Complexity    64883    65744     +861     
============================================
  Files          5311     5311              
  Lines        304903   304975      +72     
  Branches      44215    44229      +14     
============================================
+ Hits         218133   220990    +2857     
+ Misses        68896    65893    -3003     
- Partials      17874    18092     +218     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[github-actions]

❌ Gradle check result for 89e5626: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

✅ Gradle check result for 5e92828: SUCCESS