Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extensible design to add new query and field type support for Star Tree #17100

Closed
Closed
Changes from 1 commit
Commits
Show all changes
51 commits
Select commit Hold shift + click to select a range
e467a26
Initial changes
expani Jan 13, 2025
5b97c12
Initial working commit for numeric metric aggregation
expani Jan 15, 2025
b43e559
Added changes to support range query
expani Jan 17, 2025
ea5c151
Refactoring getDimension and changed ordinal fetch for keyword
expani Jan 17, 2025
931ac21
Handle conversion from string in QB to bytesref for dv lookup
expani Jan 17, 2025
f76ccae
Added support for float and double with same conversion as mapped fd …
expani Jan 20, 2025
b6d45ae
Refactoring
expani Jan 20, 2025
b7be244
Refactored code some more
expani Jan 20, 2025
4b3a867
Refactoring
expani Jan 20, 2025
6647ba6
Handle coercing properly for decimal and non-decimal types
expani Jan 20, 2025
e4e69b1
Introduced Mapper for StarTree to be used during conversion from QB t…
expani Jan 20, 2025
503881f
Removed some older code, replaced test with new interface and impleme…
expani Jan 21, 2025
b6767c4
Fixed existing tests and removed older code
expani Jan 22, 2025
bcafafc
Minor fix for range search in star tree index
expani Jan 22, 2025
f6bd9c9
Changed a condition after some pondering
expani Jan 22, 2025
8fc4b00
Moved Ordinal method into STFMapper and fixed more tests due to the same
expani Jan 23, 2025
3c18ae3
Refactoring and fixed some edge cases
expani Jan 23, 2025
be8e6e7
Small rename
expani Jan 23, 2025
f9f477b
Fixed cases where from > to in range
expani Jan 23, 2025
b5de98e
Fixed range binary search and ordinal finding logic
expani Jan 23, 2025
f412529
Skipping binary range search in ST if low doesn't exists
expani Jan 23, 2025
5959844
Fixed some boundary conditions and increased coverage
expani Jan 24, 2025
95345b0
Fixed keyword range ordinal seek status logic and added keyword in tests
expani Jan 24, 2025
bcc4c29
Removed sysout
expani Jan 24, 2025
b4e520b
Adding unchecked files
expani Jan 25, 2025
876810b
Using advanceExact instead of advance due to AssertionError at Indexe…
expani Jan 26, 2025
954ab30
Removed sysout
expani Jan 26, 2025
978d761
Fix exists queries on nested flat_object fields throw exception (#16803)
kkewwei Jan 23, 2025
6568f65
Updates version to fix BWC for SearchSourceBuilder (#17098)
junweid62 Jan 23, 2025
0b343c2
Introduce Template query (#16818)
mingshl Jan 23, 2025
6360c75
Propagate includes and excludes from fetchSourceContext to FieldsVisi…
Gankris96 Jan 24, 2025
32f8bc3
Fix exists query for flat object (#17108)
bugmakerrrrrr Jan 24, 2025
8c041bb
Added new Setting property `UnmodifiableOnRestore` to prevent updatin…
anntians Jan 24, 2025
f85f69a
Fix Binlong's name in MAINTAINERS.md (#17113)
andrross Jan 24, 2025
893a0cb
Move o.o.action.support.master classes (#17104)
andrross Jan 24, 2025
ff9fb90
Stop processing search requests when _msearch is canceled (#17005)
msfroh Jan 24, 2025
15e622f
Refactor bootstrap for JPMS support (#17117)
prudhvigodithi Jan 25, 2025
cff7284
Added support new field types and query types in metric aggregator tests
expani Jan 27, 2025
2022439
Added test support for decimal fields
expani Jan 27, 2025
df406da
Spotless and removed fixed TODOs
expani Jan 27, 2025
9f9a3e6
Trying with reducing iterations to prevent OOM
expani Jan 27, 2025
e7df311
Added support to generate star tree field config by method params
expani Jan 27, 2025
87f3988
Removed excessive instanceOf checks and added initials FLSTN test
expani Jan 27, 2025
e2f6dbc
Added more cases to make binarySearch for term/terms 100% coverage
expani Jan 27, 2025
6c99381
Added test for startreenode range match
expani Jan 27, 2025
a92f0bd
Added random test for binary exact and range match
expani Jan 27, 2025
aa48350
Added Java docs and unit tests for ordinalLookup in Keyword field
expani Jan 27, 2025
4fcde4b
Formatting
expani Jan 27, 2025
228cffa
Using Set instead of List for StarTree and Dimension Filters, Added m…
expani Jan 27, 2025
ff5c9d6
Added package info files
expani Jan 27, 2025
629866d
Removed equals and hashCode from filters and changed back to using List
expani Jan 27, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Introduced Mapper for StarTree to be used during conversion from QB t…
…o DF

Signed-off-by: expani <anijainc@amazon.com>
expani committed Jan 27, 2025
commit e4e69b18e5ad717f9677d1486b2ade10a0f34567
Original file line number Diff line number Diff line change
@@ -38,11 +38,14 @@ public class StarTreeQueryContext {
* This is used to cache the results for each leaf reader context
* to avoid reading the filtered values from the leaf reader context multiple times
*/
// TODO : Change caching to be based on aggregation specific filters.
private final FixedBitSet[] starTreeValues;

private final QueryBuilder baseQueryBuilder;
private StarTreeFilter baseStarTreeFilter;

// TODO : Implement storing and aggregating aggregation specific filters.

public StarTreeQueryContext(SearchContext context, QueryBuilder baseQueryBuilder) {
this.baseQueryBuilder = baseQueryBuilder;
// TODO : We need to select the most appropriate one from multiple star tree field types.
Original file line number Diff line number Diff line change
@@ -0,0 +1,291 @@
/*
* SPDX-License-Identifier: Apache-2.0
*
* The OpenSearch Contributors require contributions made to
* this file be licensed under the Apache-2.0 license or a
* compatible open source license.
*/

package org.opensearch.search.startree.filter.provider;

import org.apache.lucene.document.DoublePoint;
import org.apache.lucene.document.FloatPoint;
import org.apache.lucene.sandbox.document.HalfFloatPoint;
import org.apache.lucene.util.BytesRef;
import org.apache.lucene.util.NumericUtils;
import org.opensearch.common.lucene.BytesRefs;
import org.opensearch.common.lucene.Lucene;
import org.opensearch.index.mapper.KeywordFieldMapper.KeywordFieldType;
import org.opensearch.index.mapper.MappedFieldType;
import org.opensearch.index.mapper.NumberFieldMapper;
import org.opensearch.index.mapper.NumberFieldMapper.NumberFieldType;
import org.opensearch.search.startree.filter.DimensionFilter;
import org.opensearch.search.startree.filter.ExactMatchDimFilter;
import org.opensearch.search.startree.filter.MatchNoneFilter;
import org.opensearch.search.startree.filter.RangeMatchDimFilter;

import java.util.ArrayList;
import java.util.List;

import static org.opensearch.index.mapper.NumberFieldMapper.NumberType.hasDecimalPart;
import static org.opensearch.index.mapper.NumberFieldMapper.NumberType.signum;

public interface StarTreeFilterMapper {
DimensionFilter getExactMatchFilter(MappedFieldType mappedFieldType, List<Object> rawValues);

DimensionFilter getRangeMatchFilter(
MappedFieldType mappedFieldType,
Object rawLow,
Object rawHigh,
boolean includeLow,
boolean includeHigh
);

class Factory {
// TODO : Reuse instances.
public static StarTreeFilterMapper fromMappedFieldType(MappedFieldType mappedFieldType) {
if (mappedFieldType instanceof org.opensearch.index.mapper.KeywordFieldMapper.KeywordFieldType) {
return new org.opensearch.search.startree.filter.provider.KeywordFieldMapper();
} else if (mappedFieldType instanceof NumberFieldMapper.NumberFieldType) {
NumberFieldMapper.NumberType numberType = ((NumberFieldMapper.NumberFieldType) mappedFieldType).numberType();
switch (numberType) {
case BYTE:
case SHORT:
case INTEGER:
return new IntegerFieldMapper();
case LONG:
return new SignedLongFieldMapper();
case HALF_FLOAT:
return new HalfFloatFieldMapper();
case FLOAT:
return new FloatFieldMapper();
case DOUBLE:
return new DoubleFieldMapper();
default:
return null;
}
} else {
return null;
}
}
}

}

abstract class NonDecimalNumberFieldMapper implements StarTreeFilterMapper {

@Override
public DimensionFilter getExactMatchFilter(MappedFieldType mappedFieldType, List<Object> rawValues) {
if (mappedFieldType instanceof NumberFieldType) {
NumberFieldType numberFieldType = (NumberFieldType) mappedFieldType;
List<Object> convertedValues = new ArrayList<>(rawValues.size());
for (Object rawValue : rawValues) {
convertedValues.add(numberFieldType.numberType().parse(rawValue, true));
}
return new ExactMatchDimFilter(mappedFieldType.name(), convertedValues);
}
return null;
}

@Override
public DimensionFilter getRangeMatchFilter(
MappedFieldType mappedFieldType,
Object rawLow,
Object rawHigh,
boolean includeLow,
boolean includeHigh
) {
if (mappedFieldType instanceof NumberFieldType) {
NumberFieldType numberFieldType = (NumberFieldType) mappedFieldType;

Long parsedLow = rawLow == null ? defaultMinimum() : numberFieldType.numberType().parse(rawLow, true).longValue();
Long parsedHigh = rawHigh == null ? defaultMaximum() : numberFieldType.numberType().parse(rawHigh, true).longValue();

boolean lowerTermHasDecimalPart = hasDecimalPart(parsedLow);
if ((lowerTermHasDecimalPart == false && includeLow == false) || (lowerTermHasDecimalPart && signum(parsedLow) > 0)) {
if (parsedLow.equals(defaultMaximum())) {
return new MatchNoneFilter();
}
++parsedLow;
}
boolean upperTermHasDecimalPart = hasDecimalPart(parsedHigh);
if ((upperTermHasDecimalPart == false && includeHigh == false) || (upperTermHasDecimalPart && signum(parsedHigh) < 0)) {
if (parsedHigh.equals(defaultMinimum())) {
return new MatchNoneFilter();
}
--parsedHigh;
}
return new RangeMatchDimFilter(mappedFieldType.name(), parsedLow, parsedHigh, true, true);
}
return null;
}

abstract Long defaultMinimum();

abstract Long defaultMaximum();

}

class IntegerFieldMapper extends NonDecimalNumberFieldMapper {
@Override
Long defaultMinimum() {
return (long) Integer.MIN_VALUE;
}

@Override
Long defaultMaximum() {
return (long) Integer.MAX_VALUE;
}
}

class SignedLongFieldMapper extends NonDecimalNumberFieldMapper {
@Override
Long defaultMinimum() {
return Long.MIN_VALUE;
}

@Override
Long defaultMaximum() {
return Long.MAX_VALUE;
}
}

abstract class DecimalNumberFieldMapper implements StarTreeFilterMapper {

@Override
public DimensionFilter getExactMatchFilter(MappedFieldType mappedFieldType, List<Object> rawValues) {
if (mappedFieldType instanceof NumberFieldType) {
NumberFieldType numberFieldType = (NumberFieldType) mappedFieldType;
List<Object> convertedValues = new ArrayList<>(rawValues.size());
for (Object rawValue : rawValues) {
convertedValues.add(convertToDocValues(numberFieldType.numberType().parse(rawValue, true)));
}
return new ExactMatchDimFilter(mappedFieldType.name(), convertedValues);
}
return null;
}

@Override
public DimensionFilter getRangeMatchFilter(
MappedFieldType mappedFieldType,
Object rawLow,
Object rawHigh,
boolean includeLow,
boolean includeHigh
) {
if (mappedFieldType instanceof NumberFieldType) {
NumberFieldType numberFieldType = (NumberFieldType) mappedFieldType;
Number l = Long.MIN_VALUE;
Number u = Long.MAX_VALUE;
if (rawLow != null) {
l = numberFieldType.numberType().parse(rawLow, false);
if (includeLow == false) {
l = getNextLowOrHigh(l, true);
}
l = convertToDocValues(l);
}
if (rawHigh != null) {
u = numberFieldType.numberType().parse(rawHigh, false);
if (includeHigh == false) {
u = getNextLowOrHigh(u, false);
}
u = convertToDocValues(u);
}
return new RangeMatchDimFilter(numberFieldType.name(), l, u, true, true);
}
return null;
}

abstract long convertToDocValues(Number parsedValue);

abstract Number getNextLowOrHigh(Number parsedValue, boolean nextHighest);

}

class HalfFloatFieldMapper extends DecimalNumberFieldMapper {
@Override
long convertToDocValues(Number parsedValue) {
return HalfFloatPoint.halfFloatToSortableShort((Float) parsedValue);
}

@Override
Number getNextLowOrHigh(Number parsedValue, boolean nextHighest) {
return nextHighest ? HalfFloatPoint.nextUp((Float) parsedValue) : HalfFloatPoint.nextDown((Float) parsedValue);
}
}

class FloatFieldMapper extends DecimalNumberFieldMapper {
@Override
long convertToDocValues(Number parsedValue) {
return NumericUtils.floatToSortableInt((Float) parsedValue);
}

@Override
Number getNextLowOrHigh(Number parsedValue, boolean nextHighest) {
return nextHighest ? FloatPoint.nextUp((Float) parsedValue) : FloatPoint.nextDown((Float) parsedValue);
}
}

class DoubleFieldMapper extends DecimalNumberFieldMapper {
@Override
long convertToDocValues(Number parsedValue) {
return NumericUtils.doubleToSortableLong((Double) parsedValue);
}

@Override
Number getNextLowOrHigh(Number parsedValue, boolean nextHighest) {
return nextHighest ? DoublePoint.nextUp((Double) parsedValue) : DoublePoint.nextDown((Double) parsedValue);
}
}

class KeywordFieldMapper implements StarTreeFilterMapper {

@Override
public DimensionFilter getExactMatchFilter(MappedFieldType mappedFieldType, List<Object> rawValues) {
if (mappedFieldType instanceof KeywordFieldType) {
KeywordFieldType keywordFieldType = (KeywordFieldType) mappedFieldType;
List<Object> convertedValues = new ArrayList<>(rawValues.size());
for (Object rawValue : rawValues) {
convertedValues.add(parseRawKeyword(mappedFieldType.name(), rawValue, keywordFieldType));
}
return new ExactMatchDimFilter(mappedFieldType.name(), convertedValues);
}
return null;
}

@Override
public DimensionFilter getRangeMatchFilter(
MappedFieldType mappedFieldType,
Object rawLow,
Object rawHigh,
boolean includeLow,
boolean includeHigh
) {
if (mappedFieldType instanceof KeywordFieldType) {
KeywordFieldType keywordFieldType = (KeywordFieldType) mappedFieldType;
return new RangeMatchDimFilter(
mappedFieldType.name(),
parseRawKeyword(mappedFieldType.name(), rawLow, keywordFieldType),
parseRawKeyword(mappedFieldType.name(), rawHigh, keywordFieldType),
includeLow,
includeHigh
);
}
return null;
}

// TODO : Think around making TermBasedFT#indexedValueForSearch() public for reuse here.
private Object parseRawKeyword(String field, Object rawValue, KeywordFieldType keywordFieldType) {
Object parsedValue;
if (keywordFieldType.getTextSearchInfo().getSearchAnalyzer() == Lucene.KEYWORD_ANALYZER) {
parsedValue = BytesRefs.toBytesRef(rawValue);
} else {
if (rawValue instanceof BytesRef) {
rawValue = ((BytesRef) rawValue).utf8ToString();
}
parsedValue = keywordFieldType.getTextSearchInfo().getSearchAnalyzer().normalize(field, rawValue.toString());
}
return parsedValue;
}

}
Original file line number Diff line number Diff line change
@@ -8,86 +8,31 @@

package org.opensearch.search.startree.filter.provider;

import org.apache.lucene.document.DoublePoint;
import org.apache.lucene.document.FloatPoint;
import org.apache.lucene.sandbox.document.HalfFloatPoint;
import org.apache.lucene.search.MatchNoDocsQuery;
import org.apache.lucene.search.Query;
import org.apache.lucene.util.BytesRef;
import org.apache.lucene.util.NumericUtils;
import org.opensearch.common.annotation.ExperimentalApi;
import org.opensearch.common.lucene.BytesRefs;
import org.opensearch.common.lucene.Lucene;
import org.opensearch.index.compositeindex.datacube.Dimension;
import org.opensearch.index.mapper.CompositeDataCubeFieldType;
import org.opensearch.index.mapper.KeywordFieldMapper;
import org.opensearch.index.mapper.MappedFieldType;
import org.opensearch.index.mapper.NumberFieldMapper;
import org.opensearch.index.query.QueryBuilder;
import org.opensearch.index.query.RangeQueryBuilder;
import org.opensearch.index.query.TermQueryBuilder;
import org.opensearch.index.query.TermsQueryBuilder;
import org.opensearch.search.internal.SearchContext;
import org.opensearch.search.startree.filter.StarTreeFilter;
import org.opensearch.search.startree.StarTreeQueryHelper;
import org.opensearch.search.startree.filter.DimensionFilter;
import org.opensearch.search.startree.filter.ExactMatchDimFilter;
import org.opensearch.search.startree.filter.MatchNoneFilter;
import org.opensearch.search.startree.filter.RangeMatchDimFilter;
import org.opensearch.search.startree.filter.StarTreeFilter;

import java.io.IOException;
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
import java.util.Map;

import static org.opensearch.index.mapper.NumberFieldMapper.NumberType.hasDecimalPart;
import static org.opensearch.index.mapper.NumberFieldMapper.NumberType.signum;

@ExperimentalApi
public interface StarTreeFilterProvider {

StarTreeFilter getFilter(SearchContext context, QueryBuilder rawFilter, CompositeDataCubeFieldType compositeFieldType)
throws IOException;

private static long parseRawNumberToDVLong(Object rawValue, NumberFieldMapper.NumberType numberType) {
Object parsedValue;
switch (numberType) {
case BYTE:
case SHORT:
case INTEGER:
parsedValue = numberType.parse(rawValue, true);
return Long.parseLong(parsedValue.toString());
case LONG:
parsedValue = numberType.parse(rawValue, true);
return (long) parsedValue;
case HALF_FLOAT:
parsedValue = numberType.parse(rawValue, false);
return HalfFloatPoint.halfFloatToSortableShort((Float) parsedValue);
case FLOAT:
parsedValue = numberType.parse(rawValue, false);
return NumericUtils.floatToSortableInt((Float) parsedValue);
case DOUBLE:
parsedValue = numberType.parse(rawValue, false);
return NumericUtils.doubleToSortableLong((Double) parsedValue);
default:
throw new UnsupportedOperationException("Unsupported field type [" + numberType + "]");
}
}

private static Object parseRawKeyword(String field, Object rawValue, KeywordFieldMapper.KeywordFieldType keywordFieldType) {
Object parsedValue;
if (keywordFieldType.getTextSearchInfo().getSearchAnalyzer() == Lucene.KEYWORD_ANALYZER) {
parsedValue = BytesRefs.toBytesRef(rawValue);
} else {
if (rawValue instanceof BytesRef) {
rawValue = ((BytesRef) rawValue).utf8ToString();
}
parsedValue = keywordFieldType.getTextSearchInfo().getSearchAnalyzer().normalize(field, rawValue.toString());
}
return parsedValue;
}

class SingletonFactory {

private static final Map<Class<? extends QueryBuilder>, StarTreeFilterProvider> QUERY_BUILDERS_TO_STF_PROVIDER = Map.of(
@@ -115,25 +60,21 @@ public StarTreeFilter getFilter(SearchContext context, QueryBuilder rawFilter, C
TermQueryBuilder termQueryBuilder = (TermQueryBuilder) rawFilter;
String field = termQueryBuilder.fieldName();
MappedFieldType mappedFieldType = context.mapperService().fieldType(field);
Object term;
StarTreeFilterMapper starTreeFilterMapper = StarTreeFilterMapper.Factory.fromMappedFieldType(mappedFieldType);
Dimension matchedDimension = StarTreeQueryHelper.getMatchingDimensionOrNull(field, compositeFieldType.getDimensions());
if (mappedFieldType.getClass().equals(NumberFieldMapper.NumberFieldType.class)) {
if (matchedDimension == null || mappedFieldType == null) {
return null; // Indicates Aggregators to fallback to default implementation.
} else {
// FIXME : DocValuesType validation is field type specific and not query builder specific should happen elsewhere.
Query query = termQueryBuilder.toQuery(context.getQueryShardContext());
if (query instanceof MatchNoDocsQuery) {
return null; // Indicates Aggregators to fallback to default implementation.
return new StarTreeFilter(Collections.emptyMap());
} else {
return new StarTreeFilter(
Map.of(field, List.of(starTreeFilterMapper.getExactMatchFilter(mappedFieldType, List.of(termQueryBuilder.value()))))
);
}
NumberFieldMapper.NumberFieldType numFieldType = (NumberFieldMapper.NumberFieldType) mappedFieldType;
term = parseRawNumberToDVLong(termQueryBuilder.value(), numFieldType.numberType());
} else if (mappedFieldType.getClass().equals(KeywordFieldMapper.KeywordFieldType.class)) {
KeywordFieldMapper.KeywordFieldType keywordFieldType = (KeywordFieldMapper.KeywordFieldType) mappedFieldType;
term = parseRawKeyword(field, termQueryBuilder.value(), keywordFieldType);
} else {
return null;
}
// FIXME : DocValuesType validation is field type specific and not query builder specific should happen elsewhere.
return matchedDimension == null
? null
: new StarTreeFilter(Map.of(field, List.of(new ExactMatchDimFilter(field, List.of(term)))));
}
}

@@ -144,29 +85,18 @@ public StarTreeFilter getFilter(SearchContext context, QueryBuilder rawFilter, C
TermsQueryBuilder termsQueryBuilder = (TermsQueryBuilder) rawFilter;
String field = termsQueryBuilder.fieldName();
Dimension matchedDimension = StarTreeQueryHelper.getMatchingDimensionOrNull(field, compositeFieldType.getDimensions());
if (matchedDimension == null) {
MappedFieldType mappedFieldType = context.mapperService().fieldType(field);
StarTreeFilterMapper starTreeFilterMapper = StarTreeFilterMapper.Factory.fromMappedFieldType(mappedFieldType);
if (matchedDimension == null || mappedFieldType == null) {
return null; // Indicates Aggregators to fallback to default implementation.
} else {
MappedFieldType mappedFieldType = context.mapperService().fieldType(field);
Query query = termsQueryBuilder.toQuery(context.getQueryShardContext());
if (query instanceof MatchNoDocsQuery) {
return new StarTreeFilter(Collections.emptyMap());
} else {
List<Object> convertedValues = new ArrayList<>(termsQueryBuilder.values().size());
if (mappedFieldType.getClass().equals(NumberFieldMapper.NumberFieldType.class)) {
NumberFieldMapper.NumberFieldType numFieldType = (NumberFieldMapper.NumberFieldType) mappedFieldType;
for (Object rawValue : termsQueryBuilder.values()) {
convertedValues.add(parseRawNumberToDVLong(rawValue, numFieldType.numberType()));
}
} else if (mappedFieldType.getClass().equals(KeywordFieldMapper.KeywordFieldType.class)) {
KeywordFieldMapper.KeywordFieldType keywordFieldType = (KeywordFieldMapper.KeywordFieldType) mappedFieldType;
for (Object rawValue : termsQueryBuilder.values()) {
convertedValues.add(parseRawKeyword(field, rawValue, keywordFieldType));
}
} else {
return null; // Fallback to default implementation as mapped type is not yet supported.
}
return new StarTreeFilter(Map.of(field, List.of(new ExactMatchDimFilter(field, convertedValues))));
return new StarTreeFilter(
Map.of(field, List.of(starTreeFilterMapper.getExactMatchFilter(mappedFieldType, termsQueryBuilder.values())))
);
}
}
}
@@ -180,121 +110,33 @@ public StarTreeFilter getFilter(SearchContext context, QueryBuilder rawFilter, C
RangeQueryBuilder rangeQueryBuilder = (RangeQueryBuilder) rawFilter;
String field = rangeQueryBuilder.fieldName();
Dimension matchedDimension = StarTreeQueryHelper.getMatchingDimensionOrNull(field, compositeFieldType.getDimensions());
if (matchedDimension == null) {
return null;
}
MappedFieldType mappedFieldType = context.mapperService().fieldType(field);
Query query = rangeQueryBuilder.toQuery(context.getQueryShardContext());
if (query instanceof MatchNoDocsQuery) {
return new StarTreeFilter(Collections.emptyMap());
if (matchedDimension == null || mappedFieldType == null) {
return null;
} else {
if (mappedFieldType.getClass().equals(NumberFieldMapper.NumberFieldType.class)) {
NumberFieldMapper.NumberFieldType numFieldType = (NumberFieldMapper.NumberFieldType) mappedFieldType;
DimensionFilter dimensionFilter;
switch (numFieldType.numberType()) {
case BYTE:
case SHORT:
case INTEGER:
case LONG:
dimensionFilter = getRangeFilterForNonDecimals(rangeQueryBuilder, numFieldType.numberType());
break;
case HALF_FLOAT:
case FLOAT:
case DOUBLE:
dimensionFilter = getRangeFilterForDecimals(rangeQueryBuilder, numFieldType.numberType());
break;
default:
return null;
}
return new StarTreeFilter(Map.of(field, List.of(dimensionFilter)));
} else if (mappedFieldType.getClass().equals(KeywordFieldMapper.KeywordFieldType.class)) {
KeywordFieldMapper.KeywordFieldType keywordFieldType = (KeywordFieldMapper.KeywordFieldType) mappedFieldType;
Query query = rangeQueryBuilder.toQuery(context.getQueryShardContext());
if (query instanceof MatchNoDocsQuery) {
return new StarTreeFilter(Collections.emptyMap());
} else {
StarTreeFilterMapper starTreeFilterMapper = StarTreeFilterMapper.Factory.fromMappedFieldType(mappedFieldType);
return new StarTreeFilter(
Map.of(
field,
List.of(
new RangeMatchDimFilter(
field,
parseRawKeyword(field, rangeQueryBuilder.from(), keywordFieldType),
parseRawKeyword(field, rangeQueryBuilder.to(), keywordFieldType),
starTreeFilterMapper.getRangeMatchFilter(
mappedFieldType,
rangeQueryBuilder.from(),
rangeQueryBuilder.to(),
rangeQueryBuilder.includeLower(),
rangeQueryBuilder.includeUpper()
)
)
)
);
}
return null;
}
}

private static DimensionFilter getRangeFilterForNonDecimals(
RangeQueryBuilder rangeQueryBuilder,
NumberFieldMapper.NumberType numFieldType
) {
Long low = rangeQueryBuilder.from() == null ? Long.MIN_VALUE : parseRawNumberToDVLong(rangeQueryBuilder.from(), numFieldType);
Long high = rangeQueryBuilder.to() == null ? Long.MAX_VALUE : parseRawNumberToDVLong(rangeQueryBuilder.to(), numFieldType);
boolean lowerTermHasDecimalPart = hasDecimalPart(low);
if ((lowerTermHasDecimalPart == false && rangeQueryBuilder.includeLower() == false)
|| (lowerTermHasDecimalPart && signum(low) > 0)) {
if (low.equals(Long.MAX_VALUE)) {
return new MatchNoneFilter();
}
++low;
}
boolean upperTermHasDecimalPart = hasDecimalPart(high);
if ((upperTermHasDecimalPart == false && rangeQueryBuilder.includeUpper() == false)
|| (upperTermHasDecimalPart && signum(high) < 0)) {
if (high.equals(Long.MIN_VALUE)) {
return new MatchNoneFilter();
}
--high;
}
return new RangeMatchDimFilter(rangeQueryBuilder.fieldName(), low, high, true, true);
}

private DimensionFilter getRangeFilterForDecimals(RangeQueryBuilder rangeQueryBuilder, NumberFieldMapper.NumberType numFieldType) {
Number l = Long.MIN_VALUE;
Number u = Long.MAX_VALUE;
if (rangeQueryBuilder.from() != null) {
l = numFieldType.parse(rangeQueryBuilder.from(), false);
if (rangeQueryBuilder.includeLower() == false) {
l = getNextHighOrLowForDecimal(numFieldType, l, true);
}
l = parseRawNumberToDVLong(l, numFieldType);
}
if (rangeQueryBuilder.to() != null) {
u = numFieldType.parse(rangeQueryBuilder.to(), false);
if (rangeQueryBuilder.includeUpper() == false) {
u = getNextHighOrLowForDecimal(numFieldType, u, false);
}
u = parseRawNumberToDVLong(u, numFieldType);
}
return new RangeMatchDimFilter(
rangeQueryBuilder.fieldName(),
l,
u,
rangeQueryBuilder.includeLower(),
rangeQueryBuilder.includeUpper()
);
}

private static Number getNextHighOrLowForDecimal(
NumberFieldMapper.NumberType numFieldType,
Number value,
boolean returnNextHighest
) {
switch (numFieldType) {
case HALF_FLOAT:
return returnNextHighest ? HalfFloatPoint.nextUp((Float) value) : HalfFloatPoint.nextDown((Float) value);
case FLOAT:
return returnNextHighest ? FloatPoint.nextUp((Float) value) : FloatPoint.nextDown((Float) value);
case DOUBLE:
return returnNextHighest ? DoublePoint.nextUp((Double) value) : DoublePoint.nextDown((Double) value);
default:
throw new IllegalArgumentException("Invalid field type [" + numFieldType + "] for decimal");
}
}
}

}