Skip to content

Commit cc9ee9d

Browse files
harshavamsiharshavamsi
authored
Updating Ip fields to use doc_values to search (#11508)
* Updating Ip fields to use doc_values to search Signed-off-by: Harsha Vamsi Kalluri <harshavamsi096@gmail.com> * Fix IP tests Signed-off-by: Harsha Vamsi Kalluri <harshavamsi096@gmail.com> * Fix skip to allow yaml test to pass on main Signed-off-by: Harsha Vamsi Kalluri <harshavamsi096@gmail.com> * Update tests to use existing test file Signed-off-by: Harsha Vamsi Kalluri <harshavamsi096@gmail.com> * Changing skip version to match bwc Signed-off-by: Harsha Vamsi Kalluri <harshavamsi096@gmail.com> * Using exact match instead of range Signed-off-by: Harsha Vamsi Kalluri <harshavamsi096@gmail.com> * Spotless Signed-off-by: Harsha Vamsi Kalluri <harshavamsi096@gmail.com> * Fix IP field tests Signed-off-by: Harsha Vamsi Kalluri <harshavamsi096@gmail.com> * Fix spotless + precommit failure Signed-off-by: Harsha Vamsi Kalluri <harshavamsi096@gmail.com> * Get point out of query and into value Signed-off-by: Harsha Vamsi Kalluri <harshavamsi096@gmail.com> * Fix term tests Signed-off-by: Harsha Vamsi Kalluri <harshavamsi096@gmail.com> * Add skip test logic to only doc_values test Signed-off-by: Harsha Vamsi Kalluri <harshavamsi096@gmail.com> --------- Signed-off-by: Harsha Vamsi Kalluri <harshavamsi096@gmail.com>
1 parent 1f8b62f commit cc9ee9d

File tree

4 files changed

+323
-53
lines changed

4 files changed

+323
-53
lines changed

‎CHANGELOG.md

+1
Original file line numberDiff line numberDiff line change
@@ -216,6 +216,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
216216
- Ensure Jackson default maximums introduced in 2.16.0 do not conflict with OpenSearch settings ([#11890](https://github.com/opensearch-project/OpenSearch/pull/11890))
217217
- Extract cluster management for integration tests into JUnit test rule out of OpenSearchIntegTestCase ([#11877](https://github.com/opensearch-project/OpenSearch/pull/11877)), ([#12000](https://github.com/opensearch-project/OpenSearch/pull/12000))
218218
- Workaround for https://bugs.openjdk.org/browse/JDK-8323659 regression, introduced in JDK-21.0.2 ([#11968](https://github.com/opensearch-project/OpenSearch/pull/11968))
219+
- Updates IpField to be searchable when only `doc_values` are enabled ([#11508](https://github.com/opensearch-project/OpenSearch/pull/11508))
219220

220221
### Deprecated
221222

‎rest-api-spec/src/main/resources/rest-api-spec/test/search/340_doc_values_field.yml

+115-21
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,3 @@
1-
setup:
2-
- skip:
3-
features: [ "headers" ]
4-
version: " - 2.11.99"
5-
reason: "searching with only doc_values was added in 2.12.0"
61
---
72
"search on fields with both index and doc_values enabled":
83
- do:
@@ -47,18 +42,22 @@ setup:
4742
type: unsigned_long
4843
index: true
4944
doc_values: true
45+
ip_field:
46+
type: ip
47+
index: true
48+
doc_values: true
5049

5150
- do:
5251
bulk:
5352
index: test-iodvq
5453
refresh: true
5554
body:
5655
- '{"index": {"_index": "test-iodvq", "_id": "1" }}'
57-
- '{ "some_keyword": "ingesting some random keyword data", "byte": 120, "double": 100.0, "float": "800.0", "half_float": "400.0", "integer": 1290, "long": 13456, "short": 150, "unsigned_long": 10223372036854775800 }'
56+
- '{ "some_keyword": "ingesting some random keyword data", "byte": 120, "double": 100.0, "float": "800.0", "half_float": "400.0", "integer": 1290, "long": 13456, "short": 150, "unsigned_long": 10223372036854775800, "ip_field": "192.168.0.1" }'
5857
- '{ "index": { "_index": "test-iodvq", "_id": "2" }}'
59-
- '{ "some_keyword": "400", "byte": 121, "double": 101.0, "float": "801.0", "half_float": "401.0", "integer": 1291, "long": 13457, "short": 151, "unsigned_long": 10223372036854775801 }'
58+
- '{ "some_keyword": "400", "byte": 121, "double": 101.0, "float": "801.0", "half_float": "401.0", "integer": 1291, "long": 13457, "short": 151, "unsigned_long": 10223372036854775801, "ip_field": "192.168.0.2" }'
6059
- '{ "index": { "_index": "test-iodvq", "_id": "3" } }'
61-
- '{ "some_keyword": "5", "byte": 122, "double": 102.0, "float": "802.0", "half_float": "402.0", "integer": 1292, "long": 13458, "short": 152, "unsigned_long": 10223372036854775802 }'
60+
- '{ "some_keyword": "5", "byte": 122, "double": 102.0, "float": "802.0", "half_float": "402.0", "integer": 1292, "long": 13458, "short": 152, "unsigned_long": 10223372036854775802, "ip_field": "192.168.0.3" }'
6261

6362
- do:
6463
search:
@@ -162,7 +161,6 @@ setup:
162161

163162
- match: { hits.total: 1 }
164163

165-
166164
- do:
167165
search:
168166
rest_total_hits_as_int: true
@@ -174,6 +172,16 @@ setup:
174172

175173
- match: { hits.total: 1 }
176174

175+
- do:
176+
search:
177+
rest_total_hits_as_int: true
178+
index: test-iodvq
179+
body:
180+
query:
181+
term:
182+
ip_field: "192.168.0.1"
183+
184+
- match: {hits.total: 1}
177185

178186
- do:
179187
search:
@@ -186,7 +194,6 @@ setup:
186194

187195
- match: { hits.total: 2 }
188196

189-
190197
- do:
191198
search:
192199
rest_total_hits_as_int: true
@@ -264,6 +271,17 @@ setup:
264271

265272
- match: { hits.total: 2 }
266273

274+
- do:
275+
search:
276+
rest_total_hits_as_int: true
277+
index: test-iodvq
278+
body:
279+
query:
280+
terms:
281+
ip_field: ["192.168.0.1", "192.168.0.2"]
282+
283+
- match: { hits.total: 2 }
284+
267285
- do:
268286
search:
269287
rest_total_hits_as_int: true
@@ -384,6 +402,19 @@ setup:
384402

385403
- match: { hits.total: 2 }
386404

405+
- do:
406+
search:
407+
rest_total_hits_as_int: true
408+
index: test-iodvq
409+
body:
410+
query:
411+
range:
412+
ip_field:
413+
gte: "192.168.0.1"
414+
lte: "192.168.0.2"
415+
416+
- match: { hits.total: 2 }
417+
387418
---
388419
"search on fields with only index enabled":
389420
- do:
@@ -428,18 +459,22 @@ setup:
428459
type: unsigned_long
429460
index: true
430461
doc_values: false
462+
ip_field:
463+
type: ip
464+
index: true
465+
doc_values: false
431466

432467
- do:
433468
bulk:
434469
index: test-index
435470
refresh: true
436471
body:
437472
- '{"index": {"_index": "test-index", "_id": "1" }}'
438-
- '{ "some_keyword": "ingesting some random keyword data", "byte": 120, "double": 100.0, "float": "800.0", "half_float": "400.0", "integer": 1290, "long": 13456, "short": 150, "unsigned_long": 10223372036854775800 }'
473+
- '{ "some_keyword": "ingesting some random keyword data", "byte": 120, "double": 100.0, "float": "800.0", "half_float": "400.0", "integer": 1290, "long": 13456, "short": 150, "unsigned_long": 10223372036854775800, "ip_field": "192.168.0.1" }'
439474
- '{ "index": { "_index": "test-index", "_id": "2" }}'
440-
- '{ "some_keyword": "400", "byte": 121, "double": 101.0, "float": "801.0", "half_float": "401.0", "integer": 1291, "long": 13457, "short": 151, "unsigned_long": 10223372036854775801 }'
475+
- '{ "some_keyword": "400", "byte": 121, "double": 101.0, "float": "801.0", "half_float": "401.0", "integer": 1291, "long": 13457, "short": 151, "unsigned_long": 10223372036854775801, "ip_field": "192.168.0.2" }'
441476
- '{ "index": { "_index": "test-index", "_id": "3" } }'
442-
- '{ "some_keyword": "5", "byte": 122, "double": 102.0, "float": "802.0", "half_float": "402.0", "integer": 1292, "long": 13458, "short": 152, "unsigned_long": 10223372036854775802 }'
477+
- '{ "some_keyword": "5", "byte": 122, "double": 102.0, "float": "802.0", "half_float": "402.0", "integer": 1292, "long": 13458, "short": 152, "unsigned_long": 10223372036854775802, "ip_field": "192.168.0.3" }'
443478

444479
- do:
445480
search:
@@ -465,7 +500,6 @@ setup:
465500

466501
- match: { hits.total: 2 }
467502

468-
469503
- do:
470504
search:
471505
rest_total_hits_as_int: true
@@ -555,6 +589,16 @@ setup:
555589

556590
- match: { hits.total: 1 }
557591

592+
- do:
593+
search:
594+
rest_total_hits_as_int: true
595+
index: test-index
596+
body:
597+
query:
598+
term:
599+
ip_field: "192.168.0.1"
600+
601+
- match: {hits.total: 1}
558602

559603
- do:
560604
search:
@@ -567,7 +611,6 @@ setup:
567611

568612
- match: { hits.total: 2 }
569613

570-
571614
- do:
572615
search:
573616
rest_total_hits_as_int: true
@@ -645,6 +688,17 @@ setup:
645688

646689
- match: { hits.total: 2 }
647690

691+
- do:
692+
search:
693+
rest_total_hits_as_int: true
694+
index: test-index
695+
body:
696+
query:
697+
terms:
698+
ip_field: ["192.168.0.1", "192.168.0.2"]
699+
700+
- match: { hits.total: 2 }
701+
648702
- do:
649703
search:
650704
rest_total_hits_as_int: true
@@ -765,8 +819,24 @@ setup:
765819

766820
- match: { hits.total: 2 }
767821

822+
- do:
823+
search:
824+
rest_total_hits_as_int: true
825+
index: test-index
826+
body:
827+
query:
828+
range:
829+
ip_field:
830+
gte: "192.168.0.1"
831+
lte: "192.168.0.2"
832+
833+
- match: { hits.total: 2 }
768834
---
769835
"search on fields with only doc_values enabled":
836+
- skip:
837+
features: [ "headers" ]
838+
version: " - 2.99.99"
839+
reason: "searching with only doc_values was added in 3.0.0"
770840
- do:
771841
indices.create:
772842
index: test-doc-values
@@ -809,18 +879,22 @@ setup:
809879
type: unsigned_long
810880
index: false
811881
doc_values: true
882+
ip_field:
883+
type: ip
884+
index: false
885+
doc_values: true
812886

813887
- do:
814888
bulk:
815889
index: test-doc-values
816890
refresh: true
817891
body:
818892
- '{"index": {"_index": "test-doc-values", "_id": "1" }}'
819-
- '{ "some_keyword": "ingesting some random keyword data", "byte": 120, "double": 100.0, "float": "800.0", "half_float": "400.0", "integer": 1290, "long": 13456, "short": 150, "unsigned_long": 10223372036854775800 }'
893+
- '{ "some_keyword": "ingesting some random keyword data", "byte": 120, "double": 100.0, "float": "800.0", "half_float": "400.0", "integer": 1290, "long": 13456, "short": 150, "unsigned_long": 10223372036854775800, "ip_field": "192.168.0.1" }'
820894
- '{ "index": { "_index": "test-doc-values", "_id": "2" }}'
821-
- '{ "some_keyword": "400", "byte": 121, "double": 101.0, "float": "801.0", "half_float": "401.0", "integer": 1291, "long": 13457, "short": 151, "unsigned_long": 10223372036854775801 }'
895+
- '{ "some_keyword": "400", "byte": 121, "double": 101.0, "float": "801.0", "half_float": "401.0", "integer": 1291, "long": 13457, "short": 151, "unsigned_long": 10223372036854775801, "ip_field": "192.168.0.2" }'
822896
- '{ "index": { "_index": "test-doc-values", "_id": "3" } }'
823-
- '{ "some_keyword": "5", "byte": 122, "double": 102.0, "float": "802.0", "half_float": "402.0", "integer": 1292, "long": 13458, "short": 152, "unsigned_long": 10223372036854775802 }'
897+
- '{ "some_keyword": "5", "byte": 122, "double": 102.0, "float": "802.0", "half_float": "402.0", "integer": 1292, "long": 13458, "short": 152, "unsigned_long": 10223372036854775802, "ip_field": "192.168.0.3" }'
824898

825899
- do:
826900
search:
@@ -846,7 +920,6 @@ setup:
846920

847921
- match: { hits.total: 2 }
848922

849-
850923
- do:
851924
search:
852925
rest_total_hits_as_int: true
@@ -924,7 +997,6 @@ setup:
924997

925998
- match: { hits.total: 1 }
926999

927-
9281000
- do:
9291001
search:
9301002
rest_total_hits_as_int: true
@@ -936,6 +1008,16 @@ setup:
9361008

9371009
- match: { hits.total: 1 }
9381010

1011+
- do:
1012+
search:
1013+
rest_total_hits_as_int: true
1014+
index: test-doc-values
1015+
body:
1016+
query:
1017+
term:
1018+
ip_field: "192.168.0.3"
1019+
1020+
- match: { hits.total: 1 }
9391021

9401022
- do:
9411023
search:
@@ -948,7 +1030,6 @@ setup:
9481030

9491031
- match: { hits.total: 2 }
9501032

951-
9521033
- do:
9531034
search:
9541035
rest_total_hits_as_int: true
@@ -1145,3 +1226,16 @@ setup:
11451226
}
11461227

11471228
- match: { hits.total: 2 }
1229+
1230+
- do:
1231+
search:
1232+
rest_total_hits_as_int: true
1233+
index: test-doc-values
1234+
body:
1235+
query:
1236+
range:
1237+
ip_field:
1238+
gte: "192.168.0.1"
1239+
lte: "192.168.0.2"
1240+
1241+
- match: { hits.total: 2 }

0 commit comments

Comments
 (0)