Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixed swapped schema references in nodes info API buffer fields #808

Conversation

hye-on
Copy link
Contributor

@hye-on hye-on commented Jan 25, 2025

Description

Fixed incorrectly referenced schemas in nodes info API buffer fields by swapping the schema types between total_indexing_buffer and total_indexing_buffer_in_bytes .

Issues Resolved

Related to opensearch-project/OpenSearch#16910

Related PR

opensearch-project/OpenSearch#17070

Comments

This change will be effective from version 3.0.0 onwards

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: hye-on <ain0103@naver.com>
Copy link
Contributor

github-actions bot commented Jan 25, 2025

Changes Analysis

Commit SHA: d6b2b4e
Comparing To SHA: 439bf38

API Changes

Summary

└─┬Components
  └─┬nodes.info___NodeInfo
    ├─┬total_indexing_buffer_in_bytes
    │ └──[🔀] $ref (58382:11)❌ 
    └─┬total_indexing_buffer
      └──[🔀] $ref (58375:11)❌ 

Document Element Total Changes Breaking Changes
components 2 2
  • BREAKING Changes: 2 out of 2
  • Modifications: 2
  • Breaking Modifications: 2

Report

The full API changes report is available at: https://github.com/opensearch-project/opensearch-api-specification/actions/runs/14095554681/artifacts/2827865851

API Coverage

Before After Δ
Covered (%) 663 (64.94 %) 663 (64.94 %) 0 (0 %)
Uncovered (%) 358 (35.06 %) 358 (35.06 %) 0 (0 %)
Unknown 49 49 0

Copy link
Member

@dblock dblock left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Xtansia I read the related issue, this is correct right?

Copy link
Member

@dblock dblock left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually we now have a failing test. The spec needs to work correctly for all versions, how do we fix this?

@hye-on
Copy link
Contributor Author

hye-on commented Jan 26, 2025

Actually we now have a failing test. The spec needs to work correctly for all versions, how do we fix this?

@dblock I noticed that PR opensearch-project/OpenSearch#17070 will implement changes from OpenSearch 3.0.0. Can the api-specification repository define different specifications per version? Or should I wait until OpenSearch 3.0.0 is released to submit this PR?

I've reviewed the failing test files and related files (tests/default/nodes/nodes.yaml, tests/default/nodes/usage.yaml, json_schemas/test_story.schema.yaml, and others) but couldn't identify where to make changes. (Does the test payload response come from OpenSearch itself?) If I've missed any files that should be reviewed, I would greatly appreciate your guidance.

Thank you for taking the time to review this and for your help! :)

@dblock
Copy link
Member

dblock commented Jan 26, 2025

@hye-on In general we would do this:

  1. Annotate the current schema for both fields with x-version-removed: 3.0.
  2. Add the new schema with x-version-added: 3.0.

See https://github.com/opensearch-project/opensearch-api-specification/blob/main/DEVELOPER_GUIDE.md#openapi-extensions for field details.

Since the tools don't allow the same field name twice, I think we need to do something like this:

        total_indexing_buffer_in_bytes:
          oneOf:
            - $ref: '_common.yaml#/components/schemas/HumanReadableByteCount'
              x-version-removed: 3.0
            - $ref: '_common.yaml#/components/schemas/ByteCount'
              x-version-added: 3.0

You can run the merge (see https://github.com/opensearch-project/opensearch-api-specification/blob/main/DEVELOPER_GUIDE.md#spec-merger) with an option to target 3.0 and it should remove the oneOf that's not matching.

If that doesn't work, then let's do the above minus the x-version options with a comment.

…tal_indexing_buffer_in_bytes fields

Signed-off-by: hye-on <ain0103@naver.com>
Signed-off-by: hye-on <ain0103@naver.com>
Signed-off-by: hye-on <ain0103@naver.com>
Copy link
Contributor

Spec Test Coverage Analysis

Total Tested
594 593 (99.83 %)

@hye-on
Copy link
Contributor Author

hye-on commented Jan 27, 2025

@dblock
Thank you for your kind guidance.
When I added the x-version-removed and x-version-added tags, the 3.0 version test failed.
image

As you guided, after removing the tags, the 3.0 test is now working.
Thank you.

I notice that the Check Links is failing (502 Network error: Bad Gateway).
If there's anything I need to fix, please let me know.

Thank you again for taking the time to help me ! :)

@dblock
Copy link
Member

dblock commented Jan 27, 2025

@hye-on you have to update the SHA to OpenSearch 3.0 to a newer one with opensearch-project/OpenSearch#17070 merged in https://github.com/opensearch-project/opensearch-api-specification/blob/main/.github/workflows/test-spec.yml#L65.

It's not always trivial to find a complete distribution of OpenSearch 3.0 that has all the plugins, so other tests may fail, but worth a try on 1-2 newer revisions available in DockerHub. Otherwise add the x-version- headers and we can keep this open until we find one.

@Xtansia Xtansia changed the base branch from main to baseline/3.0.0-alpha1 March 26, 2025 23:33
Xtansia added 2 commits March 27, 2025 12:33
…ffer

Signed-off-by: Thomas Farr <tsfarr@amazon.com>
This reverts commit 1622f24.

Signed-off-by: Thomas Farr <tsfarr@amazon.com>
@Xtansia Xtansia merged commit b88fe06 into opensearch-project:baseline/3.0.0-alpha1 Mar 26, 2025
30 of 32 checks passed
Xtansia added a commit that referenced this pull request Mar 27, 2025
* Update 3.0.0 snapshot image to 3.0.0-alpha1

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Allow prerelease versions in semver matching

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Fixed swapped schema references in nodes info API buffer fields (#808)

* Fixed swapped schema references in nodes info API buffer fields

Signed-off-by: hye-on <ain0103@naver.com>

* Add version specific schema handling for total_indexing_buffer and total_indexing_buffer_in_bytes fields

Signed-off-by: hye-on <ain0103@naver.com>

* Fix version number format in nodes.info.yaml

Signed-off-by: hye-on <ain0103@naver.com>

* Remove version tags and add notes for schema changes

Signed-off-by: hye-on <ain0103@naver.com>

* Revert "Remove version tags and add notes for schema changes"

This reverts commit 1622f24.

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

---------

Signed-off-by: hye-on <ain0103@naver.com>
Signed-off-by: Thomas Farr <tsfarr@amazon.com>
Co-authored-by: Thomas Farr <tsfarr@amazon.com>

* Added paginationDepth to HybridQuery (#848)

* Added paginationDepth to HybridQuery

Signed-off-by: Wonjae Lee <wonjae.lee@dremio.com>

* Fixed diff and added test with pagination_depth

Signed-off-by: Wonjae Lee <wonjae.lee@dremio.com>

* Lint & changelog

Signed-off-by: Wonjae Lee <wonjae.lee@dremio.com>

---------

Signed-off-by: Wonjae Lee <wonjae.lee@dremio.com>
Co-authored-by: Thomas Farr <tsfarr@amazon.com>

* Added `warm` to `ClusterNodeCount`

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Support indices and shards on nodes stats cache

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Add total_primary_shards_per_node

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Fix flow framework provision response status code

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Fix security status codes

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

---------

Signed-off-by: Thomas Farr <tsfarr@amazon.com>
Signed-off-by: hye-on <ain0103@naver.com>
Signed-off-by: Wonjae Lee <wonjae.lee@dremio.com>
Co-authored-by: 조혜온 <68319395+hye-on@users.noreply.github.com>
Co-authored-by: Wonjae Lee <38933452+leewjae@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants