[Backport to main] Backport multiple commits from 2.x to main branch #1273
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* remote inference: add connector executor Signed-off-by: Yaliang Wu <ylwu@amazon.com> * fix null pointer bug Signed-off-by: Yaliang Wu <ylwu@amazon.com> --------- Signed-off-by: Yaliang Wu <ylwu@amazon.com>
…ch-project#1057) * refactor predictable: add method to check if model is ready Signed-off-by: Yaliang Wu <ylwu@amazon.com> * fix failed ut Signed-off-by: Yaliang Wu <ylwu@amazon.com> --------- Signed-off-by: Yaliang Wu <ylwu@amazon.com>
* add unit test for connector class in commons Signed-off-by: Yaliang Wu <ylwu@amazon.com> * add more test Signed-off-by: Yaliang Wu <ylwu@amazon.com> --------- Signed-off-by: Yaliang Wu <ylwu@amazon.com>
* Add basic connector access control classes Signed-off-by: zane-neo <zaniu@amazon.com> * Add basic connector access control classes and fformat code Signed-off-by: zane-neo <zaniu@amazon.com> --------- Signed-off-by: zane-neo <zaniu@amazon.com>
…pensearch-project#1059) Signed-off-by: Yaliang Wu <ylwu@amazon.com>
Signed-off-by: Xun Zhang <xunzh@amazon.com>
…input (opensearch-project#1061) Signed-off-by: Yaliang Wu <ylwu@amazon.com>
* Add basic connector access control classes Signed-off-by: zane-neo <zaniu@amazon.com> * Add basic connector access control classes and fformat code Signed-off-by: zane-neo <zaniu@amazon.com> * Add missing codes from pen test fix Signed-off-by: zane-neo <zaniu@amazon.com> * Add connector access control helper initialization Signed-off-by: zane-neo <zaniu@amazon.com> * Add AbstractMLSearchAction code to fix IT failure Signed-off-by: zane-neo <zaniu@amazon.com> * Fix failure UTs Signed-off-by: zane-neo <zaniu@amazon.com> * Remove code from AbstractMLSearchAction to pass PR check Signed-off-by: zane-neo <zaniu@amazon.com> * Fix UT failure Signed-off-by: zane-neo <zaniu@amazon.com> * format code Signed-off-by: zane-neo <zaniu@amazon.com> --------- Signed-off-by: zane-neo <zaniu@amazon.com>
* change to compileOnly to avoid jarhell Signed-off-by: Yaliang Wu <ylwu@amazon.com> * fix missing class Signed-off-by: Yaliang Wu <ylwu@amazon.com> * add json dependency to ml-algorithms Signed-off-by: Yaliang Wu <ylwu@amazon.com> --------- Signed-off-by: Yaliang Wu <ylwu@amazon.com>
Signed-off-by: Xun Zhang <xunzh@amazon.com>
…-project#1063) * create model group automatically with first model version Signed-off-by: Bhavana Ramaram <rbhavna@amazon.com>
Signed-off-by: zane-neo <zaniu@amazon.com>
…opensearch-project#1067) * remote inference: add unit test for create connector request/response Signed-off-by: Yaliang Wu <ylwu@amazon.com> * fix failed UT Signed-off-by: Yaliang Wu <ylwu@amazon.com> * fix failed UT Signed-off-by: Yaliang Wu <ylwu@amazon.com> --------- Signed-off-by: Yaliang Wu <ylwu@amazon.com>
Signed-off-by: Xun Zhang <xunzh@amazon.com>
…project#1068) * fix cannot specify model access control parameters error Signed-off-by: Yaliang Wu <ylwu@amazon.com> * add unit test for model group class and fix some bug Signed-off-by: Yaliang Wu <ylwu@amazon.com> * fix failed ut Signed-off-by: Yaliang Wu <ylwu@amazon.com> --------- Signed-off-by: Yaliang Wu <ylwu@amazon.com>
Signed-off-by: zane-neo <zaniu@amazon.com>
Signed-off-by: Xun Zhang <xunzh@amazon.com>
…ject#1074) * PenTest fixes: error codes and update model group fix Signed-off-by: Bhavana Ramaram <rbhavna@amazon.com> * fix get model assertion error Signed-off-by: Bhavana Ramaram <rbhavna@amazon.com> --------- Signed-off-by: Bhavana Ramaram <rbhavna@amazon.com>
* Change connector access control creation allow empty list Signed-off-by: zane-neo <zaniu@amazon.com> * Fix rare private ip address bypass SSRF issue Signed-off-by: zane-neo <zaniu@amazon.com> --------- Signed-off-by: zane-neo <zaniu@amazon.com>
Signed-off-by: Xun Zhang <xunzh@amazon.com>
…rch-project#1078) * modify error message when model group not unique is provided Signed-off-by: Bhavana Ramaram <rbhavna@amazon.com> * fix unique model group name unit test Signed-off-by: Bhavana Ramaram <rbhavna@amazon.com> --------- Signed-off-by: Bhavana Ramaram <rbhavna@amazon.com>
* init master key automatically Signed-off-by: Yaliang Wu <ylwu@amazon.com> * remove unnecessary escape Signed-off-by: Yaliang Wu <ylwu@amazon.com> * fix failed ut Signed-off-by: Yaliang Wu <ylwu@amazon.com> * tune syncup jot interval Signed-off-by: Yaliang Wu <ylwu@amazon.com> * tune syncup jot interval Signed-off-by: Yaliang Wu <ylwu@amazon.com> * remove local config file code Signed-off-by: Yaliang Wu <ylwu@amazon.com> * set master key when init remote model Signed-off-by: Yaliang Wu <ylwu@amazon.com> * move init master key to encryptor Signed-off-by: Yaliang Wu <ylwu@amazon.com> * fine tune code Signed-off-by: Yaliang Wu <ylwu@amazon.com> * fine tune code Signed-off-by: Yaliang Wu <ylwu@amazon.com> --------- Signed-off-by: Yaliang Wu <ylwu@amazon.com>
…ize to 32 (opensearch-project#1092) Signed-off-by: Yaliang Wu <ylwu@amazon.com>
Signed-off-by: Yaliang Wu <ylwu@amazon.com>
* IT Security Tests for model access control Signed-off-by: Bhavana Ramaram <rbhavna@amazon.com> * Fix assertion error Signed-off-by: Bhavana Ramaram <rbhavna@amazon.com> * Fix format violations Signed-off-by: Bhavana Ramaram <rbhavna@amazon.com> --------- Signed-off-by: Bhavana Ramaram <rbhavna@amazon.com>
* Add test keys to CI related workflows Signed-off-by: Peter Zhu <zhujiaxi@amazon.com> * test more Signed-off-by: Peter Zhu <zhujiaxi@amazon.com> * more test Signed-off-by: Peter Zhu <zhujiaxi@amazon.com> * test Signed-off-by: Peter Zhu <zhujiaxi@amazon.com> --------- Signed-off-by: Peter Zhu <zhujiaxi@amazon.com>
…arch-project#1102) Signed-off-by: Peter Zhu <zhujiaxi@amazon.com>
* 2.9 release notes Signed-off-by: Bhavana Ramaram <rbhavna@amazon.com> * move PR1065 from infrastructure to feature Signed-off-by: Bhavana Ramaram <rbhavna@amazon.com> --------- Signed-off-by: Bhavana Ramaram <rbhavna@amazon.com>
Signed-off-by: Xun Zhang <xunzh@amazon.com>
…ect#1176) Signed-off-by: Yaliang Wu <ylwu@amazon.com>
* Add model auto redeploy tutorial Signed-off-by: zane-neo <zaniu@amazon.com> * Fix minor issue Signed-off-by: zane-neo <zaniu@amazon.com> * Change to recomanded statement Signed-off-by: zane-neo <zaniu@amazon.com> --------- Signed-off-by: zane-neo <zaniu@amazon.com>
Signed-off-by: Dhrubo Saha <dhrubo@amazon.com>
…odel count bug (opensearch-project#1181) Signed-off-by: Yaliang Wu <ylwu@amazon.com>
…ect#1210) Signed-off-by: Yaliang Wu <ylwu@amazon.com>
* updating cohere blueprint doc Signed-off-by: Dhrubo Saha <dhrubo@amazon.com> * fixing the blueprint Signed-off-by: Dhrubo Saha <dhrubo@amazon.com> --------- Signed-off-by: Dhrubo Saha <dhrubo@amazon.com>
…el group (opensearch-project#1188) (opensearch-project#1218) Signed-off-by: Bhavana Ramaram <rbhavna@amazon.com> Co-authored-by: Bhavana Ramaram <rbhavna@amazon.com>
) (opensearch-project#1219) Signed-off-by: Yaliang Wu <ylwu@amazon.com>
* fixing metrics * addressing comments * addressing comments * updating test * added IllegalArgumentException in the if statement * addressing comments * fixing spotless --------- Signed-off-by: Dhrubo Saha <dhrubo@amazon.com> Co-authored-by: Dhrubo Saha <dhrubo@amazon.com>
Signed-off-by: zane-neo <zaniu@amazon.com>
…-project#1221) * add eligible node role settings * add more comment --------- Signed-off-by: Yaliang Wu <ylwu@amazon.com>
Signed-off-by: Amit Galitzky <amgalitz@amazon.com> Co-authored-by: Amit Galitzky <amgalitz@amazon.com>
Signed-off-by: Xun Zhang <xunzh@amazon.com> Co-authored-by: Xun Zhang <xunzh@amazon.com>
Signed-off-by: Sicheng Song <sicheng.song@outlook.com>
* renaming metrics Signed-off-by: Dhrubo Saha <dhrubo@amazon.com> * updating tests Signed-off-by: Dhrubo Saha <dhrubo@amazon.com> * updating test cases Signed-off-by: Dhrubo Saha <dhrubo@amazon.com> * removing the ML_NODE checking for node level stats Signed-off-by: Dhrubo Saha <dhrubo@amazon.com> * updating constructing new set Signed-off-by: Dhrubo Saha <dhrubo@amazon.com> * spotless Apply Signed-off-by: Dhrubo Saha <dhrubo@amazon.com> * updating ML_NODE_TOTAL_MODEL_COUNT to ML_DEPLOYED_MODEL_COUNT Signed-off-by: Dhrubo Saha <dhrubo@amazon.com> * fixing metrics count Signed-off-by: Dhrubo Saha <dhrubo@amazon.com> * spotless Signed-off-by: Dhrubo Saha <dhrubo@amazon.com> * fixing executing task Signed-off-by: Dhrubo Saha <dhrubo@amazon.com> * updating comment Signed-off-by: Dhrubo Saha <dhrubo@amazon.com> --------- Signed-off-by: Dhrubo Saha <dhrubo@amazon.com>
Signed-off-by: zane-neo <zaniu@amazon.com>
zane-neo
requested review from
b4sjoo,
dhrubo-os,
jngz-es,
model-collapse,
rbhavna,
wujunshen,
ylwu-amzn and
Zhangxunmt
as code owners
opensearch-trigger-bot
bot
added
documentation
ylwu-amzn
approved these changes
Sep 1, 2023
b4sjoo
approved these changes
Sep 1, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Backport multiple commits from 2.x to main branch, details can be found in involving commits.
Issues Resolved
[List any issues this PR will resolve]
Check List
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.