add tutorial for RAG with DeepSeek R1 model on Sagemaker #3455
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Yaliang Wu <ylwu@amazon.com>
ylwu-amzn
requested review from
b4sjoo,
dhrubo-os,
mingshl,
jngz-es,
model-collapse,
rbhavna,
zane-neo,
Zhangxunmt,
austintlee,
HenryL27 and
xinyual
as code owners
ylwu-amzn
had a problem deploying
to
ml-commons-cicd-env
GitHub Actions
Failure
ylwu-amzn
had a problem deploying
to
ml-commons-cicd-env
GitHub Actions
Failure
Signed-off-by: Yaliang Wu <ylwu@amazon.com>
ylwu-amzn
temporarily deployed
to
ml-commons-cicd-env
GitHub Actions
Inactive
ylwu-amzn
had a problem deploying
to
ml-commons-cicd-env
GitHub Actions
Failure
Zhangxunmt
reviewed
Jan 29, 2025
|
|
||
| ### Deploy DeepSeek R1 model to Sagemaker | ||
|
|
||
| Follow this [blog](https://community.aws/content/2sG84dNUCFzA9z4HdfqTI0tcvKP/deploying-deepseek-r1-on-amazon-sagemaker) to deploy DeepSeek R1 model to Sagemaker. |
There was a problem hiding this comment.
This is interesting. Have you followed the steps and deployed successfully in SageMaker?
There was a problem hiding this comment.
yes, the result is from test cluster, not fake one
pyek-bot
reviewed
Jan 29, 2025
Comment on lines
153
to
163
| Add DeepSeek API endpoint to trusted URLs | ||
| ``` | ||
| PUT /_cluster/settings | ||
| { | ||
| "persistent": { | ||
| "plugins.ml_commons.trusted_connector_endpoints_regex": [ | ||
| "^https://api\\.deepseek\\.com/.*$" | ||
| ] | ||
| } | ||
| } | ||
| ``` |
There was a problem hiding this comment.
this will not be needed from v2.19, so should we mention that here?
There was a problem hiding this comment.
We can add after 2.19
pyek-bot
reviewed
Jan 29, 2025
| "status": "COMPLETED" | ||
| } | ||
| ``` | ||
| 4. Predict |
There was a problem hiding this comment.
should we link the deepseek chat api here? so users can be aware of which api we are hitting internally
There was a problem hiding this comment.
The first sentence already mentioned the API
This tutorial introduces how to build RAG in Amazon managed OpenSearch with DeepSeek Chat Model.
pyek-bot
reviewed
Jan 29, 2025
| ``` | ||
| 4. Predict | ||
| ``` | ||
| POST /_plugins/_ml/models/Sym9sJQBts7fa6byEh1-/_predict |
There was a problem hiding this comment.
same here? is it possible to provide some reference so users are aware of how this parameters work
There was a problem hiding this comment.
This is somehow not easy. Sagemaker team owns how the API works. I don't have clear view about this, no docs found
Signed-off-by: Yaliang Wu <ylwu@amazon.com>
ylwu-amzn
had a problem deploying
to
ml-commons-cicd-env
GitHub Actions
Failure
ylwu-amzn
had a problem deploying
to
ml-commons-cicd-env
GitHub Actions
Failure
kolchfa-aws
reviewed
Jan 29, 2025
| ## 5. RAG | ||
|
|
||
| ### 5.1 create search pipeline | ||
| Create search pipeline with [RAG processor](https://opensearch.org/docs/latest/search-plugins/search-pipelines/rag-processor/). |
There was a problem hiding this comment.
Suggested change
| Create search pipeline with [RAG processor](https://opensearch.org/docs/latest/search-plugins/search-pipelines/rag-processor/). | |
| Create search pipeline with a [RAG processor](https://opensearch.org/docs/latest/search-plugins/search-pipelines/rag-processor/): |
| } | ||
| ``` | ||
| ### 5.2 create vector database | ||
| Follow this [neural search tutorial](https://opensearch.org/docs/latest/search-plugins/neural-search-tutorial/) to create embedding model, K-NN index `my-nlp-index`, and ingest data |
There was a problem hiding this comment.
Suggested change
| Follow this [neural search tutorial](https://opensearch.org/docs/latest/search-plugins/neural-search-tutorial/) to create embedding model, K-NN index `my-nlp-index`, and ingest data | |
| Follow the [neural search tutorial](https://opensearch.org/docs/latest/search-plugins/neural-search-tutorial/) to create an embedding model and a k-NN index. Then ingest data into the index: |
| ``` | ||
|
|
||
|
|
||
| ### 5.3 search |
There was a problem hiding this comment.
Suggested change
| ### 5.3 search | |
| ### 5.3 Search the index |
|
|
||
|
|
||
| ### 5.3 search | ||
| Run neural search to retrieve documents from vector database, then use DeepSeek model to do RAG. |
There was a problem hiding this comment.
Suggested change
| Run neural search to retrieve documents from vector database, then use DeepSeek model to do RAG. | |
| Run vector search to retrieve documents from the vector database, then use the DeepSeek model for RAG: |
| } | ||
| } | ||
| ``` | ||
| Response |
There was a problem hiding this comment.
Suggested change
| Response | |
| The response contains the matching documents: |
Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: Yaliang Wu <ylwu@amazon.com>
ylwu-amzn
had a problem deploying
to
ml-commons-cicd-env
GitHub Actions
Failure
ylwu-amzn
had a problem deploying
to
ml-commons-cicd-env
GitHub Actions
Failure
Signed-off-by: Yaliang Wu <ylwu@amazon.com>
ylwu-amzn
had a problem deploying
to
ml-commons-cicd-env
GitHub Actions
Failure
ylwu-amzn
had a problem deploying
to
ml-commons-cicd-env
GitHub Actions
Failure
jngz-es
approved these changes
Jan 29, 2025
b4sjoo
approved these changes
Jan 29, 2025
opensearch-trigger-bot bot
pushed a commit
that referenced
this pull request
Jan 29, 2025
* add tutorial for RAG with DeepSeek R1 model on Sagemaker Signed-off-by: Yaliang Wu <ylwu@amazon.com> * add tutorials for RAG with DeepSeek Chat model Signed-off-by: Yaliang Wu <ylwu@amazon.com> * add tutorial for RAG with DeepSeek R1 model on Sagemaker Signed-off-by: Yaliang Wu <ylwu@amazon.com> * Apply suggestions from code review Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: Yaliang Wu <ylwu@amazon.com> * apply comments to sagemkaer tutorial Signed-off-by: Yaliang Wu <ylwu@amazon.com> --------- Signed-off-by: Yaliang Wu <ylwu@amazon.com> Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> (cherry picked from commit 8830a43)
ylwu-amzn
added a commit
that referenced
this pull request
Jan 30, 2025
* add tutorial for RAG with DeepSeek R1 model on Sagemaker Signed-off-by: Yaliang Wu <ylwu@amazon.com> * add tutorials for RAG with DeepSeek Chat model Signed-off-by: Yaliang Wu <ylwu@amazon.com> * add tutorial for RAG with DeepSeek R1 model on Sagemaker Signed-off-by: Yaliang Wu <ylwu@amazon.com> * Apply suggestions from code review Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: Yaliang Wu <ylwu@amazon.com> * apply comments to sagemkaer tutorial Signed-off-by: Yaliang Wu <ylwu@amazon.com> --------- Signed-off-by: Yaliang Wu <ylwu@amazon.com> Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> (cherry picked from commit 8830a43) Co-authored-by: Yaliang Wu <ylwu@amazon.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This tutorial introduces how to use DeepSeek model deployed on Sagemaker to build RAG in OpenSearch
Related Issues
Resolves #[Issue number to be closed when this PR is merged]
Check List
--signoff.By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.