Skip to content

Commit de301b7

Browse files
authored
adding blue print doc for cohere multi-modal model (opensearch-project#3229)
* adding blue print doc Signed-off-by: Dhrubo Saha <dhrubo@amazon.com> * addressed comments Signed-off-by: Dhrubo Saha <dhrubo@amazon.com> * addressed comment Signed-off-by: Dhrubo Saha <dhrubo@amazon.com> --------- Signed-off-by: Dhrubo Saha <dhrubo@amazon.com>
1 parent 622f73d commit de301b7

File tree

4 files changed

+332
-8
lines changed

4 files changed

+332
-8
lines changed

common/src/main/java/org/opensearch/ml/common/connector/functions/preprocess/CohereMultiModalEmbeddingPreProcessFunction.java

+3-3
Original file line numberDiff line numberDiff line change
@@ -25,23 +25,23 @@ public CohereMultiModalEmbeddingPreProcessFunction() {
2525
public void validate(MLInput mlInput) {
2626
validateTextDocsInput(mlInput);
2727
List<String> docs = ((TextDocsInputDataSet) mlInput.getInputDataset()).getDocs();
28-
if (docs == null || docs.isEmpty() || (docs.size() == 1 && docs.get(0) == null)) {
28+
if (docs == null || docs.isEmpty() || docs.get(0) == null) {
2929
throw new IllegalArgumentException("No image provided");
3030
}
3131
}
3232

3333
@Override
3434
public RemoteInferenceInputDataSet process(MLInput mlInput) {
3535
TextDocsInputDataSet inputData = (TextDocsInputDataSet) mlInput.getInputDataset();
36-
Map<String, String> parametersMap = new HashMap<>();
36+
Map<String, Object> parametersMap = new HashMap<>();
3737

3838
/**
3939
* Cohere multi-modal model expects either image or texts, not both.
4040
* For image, customer can use this pre-process function. For texts, customer can use
4141
* connector.pre_process.cohere.embedding
4242
* Cohere expects An array of image data URIs for the model to embed. Maximum number of images per call is 1.
4343
*/
44-
parametersMap.put("images", inputData.getDocs().get(0));
44+
parametersMap.put("images", inputData.getDocs());
4545
return RemoteInferenceInputDataSet
4646
.builder()
4747
.parameters(convertScriptStringToJsonString(Map.of("parameters", parametersMap)))

common/src/test/java/org/opensearch/ml/common/connector/functions/preprocess/CohereMultiModalEmbeddingPreProcessFunctionTest.java

+1-1
Original file line numberDiff line numberDiff line change
@@ -66,7 +66,7 @@ public void testProcess_whenCorrectInput_expectCorrectOutput() {
6666
MLInput mlInput = MLInput.builder().algorithm(FunctionName.TEXT_EMBEDDING).inputDataset(textDocsInputDataSet).build();
6767
RemoteInferenceInputDataSet dataSet = function.apply(mlInput);
6868
assertEquals(1, dataSet.getParameters().size());
69-
assertEquals("imageString", dataSet.getParameters().get("images"));
69+
assertEquals("[\"imageString\"]", dataSet.getParameters().get("images"));
7070

7171
}
7272

docs/remote_inference_blueprints/cohere_connector_embedding_blueprint.md

+4-4
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
### Cohere Embedding Connector Blueprint:
22

3-
This blueprint will show you how to connect a Cohere embedding model to your Opensearch cluster, including creating a k-nn index and your own Embedding pipeline. You will require a Cohere API key to create a connector.
3+
This blueprint will show you how to connect a Cohere embedding model to your OpenSearch cluster, including creating a k-nn index and your own Embedding pipeline. You will require a Cohere API key to create a connector.
44

55
Cohere currently offers the following Embedding models (with model name and embedding dimensions). Note that only the following have been tested with the blueprint guide.
66

@@ -97,7 +97,7 @@ The last step is to deploy your model. Use the `model_id` returned by the regist
9797
POST /_plugins/_ml/models/<MODEL_ID>/_deploy
9898
```
9999

100-
This will once again spawn a task to deploy your Model, with a response that will look like:
100+
This will once again spawn a task to deploy your model, with a response that will look like:
101101

102102
```json
103103
{
@@ -113,11 +113,11 @@ You can run the GET tasks request again to verify the status.
113113
GET /_plugins/_ml/tasks/<TASK_ID>
114114
```
115115

116-
Once this is complete, your Model is deployed and ready!
116+
Once this is complete, your model is deployed and ready!
117117

118118
##### 1e. Test model
119119

120-
You can try this request to test that the Model behaves correctly:
120+
You can try this request to test that the model behaves correctly:
121121

122122
```json
123123
POST /_plugins/_ml/models/<MODEL_ID_HERE>/_predict

0 commit comments

Comments
 (0)