Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Docs] Add VLM use case #1907

Merged
merged 38 commits into from
Mar 20, 2025
Merged
Changes from 1 commit
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
6e3d7c5
Add short description of image generation pipelines
yatarkan Mar 7, 2025
09d975c
Add model preparation nav menu, add page for download pre-converted m…
yatarkan Mar 7, 2025
c02e124
Fix position
yatarkan Mar 7, 2025
e17b4c9
Add link to model preparation page in introduction
yatarkan Mar 7, 2025
bb91823
Add optimum cli command component
yatarkan Mar 10, 2025
a1eb33e
Add content for convert to openvino page
yatarkan Mar 10, 2025
dd480fd
Disable separate guides page
yatarkan Mar 10, 2025
89fb5b5
Remove tm
yatarkan Mar 10, 2025
3bb188e
Add link to dedicated model preparation guide page, reuse in llm and …
yatarkan Mar 10, 2025
5602e88
Add code examples for all image generation pipelines
yatarkan Mar 10, 2025
86639f7
Move lora adapters to separate page, add references
yatarkan Mar 10, 2025
fc99109
Fix links
yatarkan Mar 10, 2025
3261fa8
Merge branch 'master' into docs-pages-image-generation
yatarkan Mar 11, 2025
fdc38b1
Update site/docs/guides/lora-adapters.mdx
yatarkan Mar 11, 2025
d14e98f
Reuse use cases note
yatarkan Mar 11, 2025
b6a0b02
Move convert model section to shared
yatarkan Mar 11, 2025
94de0f3
Rename text generation directory
yatarkan Mar 11, 2025
cc836a9
Rename image generation directory
yatarkan Mar 11, 2025
e21d48f
Remove digits from use cases directory names
yatarkan Mar 11, 2025
d2a6fa1
Update use cases links
yatarkan Mar 11, 2025
c115768
Add link to supported llm models
yatarkan Mar 11, 2025
4b2aac0
Add initial page for VLM use case
yatarkan Mar 11, 2025
7714109
Reuse vlm code examples
yatarkan Mar 11, 2025
0e51bbb
Move basic generation configuration to separate file and reuse in llm…
yatarkan Mar 12, 2025
ed1001d
Move chat scenario and streaming to separate page, reuse in LLM and V…
yatarkan Mar 12, 2025
feb6b90
Fix link
yatarkan Mar 12, 2025
360323e
Merge branch 'master' into docs-pages-vlm
yatarkan Mar 18, 2025
c557f7d
Fix VLM run model section
yatarkan Mar 18, 2025
09baa88
Add links to model sources
yatarkan Mar 18, 2025
61899d6
Remove mentions of models
yatarkan Mar 18, 2025
d880d3e
Add links to pipeline classes api reference
yatarkan Mar 18, 2025
f4dafd7
Add example model export commands for use cases
yatarkan Mar 18, 2025
c6b50c1
Add link to KV cache page
yatarkan Mar 18, 2025
fb592eb
Add link to chat samples
yatarkan Mar 18, 2025
a463de0
Fix c++ custom streamer type
yatarkan Mar 18, 2025
6bbe8b9
Add comment
yatarkan Mar 18, 2025
58e6287
Add direct links to samples for use cases
yatarkan Mar 18, 2025
0a8c967
Merge branch 'master' into docs-pages-vlm
yatarkan Mar 19, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Move lora adapters to separate page, add references
yatarkan committed Mar 10, 2025
commit 86639f73bd90e5128e5dee319349236d12c193b2
5 changes: 0 additions & 5 deletions site/docs/guides/lora-adapters.md

This file was deleted.

101 changes: 101 additions & 0 deletions site/docs/guides/lora-adapters.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
---
sidebar_position: 4
---

# LoRA Adapters

## Overview

Low-Rank Adaptation (LoRA) is a technique for efficiently fine-tuning large models without changing the base model's weights.
LoRA adapters enable customization of model outputs for specific tasks, styles, or domains while requiring significantly fewer computational resources than full fine-tuning.

OpenVINO GenAI provides built-in support for LoRA adapters in text generation and image generation pipelines.
This capability allows you to dynamically switch between or combine multiple adapters without recompiling the model.

:::info
See [Supported Models](/docs/supported-models/) for the list of models that support LoRA adapters.
:::

## Key Features

- **Dynamic Adapter Application:** Apply LoRA adapters at runtime without model recompilation.
- **Multiple Adapter Support:** Blend effects from multiple adapters with different weights.
- **Adapter Switching:** Change adapters between generation calls without pipeline reconstruction.
- **Safetensors Format:** Support for industry-standard safetensors format for adapter files.

## Using LoRA Adapters

<LanguageTabs>
<TabItemPython>
```python
import openvino_genai as ov_genai

# Initialize pipeline with adapters
adapter_config = ov_genai.AdapterConfig()

# Add multiple adapters with different weights
adapter1 = ov_genai.Adapter("path/to/lora1.safetensors")
adapter2 = ov_genai.Adapter("path/to/lora2.safetensors")

adapter_config.add(adapter1, alpha=0.5)
adapter_config.add(adapter2, alpha=0.5)

pipe = ov_genai.LLMPipeline(
model_path,
"CPU",
adapters=adapter_config
)

# Generate with current adapters
output1 = pipe.generate("Generate story about", max_new_tokens=100)

# Switch to different adapter configuration
new_config = ov_genai.AdapterConfig()
new_config.add(adapter1, alpha=1.0)
output2 = pipe.generate(
"Generate story about",
max_new_tokens=100,
adapters=new_config
)
```
</TabItemPython>
<TabItemCpp>
```cpp
#include "openvino/genai/llm_pipeline.hpp"

int main() {
ov::genai::AdapterConfig adapter_config;

// Add multiple adapters with different weights
ov::genai::Adapter adapter1("path/to/lora1.safetensors");
ov::genai::Adapter adapter2("path/to/lora2.safetensors");

adapter_config.add(adapter1, 0.5f);
adapter_config.add(adapter2, 0.5f);

ov::genai::LLMPipeline pipe(
model_path,
"CPU",
ov::genai::adapters(adapter_config)
);

// Generate with current adapters
auto output1 = pipe.generate("Generate story about", ov::genai::max_new_tokens(100));

// Switch to different adapter configuration
ov::genai::AdapterConfig new_config;
new_config.add(adapter1, 1.0f);
auto output2 = pipe.generate(
"Generate story about",
ov::genai::adapters(new_config),
ov::genai::max_new_tokens(100)
);
}
```
</TabItemCpp>
</LanguageTabs>

## LoRA Adapters Sources

1. **Hugging Face:** Browse adapters for various models at [huggingface.co/models](https://huggingface.co/models?other=lora) using "LoRA" filter.
2. **Civitai:** For stable diffusion models, [Civitai](https://civitai.com/) offers a wide range of LoRA adapters for various styles and subjects.
Original file line number Diff line number Diff line change
@@ -1,74 +1,6 @@
### Working with LoRA Adapters

LoRA (Low-Rank Adaptation) allows you to fine-tune models for specific tasks efficiently.
OpenVINO GenAI supports dynamic loading and switching between LoRA adapters without recompilation.
LoRA adapters can be used to customize LLM outputs for specific tasks or styles.
In text generation, adapters can help models perform better at particular activities like coding, creative writing, or domain-specific knowledge.

<LanguageTabs>
<TabItemPython>
```python
import openvino_genai as ov_genai

# Initialize pipeline with adapters
adapter_config = ov_genai.AdapterConfig()

# Add multiple adapters with different weights
adapter1 = ov_genai.Adapter("path/to/lora1.safetensors")
adapter2 = ov_genai.Adapter("path/to/lora2.safetensors")

adapter_config.add(adapter1, alpha=0.5)
adapter_config.add(adapter2, alpha=0.5)

pipe = ov_genai.LLMPipeline(
model_path,
"CPU",
adapters=adapter_config
)

# Generate with current adapters
output1 = pipe.generate("Generate story about", max_new_tokens=100)

# Switch to different adapter configuration
new_config = ov_genai.AdapterConfig()
new_config.add(adapter1, alpha=1.0)
output2 = pipe.generate(
"Generate story about",
max_new_tokens=100,
adapters=new_config
)
```
</TabItemPython>
<TabItemCpp>
```cpp
#include "openvino/genai/llm_pipeline.hpp"

int main() {
ov::genai::AdapterConfig adapter_config;

// Add multiple adapters with different weights
ov::genai::Adapter adapter1("path/to/lora1.safetensors");
ov::genai::Adapter adapter2("path/to/lora2.safetensors");

adapter_config.add(adapter1, 0.5f);
adapter_config.add(adapter2, 0.5f);

ov::genai::LLMPipeline pipe(
model_path,
"CPU",
ov::genai::adapters(adapter_config)
);

// Generate with current adapters
auto output1 = pipe.generate("Generate story about", ov::genai::max_new_tokens(100));

// Switch to different adapter configuration
ov::genai::AdapterConfig new_config;
new_config.add(adapter1, 1.0f);
auto output2 = pipe.generate(
"Generate story about",
ov::genai::adapters(new_config),
ov::genai::max_new_tokens(100)
);
}
```
</TabItemCpp>
</LanguageTabs>
Refer to the [LoRA Adapters](/docs/guides/lora-adapters.mdx) for more details on working with LoRA adapters.
Original file line number Diff line number Diff line change
@@ -71,4 +71,6 @@ For the full list of generation parameters, refer to the [API reference](https:/

### Working with LoRA Adapters

TBD
For image generation models like Stable Diffusion, LoRA adapters can modify the generation process to produce images with specific artistic styles, content types, or quality enhancements.

Refer to the [LoRA Adapters](/docs/guides/lora-adapters.mdx) for more details on working with LoRA adapters.