Skip to content

Commit 8dd2a89

Browse files
authored
Udpate openvino export CLI documentation (#746)
* udpate openvino documentation * rephrase
1 parent 52875b9 commit 8dd2a89

File tree

1 file changed

+3
-3
lines changed

1 file changed

+3
-3
lines changed

docs/source/inference.mdx

+3-3
Original file line numberDiff line numberDiff line change
@@ -30,8 +30,8 @@ As shown in the table below, each task is associated with a class enabling to au
3030
| `fill-mask` | `OVModelForMaskedLM` |
3131
| `image-classification` | `OVModelForImageClassification` |
3232
| `audio-classification` | `OVModelForAudioClassification` |
33-
| `text-generation` | `OVModelForCausalLM` |
34-
| `text2text-generation` | `OVModelForSeq2SeqLM` |
33+
| `text-generation-with-past` | `OVModelForCausalLM` |
34+
| `text2text-generation-with-past` | `OVModelForSeq2SeqLM` |
3535
| `automatic-speech-recognition` | `OVModelForSpeechSeq2Seq` |
3636
| `image-to-text` | `OVModelForVision2Seq` |
3737

@@ -46,7 +46,7 @@ optimum-cli export openvino --model gpt2 ov_model
4646

4747
The example above illustrates exporting a checkpoint from the 🤗 Hub. When exporting a local model, first make sure that you saved both the model’s weights and tokenizer files in the same directory (`local_path`).
4848
When using CLI, pass the `local_path` to the model argument instead of the checkpoint name of the model hosted on the Hub and provide the `--task` argument. You can review the list of supported tasks in the 🤗 [Optimum documentation](https://huggingface.co/docs/optimum/exporters/task_manager). If task argument is not provided, it will default to the model architecture without any task specific head.
49-
Here we set the `task` to `text-generation-with-past`, with the `-with-past` suffix enabling the re-use of the pre-computed key/values hidden-states `use_cache=True`.
49+
The `-with-past` suffix enable the re-use of the pre-computed key/values hidden-states and is the recommended option, to export the model without (equivalent to `use_cache=False`), you will need to remove this suffix.
5050

5151
```bash
5252
optimum-cli export openvino --model local_path --task text-generation-with-past ov_model

0 commit comments

Comments
 (0)