Skip to content

Commit 3879e4e

Browse files
jiqing-fengecharlaix
andauthoredMar 22, 2024
Add ipex readme (#595)
* add ipex readme * Update README.md Co-authored-by: Ella Charlaix <80481427+echarlaix@users.noreply.github.com> * fix readme * fix readme * Update README.md Co-authored-by: Ella Charlaix <80481427+echarlaix@users.noreply.github.com> * fix ipex readme * Update README.md Co-authored-by: Ella Charlaix <80481427+echarlaix@users.noreply.github.com> * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md --------- Co-authored-by: Ella Charlaix <80481427+echarlaix@users.noreply.github.com>
1 parent 64d9a49 commit 3879e4e

File tree

1 file changed

+20
-0
lines changed

1 file changed

+20
-0
lines changed
 

‎README.md

+20
Original file line numberDiff line numberDiff line change
@@ -202,6 +202,26 @@ Quantization aware training (QAT) is applied in order to simulate the effects of
202202
You can find more examples in the [documentation](https://huggingface.co/docs/optimum/intel/index).
203203

204204

205+
## IPEX
206+
To load your IPEX model, you can just replace your `AutoModelForXxx` class with the corresponding `IPEXModelForXxx` class. You can set `export=True` to load a PyTorch checkpoint, export your model via TorchScript and apply IPEX optimizations : both operators optimization (replaced with customized IPEX operators) and graph-level optimization (like operators fusion) will be applied on your model.
207+
```diff
208+
from transformers import AutoTokenizer, pipeline
209+
- from transformers import AutoModelForCausalLM
210+
+ from optimum.intel import IPEXModelForCausalLM
211+
212+
213+
model_id = "gpt2"
214+
- model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.bfloat16)
215+
+ model = IPEXModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.bfloat16, export=True)
216+
tokenizer = AutoTokenizer.from_pretrained(model_id)
217+
pipe = pipeline("text-generation", model=model, tokenizer=tokenizer)
218+
results = pipe("He's a dreadful magician and")
219+
220+
```
221+
222+
For more details, please refer to the [documentation](https://intel.github.io/intel-extension-for-pytorch/#introduction).
223+
224+
205225
## Running the examples
206226

207227
Check out the [`examples`](https://github.com/huggingface/optimum-intel/tree/main/examples) directory to see how 🤗 Optimum Intel can be used to optimize models and accelerate inference.

0 commit comments

Comments
 (0)