Skip to content

Commit be6666b

Browse files
authored
Updates weight quantization section in the docs (#593)
1 parent 6e8cd3d commit be6666b

File tree

1 file changed

+11
-1
lines changed

1 file changed

+11
-1
lines changed

docs/source/optimization_ov.mdx

+11-1
Original file line numberDiff line numberDiff line change
@@ -82,7 +82,17 @@ from optimum.intel import OVModelForCausalLM, OVWeightQuantizationConfig
8282

8383
model = OVModelForCausalLM.from_pretrained(
8484
model_id,
85-
export=True,
85+
quantization_config=OVWeightQuantizationConfig(bits=4),
86+
)
87+
```
88+
89+
You can tune quantization parameters to achieve a better performance accuracy trade-off as follows:
90+
91+
```python
92+
from optimum.intel import OVModelForCausalLM, OVWeightQuantizationConfig
93+
94+
model = OVModelForCausalLM.from_pretrained(
95+
model_id,
8696
quantization_config=OVWeightQuantizationConfig(bits=4, sym=False, ratio=0.8, dataset="ptb"),
8797
)
8898
```

0 commit comments

Comments
 (0)