Skip to content

Commit 382d00f

Browse files
authored
Fix qwen export with gptq (#639)
1 parent 10ac43e commit 382d00f

File tree

1 file changed

+5
-1
lines changed

1 file changed

+5
-1
lines changed

optimum/exporters/openvino/model_patcher.py

+5-1
Original file line numberDiff line numberDiff line change
@@ -483,7 +483,11 @@ def __init__(
483483
model.config.bf16 = False
484484
model.config.fp16 = False
485485
if self.original_fp16 or self.original_bf16:
486-
model.to(torch.float32)
486+
# GPTQ models does to support casting to dtype
487+
try:
488+
model.to(torch.float32)
489+
except Exception:
490+
pass
487491
model.transformer.rotary_emb(2048)
488492

489493
def __enter__(self):

0 commit comments

Comments
 (0)