Skip to content

Commit dcf82af

Browse files
authored
Add default quantization int4 config for Mixtral-8x7B (#576)
1 parent 2d14e25 commit dcf82af

File tree

1 file changed

+1
-0
lines changed

1 file changed

+1
-0
lines changed

optimum/intel/openvino/configuration.py

+1
Original file line numberDiff line numberDiff line change
@@ -96,6 +96,7 @@
9696
"openlm-research/open_llama_3b": {"bits": 4, "sym": True, "group_size": 64, "all_layers": True},
9797
"tiiuae/falcon-7b": {"bits": 4, "sym": True, "group_size": 64, "all_layers": True},
9898
"psmathur/orca_mini_3b": {"bits": 4, "sym": True, "group_size": 64, "all_layers": True},
99+
"mistralai/Mixtral-8x7B-v0.1": {"bits": 4, "sym": True, "group_size": 128, "ratio": 0.8},
99100
}
100101

101102

0 commit comments

Comments
 (0)