Skip to content

Commit a9ab37e

Browse files
added qwen1.5-7b to model list (openvinotoolkit#316)
I've verified support of Qwen1.5-7B by OpenVINO and then added it to the github workflow and readme.md ``` (base) root@8tvt:~/openvino.genai/llm_bench/python# ../../text_generation/causal_lm/cpp/build/greedy_causal_lm qwen/pytorch/dldt/FP32/ "Why is the Sun yellow?" The Sun does not actually appear yellow to us when we look at it. In fact, it appears white because it emits light across a wide range of wavelengths, including all the colors of the visible spectrum. When this light reaches our eyes, our eyes combine the different colors to create the perception of white. ``` --------- Co-authored-by: Pavel Esir <pavel.esir@gmail.com>
1 parent 3d1e82b commit a9ab37e

File tree

2 files changed

+28
-0
lines changed

2 files changed

+28
-0
lines changed

.github/workflows/causal_lm_cpp.yml

+26
Original file line numberDiff line numberDiff line change
@@ -192,6 +192,32 @@ jobs:
192192
source ./ov/setupvars.sh
193193
convert_tokenizer ./Qwen-7B-Chat/pytorch/dldt/FP16/ --output ./Qwen-7B-Chat/pytorch/dldt/FP16/ --with-detokenizer --trust-remote-code
194194
timeout 50s ./build/beam_search_causal_lm ./Qwen-7B-Chat/pytorch/dldt/FP16/ 69 > ./pred.txt
195+
cpp-beam_search_causal_lm-Qwen1_5-7B-Chat:
196+
runs-on: ubuntu-20.04-16-cores
197+
steps:
198+
- uses: actions/checkout@v4
199+
with:
200+
submodules: recursive
201+
- uses: actions/setup-python@v4
202+
with:
203+
python-version: 3.8
204+
- name: Install OpenVINO
205+
run: |
206+
mkdir ./ov/
207+
curl https://storage.openvinotoolkit.org/repositories/openvino/packages/nightly/2024.1.0-14645-e6dc0865128/l_openvino_toolkit_ubuntu20_2024.1.0.dev20240304_x86_64.tgz | tar --directory ./ov/ --strip-components 1 -xz
208+
sudo ./ov/install_dependencies/install_openvino_dependencies.sh
209+
- name: Download, convert and build
210+
run: |
211+
source ./ov/setupvars.sh
212+
python -m pip install --upgrade-strategy eager "optimum>=1.14" -r ./llm_bench/python/requirements.txt ./thirdparty/openvino_tokenizers/[transformers] --extra-index-url https://download.pytorch.org/whl/cpu && python ./llm_bench/python/convert.py --model_id Qwen/Qwen1.5-7B-Chat --output_dir ./Qwen1.5-7B-Chat/ --precision FP16 &
213+
cmake -DCMAKE_BUILD_TYPE=Release -S ./text_generation/causal_lm/cpp/ -B ./build/
214+
cmake --build ./build/ --config Release -j
215+
wait
216+
- name: Run
217+
run: |
218+
source ./ov/setupvars.sh
219+
convert_tokenizer ./Qwen1.5-7B-Chat/pytorch/dldt/FP16/ --output ./Qwen1.5-7B-Chat/pytorch/dldt/FP16/ --with-detokenizer --trust-remote-code
220+
timeout 50s ./build/beam_search_causal_lm ./Qwen1.5-7B-Chat/pytorch/dldt/FP16/ "你好!" > ./pred_qwen15.txt
195221
cpp-beam_search_causal_lm-Phi-2:
196222
runs-on: ubuntu-20.04-16-cores
197223
steps:

text_generation/causal_lm/cpp/README.md

+2
Original file line numberDiff line numberDiff line change
@@ -130,6 +130,8 @@ To enable Unicode characters for Windows cmd open `Region` settings from `Contro
130130
6. Qwen
131131
1. https://huggingface.co/Qwen/Qwen-7B-Chat
132132
2. https://huggingface.co/Qwen/Qwen-7B-Chat-Int4 - refer to
133+
3. https://huggingface.co/Qwen/Qwen1.5-7B-Chat
134+
4. https://huggingface.co/Qwen/Qwen1.5-7B-Chat-GPTQ-Int4
133135
[Qwen-7B-Chat-Int4 - Torch not compiled with CUDA enabled](../../../llm_bench/python/doc/NOTES.md#qwen-7b-chat-int4---torch-not-compiled-with-cuda-enabled)
134136
in case of `AssertionError`
135137
7. Dolly

0 commit comments

Comments
 (0)