Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[NPU] convert phi3 model met load tokenizer issue #1932

Open
bopeng1234 opened this issue Mar 18, 2025 · 1 comment
Open

[NPU] convert phi3 model met load tokenizer issue #1932

bopeng1234 opened this issue Mar 18, 2025 · 1 comment

Comments

@bopeng1234
Copy link

I am following https://docs.openvino.ai/2025/openvino-workflow-generative/inference-with-genai/inference-with-genai-on-npu.html to convert the phi3 model, my command

pip install  nncf==2.14.1 onnx==1.17.0 optimum-intel==1.21.0
pip install openvino==2025.0 openvino-tokenizers==2025.0 openvino-genai==2025.0
optimum-cli export openvino -m microsoft/Phi-3-mini-4k-instruct --weight-format int4 --sym --ratio 1.0 --group-size -1 ov_model\Phi-3-mini-4k-instruct-intel-opt-int4-sym-channelwised

when the quantization running, it shows warning Could not load tokenizer using specified model ID or path. OpenVINO tokenizer/detokenizer models won't be generated. Exception: Invalid version: '2025.0.0.0-476-2b2420220f9'

and after quantized, load the model with ov_genai.LLMPipeline(model_path, "CPU")

met issue:

  File "C:\Users\gta\Downloads\script\npu_workspace\official_npu_ov\run.py", line 6, in <module>
    pipe = ov_genai.LLMPipeline(model_path, "CPU")#, pipeline_config)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: Check 'ov_tokenizer || ov_detokenizer' failed at C:\Jenkins\workspace\private-ci\ie\build-windows-vs2022\b\repos\openvino.genai\src\cpp\src\tokenizer.cpp:196:
Neither tokenizer nor detokenzier models were provided

which step did I make a mistake?

@bopeng1234
Copy link
Author

use 2024.6, the issue gone

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant