Skip to content

Commit db0fb9a

Browse files
authored
Replace 'CACHE_DIR' with 'NPUW_CACHE_DIR' in StatefulLLMPipeline (openvinotoolkit#1489)
Handle `CACHE_DIR` in `StatefulLLMPipeline` the same way as in `StatelessLLMPipeline`
1 parent 65e8362 commit db0fb9a

File tree

1 file changed

+4
-1
lines changed

1 file changed

+4
-1
lines changed

src/cpp/src/llm_pipeline_static.cpp

+4-1
Original file line numberDiff line numberDiff line change
@@ -739,7 +739,10 @@ std::shared_ptr<ov::CompiledModel> StatefulLLMPipeline::setupAndCompileModel(
739739

740740
rename_key(pipeline_config, "PREFILL_CONFIG", "NPUW_LLM_PREFILL_CONFIG");
741741
rename_key(pipeline_config, "GENERATE_CONFIG", "NPUW_LLM_GENERATE_CONFIG");
742-
742+
743+
// Replace CACHE_DIR option if NPUW is enabled
744+
set_npuw_cache_dir(pipeline_config);
745+
743746
return std::make_shared<ov::CompiledModel>(genai::utils::singleton_core().compile_model(model, "NPU", pipeline_config));
744747
}
745748

0 commit comments

Comments
 (0)