Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

List available backend providers (ipex, openvino) #995

Open
michaelfeil opened this issue Nov 11, 2024 · 1 comment
Open

List available backend providers (ipex, openvino) #995

michaelfeil opened this issue Nov 11, 2024 · 1 comment

Comments

@michaelfeil
Copy link

We are looking for a way how to find out which "backends" are available.

Is there an equivalent of:

import onnxruntime as ort

providers = ort.get_available_providers()
print(providers)

We don't have the luxury of trying out a OVAutoModel with openvino, notice its not available & then switch to the default Optimum model. Any way to detect this?

FYI: We are working on michaelfeil/infinity#454.

@echarlaix
Copy link
Collaborator

Hi @michaelfeil, apologies for the delay! If I'm understanding it correctly you'd like to check whether optimum-intel was correctly installed before loading your model with a OVModelXxx class ? Something that you can do is to check importlib.util.find_spec("optimum.intel") directly : here is an example of what has been done for langchain for example https://github.com/langchain-ai/langchain/blob/6c7c8a164f17194c83cece0c5e74a110c29c36a7/libs/partners/huggingface/langchain_huggingface/utils/import_utils.py#L78 and https://github.com/langchain-ai/langchain/blob/6c7c8a164f17194c83cece0c5e74a110c29c36a7/libs/partners/huggingface/langchain_huggingface/llms/huggingface_pipeline.py#L147

Also before loading your model, make sure openvino is also correctly installed, you can use directly is_openvino_available :

def is_openvino_available():

Let me know if that helps or if you'd need some additional integration in optimum to make things easier

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants