Project 4 GSoC

[raghavx1]

Dear Admins
@adrianboguszewski @mlukasze @p-wysocki

I write to express my interest in the project "Continuous Face-Detection with Automatic Device Switching on AI PCs using OpenVINO AUTO feature." Would you like to suggest any GFI related to this project?

I would be grateful if you could furnish me with the email ids of the mentors: Shivam Basia and Aishwarye Omer.

Thank You
Raghav Sharma

[raghavx1]

@sbasia @omeraish

Hi, I have a question. As inference is less compute intensive than training, even using not the most powerful resources will give us satisfactory results. In the problem that we are trying to solve is one type of processor more valuable than the other, is compute time for one unit more valuable than the other. (cpu<gpu<npu)?

Thank You
Raghav Sharma

[omeraish]

Hello @raghavx1

Thank you for reaching out and showing interest in our topic.

Through this topic, we are trying to extend OpenVINO AUTO feature through which the inference can be offloaded to any inference engine during runtime without sacrificing accuracy and performance. Moreover, it is important that during this process the system remains stable and does not cause any device/engine failure or BSOD.

AI PC has three processors available for inference: CPU (for low latency apps), GPU (for high throughput), NPU (for low power operations). One type of processor is not valuable than the other, but the use case determines the best processor for inference.

In the latest iteration of AI PC (Lunar Lake), all inference engines are powerful and capable of supporting several models and therefore using and extending AUTO feature to offload inference on any device during runtime would be interesting and would prove the consistent performance of AI PC as well

Please feel free to follow up on this

P.S. Please keep checking the deadline for applications and note that you would still need to complete the prerequisite before submitting the application/proposal.

[cyberholics]

Hello @omeraish ,

Thank you for your feedback. I’d like to clarify the project scope further. Based on the expected outcomes, it seems that we are to implement a system that can prompt the user to switch devices during runtime. Should my primary focus be on extending and modifying the underlying codebase of the AUTO feature to improve dynamic device switching, or should I concentrate on developing a demonstration face-detection application that leverages the current functionality?

Additionally, regarding the device monitoring component: could you please provide guidance on how you envision implementing it? For example, what key performance metrics or thresholds should we consider to determine when a device switch is needed? Would you recommend using built-in OpenVINO monitoring tools, or should we integrate external libraries for more detailed hardware load analysis?

Thank you for your help.

[omeraish]

Hello @cyberholics

Our main goal is to develop and demonstrate a face-detection application that leverages AUTO feature during runtime. Currently AUTO feature does support dynamic switching between processors, but it has not been extensively used to determine if the feature works as expected out of the box. As the project develops, and we feel the need to enhance and modify the underlying codebase for the feature then we will attempt that and then implement the app for demonstration.

A user prompt is one of several ways to dynamically switch the inference device. Another option here would be to disable the device under test during runtime and verify if the inferencing automatically switches to another device to complete inference. In case the original/primary device is enabled again, the app may or may not switch back inferencing but that's not decided on our end.

There can be several parameters to determine when a device switch is needed. While developing a proof of concept, we can determine and fix few key metrics like power utilization, memory available etc. which we can monitor and use in our code.

P.S. Please keep checking the deadline for applications and note that you would still need to complete the prerequisite before submitting the application/proposal.

[raghavx1]

Hi @omeraish Thank you for replying, this is exactly what I wanted to know. I'll work a bit more on the topic and will get back to you.

In the meantime can I get your email id. Is it alright if we shift this to email?

Best Regards
Raghav Sharma

[raghavx1]

Dear @sbasia and @omeraish
I don't see any GFIs up for grabs can you please see if we can raise a new issue related to our project? I have contributed to OV before and was able to get 5 PRs merged but that was last year, and I don't think I can use them as the pre-requisite task, can you suggest what I should do?

Thank You
Raghav Sharma

[omeraish]

Hello @raghavx1

That's really great if you have already contributed to OpenVINO. Please provide details of your previous contribution. We will use it to evaluate and determine if additional contribution is needed. Meanwhile, go through the application template and start drafting your approach accordingly.

We will get back to you with our email ids for future communication once we are cleared for private communication by our program coordinators

[raghavx1]

Hi @omeraish
My previous contributions are:
ONNX

1. [ONNX] Align behavior of ONNX Frontend operator ReduceMean-11, 13, 18 with original framework #23148
2. [ONNX] Extend ONNX Frontend with BitwiseXor-18 operator #21816
3. [ONNX] Extend ONNX Frontend with BitwiseAnd-18 operator #21585

TFFE
4. #21076
5. #23028

Thank You
Raghav Sharma

[raghavx1]

Hi @omeraish
Do you think additional contribution is needed?

Thank You
Raghav Sharma

[omeraish]

Hello @raghavx1

Please find below the email id for further communication. Let us know how the application/proposal is shaping up and always check the deadlines.

aishwarye.omer@intel.com
shivam.basia@intel.com

[raghavx1]

Hi @omeraish , Thank you. I'll just make a few changes and will send a first draft.