Add NPU Engine #31
Add NPU Engine #31
Conversation
fix the logic error of if else
|
@szeyu The NPU support looks good. I added some reviews. If those are resolved then it should be go to be merged. Before merging this PR, can you first merged the OpenVINO vison model PR then only merge this NPU Engine? |
merge npu merge npu merge npu merge npu merge npu merge npu merge npu merge npu
| @@ -56,6 +56,16 @@ def __init__(self, model_path: str, vision: bool, device: str = "xpu", backend: | |||
|
|
|||
| self.engine = OnnxruntimeEngine(self.model_path, self.vision, self.device) | |||
| logger.info(f"Initializing onnxruntime backend ({backend.upper()}): OnnxruntimeEngine") | |||
|
|
|||
| elif self.backend == "npu": | |||
There was a problem hiding this comment.
Can you find a way to detect if this is Intel or AMD machine?
If user specify
npudevice, you have to check if it is Intel or AMD first.
> If the machine is Intel, then you continue to load model usingintel_npu_engine.py.
> If the machine is AMD, then you throw error message saying that NPU support on AMD platform is not supported yet.
| @@ -0,0 +1,268 @@ | |||
| import contextlib | |||
There was a problem hiding this comment.
Can you rename npu_engine.py into intel_npu_engine.py as this is NPU code for Intel only?
Do DM me on Whatsapp to discuss about this if you think otherwise.
Add NPU Engine #28
Reference : Phi-3 Cookbook - Intel NPU acceleration library
Update in:
modelui.pyengine.pysetup.pyREADME.mdAdd:
npu_egnine.pyrequirements-npu.txtnpu_models.md