You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
hello guys, I'm trying to deploy LLM on SA8295 with NPU to accelerate inference. I try serval times but not success with the guidance https://github.com/quic/ai-hub-apps/tree/main/tutorials/llm_on_genie . but fail. I found Only supports Hexagon v73 and onward architectures. and the SA8295 support v68 on the doc qairt/2.27.7.241014/docs/QNN/general/htp/htp_backend.html.
I hope to know if SA8295 not support the NPU inference :(
Looking forward to your reply. Thank you.
The text was updated successfully, but these errors were encountered:
ecccccsgo
changed the title
[question] Can I deploy LLM
[question] Can I deploy LLM on SA8295 with Hexagon v68?
Nov 11, 2024
i found the new chipset Snapdragon 8cx Gen 3 (SC8280X) with v68, and it seem that it support LLM with NPU in https://docs.qualcomm.com/bundle/publicresource/topics/80-63442-50/overview.html. Although it was PC chipset with msvc.
mestrona-3
added
the
question
Please ask any questions on Slack. This issue will be closed once responded to.
label
Nov 13, 2024
hello guys, I'm trying to deploy LLM on SA8295 with NPU to accelerate inference. I try serval times but not success with the guidance
https://github.com/quic/ai-hub-apps/tree/main/tutorials/llm_on_genie
. but fail. I foundOnly supports Hexagon v73 and onward architectures.
and the SA8295 support v68 on the docqairt/2.27.7.241014/docs/QNN/general/htp/htp_backend.html
.I hope to know if SA8295 not support the NPU inference :(
Looking forward to your reply. Thank you.
The text was updated successfully, but these errors were encountered: