Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[question] Can I deploy LLM on SA8295 with Hexagon v68? #124

Open
ecccccsgo opened this issue Nov 11, 2024 · 1 comment
Open

[question] Can I deploy LLM on SA8295 with Hexagon v68? #124

ecccccsgo opened this issue Nov 11, 2024 · 1 comment
Labels
question Please ask any questions on Slack. This issue will be closed once responded to.

Comments

@ecccccsgo
Copy link

ecccccsgo commented Nov 11, 2024

hello guys, I'm trying to deploy LLM on SA8295 with NPU to accelerate inference. I try serval times but not success with the guidance https://github.com/quic/ai-hub-apps/tree/main/tutorials/llm_on_genie . but fail. I found Only supports Hexagon v73 and onward architectures. and the SA8295 support v68 on the doc qairt/2.27.7.241014/docs/QNN/general/htp/htp_backend.html.

I hope to know if SA8295 not support the NPU inference :(

Looking forward to your reply. Thank you.

@ecccccsgo ecccccsgo changed the title [question] Can I deploy LLM [question] Can I deploy LLM on SA8295 with Hexagon v68? Nov 11, 2024
@ecccccsgo
Copy link
Author

i found the new chipset Snapdragon 8cx Gen 3 (SC8280X) with v68, and it seem that it support LLM with NPU in https://docs.qualcomm.com/bundle/publicresource/topics/80-63442-50/overview.html. Although it was PC chipset with msvc.

@mestrona-3 mestrona-3 added the question Please ask any questions on Slack. This issue will be closed once responded to. label Nov 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Please ask any questions on Slack. This issue will be closed once responded to.
Projects
None yet
Development

No branches or pull requests

2 participants