Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: batch inference for nitro #41

Closed
Tracked by #1081
tikikun opened this issue Oct 2, 2023 · 1 comment · Fixed by #101
Closed
Tracked by #1081

feat: batch inference for nitro #41

tikikun opened this issue Oct 2, 2023 · 1 comment · Fixed by #101
Assignees
@tikikun
Labels
P1: important Important feature / fix
Milestone
Nitro v0.2

Comments

@tikikun
Copy link
Contributor

tikikun commented Oct 2, 2023

upstream from

ggerganov/llama.cpp#3228
@tikikun tikikun added the P0: critical Mission critical label Oct 2, 2023
@tikikun tikikun self-assigned this Oct 2, 2023
@hiro-v hiro-v moved this to In Progress in Jan & Cortex Oct 2, 2023
@freelerobot freelerobot added P1: important Important feature / fix type: enhancement and removed P0: critical Mission critical labels Oct 9, 2023
@freelerobot freelerobot added this to the Nitro v0.2 milestone Oct 9, 2023
@hiro-v hiro-v moved this from In Progress to Todo in Jan & Cortex Oct 20, 2023
@tikikun
Copy link
Contributor Author

tikikun commented Oct 26, 2023

continue working on this since the upstream is already somewhat stable

@dan-menlo dan-menlo moved this from Todo to In Progress in Jan & Cortex Nov 1, 2023
@tikikun tikikun linked a pull request Nov 3, 2023 that will close this issue
41 feat batch inference for nitro #101
Merged
@github-project-automation github-project-automation bot moved this from In Progress to Done in Jan & Cortex Nov 6, 2023
@vansangpfiev vansangpfiev mentioned this issue Sep 13, 2024
Closed
19 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@tikikun tikikun

Labels
P1: important Important feature / fix
Projects
Jan & Cortex
Archived in project
Milestone
Nitro v0.2
Development

Successfully merging a pull request may close this issue.

41 feat batch inference for nitro janhq/cortex.cpp
2 participants
@tikikun @freelerobot