Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

support Mamba(2) architecture #41

Open
CarloLucibello opened this issue Jan 9, 2025 · 1 comment
Open

support Mamba(2) architecture #41

CarloLucibello opened this issue Jan 9, 2025 · 1 comment

Comments

@CarloLucibello
Copy link

it would be nice to support the Mamba
https://arxiv.org/abs/2312.00752
and Mamba 2
https://arxiv.org/abs/2405.21060
architectures.

It is probably impossible to have efficient implementations without some heavy gpu kernel programming, but one could start with slow but correct implementations.

Some simple but not optimized code can be find here
https://github.com/johnma2006/mamba-minimal
https://github.com/tommyip/mamba2-minimal

@MartinuzziFrancesco
Copy link
Owner

Thanks for the implementation links, I was looking for something like that some time ago! I'll definitively add some modern rnn approaches soon

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants