New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Speedup ffn and gelu #15

Draft

certik wants to merge 3 commits into main from gelu

Owner

certik commented Mar 8, 2023

On my machine these changes speedup inference from 0.789s to 0.602s.

certik added 3 commits

March 7, 2023 15:47


          ffn: Fuse some of the operations

1d8c856

This provides about 4% speedup from 0.789 to 0.758s.


          Implement faster tanh(x) function

This provides about 20% speedup from 0.752s to 0.602s.


          Merge branch 'main' into gelu

04b6c43

Owner Author

certik commented Mar 17, 2023

With caching on, both main and this PR show 0.288s. With caching off, this PR is 0.543s, main is 0.716s.

certik marked this pull request as draft

March 17, 2023 15:20

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet