docs: v3.0.1 changelog

kozistr · Jun 8, 2024 · 5d924c5 · 5d924c5
1 parent 6675dfe
commit 5d924c5
Showing 1 changed file with 4 additions and 0 deletions.
diff --git a/docs/changelogs/v3.0.1.md b/docs/changelogs/v3.0.1.md
@@ -4,6 +4,10 @@
 
 * Implement `FAdam` optimizer. (#241, #242)
   * [Adam is a natural gradient optimizer using diagonal empirical Fisher information](https://arxiv.org/abs/2405.12807)
+* Tweak `AdaFactor` optimizer. (#236, #243)
+  * support not-using-first-momentum when beta1 is not given
+  * default dtype for first momentum to `bfloat16`
+  * clip second momentum to 0.999
 
 ### Bug