Skip to content

Releases: AI-Hypercomputer/maxtext

MoE v1.0.0

10 Sep 06:36
Compare
Choose a tag to compare

MoE v1.0.0 supports:

  • Megablox with Fully Sharded Data Parallelism (FSDP) and Token Parallelism (TP)
  • Dropping strategies with FSDP, TP, and Expert Parallelism (EP)