Model architecture
Mixture of Experts (MoE)
A family of routed expert architectures. Modern sparse variants activate only selected expert subnetworks for each input, increasing capacity without activating every parameter on every token.
Key terms
- experts
- router
- sparse activation
- load balancing
Connected across the project
Builds on
Related technology
Related papers
Historical context
Primary sources
This is a curated technical map, not a claim of comprehensive coverage.