Training and post-training
Distributed Training
Techniques that partition data, parameters, activations, or computation across devices and nodes. They make large-model training possible but introduce communication, synchronization, and fault-tolerance tradeoffs.
Key terms
- data parallelism
- tensor parallelism
- pipeline parallelism
- sharding
Connected across the project
Related technology
Historical context
Primary sources
This is a curated technical map, not a claim of comprehensive coverage.