Inference and deployment
KV Cache
A cache of attention keys and values from earlier tokens during autoregressive generation. It avoids recomputing the full prefix but often becomes a major memory consumer.
Key terms
- keys
- values
- prefill
- decode
Connected across the project
Builds on
Related technology
Historical context
Primary sources
This is a curated technical map, not a claim of comprehensive coverage.