TECHNOLOGY GRAPH
AI Technology
From model architecture and training to inference, deployment, and open-weight model cases
A curated map of the technical layers that turn research ideas into trainable models and usable systems. Each entry connects concepts with papers, historical eras, model cases, tools, and primary sources.
Model architecture
Structures that determine how models represent and transform information.
Transformer
An attention-based neural architecture that processes sequence positions in parallel. It became a foundation for many modern language and multimodal models.
- self-attention
- encoder
- decoder
Mixture of Experts (MoE)
A family of routed expert architectures. Modern sparse variants activate only selected expert subnetworks for each input, increasing capacity without activating every parameter on every token.
- experts
- router
- sparse activation
Diffusion Model
A generative model that learns to reverse a gradual noising process. Iterative denoising became a major approach for image, audio, video, and multimodal generation.
- forward noise
- denoising
- score matching
Frameworks and representation
Software and data interfaces used to build, train, and run models.
PyTorch
An open-source tensor and deep-learning framework with automatic differentiation, neural-network modules, compilation, and distributed execution. It is widely used from research prototypes to production training.
- tensor
- autograd
- module
Hugging Face Transformers
A model-definition library that connects pretrained architectures with training and inference ecosystems. It standardizes configuration, preprocessing, loading, generation, and many task interfaces.
- model definitions
- pretrained checkpoints
- Trainer
Tokenization
The mapping between raw text and the discrete units processed by a model. Vocabulary construction and segmentation affect sequence length, multilingual coverage, cost, and model behavior.
- token
- vocabulary
- BPE
Training and post-training
Scaling, adaptation, and preference methods that shape model capabilities and behavior.
Distributed Training
Techniques that partition data, parameters, activations, or computation across devices and nodes. They make large-model training possible but introduce communication, synchronization, and fault-tolerance tradeoffs.
- data parallelism
- tensor parallelism
- pipeline parallelism
SFT / RLHF / DPO
A family of post-training methods for teaching instruction following and preference-aligned behavior. SFT learns from demonstrations; RLHF and DPO use preference comparisons through different optimization procedures.
- supervised fine-tuning
- preference data
- reward model
Reasoning and test-time compute
Post-training that teaches a model to produce a long reasoning trace before answering, and to spend more compute at inference when a problem is harder. Reinforcement learning with verifiable rewards — where a checkable answer supplies the training signal — is what made the approach reproducible outside closed labs. The trade is worth stating plainly: accuracy bought with generation length rather than with a larger model, and paid for in latency and serving cost.
- reasoning traces
- verifiable rewards
- inference budget
LoRA / QLoRA
Parameter-efficient fine-tuning methods that train small low-rank adapters while keeping base weights frozen. QLoRA combines adapters with a quantized base model to reduce memory use further.
- low-rank adapters
- frozen base model
- 4-bit training
Inference and deployment
Memory, scheduling, compression, and runtime techniques used to serve models.
Quantization
Representing weights, activations, or caches with lower numerical precision. It can reduce memory, bandwidth, and compute costs, with accuracy and hardware-support tradeoffs.
- precision
- INT8
- INT4
KV Cache
A cache of attention keys and values from earlier tokens during autoregressive generation. It avoids recomputing the full prefix but often becomes a major memory consumer.
- keys
- values
- prefill
Continuous Batching / PagedAttention
Serving techniques that continuously admit and retire requests while managing KV-cache memory in pages. Together they improve utilization and throughput under variable-length workloads.
- request scheduling
- memory paging
- throughput
Speculative Decoding
An inference method in which a faster draft proposes multiple tokens and the target model verifies them in parallel. It can reduce generation latency without changing the target distribution.
- draft model
- verification
- latency
Open-model Inference Ecosystem
Engines and runtimes such as vLLM and llama.cpp turn downloadable model weights into local or server inference. Their priorities differ across throughput, hardware portability, model formats, and operational complexity.
- vLLM
- llama.cpp
- GGUF
APPLICATION LAYER
Application architecture
Models become products through workflows, tools, context exchange, guardrails, and evaluation. The existing MCP and Tools sections cover this application layer without duplicating it here.
Open-model cases
Model families examined through code, weights, training disclosure, and license.
Llama
Meta’s downloadable-weight model family helped expand local inference and downstream fine-tuning. Its releases use Llama-specific terms, so "open weights" is more precise than assuming OSI-defined open source.
- open weights
- community license
- model family
Qwen
Alibaba’s Qwen family spans dense, expert-routed, multilingual, and multimodal releases. Many checkpoints are openly downloadable, but license and disclosure claims should be checked at the individual model-card level.
- multilingual
- dense and MoE
- open weights
Mistral / Mixtral
Mistral AI’s family includes both dense Mistral models and expert-routed Mixtral models. Some releases use permissive licenses while others do not, so openness must be assessed per artifact.
- dense models
- MoE
- open weights
DeepSeek
DeepSeek’s downloadable releases made MoE design, reasoning post-training, and efficient serving widely studied. Code and weights may use different licenses, and training-data disclosure is a separate dimension.
- MoE
- reasoning
- open weights
Gemma
Google’s downloadable-weight family connects Gemini-derived research with local, edge, and hosted deployment. Google calls the family open models, while usage remains governed by Gemma-specific terms.
- open weights
- lightweight models
- edge deployment
OLMo
Ai2 releases the OLMo family with its pretraining data, training code, intermediate checkpoints and logs alongside the weights. The first OLMo appeared in February 2024, OLMo 2 that November, and Olmo 3 in November 2025 with 7B and 32B Base, Instruct and Think variants under Apache 2.0. It is the case where all four things worth checking separately — code, weights, training disclosure and license — are actually published, which makes it a reference point against families that release weights alone.
- fully open
- open training data
- intermediate checkpoints
gpt-oss
OpenAI’s gpt-oss-120b and 20b are downloadable MoE reasoning models designed for local and hosted inference. Their Apache 2.0 weights and reference implementations broaden deployment access, while the pretraining dataset itself is not released.
- open weights
- reasoning
- MoE
GLM-5
Z.ai’s GLM-5 series is a large MoE language-model family for reasoning, coding, and long-horizon agentic work. GLM-5 appeared in February 2026, 5.1 in April, 5.2 in June at 753 billion parameters with a one-million-token context window, and 5.3 in August on the same base with post-training changes; the smaller GLM-5.3-Flash followed days later at 320 billion total and 18 billion active parameters. The larger members’ scale makes self-hosting resource-intensive.
- open weights
- MoE
- long context
Kimi K3
Moonshot AI released Kimi K3 in July 2026 as an open-weight, native-multimodal MoE language model with 2.8 trillion total parameters, 104 billion active parameters, and a one-million-token context window. It targets long-horizon coding, knowledge work, and tool-using agents, while its scale makes local deployment demanding.
- open weights
- MoE
- long context
This is a curated technical map, not a claim of comprehensive coverage.