# Mixture of Experts (MoE)

> A mixture of experts is a model architecture split into many smaller 'expert' sub-networks, where a routing layer activates only a few experts for each token instead of running the whole model every time. This lets a model hold a very large number of parameters while keeping the compute (and cost) per answer low.

- Updated: 2026-08-01
- Canonical: https://geotoolbox.ai/glossary/mixture-of-experts

---

MoE is the trick behind the headline parameter counts you see on modern models. DeepSeek's [V4-Pro](https://geotoolbox.ai/blog/deepseek-v4), for example, holds 1.6 trillion parameters but activates only a fraction of them, on the order of tens of billions, for any given [token](https://geotoolbox.ai/blog/what-are-tokens-in-ai), which is how labs like DeepSeek and Google ship huge, capable models that are still cheap enough to run at scale.

For a publisher, MoE is mostly a "why it is cheap and fast" explanation rather than something you optimize for. It does not change how an engine decides what to cite. It is worth knowing mainly because it explains a paradox you will keep meeting: models with trillion-parameter headlines that still answer in a second and cost cents to run.