Mixture of experts (MoE)
A model built from several specialist sub-networks, with only a few switched on for each piece of text. It saves work without losing size.
The fuller definition
A mixture-of-experts architecture routes inputs through selected expert components rather than using all expert parameters for every token.
Lines represent computational connections with learned values. This tiny diagram illustrates a neural network; a transformer or mixture-of-experts model has additional structures not shown here.
A concrete example
A model may have many total parameters but activate a smaller subset for each token.
The distinction that matters
Active parameter count does not by itself describe total storage or memory requirements.
Related terms
Parameter
One of the billions of adjustable numbers inside a model. Training sets them; using the model does not change them.
FoundationsModel weights
The full set of numbers that make up a trained model. Download the weights and you have the model.
Hardware & hostingGPU — Graphics processing unit
The chip that handles graphics, and also the maths AI needs. It is what makes AI fast.
Where it is taught
What actually happens when you ask AI a question?Source and further reading
Google · Machine Learning GlossaryThe definition and example are written for this library. The link gives background, official documentation or legal authority as labelled. Legal scope depends on the jurisdiction and current text; product features depend on release and settings.