What is Encoder?
In deep learning, an encoder is a neural network module that converts raw input data into a compressed, informative representation called an embedding or latent vector.
The encoder processes the input through layers (such as RNNs, CNNs, or transformer blocks) to extract hierarchical features and reduce dimensionality while preserving key information.
It is commonly paired with a decoder in architectures like autoencoders and sequence-to-sequence models, where the encoder's output serves as a bottleneck or context for reconstruction or generation tasks.
Training encourages the encoder to learn useful abstractions, often via objectives like reconstruction loss or masked prediction, enabling downstream tasks such as classification or translation.
Example
In a machine-translation system, the encoder reads an English sentence word-by-word and produces a fixed-size context vector that summarizes its meaning for the decoder to generate the French translation.
Why it matters
Encoders are foundational to modern models such as transformers and variational autoencoders, powering efficient representation learning across NLP, vision, and multimodal AI systems.
Frequently asked questions
Its main job is to turn input data into a compact, meaningful vector that captures essential features for later use by a decoder or classifier.
Related terms
In deep learning, a decoder is a neural network module that converts an encoded representation (like a context vector or latent features) into a final output such as text, images, or sequences.
An autoencoder is a neural network that learns to compress input data into a smaller representation and then reconstruct the original data from that compressed form.
A Transformer is a neural network architecture that processes sequential data like text using self-attention to weigh relationships between all parts of the input at once.
An embedding (or vector embedding) is a way to represent words, sentences, or other data as dense numerical vectors in a high-dimensional space so that similar items end up close together.
An activation function is a mathematical operation applied to the output of a neuron in a neural network that decides whether the neuron should 'fire' and pass on a signal.
Backpropagation is an algorithm for training neural networks by calculating how much each weight contributed to the prediction error and adjusting those weights accordingly. It uses the chain rule to efficiently compute gradients of the loss function.