What is BLEU Score?
BLEU Score is an automatic metric that evaluates machine-generated text quality, mainly for machine translation, by measuring overlap with human-written reference translations.
BLEU computes modified n-gram precision (for 1- to 4-grams) between the candidate sentence and one or more references, then applies a brevity penalty to discourage overly short outputs.
The final score is the geometric mean of these precisions multiplied by the brevity penalty, producing a value between 0 and 1 where higher indicates better match to human references.
It is fast, language-independent, and correlates reasonably with human judgments, making it a standard benchmark despite known limitations such as ignoring meaning and synonymy.
Example
If the reference translation is 'the cat sat on the mat' and the model outputs 'a cat is on the mat', BLEU will award partial credit for shared bigrams like 'cat on' and 'on the' while applying a small brevity penalty.
Why it matters
BLEU remains the most widely reported automatic score for comparing translation and text-generation models, enabling reproducible progress tracking across research papers and leaderboards.
Frequently asked questions
Bilingual Evaluation Understudy.
Related terms
Precision is an evaluation metric for classification models that measures the proportion of true positive predictions among all positive predictions made.
Perplexity is an evaluation metric that measures how well a language model predicts a given sequence of text. Lower values indicate the model is less surprised by the data and thus performs better at next-token prediction.
Accuracy measures the proportion of correct predictions made by a machine learning model out of all predictions. It is calculated as the number of correct predictions divided by the total number of predictions.
A benchmark is a standardized dataset and task used to measure and compare how well different AI models perform.
A confusion matrix is a table that shows how well a classification model performs by comparing its predictions to the actual labels.
The F1 Score is a single metric that balances precision and recall to evaluate how well a classification model performs, especially when classes are uneven.