glue
VerifiedBenchmark collection for training and evaluating natural language understanding systems.
What is glue?
GLUE is the General Language Understanding Evaluation benchmark, a collection of datasets and resources for training, evaluating, and analyzing natural language understanding systems.
It supports researchers and model developers working on standardized text classification tasks with public leaderboards for comparison.
What you can build with glue
Benchmark NLU model performance
Train and evaluate transformer models across GLUE tasks like MNLI, QQP, and SST-2 to compare accuracy against published baselines.
Fine-tune for text classification
Use individual GLUE subsets such as CoLA or RTE to fine-tune models for grammatical acceptability or textual entailment in production pipelines.
Analyze model errors with AX
Load the AX diagnostic dataset to perform fine-grained analysis of model failures on linguistic phenomena like negation or coreference.
Load glue
from datasets import load_dataset
ds = load_dataset("nyu-mll/glue")- 1pip install datasets
- 2from datasets import load_dataset
- 3dataset = load_dataset('glue', 'sst2')
- 4print(dataset['train'][0])
- 5Use dataset splits for training and validation loops
glue: pros & cons
Pros
- +Standardized multi-task NLU benchmark
- +Wide task coverage with 1M-10M total examples
- +Easy loading via Hugging Face datasets
- +Includes diagnostic AX set for analysis
Cons
- –Some tasks considered saturated by modern models
- –Requires separate handling per task subset
- –License and citation rules vary by original source
Frequently asked questions
GLUE is a collection of NLU benchmarks for text classification and related tasks, hosted by nyu-mll and loadable via Hugging Face.
User reviews
Verified reviews from the community shape this listing's rating.
Loading reviews…
Similar datasets
Other text & nlp options worth comparing.