AutoMathText-V2
A 2.46 trillion token AI-curated STEM pretraining dataset.
Open dataset Updated 2026-06-18
What is AutoMathText-V2?
AutoMathText-V2 is a large-scale text collection of 2.46 trillion tokens spanning web, mathematics, code, and reasoning content.
It is useful for pretraining and fine-tuning NLP models that require broad STEM knowledge and reasoning capabilities.
What you can build with AutoMathText-V2
Pretrain STEM language models
Train or continue pretraining large models on the full 2.46T tokens to improve mathematical and scientific reasoning capabilities.
Build math question-answering systems
Fine-tune models on the reasoning and mathematics portions to create specialized QA tools for STEM problems.
Develop code generation for technical domains
Use the included code and math text to train models that generate or explain code in scientific computing contexts.
Load AutoMathText-V2
Python
from datasets import load_dataset
ds = load_dataset("OpenSQZ/AutoMathText-V2")- 1pip install datasets
- 2from datasets import load_dataset
- 3ds = load_dataset('opensqz/AutoMathText-V2')
- 4Select desired splits or columns for your training loop
- 5Tokenize and stream the data for large-scale pretraining
AutoMathText-V2: pros & cons
Pros
- +2.46 trillion tokens of scale
- +Deduplicated high-quality STEM content
- +Spans web, math, code and reasoning
- +Ready for Hugging Face loading
Cons
- –Requires massive storage and compute
- –STEM-only focus limits general use
- –License details must be verified on HF
Did you find this helpful?
Frequently asked questions
A 2.46-trillion-token deduplicated text dataset covering web, mathematics, code and reasoning for STEM language-model pretraining.
User reviews
Verified reviews from the community shape this listing's rating.
Loading reviews…
Similar datasets
Other text & nlp options worth comparing.