DS-1000
DS-1000 supplies 1,000 data science coding problems in simplified text format.
Open dataset Updated 2026-06-17
What is DS-1000?
DS-1000 is a collection of approximately one thousand data science coding problems released in simplified text format for model evaluation.
It is useful for researchers running benchmarks and comparing language models on realistic data science code tasks.
What you can build with DS-1000
Benchmarking code generation models
Run generated code against the 1000 problems to measure pass rates and compare different LLMs or fine-tuned models.
Building automated evaluation pipelines
Integrate the simplified test cases into CI workflows that score model outputs on functional correctness.
Analyzing model failure modes
Inspect incorrect solutions across problem categories to identify patterns where code generation models struggle.
Load DS-1000
Python
from datasets import load_dataset
ds = load_dataset("xlangai/DS-1000")- 1pip install datasets
- 2from datasets import load_dataset
- 3ds = load_dataset('xlangai/DS-1000')
- 4Access problem statements and reference solutions in the loaded splits
- 5Run your model and score outputs with the provided test cases
DS-1000: pros & cons
Pros
- +Roughly 1000 problems for code generation evaluation
- +Simplified format supports standard testing
- +Directly loadable via Hugging Face datasets library
- +Reformatted version credits original creators
Cons
- –Not the original fill-in-the-middle insertion style
- –Reformatted version may differ from source distribution
- –Primarily intended for evaluation rather than training
Did you find this helpful?
Frequently asked questions
A dataset of roughly one thousand problems for code generation evaluation, distributed in simplified format on Hugging Face.
User reviews
Verified reviews from the community shape this listing's rating.
Loading reviews…
Similar datasets
Other text & nlp options worth comparing.