Math, physics, chemistry, and biology
Curated to test logical structure, problem-solving accuracy, and formal rigor, grounded in real-world scientific domains.
FEATURES AND BENEFITS
STEM reasoning problems
Verifiable with deterministic answers and expert-written rubrics for improving scientific and mathematical reasoning.
Lean-based proof QA
Iterative proof generation in Lean 4 paired with informal math questions, supporting symbolic reasoning and fine-tuned verification.
Multimodality
Visual reasoning tasks with multimodal inputs including image, video, audio, and molecular structure to train and improve models in complex STEM tasks.
CAPABILITIES
Off-the-shelf and custom datasets
Browse our catalog for frontier STEM.
Frontier STEM
BenchmarkPythonSciCodeScientific Coding
Pass@3:23.68%volume
Based on SciCode: SciCode++, 5,000+ expert-designed scientific-coding tasks built to challenge frontier models, to evaluate and train advanced scientific programming and computational problem-solving.
Expert QAHLE-styleHard ReasoningSTEM
Pass@8:9.78%volume
Based on Humanity's Last Exam (HLE): self-contained, verifiable advanced STEM reasoning problems across physics, math, biology, and chemistry requiring multi-step symbolic reasoning, to evaluate frontier models.
Expert QAHLE-styleHard ReasoningMultimodal
pass@8 (range 1-4):30.4%volume
Based on Humanity's Last Exam (HLE): PhD-level multimodal STEM reasoning problems across physics, math, biology, and chemistry, designed to break frontier models, for advanced multimodal evaluation.
RL environments
MCP and computer-use environments for frontier STEM.
Experts
PhD and PhD-candidate researchers in physics, chemistry, biology, math, and materials science from leading companies and academic institutions.
