Off-the-shelf datasets

A long-horizon benchmark evaluating frontier agents across the cybersecurity lifecycle, from vulnerability exploitation and secure remediation to threat detection and incident reconstruction.

Hard terminal tasks spanning well beyond typical SWE, from distributed systems and GPU performance to security forensics, geospatial, and clinical-data workflows.

Engineer-authored Triton GPU kernel optimization tasks. Each is a live agent environment, graded against a speedup a certified human solution already reached on the same GPU.

Based on SWE-bench: verified human-in-the-loop solution trajectories for the hardest SWE-bench tasks, capturing full problem-solving paths and human corrections to train agentic software-engineering models.

CEO Bench evaluates whether AI agents can complete long-horizon financial and operational work inside a complete, expert-authored fictional company.

Long horizon benchmark and training dataset for enterprise knowledge work. Built from a real company’s historical operational data.

Evaluates agents on multi-hour operational knowledge work such as the sourcing reviews, cost roll-ups, assortment resets and referral triage that keep an enterprise running between its systems of record.

Based on FinanceBench: an enterprise-finance agent benchmark of expert-authored, multi-step tasks over real 10-Ks, 10-Qs, spreadsheets, and deal materials, requiring long-context reasoning and sandboxed tool use.

Based on SciCode: SciCode++, 5,000+ expert-designed scientific-coding tasks built to challenge frontier models, to evaluate and train advanced scientific programming and computational problem-solving.

PhD-level multimodal STEM reasoning dataset across physics, math, biology, and chemistry with deterministic, auto-gradable answers over single-image, multi-panel, and caption-free multi-image formats.

Based on Humanity's Last Exam (HLE): self-contained, verifiable advanced STEM reasoning problems across physics, math, biology, and chemistry requiring multi-step symbolic reasoning, to evaluate frontier models.

201 expert-authored, verifier-graded terminal tasks across the STEM sciences, each expertise-heavy, built natively in Harbor format and validated against frontier models.