A low pass rate is never the goal. What matters more is why the model failed.

Today, we are introducing SciCode++, a new dataset for assessing how well frontier models solve scientific problems through code.

Introducing SciCode++: Scientific coding data for frontier models

The original SciCode idea

SciCode-Verified shifted the challenge

SciCode++: Separating model failures from evaluation failures

Anatomy of a SciCode++ ‘task’

The test code for validation

What building SciCode++ at scale tells us about difficulty

Does the signal transfer? A post-training experiment

Building SciCode++ at scale

References

Introducing KernelQuest: Can agents write Triton kernels?

The challenge: speed is easy to claim and hard to verify

Why we built KernelQuest

What KernelQuest measures

Quality is part of the benchmark, not a post-processing step

One task shows why the gates matter

Leaderboard

Failure modes

Building the evidence needed to push frontier models forward

CEO Bench evaluates whether frontier agents can complete long-horizon financial and operational work inside a complete, expert-authored fictional company rather than merely crunch numbers in isolation.

Introducing CEO Bench: Evaluating frontier agents on real company work

The challenge: measuring judgment, not string matches

Introducing CEO Bench

From gold answers to graded judgment

Quality and methodology

Evaluation results

Failure-mode analysis

Next steps

Developed by Turing Frontier Research Lab, CyberStrike evaluates if agents can complete offensive, defensive, and incident response cybersecurity tasks.

Announcing CyberStrike: Measuring whether frontier agents can complete real cybersecurity work

Security work does not end at detection

The challenge: measuring completion, not partial progress

Introducing the CyberStrike benchmark

From plausible answers to verified outcomes

Task spotlight: when self-reported success does not survive replay

How CyberStrike advances the lab’s mission

Welcome to the Frontier AI Newsletter, Turing’s weekly recap of the most important AI & AGI developments...

Offensive, defensive, and incident-response work, graded on real outcomes, not plausible answers.

CyberStrike: 200 Security Tasks Where Frontier Agents Still Break

What we're delivering

Explore Turing OTS Datasets

What we're celebrating

What we're reading

Where we’ll be

Stay ahead with AGI Advance

Frontier AI Newsletter: Weekly AI & AGI Insights (Sept 1, 2026)

3,600 trials on the 45 hardest tasks, and why an agent's own checks tell you almost nothing about whether it's right.

65% of Agent Failures Are Confident False Completions

Frontier AI Newsletter: Weekly AI & AGI Insights (Aug 18, 2026)

Training general-purpose agents with real APIs, real users, and real privacy constraints.

35K+ Agent Trajectories and 120K+ RLHF Pairs Across 150+ Real Tools

Frontier AI Newsletter: Weekly AI & AGI Insights (Aug 25, 2026)

Stay ahead with the Frontier AI Newsletter

Frontier AI Newsletter: Weekly AI & AGI Insights (Sept 8, 2026)

A benchmark measuring whether AI agents can deliver correct, verifiable, expert-level GPU kernel optimization.

Introducing KernelQuest: 100 Triton Tasks with Human-Certified Targets

Frontier AI Newsletter: Weekly AI & AGI Insights (Sept 15, 2026)

Introducing SciCode++, expert-authored science data built so a model failure actually means something.

6,000 Scientific Coding Tasks Where Frontier Models Still Break

Frontier AI Newsletter: Weekly AI & AGI Insights (Sept 22, 2026)

Agents often solve the wrong problem, verify against their own misreading, and mark the task complete.

Terminal-Bench 3.0: Why capable agents fail confidently

Why self-verification fails

The data

Failure has moved upstream

The agent's own evidence is uninformative

Types of failure

Misreading the problem

Under-modeling the spec

Missing the numeric bar

Breaking under re-entry or concurrency

The traps persist across configurations

What independent verification requires

What we take from this

Welcome to AGI Advance, Turing’s weekly recap of the most important AI & AGI developments...

Five years of real enterprise history, 64 long-horizon tasks, and four failure patterns today's models still can't overcome.

GPT-5.5 Solves 44% of CompanyBench

AGI Advance: Weekly AI & AGI Insights (Aug 11, 2026)

Explore expert insights on AI at Turing’s blog. Learn how AI-powered solutions are driving innovation, model performance, and business growth.

Turing Blog: Articles, Insights, Company News and Updates

Insights, analysis, and updates from Turing research labs  on the latest evaluations and benchmarks.

Blog

Blog posts

Request dataset samples or subscribe to our weekly newsletter to stay informed of what’s happening at the frontier of AI.

Blog

Introducing SciCode++: Scientific coding data for frontier models

Blog posts

Introducing SciCode++: Scientific coding data for frontier models

Introducing KernelQuest: Can agents write Triton kernels?

Introducing CEO Bench: Evaluating frontier agents on real company work

Announcing CyberStrike: Measuring whether frontier agents can complete real cybersecurity work

Frontier AI Newsletter: Weekly AI & AGI Insights (Sept 1, 2026)

Frontier AI Newsletter: Weekly AI & AGI Insights (Aug 18, 2026)

Frontier AI Newsletter: Weekly AI & AGI Insights (Aug 25, 2026)

Frontier AI Newsletter: Weekly AI & AGI Insights (Sept 8, 2026)

Frontier AI Newsletter: Weekly AI & AGI Insights (Sept 15, 2026)

Frontier AI Newsletter: Weekly AI & AGI Insights (Sept 22, 2026)

Terminal-Bench 3.0: Why capable agents fail confidently

AGI Advance: Weekly AI & AGI Insights (Aug 11, 2026)

Contact us