Frontier AI Newsletter: Weekly AI & AGI Insights (Sept 22, 2026)

This week, we're introducing SciCode++, a production-scale scientific coding dataset built so that a model failure actually tells you something. It builds on two lessons: SciCode made scientific reasoning executable, and SciCode-Verified showed how easily a benchmark can get harder because of the task or the verifier rather than the model. SciCode++ is designed to separate genuine model failures from evaluation failures.

Here's what we built:

• ~6,000 expert-authored tasks across five domains. Physics takes the largest share, followed by materials science, chemistry, mathematics, and biology. Each task asks a model to turn a scientific method into working Python, decomposed into sub-problems that feed a main problem, with every constant, sign, and unit convention pinned down so the task is well-posed and the solution verifiable.
• Difficulty calibrated against the current frontier. Each task is run eight times against a state-of-the-art model. Mean main-problem pass rates range from just 4% in biology to 14% in mathematics, with no domain above 15%, and roughly 70% of main problems went unsolved across all eight attempts.
• Built and reviewed for trust. Tasks are written by working scientists (PhDs and professors in the relevant fields), then pass through an expert trainer, independent L1 review, independent L2 review, and a frontier-model pass-rate gate, because task difficulty depends on the problem, the verifier, and the current model frontier all lining up.

💡 In an early post-training experiment, training on private SciCode++ data improved a baseline model by 10.3% on the public SciCode benchmark and 9.1% on SciCode-Verified, with error analysis showing fewer failures tied to incorrect scientific reasoning. For model teams, the real value is failures that trace back to a specific capability, pointing clearly to what to improve next.

→ Request Sample Data

🔬 Need better eval data this quarter?

Turing’s off-the-shelf (OTS) datasets are built for teams that need verifiable, high-signal data where frontier models still break — across multimodal STEM, HLE++ STEM, coding evaluation, and rubric-based reasoning.

Use them for reward modeling, RL post-training, outcome-supervised fine-tuning, frontier benchmarking, and failure-mode analysis.

→ Request sample data

🎉Our CTO, Ece Kamar, named one of the Top 100 Women in AI for 2026

Ece Kamar, who recently joined Turing as Chief Technology Officer after leading the AI Frontiers Lab at Microsoft Research, has been recognized on the Top 100 Women in AI 2026 list. At Turing, Ece leads technology and research strategy across our frontier AI and enterprise AI businesses, sharpening the link between what frontier labs need next and how enterprises put that innovation to work. A proud moment for the whole team.

→ Read the announcement

• SciCode: A Research Coding Benchmark Curated by Scientists The paper that started it all, and a direct ancestor of the dataset above. Curated by scientists across 16 natural-science subfields, SciCode turned 80 main problems into 338 executable sub-problems, asking models to implement a scientific method as Python rather than pick a multiple-choice answer.
  
  Its headline finding still frames the challenge: the best model tested at the time solved just 4.6% of main problems in the most realistic setting, showing how far scientific coding sits beyond ordinary code generation.
• How Claude is uplifting biomolecular modeling Anthropic reports that Claude, working within Claude Science, optimized more than 30 open-source biomolecular models in under four weeks, delivering a roughly 4x average speedup with minimal precision loss.
  
  It also built a low-memory "Big" mode that folds systems larger than 10,000 tokens (including a bacterial ribosome and a proteasome) accurately on a single GPU node, and matched an earlier protein-design campaign's in silico results using about two orders of magnitude fewer GPU hours. The optimized code is open-sourced, alongside a protein-design competition co-sponsored with Adaptyv Bio.
• Bypassing inference bottlenecks: Accelerating complex AI search with Retrieve-for-Train Google Research uses offline RL as a one-time "objective transducer", discovering reward-aligned query fan-outs and distilling them into a lightweight (~54M-parameter) diffusion retriever that does the work in a single parallel pass.
  
  The result is a 12-20x speedup (cutting fan-out latency from nearly 50 seconds to sub-second) while still optimizing set-level properties like diversity and groundedness, a nice illustration of moving heavy reward-driven behavior out of inference and into training.

🔹 COLM 2026

📍 San Francisco, CA United States |  🗓️ Oct 6-9

Turing is headed to San Francisco for COLM 2026! Come connect with us and other researchers, stay tuned for specific Turing-hosted events.

Turing is leading the charge in bridging AI research with real-world applications. Subscribe to the Frontier AI Newsletter for weekly insights into breakthroughs, research, and industry shifts that matter.

[Subscribe & Read More]