Models generating physics simulation code often produce outputs that execute but are logically incorrect, visually inconsistent, or unresponsive. The client required:

• High-diversity simulation prompts, authored from scratch
• Critiques of model-generated code for adherence and correctness
• Executable rewrites that corrected all functional, visual, and performance issues, ensuring simulations ran smoothly, adhered to physical laws, behaved logically, and rendered with visual clarity
• A labeled corpus of failure types to inform model analysis or finetuning

The solution needed to span Python and JavaScript, encompass both 2D and 3D simulations, and evolve with feedback.

Dataset

Turing established a structured, repeatable process to generate simulation QA tasks at scale. Each task included:

• A unique simulation prompt testing physical behavior, constraints, and interactivity
• A critique of the model's output with a detailed error breakdown
• A rewritten version of the simulation validated to: 
  - Fully meet all functional prompt constraints
  - Reflect physically realistic behavior and consistent logic
  - Render with clarity and intuitive design
  - Execute smoothly without lag, freezes, or glitches
• Failure mode tagging across four defined categories

Coverage areas

• Languages: Python (2D), JavaScript (2D and 3D)
• Frameworks:
  - Python: PyGame, Matplotlib
  - JavaScript: P5.js, Matter.js, Three.js, Cannon.js
• Simulation dimensions: 2D and 3D
• Interactivity metadata: Annotated per prompt
• Execution format: Python scripts and standalone HTML files (module-based)

All outputs adhered to strict internal QA standards and annotation guidelines. Each rewrite underwent independent review for prompt completeness, visual fidelity, and execution performance.

• Delivered 3,800+ simulation tasks across Python and JavaScript ecosystems
• Authored 100% of prompts and rewrites from scratch with no client code reuse
• Achieved 90% QA acceptance rate across all client-reviewed tasks
• Validated each rewrite for physics realism, visual appeal, and prompt completeness, ensuring stable execution without performance bottlenecks
• Tagged thousands of failure modes across runtime, logic, visual, and performance categories
• Structured all outputs with metadata for interactivity, simulation dimension, and framework type

The final dataset enabled the client to:

• Evaluate frontier models on diverse simulation prompts and verify their outputs
• Analyze where and how model-generated code fails under real-world conditions
• Identify recurring issues in syntax, logic, rendering, and runtime behavior
• Build a rewrite-backed corpus to improve model instruction-following and realism

When models learn to simulate physical behavior, not just generate code or images, they begin to reason about the world in ways that resemble human intuition and causality. The implications extend far beyond QA benchmarks:

• Predictive reasoning: Forecast object motion, interaction, and collision
• Causal inference: Understand not just outcomes, but underlying causes
• Embodied understanding: Enable intelligent robotics and spatial reasoning
• Generative design: Optimize structures in aerospace, automotive, or architecture
• Simulation-based optimization: Test and refine thousands of virtual prototypes rapidly
• Molecular and material modeling: Simulate physical behavior before manufacturing
• Training and transfer: Create safe, digital twin environments for robotics and control systems
• Immersive world-building: Enhance realism in games, AR/VR, and film production
• Educational tools: Teach physics and engineering through interactive simulations
• Safety-critical systems: Model edge cases for autonomous vehicles and drones
• Embodied AGI: Anchor abstract reasoning in time, space, and physical law

By enabling models to learn through simulation, we unlock foundational capabilities in planning, design, prediction, and embodied cognition, required for the next generation of general-purpose AI.