The client needed a high-quality agentic dataset to train and evaluate their AI agent's ability to handle user queries across multiple languages and regions. The core challenges included:

• Extending reliable tool-calling behavior beyond English: the agent needed to select the correct tools, supply locale-appropriate arguments, and chain calls logically across distinct language environments, each with its own formatting conventions, punctuation rules, and cultural context.
• Maintaining locale consistency throughout multi-turn interactions:  responses, tool arguments, and conversational references had to remain strictly aligned to the user's locale.
• Designing system prompts that governed nuanced tool behavior: each task required a detailed, conditional system prompt defining precise tool invocation policies, agent persona, contextual information, and behavioral rules that the agent had to follow throughout the conversation.
• Capturing realistic, natural interactions at scale: user prompts had to reflect genuine, human-like behavior rather than scripted or tool-aware phrasing, while still implicitly requiring multi-step tool reasoning across every turn.
• Enforcing structural compliance across thousands of tasks: each conversation had to meet strict turn count requirements, including a minimum number of sequential tool chains and corrected responses.

Turing deployed a team of trained multilingual evaluators, each assigned to their native locale, operating within a structured task design and review workflow purpose-built for agentic, tool-calling data.

1. Locale-native task design

Each task was assigned to an expert with native-level fluency in the target locale. Experts designed all user prompts and evaluated all agent responses in their assigned language, ensuring that conversational tone, cultural references, and linguistic conventions were authentic rather than translated.

System prompts were written in English but included locale-specific contextual elements, such as locally relevant place names, currencies, entity names, and screen context, to avoid ambiguity without compromising the English-first instruction format.

2. Structured system prompt design

Every system prompt was required to include at least two valid context types, such as current location, device state, user identity, or active application state, alongside at least two explicit tool invocation policies defining how the agent should behave when using specific tools. System prompts also established agent persona, tone, fallback behavior, and conditional logic governing edge cases.

This design ensured that system prompts provided precise, unambiguous instructions that the agent had to follow throughout the conversation, and that evaluators could use as a consistent reference when accepting or rejecting responses.

3. Multi-turn conversation construction

Each conversation was structured as a sequence of 10 to 15 turns, with user prompts crafted to naturally require multiple tool calls, including sequential chains where one tool's output fed the next, and parallel calls where independent requests could be resolved simultaneously.

Tasks were required to include turns demonstrating sequential tool chaining, corrected agent responses, and without tool calls. This structure ensured that every delivered conversation provided dense, high-signal training data for both tool selection and multi-step reasoning.

4. Locale-aware tool argument handling

Tool usage was grounded in the target locale throughout. Argument values, search queries, and conversational references were authored in the user's language, while fallback behavior for edge cases was governed by explicit system prompt instructions, keeping the interaction consistent and locale-appropriate.

5. Three-layer human-in-the-loop quality assurance

Every task passed through a structured three-stage QA process.

• An automated reviewer ran first, evaluating the task against rubric dimensions including tool use accuracy, hallucination, task completeness, system context adherence, grammar and language quality, naturalness of dialogue, datetime reasoning, tool output integration, and system prompt compliance. Structural validations, such as turn count, rejection counts, cell ordering, and similarity checks were also applied at this stage
• Human reviewers then validated rubric alignment, locale consistency, and compliance with all system prompt instructions with 100% review coverage across all tasks
• A calibration layer covered 20–30% of the dataset, where domain leads audited tasks to catch scoring drift, resolve edge cases, and maintain consistent quality standards across locales and evaluator cohorts

• More than 3,500 agentic conversations delivered across 15+ locales, each comprising multi-turn interactions with tool calls, sequential reasoning chains, corrected responses, and locale-consistent outputs
• 10+ rubric dimensions evaluated per task, combining manual expert review with automated LLM-assisted checks across grammar, tool accuracy, hallucination, system adherence, and dialogue naturalness
• 100% review coverage enforced, with every task passing through both automated pre-submission checks and human reviewer validation before delivery
• 300+ multilingual evaluators onboarded, spanning software engineering, data science, and ML backgrounds, with experience exceeding five years across technical domains

The client received a production-ready multilingual agentic dataset with locale-native conversation design, structured system prompts, and a rigorous dual-layer QA pipeline. The dataset provides high-signal supervision for agents learning to interpret user requests, invoke appropriate tools, and deliver locale-consistent responses in real-world multi-turn workflows.

This foundation supports:

• Training agents to handle tool-calling across diverse languages without defaulting to English-centric reasoning or formatting conventions
• Evaluating locale-aware instruction following across a broad range of tool types, invocation sequences, and conversational contexts
• Calibrating agent behavior under detailed, conditional system prompts that govern persona, tone, fallback logic, and tool use policies
• Scaling multilingual agentic data production using a validated, rubric-driven workflow with demonstrated quality across 15+ distinct locales