Building Reliable Data Analytics Agents: Lessons from the KDD Cup

PUBLISHER ARTICLE PREVIEW

From the original article

The NVIDIA KGMON team placed second in the KDD Cup 2026 Data Agents competition with a system built around a simple idea of making an agent’s harness smaller, clearer, and easier to verify.

The competition asked agents to answer natural-language questions over heterogeneous data sources, including databases, CSV and JSON files, prose documents, PDFs, and briefing videos. Every task required more than retrieval, with the agent inspecting available data, choosing the right tools, reasoning across sources, producing a final answer file, and handling traps that appear in analytical workflows.

KDD also required teams to work with a small, fixed LLM to power the agent, making the harness the main optimization surface. The team aimed to make the task easier for the available model through preprocessing, constrained tools, persistent state, and evaluation. These techniques are especially useful when building reliable systems around smaller open models.

This post shares the playbook behind their result. It isn’t a general recipe for every data science agent, but the broader lesson applies to many agent systems. Reliability often comes less from making the model more open-ended and more from building the right harness around it.

First, constrain the action space. Too many ways to inspect data, call tools, write files, or recover from mistakes can lead to failures. KGMON unified data access, exposed a small tool set, and required final answers to follow a single output path.

Second, make every attempt inspectable. A run can fail because of a bad tool call, an incorrect join, a missed document rule, or an answer-format issue. Execution traces, repeated attempts, and trajectory inspection helped the team identify failures and refine the harness. Figure 1 shows the overall workflow.

The following techniques offer practical guidance for building reliable data analytics agents.

Each KDD task could combine SQL databases, CSV and JSON files, documents, and videos. KGMON converted CSV and JSON files into tables in the existing SQLite database, giving the agent one SQL interface for all structured data.

KGMON’s custom, persistent Python environment exposed two built-in functions for inspecting and querying that unified data surface:

These custom functions gave the agent a single way to discover and query structured data. They were built into KGMON’s environment rather than provided by Python or SQLite.

The single SQL interface reduced routing failures and wasted turns, leaving the fixed model more room to reason over the data.

Apply this approach: Normalize access to structured data before the agent starts. A temporary SQLite database, a virtualized query layer, or a governed warehouse interface can provide a stable, narrow, and well-documented query surface.