Research

NVIDIA reports two gold-level Nemotron results, with different verification limits

By Saint Zero Day · 2026-10-08 · 4 min read

Nemotron specialists reportedly crossed two olympiad gold thresholds. Released methods offer research value, while an unsupervised coding run and proof-checking errors limit the conclusions.

NVIDIA’s October 7 announcement brings together two reported achievements for specialist systems built from Nemotron 3: 535.4 points out of 600 on the 2026 International Olympiad in Informatics problems and 30 out of 42 at the International Mathematical Olympiad. Both exceed the reported gold thresholds. The distinction behind the headline matters: the coding run was unofficial and unsupervised, while NVIDIA says official IMO graders marked the mathematical submissions. [1]

The coding paper describes a prospective run conducted before the competition problems became public. GenCorrect generates candidate programs, uses evaluation feedback and refines subsequent attempts within a submission budget. The researchers report matching contestants’ time, internet restrictions and submission limits. That makes the experiment more informative than testing a model on familiar, publicly available questions, although the account remains the developers’ report. [2]

Matching those rules did not mean matching resources. The coding system used a peak allocation of 760 NVIDIA GB300 GPUs. Its reported score exceeded the leading human contestant’s 498.27, but the authors explicitly describe this as a system comparison rather than an equal-resource contest. They also warn that the findings may not generalize beyond competitive programming. [2]

Why it matters

The mathematics implementation makes the machinery behind its score unusually visible. Three model checkpoints generate proofs, two specialist checkpoints scrutinize them, and later rounds revise promising candidates using critiques. The first round creates 384 attempts per problem. Acceptance requires all 16 verification judgments to approve a proof; final selection adds 48 judgments per finalist. These are repeated model assessments, whose agreement still requires checking against mathematical correctness. [4]

The IMO paper reports full marks on four problems and one point each on the remaining two. It attributes the 30-point total to official graders, against a 29-point gold cutoff. Finding the submitted proofs consumed approximately 1,464 GB200 GPU-hours; completing computation already underway brought the run to about 4,800. This was a substantial search system, rather than a single response from a chatbot. [3]

The strongest warning comes from that same experiment. Internal verification and a separate model jury estimated roughly 32 points, overcrediting the two weak submissions. The authors identify a shared verification blind spot. Continued search later improved one proof to an unofficial human-assessed four points, but that happened beyond the contest deadline and without the official marking scheme. It does not replace the reported 30-point competition result. [3]

The verification approach has a concrete antecedent. NVIDIA cites DeepSeekMath-V2 as inspiration for its proof search and reinforcement-learning rewards. DeepSeek’s researchers trained a verifier and then a second evaluator to inspect the verifier’s explanations. Their motivation was a specific failure: a checker could predict the right score while inventing defects that did not exist. This earlier work helps explain why checking the checker matters; it does not independently reproduce or validate Nemotron’s results. [3] [5]

Another directly relevant source is MathArena, whose proof-evaluation methodology NVIDIA adapted for final selection. Its ETH Zurich and INSAIT researchers developed grading rubrics, combined multiple model judges and reviewed results with humans. They found that rubric wording could mislead judges and that some individual judges awarded excessive credit. Their procedure used reference solutions and manual corrections; NVIDIA’s selection procedure is reference-free. That difference limits how much MathArena’s validation can establish about Nemotron’s judgments. [3] [6]

MathArena also exposes limits in its own validation. For two computation-heavy geometry problems, reviewers assumed the model jury had identified the relevant issues, then concentrated on consistent grading. Its findings therefore offer methodological context, rather than an unrestricted audit of Nemotron. The independent researchers also reported models attempting to prove false research statements, illustrating a practical harm: apparently rigorous output can consume expert checking time and spread unsupported mathematical claims. [6]

Useful mathematical assistance requires more than a high score. ProofRank, an overlapping ETH Zurich and INSAIT team’s separate study, evaluates conciseness, computational burden, accessible techniques and other proxies for proof quality. It finds differences that correctness scores miss. Its reliance on automated proxies and final-answer competition problems limits generalization. Applied here as an editorial inference, this suggests that educators and researchers would need evidence of understandable, transferable explanations before treating olympiad performance as demonstrated everyday usefulness. [7]

The immediate research benefit is inspectability. NVIDIA’s announcement points to specialist checkpoints, training data, submitted proofs and a new benchmark, while the opened implementation documentation describes a runnable inference workflow. The mathematics paper specifies OpenMDW-1.1 for its specialist weights and CC BY 4.0 for released datasets. Access is still incomplete for reconstructing the coding project: its authors say third-party restrictions prevent redistribution of the full training corpus. [1] [2] [3] [4]

The editorial assessment is cautiously hopeful about research access and narrowly reported capability. These sources support studying how specialization and repeated feedback improve difficult problem solving; they do not demonstrate affordable deployment or dependable autonomous verification. Independent reruns using frozen checkpoints and complete logs, externally checked proofs and explicit compute budgets would strengthen the assessment. Trials measuring useful work, checking burden and failures outside contests would be needed to establish practical benefits. [2] [4] [5] [7]