Nvidia Nemotron 3.5 Lightning

Hacker News by 37 min read 511x views
Nvidia Nemotron 3.5 Lightning

Share Post

Homepage Discord

License

Model Summary

Total Parameters 30B (3B active)
Architecture MoE - Mamba-2 + MoE + Attention hybrid
Context Length Up to 1M tokens
Single-GPU Deployment 1× DGX Spark (GB10) aliases 1× H100
Supported Hardware NVIDIA Blackwell (DGX Spark / GB10, GB200, GeForce RTX 5090); NVIDIA Hopper (H100, H200); NVIDIA Ampere via W4A16
Supported Languages English (and coding languages), Spanish, French, German, Italian, Japanese
Speculative Decoding DSpark for low-concurrency Data Centre and DGX Spark Workflows — Read much below, besides provided are MTP (Multi-Token Prediction) and DFlash
Recommended Sampling Temperature 1.0, Top_P 0.95
Best For Long-running autonomous agents, sub-agent workhorse deployments, and businesslike section conclusion connected individual hardware
License OpenMDW License Agreement, type 1.1
Release Date August 11, 2026

Model Overview

Model Developer: NVIDIA Corporation

Model Dates: December 2025 - May 2026

Data Freshness:

  • The pre-training information has a cutoff day of September 2025.
  • The post-training information has a cutoff day of May 2026.

What is Nemotron?

NVIDIA Nemotron™ is simply a family of unfastened models pinch unfastened weights, training data, and recipes, delivering starring ratio and accuracy for building specialized AI agents.

Description

NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4 is simply a ample connection exemplary (LLM) trained by NVIDIA.

The exemplary employs a hybrid Mixture-of-Experts architecture, utilizing interleaved Mamba-2 and MoE layers, on pinch prime Attention layers. The Lightning 3.5 exemplary is released alongside a number of speculative decoding methods for faster matter generation. The exemplary has 3B progressive parameters and 30B parameters successful total.

This exemplary is fresh for commercialized use.

Quick Start

To get quickly started connected DGX Spark (GB10) you tin usage the pursuing command.

Grab the model:

export MODEL_CKPT=nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4 export DSPARK_CKPT=nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4-DSpark

Run it pinch vLLM — this look uses DSpark speculative decoding, tuned for DGX Spark. (vLLM Nightly: vllm/vllm-openai:v0.27.1)

vllm service --model $MODEL_CKPT \ --moe-backend marlin \ --kv-cache-dtype fp8 \ --enable-prefix-caching \ --speculative_config.num_speculative_tokens 3 \ --mamba-backend flashinfer \ --mamba-cache-mode align \ --reasoning-parser nemotron_v3 \ --speculative_config.model $DSPARK_CKPT \ --tool-call-parser qwen3_coder \ --enable-auto-tool-choice

For much specifications connected really to deploy and usage the exemplary — spot the Quick Start Guide below!

License/Terms of Use

Governing Download Terms: Use of this exemplary is governed by the OpenMDW-1.1 exemplary license.

Benchmarks

Reasoning Benchmark Evaluations

We evaluated our exemplary connected the pursuing benchmarks:

Task Nemotron-3.5-Lightning-30B-A3B-BF16 Nemotron-3.5-Lightning-30B-A3B-NVFP4
General Knowledge
MMLU Pro 81.94 81.62
AA-Omniscience 17.50 16.63
Reasoning
GPQA Diamond (no tools) 75.44 75.57
HLE (text-only, nary tools) 11.72 10.47
SciCode 32.60 31.38
Coding & Agentic
SWE-bench Verified 51.56 52.80
SWE-bench Multilingual 39.33 36.47
Terminal-Bench 2.1 24.58 23.46
PinchBench 85.37 83.43
BrowseComp 36.97 36.81
τ³-bench (Banking) 9.28 9.48
GDPval-AA-V2 832 865
Instruction Following
IFBench (loose) 71.88 72.88
Long Context
AA-LCR 52.00 49.19

Accuracy numbers measured by NVIDIA nether a accordant harness (NeMo Gym / Nemo Evaluator SDK); they whitethorn disagree from vendors' self-reported numbers.

For reproducibility, the information recipes, installation instructions, and commands for NVIDIA Nemotron 3.5 Lightning were collected and published successful NeMo Gym. The reported results screen the merchandise information suite, including knowledge and reasoning, instruction following, coding, agentic, tool-use, and long-context. Most evaluations usage NeMo Gym-native harnesses while a mini subset, including SWE-Bench and Terminal-Bench, utilized NeMo Evaluator natively. The published recipes specify the benchmark-specific containers, prompts, conclusion parameters, parser configurations, and scoring settings utilized to nutrient the results.

These numbers were measured pinch and use to the charismatic NVFP4 checkpoint

Agentic Coding Benchmarks

Additional harness-level coding-agent results for SWE-Bench Verified and Terminal-Bench 2.1 are shown below.

Agentic Coding Benchmarks

Deployment Geography: Global

Use Case

NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4 is simply a wide intent reasoning and chat exemplary intended to beryllium utilized successful English and coding languages. Other non-English languages (Spanish, French, German, Italian, Japanese) are besides supported. Intended for developers designing AI Agent systems, chatbots, RAG systems, and different AI-powered applications. Also suitable for emblematic instruction-following tasks.

Release Date

Hugging Face — 08/11/2026

Model Architecture

  • Architecture Type: Mixture-of-Experts Hybrid (Mamba + Transformer)
  • Network Architecture: Nemotron-3-Lightning + Multi-Token Prediction (MTP)
  • Number of exemplary parameters: 30B Total / 3B Active

Model Design

The exemplary was pre-trained pinch complete 20T tokens and supports up to 1M discourse length. The pre-training shape utilized an NVFP4 recipe. The exemplary includes Multi-Token Prediction (MTP) layers, which foretell aggregate early tokens to supply richer training signals.

Training Methodology

Stage 1: Pre-Training

  • NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4 exemplary was pre-trained utilizing an NVFP4 look pinch crawled and synthetic code, math, science, and wide knowledge data.
  • Software utilized for pre-training: Megatron-LM

Stage 2: Continued Pre-Training for Multi-Token Prediction (MTP)

  • The exemplary underwent a continued pre-training shape to train its Multi-Token Prediction (MTP) layers. In this stage, MTP heads study to foretell aggregate early tokens, providing richer training signals to the guidelines model. This shape aligns the MTP layers pinch the guidelines model's distribution.

Stage 3: Supervised Fine-Tuning

  • The exemplary was further fine-tuned connected synthetic code, math, science, instrumentality calling, instruction following, system outputs, and wide knowledge data. This shape incorporated information designed to support long-range retrieval and multi-document aggregation.

Stage 4: Reinforcement Learning

  • The exemplary underwent multi-environment reinforcement learning utilizing GRPO (Group Relative Policy Optimization) crossed math, code, science, instruction following, multi-step instrumentality use, multi-turn conversations, and system output environments. It utilized an asynchronous RL architecture that decouples training from conclusion and leverages MTP to accelerate rollout generation.
  • Software utilized for reinforcement learning: NeMo RL, NeMo Gym

Stage 5: Post-training Quantization (PTQ)

  • We performed post-training quantization (PTQ) pinch Nvidia Model Optimizer utilizing the pursuing recipe: Four Over Six NVFP4 (a version of fixed MSE calibration) W4A16 connected routed and shared experts, FP8 per-tensor move scales connected mamba in_proj/out_proj and KV cache. We utilized a subset of the Nemotron Ultra validation group for calibration pinch 1000 samples astatine 32k token length.

NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4 is simply a consequence of the supra work.

Input

  • Input Type(s): Text
  • Input Format(s): String
  • Input Parameters: One-Dimensional (1D): Sequences
  • Other Properties Related to Input: Maximum discourse magnitude up to 1M tokens. Supported languages see English, Spanish, French, German, Italian, and Japanese.

Output

  • Output Type(s): Text
  • Output Format: String
  • Output Parameters: One-Dimensional (1D): Sequences
  • Other Properties Related to Output: Maximum discourse magnitude up to 1M tokens

Our AI models are designed and/or optimized to tally connected NVIDIA GPU-accelerated systems. By leveraging NVIDIA's hardware (e.g. GPU cores) and package frameworks (e.g., CUDA libraries), the exemplary achieves faster training and conclusion times compared to CPU-only solutions.

Software Integration

  • Runtime Engine(s): PyTorch
  • Supported Hardware Microarchitecture Compatibility: NVIDIA Blackwell; NVIDIA Hopper (NVFP4 / W4A16); NVIDIA Ampere (W4A16)
  • Preferred/Supported Operating System(s): Linux

The integration of instauration and fine-tuned models into AI systems requires further testing utilizing use-case-specific information to guarantee safe and effective deployment. Following the V-model methodology, iterative testing and validation astatine some portion and strategy levels are basal to mitigate risks, meet method and functional requirements, and guarantee compliance pinch information and ethical standards earlier deployment.

Model Version(s)

  • GA (08/11/2026)

Quick Start Guide

All deployment snippets beneath assume:

export MODEL_CKPT=nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4

And for DSpark:

export DSPARK_CKPT=nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4-DSpark

Speculative Decoding Strategies

Lightning 3.5 ships pinch 2 outer draught models for speculative decoding arsenic good arsenic MTP (Multi-Token Prediction). While we presently urge DSpark for each cases - your usecase whitethorn align pinch DFlash and MTP:

  • DSpark: A semi-autoregressive speculative-decoding drafter that proposes a full artifact of campaigner tokens successful a azygous guardant walk from a parallel backbone. This is recommended for DGX Spark, arsenic good arsenic low-concurrency information centre deployments.
  • DFlash: A speculative-decoding drafter that uses a lightweight block-diffusion exemplary to make an full draught artifact successful 1 guardant pass.
  • MTP: A modeling method that trains the web to foretell respective early tokens astatine each position alternatively of only the adjacent one.

vLLM

For much indepth instructions connected really to deploy done vLLM, caput here

  • vLLM Nightly: vllm/vllm-openai:v0.27.1

1x DGX Spark (GB10)

Specdec method - DSpark:

vllm service --model $MODEL_CKPT \ --moe-backend marlin \ --kv-cache-dtype fp8 \ --max-model-len 1048576 \ --enable-prefix-caching \ --speculative_config.num_speculative_tokens 3 \ --mamba-backend flashinfer \ --mamba-cache-mode align \ --reasoning-parser nemotron_v3 \ --speculative_config.method dspark \ --tool-call-parser qwen3_coder \ --enable-auto-tool-choice

1x H100

For max throughput deployments, usage the pursuing configuration, nary speculative decoding strategy is champion for this serving configuration, and owed to representation constraints the Mamba cache dtype is group arsenic FP16:

vllm service --model $MODEL_CKPT \ --max-num-seqs 256 \ --max-num-batched-tokens 16384 \ --enable-prefix-caching \ --async-scheduling \ --mamba-backend flashinfer \ --moe-backend humming \ --linear-backend humming \ --mamba-ssu-algorithm horizontal \ --mamba-cache-mode align \ --mamba-ssm-cache-dtype float16 \ --enable-mamba-cache-stochastic-rounding \ --mamba-cache-philox-rounds 5 \ --reasoning-parser nemotron_v3 \ --tool-call-parser qwen3_coder \ --enable-auto-tool-choice

For interactive usage scenarios (achieving 40+ TPS/User) usage a little concurrency (<=128) pinch DSpark:

vllm service --model $MODEL_CKPT \ --max-num-seqs 128 \ --enable-prefix-caching \ --async-scheduling \ --speculative_config.model $DSPARK_CKPT \ --speculative_config.num_speculative_tokens 3 \ --mamba-ssu-algorithm horizontal \ --mamba-backend flashinfer \ --mamba-ssm-cache-dtype float16 \ --enable-mamba-cache-stochastic-rounding \ --mamba-cache-philox-rounds 5 \ --reasoning-parser nemotron_v3 \ --tool-call-parser qwen3_coder \ --enable-auto-tool-choice

8x H100

For long-context, multi-GPU serving (TP8 pinch master parallelism):

vllm service --model $MODEL_CKPT \ --mamba-backend flashinfer \ --async-scheduling \ --enable-prefix-caching \ --mamba-cache-mode align \ --enable-expert-parallel \ --tensor-parallel-size 8 \ --reasoning-parser nemotron_v3 \ --tool-call-parser qwen3_coder \ --enable-auto-tool-choice

1x GB200

vllm service --model $MODEL_CKPT \ --max-num-batched-tokens 10240 \ --no-enable-prefix-caching \ --async-scheduling \ --speculative_config.model $DSPARK_CKPT \ --speculative_config.num_speculative_tokens 5 \ --mamba-backend flashinfer \ --reasoning-parser nemotron_v3 \ --tool-call-parser qwen3_coder \ --enable-auto-tool-choice

W4A16 — Ampere

The aforesaid checkpoint besides serves via W4A16 kernels, extending sum to Ampere-class GPUs:

vllm service --model nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4 \ --moe-backend humming \ --linear-backend humming \ --max-num-seqs 256 \ --max-num-batched-tokens 32768 \ --enable-prefix-caching \ --async-scheduling \ --quantization modelopt_fp4 \ --mamba-backend flashinfer \ --mamba-cache-mode align \ --mamba-ssu-algorithm elemental \ --reasoning-parser nemotron_v3 \ --tool-call-parser qwen3_coder \ --enable-auto-tool-choice
  • Context Length: The H100 and GB200 snippets supra service the model's afloat 1M-token discourse model by default. If you're memory-constrained — aliases want much KV-cache headroom astatine precocious concurrency — little --max-model-len to lucifer your workload.

TensorRT-LLM

For much indepth instructions connected really to deploy done TensorRT-LLM, caput here

Container: nvcr.io/nvidia/tensorrt-llm/release:1.3.0rc24

1x H100

cat > nemotron-35-lightning-nvfp4-mtp.yaml << EOF kv_cache_config: dtype: fp8 enable_block_reuse: false mamba_state_config: periodic_snapshot_interval: 8192 free_gpu_memory_fraction: 0.8 mamba_ssm_cache_dtype: float16 mamba_ssm_stochastic_rounding: true mamba_ssm_philox_rounds: 5 moe_config: backend: MARLIN nvfp4_gemm_config: allowed_backends: [marlin, cutlass, cublaslt, cuda_core] cuda_graph_config: enable_padding: true max_batch_size: 8 speculative_config: decoding_type: MTP max_draft_len: 3 allow_advanced_sampling: true enable_chunked_prefill: true num_postprocess_workers: 4 print_iter_log: true stream_interval: 10 disable_overlap_scheduler: false EOF trtllm-serve \ $MODEL_CKPT \ --max_batch_size 8 \ --max_num_tokens 8192 \ --reasoning_parser nemotron-v3 \ --tool_parser qwen3_coder \ --config nemotron-35-lightning-nvfp4-mtp.yaml
  • Context length: The bid supra serves the model's afloat 1M-token discourse model by default. If you're memory-constrained — aliases want much KV-cache headroom astatine higher concurrency — little --max_seq_len to lucifer your workload.

SGLang

For much indepth instructions connected really to deploy done SGLang, caput here

  • Container: lmsysorg/sglang:dev-nemotron3-5-lighting

1x H100

sglang service \ --model-path nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4 \ --max-running-requests 256 \ --trust-remote-code \ --chunked-prefill-size 32768 \ --mem-fraction-static 0.9 \ --speculative-algorithm EAGLE \ --speculative-draft-model-path nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4 \ --speculative-num-steps 3 \ --speculative-eagle-topk 1 \ --speculative-num-draft-tokens 4 \ --mamba-backend flashinfer \ --mamba-radix-cache-strategy extra_buffer \ --reasoning-parser nemotron_3 \ --tool-call-parser qwen3_coder
  • Context length: The bid supra serves the model's afloat 1M-token discourse model by default. If you're memory-constrained — aliases want much KV-cache headroom astatine higher concurrency — group --context-length to a smaller value.

API Client

The examples beneath usage the OpenAI-compatible customer and activity pinch immoderate of the serving backends above. All backends service connected larboard 8000 (vLLM and TRT-LLM by default; SGLang via $PORT=8000), truthful the base_url useful as-is. Recommended sampling settings are Temperature 1.0 and Top_P 0.95.

The vLLM snippets supra registry the exemplary arsenic nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4 via --served-model-name. For the different backends — aliases if you alteration that emblem — transcript the identifier returned by GET /v1/models into MODEL below.

from openai import OpenAI client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY") MODEL = "nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4"

Lightning 3.5 exposes reasoning power done chat-template kwargs: reasoning enabled (the default), and reasoning abnormal for nonstop answers.

Reasoning ON / OFF and streaming examples: Click to expand!

Reasoning ON (default)

response = client.chat.completions.create( model=MODEL, messages=[{"role": "user", "content": "Write a haiku astir GPUs"}], max_tokens=16000, temperature=1.0, top_p=0.95, extra_body={"chat_template_kwargs": {"enable_thinking": True}} ) print(response.choices[0].message.content)

Reasoning OFF

response = client.chat.completions.create( model=MODEL, messages=[{"role": "user", "content": "What is the superior of Japan?"}], max_tokens=16000, temperature=1.0, top_p=0.95, extra_body={"chat_template_kwargs": {"enable_thinking": False}} ) print(response.choices[0].message.content)

Streaming

stream = client.chat.completions.create( model=MODEL, messages=[{"role": "user", "content": "Explain speculative decoding successful 2 sentences"}], max_tokens=16000, temperature=1.0, top_p=0.95, stream=True, ) for chunk in stream: print(chunk.choices[0].delta.content or "", end="", flush=True)

Tool Calling

The TRT-LLM snippet supra already launches pinch the required parsers (--reasoning_parser nemotron-v3 --tool_parser qwen3_coder). For vLLM, adhd the pursuing to immoderate service bid above:

--enable-auto-tool-choice \ --tool-call-parser qwen3_coder \ --reasoning-parser nemotron_v3

NOTE: For coding agents, adhd extra_body={"chat_template_kwargs": {"force_nonempty_content": True}} to the API call, arsenic shown below.

tools = [{ "type": "function", "function": { "name": "get_weather", "description": "Get the existent upwind for a city", "parameters": { "type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"], }, }, }] response = client.chat.completions.create( model=MODEL, messages=[{"role": "user", "content": "What's the upwind successful Santa Clara?"}], tools=tools, max_tokens=16000, temperature=1.0, top_p=0.95, extra_body={"chat_template_kwargs": {"force_nonempty_content": True}}, ) print(response.choices[0].message.tool_calls)

Training, Testing, and Evaluation Datasets

Data Modality: Text Training Data Size: More than 20 Trillion Tokens Dataset partition: Training [100%], testing [0%], validation [0%] Time play for training information collection: 2013 to December 2025 Time play for testing information collection: 2013 to December 2025 Time play for validation information collection: 2013 to December 2025 Data Collection Method by dataset: Hybrid: Automated, Manually-Collected, Synthetic Labeling Method by dataset: Hybrid: Automated, Manually-Labeled, Synthetic

NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4 is pre-trained connected a ample corpus of high-quality curated and synthetically-generated data. It is trained successful the English language, arsenic good arsenic 19 different spoken languages and 43 programming languages. Our sources screen a assortment of archive types specified as: webpages, dialogue, articles, and different written materials. The corpus spans domains including legal, math, science, finance, and more. We besides see a mini information of question-answering, and alignment style information to amended exemplary accuracy. The exemplary was pre-trained for much than 20 trillion tokens.

The post-training corpus for NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4 consists of high-quality curated and synthetically-generated data. Primary languages utilized for post-training see English, French, German, Italian, Japanese, Spanish, and Chinese.

These datasets, specified arsenic FinePDFs, EssentialWeb, HotpotQA, SQuAD, and HelpSteer3, do not collectively aliases exhaustively correspond each demographic groups (and proportionally therein). For instance, these datasets do not incorporate definitive mentions of demographic classes specified arsenic age, gender, aliases ethnicity successful 64-99% of samples, depending connected the source. In the subset wherever specified position are present, document-based datasets (FinePDFs and EssentialWeb) incorporate representational skews, specified arsenic references to "male" outnumbering those to "female", and mentions of "White" arsenic the astir predominant among taste identifiers (comprising 43-44% of ethnicity mentions). To mitigate these imbalances, we urge considering information techniques specified arsenic bias audits, fine-tuning pinch demographically balanced datasets, and mitigation strategies for illustration counterfactual information augmentation to align pinch the desired exemplary behavior. This information utilized a 3,000-sample subset per dataset, identified arsenic the optimal period for maximizing embedder accuracy.

During post-training, we make synthetic information by distilling trajectories, solutions, and translations from beardown coach models and supplier systems, often grounded successful existent tasks aliases documents and aggressively filtered for quality. For math, code, and science, we commencement from curated problem sets and usage unfastened root permissive models specified arsenic GPT-OSS-120B to nutrient step-by-step reasoning traces, campaigner solutions, best-of-n action traces, and verified CUDA kernels. For long-context and science, we build synthetic QA and reasoning information by retrieving passages from agelong documents, generating MCQ/OpenQA questions and answers, and paraphrasing them into aggregate prompt/response formats to guarantee diversity. Across each pipelines we stack automated verification—compilers, numerical checks, connection identification—to guarantee our information is precocious quality.

For each domains, we use a unified information filtering pipeline to guarantee that only high-quality, license-compliant, and verifiable samples are utilized for post-training. We first discard malformed examples utilizing structural checks (e.g., missing instrumentality definitions erstwhile instrumentality calls are present). We past aggressively select reasoning traces exhibiting pathological repetition, specified arsenic repeated n-grams wrong a sliding model aliases crossed the full trajectory, which we recovered to beryllium a beardown parameter of malformed aliases low-quality reasoning. Finally, based connected soul audits of synthetically generated datasets, we observed that immoderate coach models occasionally nutrient reasoning traces and last responses that implicitly align pinch circumstantial governmental entities aliases beforehand nationalistic narratives. To mitigate this, we use targeted keyword- and regex-based filters and region each trajectories matching specified behavior.

Alongside the model, we merchandise our last pre-training and post-training data, arsenic outlined successful this section. For easiness of analysis, location is simply a sample group that is ungated. For each remaining code, mathematics and multilingual data, gating and support is required, and the dataset is permissively licensed for exemplary training purposes.

For Detailed Dataset Information: Click here!

Base Pre-Training Corpus (Nemotron 3 Foundation)

The instauration of the exemplary is trained connected the Nemotron 3 corpus, comprising the pursuing datasets from the Nemotron Pretraining Datasets collection:

Dataset Collection Token Counts Description
Nemotron-CC-v2 & v2.1 9.1T A monolithic postulation of English web information filtered from Common Crawl, including 2.5T+ tokens of caller organic, translated, and synthetically rephrased content.
Nemotron-CC-Code-v1 427.9B High-quality codification tokens extracted from Common Crawl utilizing the Lynx + LLM pipeline to sphere building and equations.
Nemotron-Pretraining-Code-v1 & v2 & v3 1.7T Curated GitHub codification references pinch multi-stage filtering, deduplication, and large-scale synthetic codification data.
Nemotron-CC-Math-v1 133.3B High-quality mathematics pre-training dataset preserving LaTeX formatting and mathematical structures.
Nemotron-Pretraining-Specialized-v1 & v1.1 & v1.2 & Nemotron-Pretraining-SFT-v1 660.0B Synthetic datasets targeting specialized domains specified arsenic STEM reasoning and technological coding.
Nemotron-Pretraining-Legal-v1 4.3B Synthetic datasets targeting the ineligible domain.

Public Datasets

Crawled and Scraped from Online Sources by NVIDIA

The English Common Crawl information was downloaded from the Common Crawl Foundation (see their FAQ for specifications connected their crawling) and includes the snapshots CC-MAIN-2013-20 done CC-MAIN-2025-13. The information was subsequently deduplicated and filtered successful various ways described successful the Nemotron-CC paper. Additionally, we extracted information for 15 languages from the pursuing 3 Common Crawl snapshots: CC-MAIN-2024-51, CC-MAIN-2025-08, CC-MAIN-2025-18. The 15 languages included were Arabic, Chinese, Danish, Dutch, French, German, Italian, Japanese, Korean, Polish, Portuguese, Russian, Spanish, Swedish, and Thai. As we did not person reliable multilingual model-based value classifiers available, we applied conscionable heuristic filtering instead—similar to what we did for little value English information successful the Nemotron-CC pipeline, but selectively removing immoderate filters for immoderate languages that did not activity well. Deduplication was done successful the aforesaid measurement arsenic for Nemotron-CC.

The GitHub Crawl was collected utilizing the GitHub REST API and the Amazon S3 API. Each crawl was operated successful accordance pinch the complaint limits group by its respective source, either GitHub aliases S3. We cod earthy root codification and subsequently region immoderate having a licence which does not beryllium successful our permissive-license set.

Dataset Modality Dataset Size Collection Period Collecting Organisation
English Common Crawl Text 3.36T 4/8/2025 NVIDIA Advanced Deep Learning Research
English Common Crawl 1.1 Text Not disclosed 10/2/2025 NVIDIA Advanced Deep Learning Research
Multilingual Common Crawl Text 812.7B 5/1/2025 NVIDIA Advanced Deep Learning Research
GitHub Crawl Text 747.4B 4/29/2025 NVIDIA Advanced Deep Learning Research
GitHub Crawl 1.1 Text 172.7B 9/30/2025 NVIDIA Advanced Deep Learning Research

Private Non-publicly Accessible Datasets of Third Parties

Dataset Model(s) used
Global Regulation Unknown
TAUS Translation Memory Unknown
Scale HLE Unknown
HackerRank Coding Unknown
RL information for Search Gemini 3; GPT-5
Mercor SWE-AgentsV1 Undisclosed

Private Non-publicly Accessible Datasets by NVIDIA

Dataset Model(s) used
Simple Minesweeper Undisclosed
Simple Sudoku Undisclosed
Multitool Typewriter Hard Undisclosed
Machine Translation of News Commentary and TAUS Translation Memory Undisclosed
Machine Translation of STEM - Qwen2.5-14B-Instruct
Competitive Coding RL information from Nemotron Cascade Undisclosed
Long discourse RL Undisclosed
Single-step SWE RL for spot generation Undisclosed
OpenHands SWE Undisclosed

NVIDIA-Sourced Synthetic Datasets (Pre-Training)

Dataset Modality Dataset Size Seed Dataset Model(s) utilized for generation
Nemotron-Pretraining-Fact-Seeking Text 35.0B FineWiki Qwen3-30B-A3B-Instruct-2507
Nemotron-Pretraining-Legal Text 4.3B CommonPile (caselaw_access_project_filtered); California Code of Regulations; Judicial Ethics Opinions; GLOBALCIT; CUAD; Nemotron Personas; ToSDR Terms of Service Corpus; CodeHima/TOS_Dataset; ContractNLI; CaseHOLD; Code of Federal Regulations; Canadian Case Law (subsets that let commercialized use) Qwen3-235B-A22B-Thinking-2507
Nemotron-Pretraining-Formal-Logic Text 128M Nemotron Personas Qwen3-235B-A22B-Thinking-2507
Nemotron-Pretraining-Economics Text 73.4M - Qwen3-235B-A22B-Thinking-2507
Nemotron-Pretraining-Multiple-Choice Text 1.6B MMLU Auxiliary Train DeepSeek-V3; Qwen3-235B-A22B
Nemotron-Pretraining-Code-Concepts Text 7.3B - gpt-oss-20b; gpt-oss-120b
Nemotron-Pretraining-Unconditional-Algorithmic Text 196.5M - gpt-oss-120b; Qwen3-235B-A22B
More Synthetic Tasks from DeepSeek-V3 and Qwen3-235B-A22B Text 1.1B train splits of acp_bench; ai2_arc; babi; gsm8k; hendrycks_math; IFEval; MedText; mediqa_qa; mlqa; MMLU-Pro; mmlu-pro-plus; MMLU-ProX; nq_open; tinyGSM8k; truthful_qa; truthfulqa-multi; MATH-lighteval; mmlu; awesome-chatgpt-prompts; super_glue DeepSeek v3; Qwen3-235B-A22B
Synthetic Tasks from DeepSeek-V3 and Qwen3-235B-A22B Text 6.7B train splits of Into the Unknown; AI2 ARC (AI2 Reasoning Challenge); BLiMP (Benchmark of Linguistic Minimal Pairs); CommonSenseQA; GLUE; HeadQA; Hendrycks Ethics; Memo Trap; modus-tollens; NeQA; pattern-matching-suppression; mastermind_24_mcq_random; mastermind_24_mcq_close; quote-repetition; redefine-math; Repetitive Algebra; sig-figs; MMLU-Pro; MC-TACO; MedConceptsQA; MMLU_dataset; OpenbooksQA; PIQA (Physical Interaction Question Answering); SocialIQA; SuperGLUE; tinyAI2_arc; tinyMMLU; tinyWinogrande; TruthfulQA; WebQuestions; Winogrande; GPQA; MBPP DeepSeek v3; Qwen3-235B-A22B
Synthetic Art of Problem Solving from DeepSeek-R1 Text 40B Art of Problem Solving; American Mathematics Competitions 8; American Mathematics Competitions 10 DeepSeek-R1
Synthetic Moral Stories and Social Chemistry from Qwen3-235B-A22B-Thinking-2507 and Mixtral-8x22B-v0.1 Text 15.2M social-chemestry-101; Moral Stories Qwen3-235B-A22B-Thinking-2507; Mixtral-8x22B-v0.1
Synthetic Moral Stories and Social Chemistry from Mixtral-8x22B-v0.1 Text 327M social-chemestry-101; Moral Stories Mixtral-8x22B-v0.1
Synthetic Social Sciences seeded pinch OpenStax from DeepSeek-V3, Mixtral-8x22B-v0.1, and Qwen2.5-72B Text 83.6M OpenStax - CC BY-SA subset DeepSeek-V3; Mixtral-8x22B-v0.1; Qwen2.5-72B
Synthetic Health Sciences seeded pinch OpenStax from DeepSeek-V3, Mixtral-8x22B-v0.1, and Qwen2.5-72B Text 9.7M OpenStax - CC BY-SA subset DeepSeek-V3; Mixtral-8x22B-v0.1; Qwen2.5-72B
Synthetic STEM seeded pinch OpenStax, Open Textbook Library, and GSM8K from DeepSeek-R1, DeepSeek-V3, DeepSeek-V3-0324, and Qwen2.5-72B Text 175M OpenStax - CC BY-SA subset; GSM8K; Open Textbook Library - CC BY-SA & GNU subset DeepSeek-R1, DeepSeek-V3; DeepSeek-V3-0324; Qwen2.5-72B
Nemotron-PrismMath Text 4.6B Big-Math-RL-Verified; OpenR1-Math-220k Qwen2.5-0.5B-instruct, Qwen2.5-72B-Instruct; DeepSeek-R1-Distill-Qwen-32B
Synthetic Question Answering Data from Papers and Permissible Books from Qwen2.5-72B-Instruct Text 350M arXiv; National Institutes of Health ExPorter; BioRxiv; PMC Article; USPTO Backgrounds; peS2o; Global Regulation; CORE; PG-19; DOAB CC BY & CC BY-SA subset; NDLTD Qwen2.5-72B-Instruct
Synthetic Rephrased Math Data from Common Crawl from phi-4 Text 73B Common Crawl phi-4
Synthetic Math Data from Common Crawl 4plus Text 52.3B Common Crawl phi-4
Synthetic Math Data from Common Crawl 3 Text 80.9B Common Crawl phi-4
Synthetic AGIEval seeded pinch AQUA-RAT, LogiQA, and AR-LSAT from DeepSeek-V3 and DeepSeek-V3-0324 Text 4.0B AQUA-RAT; LogiQA; AR-LSAT DeepSeek-V3; DeepSeek-V3-0324
Synthetic AGIEval seeded pinch AQUA-RAT, LogiQA, and AR-LSAT from Qwen3-30B-A3B Text 4.2B AQUA-RAT; LogiQA; AR-LSAT Qwen3-30B-A3B
Synthetic Art of Problem Solving from Qwen2.5-32B-Instruct, Qwen2.5-Math-72B, Qwen2.5-Math-7B, and Qwen2.5-72B-Instruct Text Undisclosed Art of Problem Solving; American Mathematics Competitions 8; American Mathematics Competitions 10; GSM8K; PRM800K Qwen2.5-32B-Instruct; Qwen2.5-Math-72B; Qwen2.5-Math-7B; Qwen2.5-72B-Instruct
Synthetic MMLU Auxiliary Train from DeepSeek-R1 Text 0.5B MMLU Auxiliary Train DeepSeek-R1
Synthetic Long Context Continued Post-Training Data from Papers and Permissible Books from Qwen2.5-72B-Instruct Text Undisclosed arXiv; National Institutes of Health ExPorter; BioRxiv; PMC Article; USPTO Backgrounds; peS2o; Global Regulation; CORE; PG-19; DOAB CC BY & CC BY-SA subset; NDLTD Qwen2.5-72B-Instruct
Synthetic Common Crawl from Qwen3-30B-A3B and Mistral-Nemo-12B-Instruct Text 415.8B Common Crawl Qwen3-30B-A3B; Mistral-NeMo-12B-Instruct
Synthetic Multilingual Data from Common Crawl from Qwen3-30B-A3B Text Undisclosed Common Crawl Qwen3-30B-A3B
Synthetic Multilingual Data from Wikimedia from Qwen3-30B-A3B Text Undisclosed Wikimedia Qwen3-30B-A3B
Synthetic Math Data from Wikimedia from Nemotron-4-340B-Instruct Text Undisclosed - Nemotron-4-340B-Instruct
Synthetic Common Crawl Code from phi-4 Text 427.9B Common Crawl phi-4
Synthetic Scientific Coding from Qwen3-235B-A22B Text 1.2B Wikimedia Qwen3-235B-A22B
Tool Calling Data Text 26.2B - Qwen3-235B-A22B-2507; gpt-oss-120b
Synthetic Essential-Web from QwQ-32B Text 28.1B Essential-Web QwQ-32B
Translated Synthetic Crawl Text 389.9B Common Crawl Qwen3-30B-A3B
Translated Synthetic Wikipedia Text 7.9B Wikimedia Qwen3-30B-A3B
Synthetic Long Context from Qwen3-235B-A22B-Instruct-2507 Text Undisclosed CORE; PG-19; DOAB CC BY & CC BY-SA subset; NDLTD Qwen3-235B-A22B-Instruct-2507
Synthetic Search STEM OPENQ from DeepSeek-R1-0528 Text Undisclosed - DeepSeek-R1-0528
Synthetic MCQ from Qwen2.5-32B-Instruct and DeepSeek-R1-0528 Text Undisclosed - Qwen2.5-32B-Instruct; DeepSeek-R1-0528
Synthetic Offline Search MCQA HLE from DeepSeek-R1-0528 Text Undisclosed - DeepSeek-R1-0528
Synthetic Offline Search MCQA GPQA from Qwen3-235B-A22B and DeepSeek-R1-0528 Text Undisclosed - Qwen3-235B-A22B; DeepSeek-R1-0528
Synthetic Human Preference from QwQ-32B, Qwen3-30B-A3B, Qwen3-235B-A22B, Qwen3-235B-A22B-Instruct-2507, Mistral-Small-3.1-24B-Instruct-2503, Mistral-Small-3.2-24B-Instruct-2506, MiniMax-M1-80k, MiniMax-M1-40k, Kimi-K2-Instruct, DeepSeek-V3-0324, DeepSeek-R1-0528 Text Undisclosed - QwQ-32B; Qwen3-30B-A3B; Qwen3-235B-A22B; Qwen3-235B-A22B-Instruct-2507; Mistral-Small-3.1-24B-Instruct-2503; Mistral-Small-3.2-24B-Instruct-2506; MiniMax-M1-80k; MiniMax-M1-40k; Kimi-K2-Instruct; DeepSeek-V3-0324; DeepSeek-R1-0528
Synthetic WildChat-1M and arena-human-preference-140k from DeepSeek-R1, gemma-2-2b-it, gemma-3-27b-it, gpt-oss-20b, gpt-oss-120b, Mistral-7B-Instruct-v0.3, Mixtral-8x22B-Instruct-v0.1, Nemotron-4-340B-Instruct, NVIDIA-Nemotron-Nano-9B-v2, Phi-4-mini-instruct, Phi-3-small-8k-instruct, Phi-3-medium-4k-instruct, Qwen3-235B-A22B, QwQ-32B Text Undisclosed WildChat-1M; arena-human-preference-140k DeepSeek-R1; gemma-2-2b-it; gemma-3-27b-it; gpt-oss-20b; gpt-oss-120b; Mistral-7B-Instruct-v0.3; Mixtral-8x22B-Instruct-v0.1; Nemotron-4-340B-Instruct; NVIDIA-Nemotron-Nano-9B-v2; Phi-4-mini-instruct; Phi-3-small-8k-instruct; Phi-3-medium-4k-instruct; Qwen3-235B-A22B; QwQ-32B
Synthetic Code from Qwen3-32B Text Undisclosed English Common Crawl; English Common Crawl 1.1 Qwen3-32B
Synthetic OpenCodeReasoning from DeepSeek-R1 Text Undisclosed OpenCodeReasoning DeepSeek-R1
Synthetic OpenCodeReasoning from DeepSeek-R1-0528 Text Undisclosed OpenCodeReasoning DeepSeek-R1-0528
Synthetic HackerRank Coding from DeepSeek-R1-0528 Text Undisclosed HackerRank Coding Dataset DeepSeek-R1-0528
Synthetic LIMO from DeepSeek-R1-0528 Text Undisclosed LIMO DeepSeek-R1-0528
Synthetic SCP from DeepSeek-R1-0528 Text Undisclosed SCP-116K DeepSeek-R1-0528
Synthetic Stack Exchange from DeepSeek-R1-0528 Text Undisclosed Stack Exchange DeepSeek-R1-0528
Synthetic Stack Exchange from gpt-oss-120b and Qwen2.5-32B-Instruct Text Undisclosed Stack Exchange gpt-oss-120b; Qwen2.5-32B-Instruct
Synthetic Stack Exchange from gpt-oss-120b Text Undisclosed Stack Exchange gpt-oss-120b
Synthetic Art of Problem Solving from gpt-oss-120b and Qwen2.5-32B-Instruct Text Undisclosed Art of Problem Solving; American Mathematics Competitions 8; American Mathematics Competitions 10 gpt-oss-120b; Qwen2.5-32B-Instruct
Synthetic Common Crawl from Qwen3-30B-A3B Text Undisclosed Common Crawl Qwen3-30B-A3B
Synthetic Wikipedia from Qwen3-30B-A3B Text Undisclosed Wikimedia Qwen3-30B-A3B
Synthetic Essential-Web from Qwen3-30B-A3B and Qwen3-235B-A22B-Thinking-2507 Text Undisclosed Essential-Web Qwen3-30B-A3B; Qwen3-235B-A22B-Thinking-2507
Synthetic Essential-Web from gpt-oss-120b Text Undisclosed Essential-Web gpt-oss-120b
Synthetic Textbook Math from Qwen3-30B-A3B, Qwen3-235B-A22B, phi-4 Text Undisclosed Common Crawl; FineMath Qwen3-30B-A3B; Qwen3-235B-A22B; phi-4
Synthetic Math and Code from DeepSeek-R1 and DeepSeek-R1-0528 Text Undisclosed Magicoder-Evol-Instruct-110K; opc-sft-stage2; TACO; OpenCodeReasoning; OpenMathReasoning; NuminaMath CoT DeepSeek-R1; DeepSeek-R1-0528
Synthetic Math from gpt-oss-120b and Qwen2.5-32B-Instruct Text Undisclosed - gpt-oss-120b; Qwen2.5-32B-Instruct
Synthetic OpenMathReasoning from gpt-oss-120b and Qwen2.5-32B-Instruct Text Undisclosed OpenMathReasoning gpt-oss-120b; Qwen2.5-32B-Instruct
Synthetic KernelBook from DeepSeek-R1-0528 Text Undisclosed KernelBook DeepSeek-R1-0528
Synthetic Scale HLE from gpt-oss-120b Text Undisclosed Scale HLE gpt-oss-120b
Synthetic CDQuestions from gpt-oss-120b Text Undisclosed CDQuestions gpt-oss-120b
Synthetic GPQA from gpt-oss-120b and Qwen2.5-32B-Instruct Text Undisclosed Stack Exchange gpt-oss-120b; Qwen2.5-32B-Instruct
Synthetic Vedantu from gpt-oss-120b Text Undisclosed Vedantu gpt-oss-120b
Synthetic Search STEM MCQ from Qwen3-235B-A22B and DeepSeek-R1-0528 Text Undisclosed - Qwen3-235B-A22B; DeepSeek-R1-0528
Synthetic OpenSTEM from Qwen2.5-32B-Instruct and DeepSeek-R1-0528 Text Undisclosed - Qwen2.5-32B-Instruct; DeepSeek-R1-0528
Synthetic MCQ10 from DeepSeek-R1-0528 Text Undisclosed - DeepSeek-R1-0528
Synthetic MCQ4 from Qwen3-235B-A22B, DeepSeek-R1-0528, and Qwen3-235B-A22B-Instruct-2507 Text Undisclosed - Qwen3-235B-A22B; DeepSeek-R1-0528; Qwen3-235B-A22B-Instruct-2507

NVIDIA-Sourced Synthetic Datasets (Post-Training)

Dataset Modality Dataset Size Seed Dataset Model(s) utilized for generation
Synthetic Competitive MATH Proofs from DeepSeek-V4-Pro Text Undisclosed [AMC 8 Problems and Solutions, AMC 10 Problems and Solution, and AIME Problems and Solutions] [deepseek-ai/DeepSeek-V4-Pro]
Synthetic Hermes Agent Reasoning Traces Text Undisclosed [lambda/hermes-agent-reasoning-traces] [hermes-agent-generator]
Synthetic Competitive Coding from DeepSeek-V4-Pro Text Undisclosed [NVCompetitiveCodingV1] [deepseek-ai/DeepSeek-V4-Pro]
Synthetic Competitive Science Reasoning from DeepSeek-V4-Pro Text Undisclosed [AMC 8 Problems and Solutions, AMC 10 Problems and Solution, and AIME Problems and Solutions]; [EssentialAI/essential-web-v1.0]; [cdquestions.com]; [Pile-FreeLaw]; [Vedantu]; [askfilo]; [doubtnut]; [ICHO-IPH0 Dataset]; [LIMO dataset (Less is More for Reasoning)]; [AAPT]; [ChemData 700K]; [oMeBench]; [Flavor Analysis and Recognition Transformer]; [ChemCoTBench]; [Llama Nemotron Dataset] [deepseek-ai/DeepSeek-V4-Pro]
Synthetic Competitive MATH CoT and TIR from Nemotron 5.5 Text Undisclosed [Pile-FreeLaw]; [AMC 8 Problems and Solutions, AMC 10 Problems and Solution, and AIME Problems and Solutions] [Nemotron 5.5]
Vendor Terminal Bench-like Tasks from Mercor Text Undisclosed [Terminal chair for illustration tasks curated by the vendor] [Undisclosed - purchased dataset]
Turing Math Data Pack Text Undisclosed [Turing Math Data Pack dataset] [Undisclosed - purchased dataset]
Synthetic Holdout, Skywork, DAPO, and Turing Math from GPT-5.5 Text Undisclosed [DocQA-RL-1.6K]; [DAPO-Math-17k] [GPT-5.5]
Synthetic Long Context RL from QwenLong L1 and DocQA-RL-1.6K Text Undisclosed [DocQA-RL-1.6K] Undisclosed
Synthetic Competitive Coding Gym Tasks Text Undisclosed [NVCompetitiveCodingV1.1] Undisclosed
Synthetic Finance SEC Search Agent from GPT-OSS-120B and Qwen3 Text Undisclosed [SEC filings from sec.gov] [GPT-OSS-120B]; [Qwen3-235B-A22B-Instruct]; [Qwen3-4B-Instruct]
Synthetic Structured Outputs from Qwen3-30B-A3B-Instruct-2507, Qwen3-30B-A3B-Thinking-2507, Qwen3-235B-A22B-Instruct-2507, and Qwen3-235B-A22B-Thinking-2507 Text Undisclosed [Nemotron-RL-agent-structured-outputs-v1] [Qwen3-30B-A3B-Instruct-2507]; [Qwen3-235B-A22B-Instruct-2507]
Synthetic Long Context Equivalence Rule from Qwen3-235B-A22B-Thinking-2507 and DeepSeek-R1 Text Undisclosed [Long-context SFT data] [Qwen/Qwen3-235B-A22B-Thinking-2507]; [Deepseek-ai/DeepSeek-R1]
Synthetic Science RL Data Blend from Qwen2.5-32B Text Undisclosed [doubtnut]; [Pile-FreeLaw]; [Llama Nemotron Dataset]; [askfilo]; [EssentialAI/essential-web-v1.0]; [Vedantu]; [auxiliary_train]; [cdquestions.com]; [AMC 8 Problems and Solutions, AMC 10 Problems and Solution, and AIME Problems and Solutions]; [AAPT]; [ICHO-IPH0 Dataset]; [LIMO dataset (Less is More for Reasoning)] [Qwen2.5-32B]
Synthetic Abstention Data from Nemotron Super v3 Text Undisclosed [Go abstention Dataset] [nvidia/nvidia/nemotron-3-super-v3]
Synthetic Chemistry Data from Nemotron Super v3 Text Undisclosed [ChemData 700K] [nvidia/nvidia/nemotron-3-super-v3]
Synthetic Tool Call Schema for RL Text 469,983 [UltraTool]; [ToolEyes]; [AutoTools]; [API-Bank]; [Nemotron-Personas-USA]; [Salesforce xLAM function-calling]; [Glaive function-calling-v2]; [Agent-Ark/Toucan-1.5M] [DeepSeek-V3.2]; [GLM-4.6]; [gpt-oss-120b]; [Kimi-K2-Instruct]
Synthetic Freeform Text Formatting from GPT-OSS-120B Text Undisclosed [In-house data] [GPT OSS 120B - Apache 2.0]
Synthetic Citation Formatting from GPT-OSS-120B Text Undisclosed [In-house data] [GPT OSS 120B - Apache 2.0]
Droid Harness Pivot Vendor Data Text Undisclosed [Droid Harness Pivot vendor data] Undisclosed
Synthetic HotpotQA Training Data from Qwen3-235B Text Undisclosed [HotpotQA] [Qwen3-235B]
Synthetic Natural Language Math Proofs from Nemotron 5.5 Text Undisclosed [AMC8, AMC10, and AIME problem sets hosted connected Art of Problem Solving]; [Pile-StackExchange] [Nemotron 5.5]
Synthetic Stack Overflow OpenQ Text Undisclosed [Pile-FreeLaw] Undisclosed
Chemistry Ether0 Vendor Data Text Undisclosed [Chemistry ether0 vendor data] Undisclosed
Synthetic Litmus-Bench Chemistry from ChEMBL Text Undisclosed [ChEMBL]; [Nemo Gym RL dataset generated from ChEMBL pinch RDKit] Undisclosed
Synthetic ZINC Chemistry from Nemotron Super v3 Text Undisclosed [ZINC] [Nemotron Super v3]
ARC-AGI Gym Environment Text Undisclosed [ARC-AGI-2] [ARC-AGI-2]
Synthetic Agentic Search Tool-Use from DeepSeek-V3.2 Text Undisclosed [Mercor Data] [DeepSeek-V3.2]
Synthetic Text-To-SQL Text 96,564 [In-house Text-to-SQL data] [gpt-oss-120b]
Dialog Memory Vendor Data Text Undisclosed [Patronus outer vendor agreement] Undisclosed
Synthetic Indirect Prompt Injection from Nemotron Super v3 and Qwen3-Next-80B-A3B-Instruct Text Undisclosed [In-house indirect punctual injection data] [nvidia/nemotron-3-super-v3, qwen/qwen3-next-80b-a3b-instruct]
Synthetic Malicious Code and Agentic Security Text Undisclosed [In-house malicious-code / agentic-security data] Undisclosed
Synthetic Single-Step SWE Patch Selection Text Undisclosed [SWE-Gym Dataset]; [SWE Bench Verified Benchmark] [ground truth and task checks]
Synthetic Natural Language Math Final Answers from Nemotron 5.5 Text Undisclosed [AMC8, AMC10, and AIME problem sets hosted connected Art of Problem Solving]; [Pile-StackExchange] [nemotron 5.5]
Synthetic Simple Math Prompts for Token Efficiency Text Undisclosed [In-house elemental mathematics prompts] Undisclosed
Synthetic Abstention Data from Nemotron Super v3 (CRAG) Text Undisclosed [CRAG] [nvidia/nvidia/nemotron-3-super-v3]
Synthetic Agentless SWE Text 242,536 [SWE-Rebench-V2]; [SWEbench Training Set]; [R2E-Gym/R2E-Gym-Subset]; [SWE-Gym/SWE-Gym]; [SWE-Rebench] [openai/gpt-oss-120b]
Synthetic Agentless SWE from DeepSeek-R1-0528 Text 209,976 [SWE-Bench-Train]; [SWE-Fixer-Train]; [SWE-reBench]; [SWE-Smith] [deepseek-ai/DeepSeek-R1-0528]
Synthetic Agentic CUDA Traces from GLM-4.7 Text 2,276 [Internal CUDA task data] [GLM-4.7]
Synthetic Math Proofs from DeepSeek-V3.2-Speciale Text 820,772 [Nemotron-Math-Proofs-v1] [SDG: DeepSeek-V3.2-Speciale]; [Filter: impervious validation]
Synthetic Multilingual SFT from DeepSeek-V3 Text 1,245,284 [Nano v3 SFT data] [DeepSeek-V3]
Synthetic Agentic Code from gpt-oss-120b Text 109,086 [NVAgenticCLIPrompts-v1]; [NVAgenticSkills-v1]; [NVAgenticCLIMultiTurnPrompts-v1] [openai/gpt-oss-120b]
Synthetic Agentic CLI and Web Skills from gpt-oss-120b Text 27,418 [NVAgenticCLIPrompts-v1]; [NVAgenticSkills-v1]; [NVAgenticCLIMultiTurnPrompts-v1]; [NVAgenticCLIPrompts-Web-v1] [openai/gpt-oss-120b]
Synthetic Agentic Coding from gpt-oss-120b Text 160,531 [NVAgenticCLIPrompts-v1]; [NVAgenticSkills-v1]; [NVAgenticCLIMultiTurnPrompts-v1]; [NVAgenticCLIPrompts-Web-v1] [openai/gpt-oss-120b]
Synthetic OpenCode Agentic Tasks from gpt-oss-120b Text 614,773 [NVAgenticCLIPrompts-v1]; [NVAgenticSkills-v1]; [NVAgenticCLIMultiTurnPrompts-v1]; [NVAgenticCLIPrompts-Web-v1] [openai/gpt-oss-120b]
Synthetic SWE Unverified Text Undisclosed [NVAgenticCLIPrompts-v1]; [NVAgenticSkills-v1]; [NVAgenticCLIMultiTurnPrompts-v1]; [NVAgenticCLIPrompts-Web-v1] [gpt-oss-120b]; [Qwen/Qwen3-Coder-480B-A35B-Instruct]; [GLM-4.7-Flash]
Synthetic ARC-AGI Ultra Data Text 192,016 [ARC-AGI-2]; [arc dataset collection] [ARC-AGI-2]
Synthetic LiveCodeBench TIR from DeepSeek-R1-0528 Text 1,283,398 [Nemotron-X training datasets] [DeepSeek-R1-0528]
Synthetic Verilog and SystemVerilog Code from DeepSeek-R1-0528 and GPT-OSS-120B Text 1,233,247 [Verilog/SystemVerilog seed code] [SDR: DeepSeek R1 0528 and GPT-OSS-120B]; [Filtering: Claude 4 Sonnet]
Synthetic Aider Python Tasks from DeepSeek-R1-0528 Text 236,099 [Exercism (GitHub Python)] [Deepseek R1 0528]
Synthetic Chat Reasoning-Off Data from GLM-5 Text 646,738 [lmarena-ai/repochat-arena-preference-4k personification prompts] [Multi-turn conversations generated by GLM-5 pinch best-of-4 action via Qwen3-Nemotron-235B-A22B-GenRM]
Synthetic Chat Reasoning-On Data from GLM-5 Text 644,286 [lmarena-ai/repochat-arena-preference-4k personification prompts]; [lmarena-ai/arena-expert-5k personification prompts]; [lmarena-ai/arena-human-preference-55k personification prompts]; [lmarena-ai/arena-human-preference-100k personification prompts]; [lmarena-ai/arena-human-preference-140k personification prompts] [Multi-turn conversations generated by GLM-5 pinch best-of-4 action via Qwen3-Nemotron-235B-A22B-GenRM]
Synthetic Multilingual Safety from Riva-Translate-4B-Instruct-v1.1 Text 132,067 [Safety SFT Data: Ultra] [nvidia/Riva-Translate-4B-Instruct-v1.1]
Synthetic Science Reasoning Effort Medium Text 502,722 [science-reasoning-effort-medium-v0] Undisclosed
Synthetic Telecom Tool-Use Trajectories from gpt-oss-120b Text 12,455 [Existing Tau2 telecom trajectories primitively generated pinch DeepSeek V3.2] [gpt-oss-120b]
Synthetic Terminal Bench Data from OpenReasoningv2 Text Undisclosed [OpenCodeReasoningv2]; [OpenMathReasoning]; [nemo-swe-bench-repos]; [SWE-Rebench]; [SWE-Fixer-110K] [OpenReasoningv2]
Synthetic Tulu Instruction Following from DeepSeek-R1-0528 Text 105,361 [Nemotron-X training datasets] [DeepSeek-R1-0528]
Synthetic Instruction Following from gpt-oss-120b Text 151,988 [IFEval]; [IFEvalG] [gpt-oss-120b]
Synthetic Instruction Following for RL Text Undisclosed [WildChat-1M]; [LMSYS-340B-Eval Dataset]; [LMSYS-Chat-1M Prompts]; [IFEval]; [IFEvalG] [Qwen/Qwen3-235B-A22B-Thinking-2507]; [gpt-oss-120b]; [Qwen3-235B-A22B-Instruct-2507]
Synthetic Identity Data from Qwen3-Next-80B-A3B-Instruct and Qwen3-235B-A22B-Instruct-2507 Text 25,992 [Hand-written prompts] [Qwen3-Next-80B-A3B-Instruct]; [Qwen3-235B-A22B-Instruct-2507]
Synthetic Terminus Ultra Agentic Reasoning Blend Text 96,881 [ARC-AGI-2]; [OpenCodeReasoningv2]; [OpenMathReasoning]; [SWE-Fixer-110K]; [SWE-Rebench]; [SWE-Smith] [DeepSeek-V3.2]; [Qwen3-235B-A22B-Thinking-2507]; [Ring-1T]; [Kimi-K2.5]; [GLM-4.7-FP8]; [Qwen3-Next-80B-A3B-Thinking]; [gpt-oss-120b]; [Ministral-3-14B-Reasoning-2512]; [LM-4.5-Air-FP8]
Synthetic STEM from Qwen3-235B-A22B-Thinking-2507 Text 1,174,694 [IChO-IPhO-RL-v2]; [Physics-Big Dataset]; Scale HLE; [OpenMathReasoning]; [OpenCodeReasoning] [Qwen3-235B-A22B-Thinking-2507]
Synthetic STEM from Qwen3-235B-A22B-Instruct-2507 and gpt-oss-120b Text Undisclosed [arXiv]; [National Institutes of Health ExPorter]; [BioRxiv]; [PMC Article]; [USPTO Backgrounds]; [peS2o]; Global Regulation; [CORE]; [PG-19]; [DOAB CC BY & CC BY-SA subset]; [NDLTD] [Qwen3-235B-A22B-Instruct-2507]; [gpt-oss-120b]
Translation Data from TAUS Text 1,618,055 [TAUS proprietary dataset] Undisclosed
Synthetic Art of Problem Solving and Stack Exchange from gpt-oss-120b, Qwen2.5-32B-Instruct, and Goedel-Prover-V2-32B Text 860,469 [Nemotron-Math-Proofs-v1] [Goedel-Prover-V2-32B]
Synthetic Art of Problem Solving and Stack Exchange from gpt-oss-120b Text 1,201,815 [Upstream released mathematics dataset]; [AoPS]; [StackOverflow / StackExchange] [gpt-oss-120b]
Synthetic Multilingual Science and Code information from DeepSeek-R1, DeepSeek-R1-0528, Qwen2.5-32B-Instruct, and Qwen3-235B-A22B, translated pinch Qwen2.5-32B-Instruct and Qwen2.5-14B-Instruct Text Undisclosed [Nano-V3 SFT Data (without instrumentality call)] [Qwen/Qwen2.5-14B-Instruct]; [Qwen/Qwen3-4B-Thinking-2507]
Synthetic Multilingual Science and Code information from DeepSeek-R1, DeepSeek-R1-0528, Qwen2.5-32B-Instruct, and Qwen3-235B-A22B, translated pinch Qwen2.5-32B-Instruct and Qwen2.5-14B-Instruct (Stack Exchange lineage) Text Undisclosed [Stack Exchange]; [SCP-116K]; [LIMO]; [TACO]; Code Contest; Codeforces [DeepSeek-R1]; [DeepSeek-R1-0528]; [Qwen2.5-32B-Instruct]; [Qwen3-235B-A22B]
Synthetic Search Graph Walk Text 6,977 [Wikidata / Wikipedia KnowledgeBase] [MiniMaxAI/MiniMax-M2]
Synthetic Agentic Diverse Domains Text 281,537 [Handwritten prompts (synthetic; nary outer seed information used)] [SDG model: deepseek-ai/DeepSeek-V3.2, deepseek-ai/DeepSeek-R1-0528, Qwen/Qwen3-235B-A22B-Thinking-2507, Qwen/Qwen3-32B]; [Filtering model: openai/gpt-oss-120b, Qwen/Qwen3-32B, Qwen/Qwen3-235B-A22B-Instruct-2507]
Synthetic Long Context from Qwen3-235B-A22B-Instruct-2507 Text Undisclosed [Long-context SFT seed blend (pre-training blend + nano-v1 post-training data)]; [Long-context SFT data: lc_nothink 256k, MRCR 200k, RULER 256k]; [AALCR seed blend: SEC Filings, CC, Wikipedia, FinePDFs, ArXiv, Pile-NIH ExPorter, BioRxiv, PMC Article, USPTO Backgrounds, peS2o, Global Regulations, CORE, Gutenberg (PG-19), DOAB CC-BY, NDLTD, Amps, StackExchange, MathPile, Numinas] [Qwen/Qwen3-235B-A22B-Thinking-2507]; [deepseek-ai/DeepSeek-R1]; [Qwen3-30B-A3B]
Synthetic Nemotron Math SFT from DeepSeek-V3.2-Speciale Text 1,900,553 [Nemotron-Math-v2 (AOPS and StackExchange-math problems)] [DeepSeek-V3.2-Speciale]
Synthetic Nemotron Math TIR from DeepSeek-V3.2 Text 1,789,258 [Nemotron-Math-v2 (AOPS and StackExchange-math problems)] [DeepSeek-V3.2]
Synthetic NemoCascade OCR Distillation from gpt-oss-120b Text 682,864 [Nemotron-X training datasets] [gpt-oss-120b]
Synthetic CUDA 100k Text 93,086 [KernelBook]; [HuggingFace Transformers]; [FlashInfer] [gpt-oss-120b]; [DeepSeek-R1-0528]
Synthetic Science MCQ and QA Diversity from GPT-OSS and Kimi-K2 Text 30,358 [doubtnut]; [Pile-FreeLaw]; [Llama Nemotron Dataset]; [askfilo]; [EssentialAI/essential-web-v1.0]; [Vedantu]; [auxiliary_train]; [cdquestions.com]; [AMC 8 Problems and Solutions, AMC 10 Problems and Solution, and AIME Problems and Solutions]; [AAPT]; [ICHO-IPH0 Dataset]; [LIMO dataset (Less is More for Reasoning)] [GPT-OSS]; [Kimi-K2]
Synthetic Science HLE pinch Python from GPT-OSS and Kimi-K2 Text 85,184 [doubtnut]; [Pile-FreeLaw]; [Llama Nemotron Dataset]; [askfilo]; [EssentialAI/essential-web-v1.0]; [Vedantu]; [auxiliary_train]; [cdquestions.com]; [AMC 8 Problems and Solutions, AMC 10 Problems and Solution, and AIME Problems and Solutions]; [AAPT]; [ICHO-IPH0 Dataset]; [LIMO dataset (Less is More for Reasoning)] [GPT-OSS]; [Kimi-K2]
Synthetic Science Search and Python from GPT-OSS and Kimi-K2 Text 6,179 [doubtnut]; [Pile-FreeLaw]; [Llama Nemotron Dataset]; [askfilo]; [EssentialAI/essential-web-v1.0]; [Vedantu]; [auxiliary_train]; [cdquestions.com]; [AMC 8 Problems and Solutions, AMC 10 Problems and Solution, and AIME Problems and Solutions]; [AAPT]; [ICHO-IPH0 Dataset]; [LIMO dataset (Less is More for Reasoning)] [GPT-OSS]; [Kimi-K2]
Synthetic Science Search from GPT-OSS and Kimi-K2 Text 32,554 [doubtnut]; [Pile-FreeLaw]; [Llama Nemotron Dataset]; [askfilo]; [EssentialAI/essential-web-v1.0]; [Vedantu]; [auxiliary_train]; [cdquestions.com]; [AMC 8 Problems and Solutions, AMC 10 Problems and Solution, and AIME Problems and Solutions]; [AAPT]; [ICHO-IPH0 Dataset]; [LIMO dataset (Less is More for Reasoning)] [GPT-OSS]; [Kimi-K2]
Synthetic Finance Reasoning from GPT-OSS-120B and Qwen3-235B-A22B-Instruct-2507 Text 326,700 [SEC filings] [GPT-OSS-120B, Qwen3-235B-A22B-Instruct-2507]
Synthetic Science Diversity MCQ from GPT-OSS and Kimi-K2 Text 532,942 [doubtnut]; [Pile-FreeLaw]; [Llama Nemotron Dataset]; [askfilo]; [EssentialAI/essential-web-v1.0]; [Vedantu]; [auxiliary_train]; [cdquestions.com]; [AMC 8 Problems and Solutions, AMC 10 Problems and Solution, and AIME Problems and Solutions]; [AAPT]; [ICHO-IPH0 Dataset]; [LIMO dataset (Less is More for Reasoning)] [GPT-OSS]; [Kimi-K2]
Synthetic Science Diversity OpenQ from GPT-OSS and Kimi-K2 Text 131,045 [doubtnut]; [Pile-FreeLaw]; [Llama Nemotron Dataset]; [askfilo]; [EssentialAI/essential-web-v1.0]; [Vedantu]; [auxiliary_train]; [cdquestions.com]; [AMC 8 Problems and Solutions, AMC 10 Problems and Solution, and AIME Problems and Solutions]; [AAPT]; [ICHO-IPH0 Dataset]; [LIMO dataset (Less is More for Reasoning)] [GPT-OSS]; [Kimi-K2]
Synthetic Science Reasoning No-Tool from GPT-OSS and Kimi-K2 Text 2,085,600 [doubtnut]; [Pile-FreeLaw]; [Llama Nemotron Dataset]; [askfilo]; [EssentialAI/essential-web-v1.0]; [Vedantu]; [auxiliary_train]; [cdquestions.com]; [AMC 8 Problems and Solutions, AMC 10 Problems and Solution, and AIME Problems and Solutions]; [AAPT]; [ICHO-IPH0 Dataset]; [LIMO dataset (Less is More for Reasoning)] [GPT-OSS]; [Kimi-K2]
Synthetic Text-To-SQL from gpt-oss-120b Text Undisclosed [In-house Text-to-SQL data] [gpt-oss-120b]
Synthetic Tool Call Schema for RL (extended) Text 707,967 [UltraTool]; [ToolEyes]; [AutoTools]; [API-Bank]; [Nemotron-Personas-USA]; [Salesforce xLAM function-calling]; [Glaive function-calling-v2]; [Agent-Ark/Toucan-1.5M] [DeepSeek-V3.2]; [GLM-4.6]; [gpt-oss-120b]; [Kimi-K2-Instruct]
Synthetic Safety from gemma-3-4b-it, Nemotron-Nano-9B-v2, and gpt-oss-120b Text 44,091 [Safety SFT Data] [google/gemma-3-4b-it]; [Nemotron-Nano-9B-v2]; [gpt-oss-120b]
Synthetic Safety from DeepSeek-R1-0528, gpt-oss-120b, DeepSeek-R1-Distill-Qwen-7B, and Mixtral-8x7B-v0.1 Text Undisclosed [Nemotron Content Safety Dataset V2]; [Gretel Synthetic Safety Alignment Dataset]; [RedTeam-2K]; [Malicious Tasks]; [Nemotron-Personas-USA] [DeepSeek-R1-0528]; [gpt-oss-120b]; [DeepSeek-R1-Distill-Qwen-7B]; [Qwen3-30B-A3B-Thinking-2507]; [Qwen3-235B-A22B-Instruct-2507]; [Mixtral-8x7B-v0.1]
Synthetic Tool Calling from Qwen3-235B-A22B-Thinking-2507 and Qwen3-Next-80B-A3B-Thinking Text Undisclosed [ToolBench]; [glaive-function-calling-v2]; [APIGen Function-Calling]; [Nemotron-Personas-USA] [Qwen3-235B-A22B-Thinking-2507]; [Qwen3-Next-80B-A3B-Thinking]
Synthetic Chat from gpt-oss-120b, Mixtral-8x22B-Instruct-v0.1, Qwen3-235B-A22B-Instruct-2507, and Qwen3-235B-A22B-Thinking-2507 Text Undisclosed [C4]; [LMSYS-Chat-1M]; [ShareGPT]; [GSM8K]; [PRM800K]; [FinQA]; [WikiTableQuestions]; [Riddles]; [glaive-function-calling-v2]; [SciBench]; [tigerbot-kaggle-leetcodesolutions-en-2k]; [OpenBookQA]; [Advanced Reasoning Benchmark]; Software Heritage; [Khan Academy Math Keywords]; [WildChat-1M]; [Nemotron-Personas-USA] [gpt-oss-120b]; [Mixtral-8x22B-Instruct-v0.1]; [Qwen3-235B-A22B-Instruct-2507]; [Qwen3-235B-A22B-Thinking-2507]
Synthetic Tool Use Interactive Agent from gpt-oss-120b, DeepSeek-R1-0528, Qwen3-32B, and Qwen3-235B-A22B-Thinking-2507 Text Undisclosed NVIDIA Internal [gpt-oss-120b]; [DeepSeek-R1-0528]; [Qwen3-32B]; [Qwen3-235B-A22B-Thinking-2507]
Synthetic DocFinQA and SWE-smith from Qwen3-Coder-480B-A35B-Instruct and Kimi-K2-Thinking Text Undisclosed [DocFinQA]; [SWE-smith] [Qwen3-Coder-480B-A35B-Instruct]; [Kimi-K2-Thinking]
Synthetic SWE-Gym from Qwen3-Coder-480B-A35B-Instruct Text Undisclosed [SWE-Gym] [Qwen3-Coder-480B-A35B-Instruct]
Synthetic SWE-Gym and R2E-Gym-Subset from Qwen3-Coder-480B-A35B-Instruct Text Undisclosed [SWE-Gym]; [R2E-Gym-Subset] [Qwen3-Coder-480B-A35B-Instruct]
Synthetic SWE-Gym and R2E-Gym-Subset from DeepSeek-R1-0528 Text Undisclosed [SWE-Gym]; [R2E-Gym-Subset] [DeepSeek-R1-0528]
Synthetic HelpSteer, LMSYS-Chat-1M, and Nemotron-Personas-USA from gpt-oss-120b, Qwen3-235B-A22B-Instruct-2507, and Qwen3-235B-A22B-Thinking-2507 Text Undisclosed [HelpSteer2]; [HelpSteer3]; [LMSYS-Chat-1M]; [Nemotron-Personas-USA] [gpt-oss-120b]; [Qwen3-235B-A22B-Instruct-2507]; [Qwen3-235B-A22B-Thinking-2507]
Synthetic Nemotron-Personas-USA from gpt-oss-120b and Qwen3-8B Text Undisclosed [Nemotron-Personas-USA] [gpt-oss-120b]; [Qwen3-8B]
Vendor Terminal Bench-like Tasks (Droid) Text Undisclosed [Droid Harness Pivot vendor data] Undisclosed

Language Distribution successful Post-Training

For our post-training recipe, we focused connected the pursuing languages successful summation to English: French, German, Italian, Japanese, Spanish, and Chinese. Those languages were represented successful the shape of multilingual reasoning and translator tasks.

Testing Datasets:

Data Collection Method by dataset

  • Hybrid: Automated, Manually-Collected, Synthetic Labeling Method by dataset
  • Hybrid: Automated, Manually-Labeled, Synthetic Properties: This corpus comprises a operation of high-quality modular benchmarks and trial suites for modern agentic AI. These benchmarks trial exemplary capabilities connected tasks specified arsenic tool-calling and instruction following.

Evaluation Datasets:

Data Collection Method by dataset

  • Hybrid: Automated, Manually-Collected, Synthetic Labeling Method by dataset
  • Hybrid: Automated, Manually-Labeled, Synthetic Properties: This corpus comprises a operation of high-quality modular benchmarks and trial suites for modern agentic AI. These benchmarks trial exemplary capabilities connected tasks specified arsenic tool-calling and instruction following.

Inference

  • Acceleration Engine: PyTorch
  • Test Hardware:
    • NVIDIA Hopper
      • 1-8x H100
      • 1-8x H200
    • NVIDIA Blackwell
      • GB200
      • DGX Spark (GB10)
      • GeForce RTX 5090

Ethical Considerations

NVIDIA believes Trustworthy AI is simply a shared work and we person established policies and practices to alteration improvement for a wide array of AI applications. Developers should activity pinch their soul exemplary squad to guarantee this exemplary meets requirements for the applicable manufacture and usage lawsuit and addresses unforeseen merchandise misuse.

We counsel against circumvention of immoderate provided information guardrails contained successful the Model without a substantially akin guardrail due for your usage case. For much details: Safety and Explainability Subcards.

For much elaborate accusation connected ethical considerations for this model, please spot the Model Card++ Bias, and Privacy Subcards.

Please study exemplary quality, risk, information vulnerabilities aliases NVIDIA AI Concerns here.

Other Article Hacker News
↑
Close Right Ads
Close Left Ads