An iterative synthetic data pipeline has produced 502,536 unique financial news headlines across 13 categories-including rare events like credit-rating changes and product approvals that real-world feeds underrepresent. The dataset, generated over 82 iterations on a single 8-way NVIDIA B200 node, gives trading researchers and risk modelers a labeled corpus that would be difficult to assemble from live news sources alone.
A naive single-pass run of 50,000 headlines lost 65% to near-duplicate removal. The team behind the pipeline instead built a closed loop: generate a batch, filter malformed outputs, deduplicate against the full accumulated corpus, select diverse few-shot examples for the next round, and correct category weights before repeating. After roughly six days of compute with checkpointing and crash recovery, the final corpus crossed the 500,000 mark with only 2,536 excess headlines trimmed from the overrepresented "Other" category.
The pipeline at a glance
The workflow combines three NVIDIA components. NeMo Data Designer orchestrates structured generation through a declarative configuration that samples categories by weight and conditions a language model on few-shot examples. NeMo Curator handles semantic deduplication by embedding headlines, clustering them with K-means, and comparing pairwise cosine similarity within clusters at a 90% threshold. The Nemotron-3-Nano-30B-A3B model-a mixture-of-experts architecture with 3B active parameters per forward pass-runs on vLLM with 4-way tensor parallelism and 448 concurrent requests.
Category weights start from a target distribution but drift quickly because the model favors easier classes like "Stock Movement" and "Other." After each iteration, the pipeline compares actual proportions against targets, boosts underrepresented categories, clamps extremes, and normalizes weights. By iteration 82, rare categories such as "Credit Ratings" and "Product Approval" closely matched their 1% targets.
Why single-pass generation falls short
Financial news feeds skew heavily toward earnings reports and stock movements. Rarer events-credit-rating changes, regulatory product approvals, labor disputes-appear too infrequently to train reliable classifiers. A single large generation run does not solve this because language models converge on common patterns. In the baseline test, semantic deduplication retained only 17,348 unique headlines from 50,000 generated, confirming that scaling a single batch mostly adds near-duplicates rather than broader coverage.
The pipeline's key design choice is global deduplication. Each new batch is compared against all previously retained headlines, not just itself. This prevents cross-batch duplicates that would otherwise accumulate as the corpus grows. At 500 clusters with K-means, pairwise comparisons stay near 500 million-tractable even at the full 500K scale. Using fewer clusters would push comparisons toward 19 billion.
Few-shot selection as the steering mechanism
After each deduplication pass, the pipeline selects three few-shot examples per category for the next iteration. The selection uses a farthest-from-centroid strategy: headlines that sit at the edge of their K-means cluster are more likely to steer the model toward novel outputs. A secondary semantic filter rejects any candidate with 80% or higher cosine similarity to any previously used example, preventing the same prompt signals from repeating across iterations.
This approach shows measurable drift over the run. Early examples read like standard wire copy-"Google Parent Alphabet Posts $1.84 EPS, $63.5B Revenue"-while later iterations produce more specific, niche headlines: "ThermoGlow Posts Q2 Beat, Lifts FY Guidance on Record Orders for Ultra-Low-VOC Automotive Adhesive." The evolving prompts push generation into frontier content even as the accumulated corpus makes novelty harder to achieve.
Deduplication yield fell from roughly 35% in early iterations to about 15% by iteration 82, with each 35,000-headline batch contributing roughly 5,000 to 6,000 net-new entries. The steady nonzero yield near the 500K endpoint confirms that updated few-shot examples continued to find unexplored semantic territory.
From synthetic headlines to distilled models
The resulting dataset-FinHeadlineMix-directly supports model distillation workflows where a 49B-70B parameter teacher model labels the headlines and smaller student models are fine-tuned with LoRA. In tests, a 3B-parameter student reached 95% of the teacher's F1-score on financial headline classification using only 25,000 labeled examples. Semantic uniqueness and category balance expose the student to edge cases that real-world sources rarely provide at scale, which matters for professionals building classifiers that must handle infrequent but high-stakes event types.
Professionals working with Generative AI and LLM techniques in finance will recognize the broader pattern: synthetic data pipelines need closed-loop quality control, not just larger generation runs. The same principles-iterate rather than batch, deduplicate globally, steer with diverse few-shot examples, and correct distributions dynamically-apply to any domain where real-world data is imbalanced and semantic novelty is the goal.
Why this matters for finance professionals
Financial NLP projects often stall on data scarcity for the exact events that matter most to risk and compliance teams. A credit-rating downgrade, a product recall, or a regulatory approval can move markets, but these events appear too rarely in training corpora to build reliable detection systems. The pipeline described here offers a repeatable method for generating balanced, semantically diverse training data that covers the long tail of financial news. For quants and data scientists, the open-sourced dataset provides a starting point for headline classification, sentiment analysis, or fine-tuning smaller models that run on institutional infrastructure without the cost of large-model inference. For those exploring AI for Finance workflows, the distillation results suggest that compact models trained on high-quality synthetic data can approach teacher-level accuracy at a fraction of the compute cost.
Your membership also unlocks:
