ByteDance Ltd. is pretraining a new artificial intelligence model with approximately 10 trillion parameters, according to a Financial Times report citing multiple sources.
The scale places the company ahead of currently disclosed Chinese models and directly challenges the parameter counts of top U.S. systems like Anthropic's Mitos series.
Training a base model at this size typically requires three to six months before fine-tuning and release.
Scaling past open-weight strategies
The 10 trillion parameter architecture represents a deliberate move away from the open-weight approach favored by most Chinese tech firms. ByteDance has instead focused on closed models backed by aggressive infrastructure spending. The company recently instructed engineers to stop using distillation techniques that reference external American models. This directive aims to build capabilities independently, though some analysts note the policy has slowed development velocity compared to competitors who adopted faster iterative methods.
Leadership and organizational structure
Development falls under Seed, a 2,000-person research division led by Wu Yonghui, formerly of Google DeepMind. The push reflects a calculated long-term bet rather than a sprint to match quarterly releases. Founder Zhang Yiming told staff, according to The Information, that they should "not worry about falling behind in the short term and set a goal of securing 'world-class model capabilities' in the long term." Current products already show the payoff of this infrastructure buildout. Doubao serves 324 million monthly users in China, while the S-Dance video generation system ranks among the global leaders.
Teams managing Generative AI and LLM workloads will need to adjust compute allocation when shifting from lightweight fine-tuning to massive base training runs. Engineers overseeing distributed systems should also consult our AI for IT & Development materials to understand the storage bandwidth requirements for checkpointing trillion-parameter weights.
Why this matters for IT and Development professionals
Trillion-parameter pretraining exposes bottlenecks in data pipeline throughput, checkpointing frequency, and distributed gradient synchronization. Teams that manage GPU clusters will need to adjust capacity planning around multi-month base training cycles rather than relying on shorter fine-tuning jobs. Monitoring frameworks must track memory fragmentation across thousands of nodes, and data engineers should prepare schema validation pipelines that can handle the volume required to avoid catastrophic forgetting during early training stages.
Your membership also unlocks:
![[Wafer: response was truncated before the model finished its internal reasoning. Increase max_tokens, or disable thinking on this model (e.g. chat_template_kwargs.enable_thinking=false), then retry.]](https://cdn.completeaitraining.com/news_images/wafer-response-was-truncated-before-the-model-finished-its-internal-reasoning-in_2026-08-09.jpg)