Researchers led by the University of Hong Kong have released RoboDojo, a benchmark that evaluates physical AI systems across both simulated and real-world robot tasks, and found that even the strongest models remain far behind human performance. The platform is designed to give researchers a standardized way to compare robot-learning systems that currently rely on different hardware, environments, and scoring methods.
RoboDojo was developed by the Multimedia Laboratory at HKU with researchers from nearly 20 universities, including the University of California, Berkeley, and Tsinghua University. The project was co-initiated by Ping Luo, associate director of AI research and tech transfer at the HKU School of Computing and Data Science, and doctoral student Tianxing Chen.
What RoboDojo tests
RoboDojo combines simulation testing, physical-robot testing, and evaluation of robot control policies in a single framework. The platform includes 30 representative robot policies, 42 simulation tasks, and 18 real-world tasks.
The tests examine capabilities considered important for physical AI, including generalization to new situations, memory, precision, and multi-step execution over longer periods.
HKU said systems are often evaluated under incompatible conditions, making it difficult to determine whether results carry over to real-world settings. RoboDojo is intended to provide a reproducible way to compare approaches in both virtual and physical environments rather than relying on isolated demonstrations.
"To the best of our knowledge, RoboDojo is the first Hong Kong-led benchmark to unify simulation and standardised real-robot evaluation," said Luo. "It moves embodied AI beyond impressive demonstrations towards progress that can be measured, compared and trusted."
Human performance gap
Early results show a large gap between AI systems and humans. The top-performing AI model achieved a success rate of 8.8% on simulated tasks and 12.8% during real-world robot testing. Human experts recorded success rates of 76.03% in simulation and 100% on physical tasks.
The results highlight how a system that completes a task under carefully selected conditions may struggle when environments, objects, or sequences of actions change. Standardized benchmarks can make those weaknesses easier to identify by giving researchers a common set of tasks and measurements.
For developers working in AI for IT & Development, the benchmark offers a concrete way to evaluate how well robot-learning models generalize beyond narrow test conditions. The gap between simulation and real-world performance also underscores why evaluation methods matter when comparing Generative AI and LLM systems.
The researchers have made RoboDojo available to the broader robotics community. According to HKU, the open-source resources have been downloaded more than 100,000 times from Hugging Face since launch, and posts about the project received more than 100,000 views on X during its first week.
Why this matters for IT and development professionals
RoboDojo gives developers a common yardstick for measuring robot-learning systems across simulation and physical deployment. For teams building or evaluating physical AI, the benchmark's standardized tasks and open-source resources make it possible to compare models on equal terms, rather than trusting results produced under incompatible testing conditions. The wide gap between AI and human performance also signals that claims about robot capabilities should be checked against standardized benchmarks before deployment decisions are made.
Your membership also unlocks:
