MIT researchers speed up federated learning by 81 percent
MIT researchers have developed a method that accelerates federated learning-a privacy-preserving approach to training AI models-by about 81 percent. The technique enables resource-constrained devices like smartwatches and wireless sensors to deploy accurate AI models while keeping user data secure.
Federated learning works by distributing model training across a network of connected devices. Each device trains the model using its local data, then sends updates back to a central server. The server averages these updates and repeats the process. Because data never leaves individual devices, the approach protects user privacy.
The problem: not all devices have the memory, processing power, or reliable connectivity needed to handle full models. Smartwatches, older phones, and sensors often struggle. When devices fall behind, the server waits for stragglers, creating delays that degrade training performance and drain battery life.
Three changes to reduce bottlenecks
MIT researchers created FTTE (Federated Tiny Training Engine) to address these constraints. The framework makes three key changes.
- Smaller model subsets: Instead of sending the entire model to every device, FTTE sends only a subset of parameters-the internal variables the model adjusts during training. A search procedure identifies which parameters matter most for accuracy within each device's memory budget.
- Asynchronous updates: The server no longer waits for all devices to respond. Instead, it accumulates incoming updates until reaching a fixed capacity, then proceeds with the next training round.
- Weighted updates: The server weights incoming updates based on when they arrive. Older updates contribute less to training, preventing stale data from slowing progress and reducing accuracy.
The semi-asynchronous approach keeps less powerful devices involved while preventing more capable devices from sitting idle, said Irene Tenison, the lead researcher.
Performance gains in testing
In simulations with hundreds of heterogeneous devices, FTTE reduced on-device memory overhead by 80 percent and communication payload by 69 percent. Training completed 81 percent faster than standard federated learning approaches, with only a small accuracy tradeoff.
The researchers also tested FTTE on real devices with varying computational capabilities. The method scaled effectively as network size increased.
"Not everyone has the latest phone," Tenison said. "In many developing countries, users have less powerful mobile devices. With this technique, we can bring federated learning to these settings."
Real-world applications
The approach opens federated learning to high-stakes applications like healthcare and finance, where privacy and security requirements are strict. Banks and hospitals can now train models on sensitive data without centralizing it on powerful servers.
Future work will focus on personalizing model performance for individual devices rather than optimizing average performance across the network, and conducting larger experiments on real hardware.
For IT professionals and developers, understanding efficient model training methods matters. As AI moves to edge devices in production environments, techniques like FTTE become practical necessities rather than research curiosities. Learn more about AI for IT & Development to stay current with deployment strategies.
Your membership also unlocks:
