Enterprises in retail, logistics and manufacturing are shifting from AI experimentation to scaled adoption, with on-device AI emerging as a practical way to cut latency, manage costs and strengthen data security, according to Stuart Hubbard, Senior Director of AI and Advanced Development at Zebra Technologies. Hubbard spoke about how small language models, AI-ready processors and real-time data capture are reshaping frontline operations.
On-device AI tackles security and cost challenges
May 2026 research by S&P found that the most common challenges with generative AI are data privacy (38%), security risks (38%) and costs (37%). "On-device AI goes some way towards overcoming these challenges," Hubbard said. Small language models (SLMs) running on devices are well-suited for high-volume tasks like translation, transcription and label scanning. SLMs can be highly tailored to specific jobs and sectors, so data is accessed quickly and intelligence is reliable and up to date.
On-device AI eliminates latency because no data travels to the cloud. That matters in environments where speed is critical, such as fast-paced retail aisles, warehouse picking zones and rural locations with limited connectivity. Edge AI scales across fleets of devices, cameras and sensors, turning existing IoT data capture tools into new sources of intelligence.
Financial and operational advantages
Financially, on-device AI saves on token API costs because data is processed directly on the device. "We've seen headlines about 'tokenmaxxing' as a proxy for adoption and ROI from AI, and the spiralling costs it has caused, which budget holders want to avoid," Hubbard said. Since no data leaves the device, concerns around end-to-end encryption, API security and identity management are reduced. For complex AI agent chains, edge servers or hybrid architectures can provide the security and compute needed.
Businesses need to invest in workforce devices that are AI-ready, with appropriate endpoint cybersecurity. "IT and operational leaders should be thinking about AI readiness, security, fleet management and total cost of ownership rather than cheap per device unit costs," Hubbard said. AI becomes a new type of infrastructure underpinning revenue-generating work on the frontline, improving worker autonomy and efficiency. Inventory management visibility means shrink can be detected and overstocking or understocking avoided.
Hardware advances enable real-time on-device AI
Today's mobile computers come with advanced system-on-chip designs that include a dedicated neural processing unit, GPUs and CPUs. Zebra's partnership with Qualcomm delivers AI-enabled processors like the Qualcomm Dragonwing Q-6690 (up to 2.9 GHz). This shift is part of a broader trend in AI for Operations, where on-device intelligence is becoming a practical tool.
AI models on-device draw on exceptional data capture sensors: microphones for natural language interfaces, cameras that prompt users to clean the lens and blur personal data, scanners, OCR, 3D time-of-flight sensors and RFID. "A living digital twin of the frontline is becoming a reality, what we call 'ambient intelligence' where agents act upon the data and within the digitised environment," Hubbard said.
Using existing device data for AI
Organisations need to invest in AI-ready hardware and work with providers on use cases and application development. Zebra offers AI blueprints-ready-made frameworks that modernise manual work-along with AI tools like trained vision models, sample applications and APIs. "It means buying and creating AI applications are both options," Hubbard said.
Businesses are finding new value from their current IoT, software stack and data capture technologies. Machine performance metrics, defects in goods, RFID tags, inventory lists, SOPs and worker interactions all become valuable sources of data for training, workflow improvement and predictive operations. Digitised physical operations and frontline IoT data create the environment for AI agents to act upon.
Cloud and edge will coexist
"Both are needed. On-device has definitely grown in importance, with 2026 seeing a range of product launches aimed at using on-device AI," Hubbard said. Private cloud, edge servers and hybrid architectures remain, giving organisations choices based on security requirements and compute complexity. Capital continues to flow into hyperscaler data centres, even as some discuss data centres in space.
Enterprise leaders should prioritise clear business cases with realistic timelines, metrics of success and culture change. A report from MIT's NADA found businesses that partner with AI providers had a success rate of 66% compared to those who relied on internal development alone (33%). Stanford's Digital Lab research concluded that the top factor for AI success (43%) was executive sponsorship and a culture of experimentation and permission to fail.
Why this matters for operations
For operations managers, the shift to on-device AI means faster, more secure tools that run on mobile devices in warehouses, store aisles and delivery routes. Cutting reliance on cloud connectivity and token costs directly impacts the bottom line, while improving inventory accuracy and worker productivity. The immediate priority is to evaluate AI-ready devices, strengthen endpoint security and define clear use cases. Pairing with experienced AI providers can accelerate adoption and reduce risk. Operations managers can explore an AI Learning Path for Operations Managers to deepen their understanding of these technologies.
Your membership also unlocks:
