Making Edge AI work: why industry collaboration is key
Edge AI is ready for prime time. The blockers aren't ideas or demand-they're gaps in silicon, tools, and shared standards. Product teams feel it first: promising prototypes that don't scale, or viable products that never get built because the dev path is unclear.
The fix is collaboration across the stack. Co-opetition-competing for customers while aligning on the basics-will grow the market and reduce risk for everyone building real products.
The gap: mid-range TOPS is underserved
AI acceleration is often measured in TOPS. Today, you can buy low single-digit TOPS parts and high-teens (and beyond), but the middle is thin. That's exactly where many IoT vision workloads land.
This gap pushes teams to prototype on a platform that's too powerful and too expensive, then struggle to scale down. The opportunity is clear: efficient silicon in the mid-range that hits performance per watt and cost targets for edge devices.
Dev kits: overpowered prototypes = overpriced products
Dev kits for Edge AI are scarce. So teams default to what's available-even if it's oversized. Prototypes work, but the business case fails once you price the production unit.
You need dev kits that map directly to production: the same class of CPU/NPU/DSP, the same memory constraints, the same I/O. If your prototype can't be dropped into a bill of materials with minimal rework, you're wasting cycles.
- Prioritize right-sized kits over "safe" workstations.
- Insist on production-ready reference designs and BOM guidance.
- Validate energy per inference, not just FPS or raw TOPS.
- Test two silicon targets early to avoid a single-vendor dead end.
Software choices without lock-in
Linux or RTOS? C++ or Python? The wrong choice isn't about syntax-it's about portability and long-term agility. Your stack should make it easy to update models, ship variants, and move across silicon.
Favor open formats and tooling. Use model interchange standards and ensure your pipeline survives vendor changes.
- Standardize on portable model formats (e.g., ONNX or TFLite) and target multiple runtimes.
- For constrained devices, consider an open RTOS like Zephyr; for richer apps, lock in a repeatable Linux build system.
- Automate quantization, compilation, and deployment as CI/CD steps.
- Require clear migration paths for SDK versions and model updates.
Group problems, group solutions
The AI stack is already crowded: datasets, model hubs, compilers, toolchains, device lifecycle tools. Expecting each product team to stitch this together from scratch slows the market.
- Shared reference designs for common edge use cases (vision, audio, sensor fusion).
- Cross-vendor SDK compatibility and sample apps that run on more than one chip.
- Open, well-documented data pipelines and evaluation suites for accuracy, latency, and cost.
- A public library of production-grade MLOps patterns for over-the-air model updates at the edge.
The opportunity
Analysts estimate AI semiconductor revenue will reach around $159B by 2028, driven by AI moving from data centers into PCs, phones, and edge devices. The demand is there across consumer, industrial, healthcare, telecom, agriculture, and robotics.
The IoT market is ready to add AI/ML. The constraint is execution: fit-for-purpose platforms, accessible tools, and a shared open-source base.
Action plan for product development teams
- Define the workload: model size, latency targets, accuracy thresholds, and energy per inference. Set a TOPS budget based on reality, not marketing slides.
- Select two right-sized dev kits that bracket your target. Prototype on both to de-risk portability.
- Adopt a portable model workflow (e.g., PyTorch/ONNX or TensorFlow/TFLite) with automated quantization and hardware-specific compilation.
- Measure what matters: latency at target accuracy, energy per inference, memory footprint, and total BOM impact.
- Make lock-in visible: require vendor-agnostic runtimes, documented kernels, and a clear exit path in contracts.
- Plan lifecycle early: model retraining cadence, over-the-air updates, rollback, and device telemetry for model drift.
- Publish internal benchmarks and checklists so engineering, product, and procurement align on the same criteria.
What the ecosystem should align on
- A mid-range Edge AI silicon profile that balances cost, performance, and power.
- Dev kits that map 1:1 to production constraints, plus open reference designs.
- Open-source SDKs and model formats as the default, with clear migration guides.
- Standard evaluation metrics and test suites that any vendor can run and publish.
Bottom line
Edge AI doesn't need more promise. It needs fit-for-purpose silicon, production-ready tools, and a willingness to build common rails while competing on product. Co-opetition turns prototypes into shipments.
If your team needs a fast shared baseline on AI/ML skills by role, explore curated learning paths here: AI Courses by Job.
Your membership also unlocks:
