AI Speeds Up Real-Time Data Processing on Devices

Real-time artificial intelligence on devices like drones and smartphones is crucial for applications from autonomous driving to health monitoring, but it often struggles with delays that can compromise safety and responsiveness. A new system called MMEdge, developed by researchers at the Hong Kong University of Science and Technology, tackles this by rethinking how AI processes data from multiple sensors simultaneously. This innovation allows devices to analyze incoming information incrementally, rather than waiting for all data to arrive, significantly cutting down latency without sacrificing accuracy.

The key finding from the study is that MMEdge reduces end-to-end latency by up to 75.83% compared to traditional methods, while maintaining performance close to baseline systems. For example, in tests on an unmanned aerial vehicle (UAV) tracking humans, MMEdge achieved a mean Intersection over Union (IoU) score of 70.47%, nearly matching the accuracy of slower, blocking inference approaches. This means devices can react faster in dynamic environments, such as avoiding obstacles in real-time, which is vital for time-sensitive tasks.

Methodologically, MMEdge decomposes the data processing pipeline into fine-grained units, where each unit corresponds to the smallest segment of data from sensors like cameras or microphones. Instead of processing a full window of data at once, the system begins encoding each unit immediately upon arrival. This pipelined approach allows computation to overlap with data acquisition, eliminating idle periods. Additionally, the system incorporates an adaptive configuration module that selects optimal sensor and model settings based on runtime conditions, and a cross-modal speculative skipping mechanism that bypasses slower modalities when faster ones provide confident predictions.

Results from evaluations on public datasets and a real-world UAV testbed show that MMEdge consistently outperforms baselines in balancing latency and accuracy. On the Lip Reading in the Wild dataset, it reduced latency to around 137 milliseconds with less than a 3% drop in accuracy compared to the highest-performing baseline. The system's robustness was tested under varying resource constraints, such as limited CPU availability, where it maintained stable performance by dynamically adjusting configurations. For instance, even with CPU capped at 50%, MMEdge adapted effectively, ensuring reliable inference without significant degradation.

In practical terms, this matters because it enables more efficient on-device AI for everyday technologies, enhancing privacy by reducing reliance on cloud processing and improving responsiveness in applications like interactive assistants or emergency response systems. The approach addresses inherent dependencies between sensing and inference stages, which previously caused bottlenecks in multimodal systems.

Limitations noted in the paper include potential performance drops in highly variable environments and the need for further generalization to unseen data distributions. The learning-based components, such as the accuracy predictor, may require calibration to maintain effectiveness across different scenarios. Future work could explore transfer learning to adapt the system to new domains with minimal supervision, ensuring broader applicability.