AI Learns to Fly Through Crowds Without Seeing People

A new autonomous flight system enables drones to navigate through dense, dynamic environments without needing to identify or track individual moving objects. Developed by researchers at the University of Hong Kong, this approach combines LiDAR sensing with reinforcement learning to create a lightweight, end-to-end solution that maps raw point clouds directly to motion commands. The system demonstrates a superior success rate in simulations, maintaining 40% reliability in highly cluttered scenarios with both static and dynamic obstacles, outperforming existing s that often fall below 30%. This breakthrough addresses a critical in robotics: how to safely traverse unpredictable settings like crowded streets or warehouses without heavy computational overhead.

The key finding is that drones can learn to avoid dynamic threats by sensing environmental changes through a novel representation called 'point flow,' rather than relying on object detection and tracking. The researchers encoded LiDAR data into a low-resolution distance map that preserves obstacle details for safety, while extracting motion features from multi-frame observations using a pre-trained neural optical flow estimator. This combined input allows the policy to implicitly drive evasive maneuvers based on relative motion, as shown in Figure 4, where the distance field is reshaped to account for extra threat regions from moving obstacles. In simulations, the system achieved a 95% success rate in environments with 7 dynamic and 7 static obstacles, dropping to 40% in more complex settings with 25 dynamic and 19 static obstacles, but still outperforming benchmarks like FAPP, NavRL, and Obsnet.

Ology centers on a reinforcement learning framework that trains a neural controller using proximal policy optimization. Observations include the distance map and point flow concatenated into a tensor, along with state inputs like goal direction, velocity, and last action. The reward function encourages safe, goal-oriented flight with components for state limits, goal alignment, safety distance, and dynamic obstacle avoidance, where the latter reshapes the distance field based on obstacle relative motion. Training occurred in simulated environments with static columns and walls, and dynamic obstacles moving in uniform linear motions at speeds up to 5 m/s, as depicted in Figure 5. The system outputs acceleration commands at the kinematic level, enabling deployment on physical quadrotors with different dynamics, and inference runs at 50 Hz on an Nvidia Jetson Orin NX.

From extensive simulations and real-world experiments validate the system's effectiveness. In benchmark tests, the proposed showed higher success rates and safety distances compared to alternatives, with an average flight speed of 9.58 m/s in less cluttered scenes and planning latency of just 3.14 ms. Ablation studies in Table II reveal that removing point flow or the dynamic obstacle avoidance reward reduces success rates significantly, especially at higher obstacle speeds, with metrics like αc dropping to 0.58 without flow in high-speed scenarios. Real-world deployments, illustrated in Figure 8, demonstrated the drone navigating through box environments, interacting with pedestrians, and reacting to fast-moving thrown objects, achieving speeds up to 14.57 km/h while maintaining safe distances.

Are substantial for applications in delivery, surveillance, and search-and-rescue, where drones must operate autonomously in unpredictable settings. By eliminating the need for object detection and tracking modules, the system reduces computational burden and latency, making it more adaptable to real-time changes. The researchers note that the policy generalizes well to unseen motion patterns, such as chaotic pedestrian trajectories, though limitations exist, including difficulties with obstacles emerging from occlusions or abrupt motion changes. Future work will focus on incorporating risk awareness and diversifying training conditions to handle out-of-view threats, potentially expanding the system's robustness for broader use in natural clutters and urban environments.