Threat-Aware UAV Dodging of Human-Thrown Projectiles with an RGB-D Camera
Abstract
Uncrewed aerial vehicles (UAVs) performing tasks such as transportation and aerial photography are vulnerable to intentional projectile attacks from humans. Dodging such a sudden and fast projectile poses a significant challenge for UAVs, requiring ultra-low latency responses and agile maneuvers. Drawing inspiration from baseball, in which pitchers’ body movements are analyzed to predict the ball’s trajectory, we propose a novel real-time dodging system that leverages an RGB-D camera. Our approach integrates human pose estimation with depth information to predict the attacker’s motion trajectory and the subsequent projectile trajectory. Additionally, we introduce an uncertainty-aware dodging strategy to enable the UAV to dodge incoming projectiles efficiently. Our perception system achieves high prediction accuracy and outperforms the baseline in effective distance and latency. The dodging strategy addresses temporal and spatial uncertainties to ensure UAV safety. Extensive real-world experiments demonstrate the framework’s reliable dodging capabilities against sudden attacks and its outstanding robustness across diverse scenarios.
Method Overview
Our framework consists of two tightly coupled modules: Pose-Aware Projectile Trajectory Prediction (PAPT) and Uncertainty-Aware Dodging (UAD). Instead of waiting until the projectile becomes clearly visible in flight, the UAV anticipates a potential attack directly from the thrower’s body motion using only an onboard RGB-D camera.
PAPT: Pose-Aware Projectile Trajectory Prediction
Synchronized RGB-D streams are first used to estimate 3D human keypoints in real time. The recent motion of the throwing joint is smoothed with a pre-release trajectory model, and multiple plausible release instants are detected from the joint’s velocity and acceleration dynamics. For each candidate release moment, the system predicts a post-release projectile trajectory using a physics-based motion model.
UAD: Uncertainty-Aware Dodging
Because both the release timing and the predicted trajectory are uncertain, each candidate projectile path is surrounded by an expanding ivory-shaped uncertainty region. A collision threat is identified when the UAV’s current position or planned trajectory intersects this region, enabling conservative but timely risk assessment.
Threat-Aware Motion Planning
The UAV trajectory is parameterized by MINCO and optimized in a GCOPTER-style framework with smoothness, feasibility, obstacle, and time objectives. On top of these standard terms, we introduce two dodging-oriented penalties: a surviving-trajectory penalty that pushes the UAV away from all still-threatening candidate projectile paths, and a relative-velocity penalty that encourages safer avoidance directions.
Long-Range, Low-Latency Onboard Response
By leveraging human pose instead of relying only on direct projectile appearance, the framework enables earlier threat anticipation, longer effective detection range, and real-time onboard dodging using only an RGB-D camera, without requiring expensive event cameras or heavy perception hardware.
In our experiments, the perception module achieves an effective detection distance of 6.0 m with an average latency of 26.4 ms on CPU-only hardware. For 3D keypoint tracking, the position RMSE is 0.037 m and the velocity RMSE is 0.154 m/s, supporting reliable early warning and real-time evasive motion in diverse real-world attack scenarios.