Fast, Scalable, Model-free Trajectory Optimization for Wireless Data Ferries

Given multiple widespread stationary data sources such as ground-based sensors, an unmanned aircraft can fly over the sensors and gather the data via a wireless link. To minimize delays and system resources, the aircraft should collect the data at each sensor node via the shortest trajectory. Trajectory planning is hampered by the complex vehicle and communication dynamics and by uncertainty in the locations of sensors, so we develop a technique based on model-free learning. Previous work showed that model-free stochastic optimization can find good trajectories quickly enough for use in the field, but scaled poorly as the number of sensors increased, requiring roughly O(n) flights for n sensors. Here we modify the gradient computation, combining the global optimization criterion with multiple overlapping local ones, introduce data-mule-specific credit assignment, and use observed behavior to redistribute global rewards to local regions in the trajectory. This improves scalability of the initial trajectory learning phase nearly to O(1). We target a scenario in which sensors are known to lie somewhere near a known trajectory, for example after having been parachuted out of a deployment aircraft.