A deep-learning, vision-based framework for testing a swarm algorithm using inexpensive mini drones

The ability to explore dangerous buildings or hostile landscapes using a swarm of inexpensive mini drones is relevant to many search and rescue or surveillance scenarios encountered by civilian first responders and military personnel. Swarms of mini drones, implementing various path planning algorithms, provide a unique solution in situations where there is the risk to human life or use of expensive Unmanned Aerial Vehicle technology would be cost-prohibitive or both. Although inexpensive, off-the-shelf drones contain stabilization circuitry and onboard cameras, they suffer restricted flying time and lack GPS systems. The limited capability of such drones has curtailed their use by researchers investigating practical search and genetic algorithms, and many researchers rely on simulation, rather than testing with actual drones. In this paper, we describe an ad hoc framework for testing swarm algorithms while taking the first step toward implementing swarm intelligence using low-cost, off-the-shelf drones and an inexpensive network router. We initially created a public dataset, MINIUAV, including images of Tello and TelloEdu mini-drones taken from our live drone video recordings and photos scraped from various internet resources. Using the images, we then trained a deep-learning-based YOLOv4-Tiny (You Only Look Once) object detector allowing us to implement a swarm intelligence rule where drones act collectively based on a swarm alignment rule. Our results show the object detector allows a drone to identify a neighboring drone with greater than 90% accuracy. Finally, the dataset used to train the object detector will be made available on request.