We present a framework to search for and track a target within an urban environment by fusing data from an Unmanned Aerial Vehicle and Unattended Ground Sensors. The target and UAV are restricted to a road network modeled as a directed graph with the ground sensors deployed along selected edges. The UAV is equipped with an onboard camera capable of detecting the target, and it is guided by an information theoretic planner that uses a particle filter estimate of the target state as its input. We introduce a method to process out-of sequence measurements that exploits the time-sparseness of the UGS readings to reduce the computational complexity. Finally, we present simulation results on real road networks that show the target tracking performance and the gains in computation time of our approach.

Target search with unconstrained mobile sensor

Our initial work focused on the use of a multirotor aircraft equipped with a gimbaled camera to search for a target within an environment with obstacles. We proposed an algorithm that seeks to reduce the entropy of the target’s probability density function over time, resulting in target acquisition and tracking. Our approach outperforms exhaustive, sequential searches.

At the heart of our approach is a cost function to be minimized containing two terms: The posterior conditional entropy of the target pdf and the distance that the sensor needs to move.

Incorporating graph-based maps and out-of-sequence measurements

Fixed-wing UAVs have longer flight times that make them better suited for surveillance applications. Their downside is a more limited set of motions and their inability to hover. We applied mutual information maximization to achieve target acquisition and tracking within a road network in which the UAV is constrained to follow the roads. We also consider an additional information source, Unattended Ground Sensors. These sensors can detect the presence of a target but are not connected to a central estimator, so their measurements must be retrieved by the UAV coming in close proximity to them. This gives rise to Out-of-Sequence Measurements that must be handled by the estimator. We developed a modified particle filter with a limited time history that efficiently processes the OOSM and enhances the estimation results. Our method locates and tracks a target vehicle even with the sensor coverage gaps arising from the UAV kinematic constraints. This is achieved by using the particle filter information in a path-planner that keeps the UAV re-visiting the target.

Soft information and human-machine interaction

The final stage of this project will consist of incorporating “soft” information from human observers into the target state estimation. The observers will be able to make imprecise reports of target sightings that will influence the robots pursuing the target. We are planning outdoor experiments to validate our approach.

Research products