A human visual attention analysis model for remote interaction interface of unmanned agricultural vehicles

When unmanned agricultural vehicles (UAVs) encounter unexpected failures or incidents during autonomous operations, a highly efficient remote human-machine interaction (HMI) interface is needed to assist in the manual joint control process. The highly complex cognitive mechanisms and visual characteristics of humans make it difficult to design efficient human-machine interfaces. Therefore, a human visual attention analysis model was proposed based on fully convolutional networks for the remote interaction interface of unmanned agricultural vehicles. First, visual attention experimental software combined with gaze-contingent real-time simulation methods was developed to build the dataset. Subsequently, the human visual attention analysis model (HVAAM) is constructed, and the corresponding training strategy is proposed, which is trained on the ILSVRC2012 standard and self-constructed datasets. The accuracy of the HVAAM analysis on the test dataset was validated. Ultimately, a human-machine interaction test of random failure warning for remote monitoring interface of unmanned plowing operation is performed with a self-developed unmanned electric tractor as the carrier. In the test dataset, the average analysis accuracy of HVAAM was 79.2 %, and the proposed training strategy led to a 20.7 % improvement in the model accuracy. In the unmanned electric tractor human-machine interaction test, HVAAM achieved 77.9 % accuracy in the visual attention analysis of the monitoring interface. This study can effectively facilitate the analysis, design, and optimization of remote monitoring interfaces for unmanned agricultural vehicles.