An analysis of attention-based multi-task learning for end-to-end autonomous driving

Abstract

The escalating adoption of autonomous vehicles (AVs) underscores the imperative of ensuring their safety. To mitigate issues like training computational overhead, error propagation, and generalization, multi-task end-to-end learning models have

been proposed. These models leverage Multi-task Learning (MTL) to enhance interpretability and feature extraction by concurrently training on multiple tasks. This

approach integrates primary vehicle control with auxiliary functions, such as object detection, aiming to improve upon single-task models in efficiency and effectiveness. However, the advantages of MTL beyond generalization remain underexplored. This investigation contrasts MTL with traditional end-to-end models, discovering

that while both MTL and baseline models adeptly learn steering control, acquiring mastery over throttle and brake control proves more complex. The comparative

analysis indicates that MTL models do not markedly outperform their baseline counterparts, highlighting that the advantages of MTL may hinge on the distinct characteristics of the training data and the complexity of the tasks involved. Through the

deployment of saliency maps to decode model attention, it is revealed that MTL and single-task end-to-end models prioritize different image features for vehicle control. This differentiation suggests that leveraging such insights could guide the selection of auxiliary tasks in future research, potentially enhancing model performance and interpretability.