The existing fault diagnosis models are limited to specific mechanical devices and specific signal types, hindering their use in industrial applications. This study aims to address this limitation by proposing a universal method compatible with different devices and multimodal sensing, while considering its suitability under different working conditions. First, a joint distribution adaptation method based on lightweight networks (JDALNs) is proposed to reduce data distribution differences between source and target domains and avoid pattern collapse problems. Second, a lightweight network block constructed by partial convolution (PConv) and pointwise convolution (PW) is proposed to enhance the feature extraction capability, and the classification model is designed based on this block and grouped convolution. Finally, experimental evaluations are conducted on current signals of industrial robots and vibration signals of bearings, demonstrating an extremely high level of model accuracy. Remarkably, the proposed model achieves the good performance while maintaining a compact parameter size and computational effort.