Model-Driven Deep-Learning for End-to-End Optimization in Fiber-Terahertz Communication Systems

We propose a novel deep learning-based end-to-end (E2E) optimization framework for fiber-terahertz (THz) integrated communication system. Our framework facilitates the development of advanced transceiver, optimizing both the achievable information rate and power efficiency of the waveform for THz communication. The framework utilizes a model-driven approach and is constructed using a bitwise autoencoder (BAE) based on the structure of the single-carrier communication system (SC-BAE). It consists of artificial neural networks (ANNs) serving as transmitter (T-ANN), channel models, and receiver (R-ANN). The T-ANN incorporates conventional single-carrier transmitter functionalities, including bit-to-symbol mapping, geometric shaping (GS), pulse shaping (PS), and digital pre-distortion (DPD). This model-driven design preserves the explainable architecture and facilitates control over the peak-to-average power ratio (PAPR) and resistance to distortion-limited communication environments. We conduct simulation and experimental studies, analyzing the performance gain contributed by the trainable GS, PS, and DPD blocks. The results demonstrate that the learnable PS effectively combats linear frequency fading, while the nonlinear DPD block provides additional optimization freedom to meet both the PAPR constraint and the desired data rate simultaneously. Our deep learning-based E2E THz communication system achieves a data rate of 77 Gbit/s, a sensitivity gain of 3.5 dB, and a 12 Gbit/s improvement compared to the conventional single-carrier baseline.