Terahertz (THz) ultra-massive multiple-input multiple-output (UM-MIMO) is envisioned as a key technology to support ever-increasing data rates in 6G communication systems. To make the most of THz UM-MIMO systems, acquisition of accurate channel information is crucial. However, the THz channel acquisition is not easy due to the humongous pilot overhead that scales linearly with the number of antennas. In this article, we propose a novel deep learning (DL)-based channel acquisition technique called <italic>Transformer-based parametric THz channel acquisition</italic> (T-PCA) for the THz UM-MIMO systems. By learning the complicated mapping function between the received pilot signal and the sparse channel parameters (e.g., angles, distances, path gains) using Transformer, T-PCA can make a fast yet accurate channel estimation with a relatively small amount of pilot resources. Moreover, using the attention mechanism of Transformer, we can promote the correlation structure of the received pilot signals in the feature extraction, thereby improving the channel parameter estimation quality significantly. From the simulation results, we demonstrate that T-PCA is effective in acquiring the THz channel information and reducing the pilot overhead.