Two-dimensional chalcogenide-based materials of group 14 elements are predicted as potential thermoelectric (TE) materials, though the figure of merit (ZT) obtained requires improvement to be commercially accessible. Herein, we have computationally modeled synthesized gamma-GeSe and reduced dimension 2D layers (monolayer, bilayer, trilayer, and quad layer) and subjected them to first-principles calculations to extract essential properties pertaining to TE. The ZT values obtained for the considered systems are found to be remarkably high (quad layer: 2.8; trilayer: 3.1; bilayer: 3.8), even at a high temperature of 900 K. The dimensionality reduction (3D to 2D) as well as reducing layers (quad-layer to bilayer) improved the ZT considerably in comparison to that of bulk gamma-GeSe (0.8 at 900 K). Even though the power factor decreases with decreasing layers, ultralow lattice thermal conductivities (KL) are responsible for the high ZT. Ultralow KL (> 1 W m-1 K-1) was observed in 2D gamma-GeSe at all temperature ranges, with the lowest KL observed in the bilayer (0.15 W m-1 K-1) and trilayer (0.17 W m-1 K-1) at 900 K. The low KL is also supported by the presence of high anharmonicity, high phonon scattering rates, low elastic constants, low group velocity, and low Debye temperature. We envisage that these findings will motivate investigations on similar low-dimensional materials for improved thermoelectric performance.