Thermoelectric effect is the most efficient way to convert electric energy directly from the temperature gradient. Thermoelectric effect-based power generation, cooling and heating devices are solid-stated, environmentally friendly, reliable, long-lived, easily maintainable, and easy to achieve miniaturization and integration. So they have unparalleled advantages in the aerospace, vehicle industry, waste heat recovery, electronic cooling, etc. This paper reviews the progress in thermodynamic analyses and optimizations for single- and multiple-element, single- and multiple-stage, and combined thermoelectric generators, thermoelectric refrigerators and thermoelectric heat pumps, especially in the aspects of non-equilibrium thermodynamics and finite time thermodynamics. It also discusses the developing trends of thermoelectric devices, such as the heat sources of thermoelectric generators, multi-stage thermoelectric devices, combined thermoelectric devices, and heat transfer enhancement of thermoelectric devices.