We study the coefficients of a natural basis for the space of harmonic weak Maass forms of weight 5/2 on the full modular group. The non-holomorphic part of the first element of this infinite basis encodes the values of the partition function p(n). We show that the coefficients of these harmonic Maass forms are given by traces of singular invariants. These are values of non-holomorphic modular functions at CM points or their real quadratic analogues: cycle integrals of such functions along geodesics on the modular curve. The real quadratic case relates to recent work of Duke, Imamoglu, and Toth on cycle integrals of the j-function, while the imaginary quadratic case recovers the algebraic formula of Bruinier and Ono for the partition function.