A numerical scheme is described for including radiation in multidimensional general relativistic conservative fluid dynamics codes. In this method, a covariant form of the M1 closure scheme is used to close the radiation moments, and the radiative source terms are treated semi-implicitly in order to handle both optically thin and optically thick regimes. The scheme has been implemented in a conservative general relativistic radiation hydrodynamics code KORAL. The robustness of the code is demonstrated on a number of test problems, including radiative relativistic shock tubes, static radiation pressure supported atmosphere, shadows, beams of light in curved space-time and radiative Bondi accretion. The advantages of M1 closure relative to other approaches such as the Eddington closure and flux-limited diffusion are discussed, and its limitations are also highlighted.