Self-phase modulation has been observed for ultrashort pulses of wavelength 800 nm propagating through a 1 cm-long Ta2O5 rib waveguide. The associated nonlinear refractive index n(2) was estimated to be 7.23 x 10(-19) m(2)/W, which is higher than silica glass by more than one order of magnitude. Femtosecond time of flight measurements based on a Kerr shutter configuration show that the group velocity dispersion is small at a wavelength of 800 nm, confirming that dispersion may be neglected in the estimation of n(2) so that a simplified theory can be used with good accuracy. (C) 2004 Optical Society of America.