We have studied the thermal, magnetic, and electrical properties of the ternary intermetallic system CeNiGe3 by means of specific heat, magnetization, and resistivity measurements. The specific heat data, together with the anisotropic magnetic susceptibility, was analyzed on the basis of the point charge model of crystalline electric field. The J=5/2 multiplet of the Ce3+ is split by the crystalline electric field into three Kramers doublets, where the second and third doublets are separated from the first (ground state) doublet by Delta(1) similar to 100 K and Delta(2) similar to 170 K, respectively. In zero field CeNiGe3 exhibits an antiferromangeic order below T-N = 5.0 K. For H parallel to a two metamagnetic transitions are clearly evidenced between 2-4 K from the magnetization isotherm and extended down to 0.4 K from the magnetoresistance measurements. For H parallel to a, T-N shifts to lower temperature as magnetic field increases, and ultimately disappears at H-c similar to 32.5 kOe. For H > H-c, the electrical resistivity shows the quadratic temperature dependence (Delta p=AT(2)). For H >> H-c, an unconventional T-n dependence of Delta p with n > 2 emerges, the exponent n becomes larger as magnetic field increases. Although the antiferromagnetic phase transition temperature in CeNiGe3 can be continuously suppressed to zero, it provides an example of field tuning that does not match current simple models of quantum criticality.