This paper describes a manipulation system composed of a multi-fingered hand and multi-sensors. The system is capable of dexterous manipulation by rolling contact between fingertips and a grasped object. First, a mathematical model of the kinematics of rolling contact for two surfaces of arbitrary shape rolling on each other is formulated. Then, a trajectory/force planning algorithm for the fingers is developed assuming that each fingertip is equipped with a three-axis force sensor, the geometric shape of the object and each fingertip are known, and the initial position and orientation of the object are known. Finally, actual implementation of the system are explained together with results of experiments verifying our approach.