We are constructing and studying a unicycle robot which balances and steers itself by swinging its pendulum sidewards. Our unicycle robot, automatic lateral-pendulum unicycle (ALP Cycle), has three main parts: wheel, chassis and pendulum. It is an underactuated robot with only two control inputs and five degrees of freedom. The wheel motor provides balancing torque in forward-backward direction while the pendulum motor provides balancing torque in the left-right direction. For balancing the ALP Cycle either from a non-upright static initial posture or from external disturbances, both the wheel and pendulum motors need to provide the necessary recovery torques to counter the falling motion of ALP Cycle's bodies in two directions. In this paper, we discuss qualitatively the issues of balancing of ALP Cycle including the different types of static initial postures which pose different difficulty levels. Next, three linear controllers are designed for longitudinal and lateral modes for balancing ALP Cycle based on the idea of separate regulations. The performances of the controllers for different initial postures are evaluated by numerical simulation in MATLAB.