In a display system with a line-of-gaze (LOG) controller, it is difficult to make the directions and motions of a LOG-controlled object coincide as closely as possible in the display with the user's intended LOG-directions and motions. This is because LOG behavior is not only smooth, but also saccadic due to the problem of involuntary eye movement. This article introduces a flexible on-line LOG-control scheme to realize nearly perfect LOG operation. Using a mesh-wise cursor pattern, the first visual experiment elucidates subjectively that a Kalman Filter (KF) for smoothing and predicting is effective in filtering out macro-saccadic changes of the LOG and in predicting sudden changes of the saccade while movement is in progress. It must be assumed that the LOG trajectory can be described by a linear position-velocity-acceleration approximation of Sklansky Model (SM). Furthermore, the second experiment uses a four-point pattern and simulations to scrutinize the two physical properties of velocity and direction-changes of the LOG in order to quantitatively and efficiently resolve "moving" and "gazing". In order to greatly reduce the number of LOG-small-position changes while gazing, the proposed Gaze-Holding algorithm (GH) with a gaze-potential function is combined with the KF. This algorithm allows the occurrence frequency of the micro-saccade to be reduced from approximately 25 Hz to 1 or 2 Hz. This great reduction in the frequency of the LOG-controlled object moves is necessary to achieve the user's desired LOG-response while gazing. Almost perfect LOG control is accomplished by the on-line SM+KF+GH scheme while either gazing or moving. A menu-selection task was conducted to verify the effectiveness of the proposed on-line LOG-control method.