We propose a speech recognition method using fusion of auditory and visual information for accurate speech recognition. Since we use both auditory information and visual information, we can perform speech recognition more accurately in comparison with the case of either auditory information or visual information. After processing each information by HMM, they are fused by linear combination with weight coefficient. We performed experiments and confirmed the validity of the proposed method.