We propose an architecture of Intrusion Detection System (IDS) for VoIP using a protocol specification-based detection method to monitor the network traffics and alert administrator for further analysis of and response to suspicious activities. The protocol behaviors and their interactions are described by state machines. Traffic that behaves differently from the standard specifications are considered to be suspicious. The IDS has been implemented and simulated using OPNET Modeler, and verified to detect attacks. It was found that our system can detect typical attacks within a reasonable amount of delay time.