In this paper an algorithm for numerical investigation of the transmission line having a generalized polygonal cross-section and open interface is proposed. Solution of the eigenmode problem is based on the method called the domain product technique, which employs a Mathieu function expansion and provides an efficient technique to the analysis of the structures with multiangular boundaries. An agreement at the obtained numerical results with existing data confirms the applicability of the theoretical analysis given in the paper.