The core of the support vector machine (SVM) problem is a quadratic programming problem with a linear constraint and bounded variables. This problem can be transformed into the second order cone programming (SOCP) problems. An interior-point-method (IPM) can be designed for the SOCP problems in terms of storage requirements as well as computational complexity if the kernel matrix has low-rank. If the kernel matrix is not a low-rank matrix, it can be approximated by a low-rank positive semi-definite matrix, which in turn will be fed into the optimizer. In this paper we present two SOCP formulations for each SVM classification and regression problem. There are several search direction methods for implementing SOCPs. Our main goal is to find a better search direction for implementing the SOCP formulations of the SVM problems. Two popular search direction methods: HKM and AHO are tested analytically for the SVM problems, and efficiently implemented. The computational costs of each iteration of the HKM and AHO search direction methods are shown to be the same for the SVM problems. Thus, the training time depends on the number of IPM iterations. Our experimental results show that the HKM method converges faster than the AHO method. We also compare our results with the method proposed in Fine and Scheinberg (2001) that also exploits the low-rank of the kernel matrix, the state-of-the-art SVM optimization softwares SVMTorch and SVMlight. The proposed methods are also compared with Joachims 'Linear SVM' method on linear kernel.

The choice of kernel is an important issue in the support vector machine algorithm, and the performance of it largely depends on the kernel. Up to now, no general rule is available as to which kernel should be used. In this paper we investigate two kernels: Gaussian RBF kernel and polynomial kernel. So far Gaussian RBF kernel is the best choice for practical applications. This paper shows that the polynomial kernel in the normalized feature space behaves better or as good as Gaussian RBF kernel. The polynomial kernel in the normalized feature space is the best alternative to Gaussian RBF kernel.

Structural learning algorithms are obtained by adding a penalty criterion (usually comes from the network structure) to the conventional criterion of the sum of squared errors and applying the backpropagation (BP) algorithm. This problem can be viewed as a constrained minimization problem. In this paper, we apply the Lagrangian differential gradient method to the structural learning based on the backpropagation-like algorithm. Computational experiments for both artificial and real data show that the improvement of generalization performance and the network optimization are obtained applying the proposed method.