SMO算法，platt论文的原始算法及优化算法

 platt论文：[PDF] Sequential Minimal Optimization : A Fast Algorithm for Training Support Vector Machines | Semantic Scholar
算法优化：[PDF] Improvements to Platt's SMO Algorithm for SVM Classifier Design | Semantic Scholar
包罗个人platt版SMO代码实现、SMO 优化算法、libsvm：yanzhi0922/SVM: 2024.07.12 (github.com)
数据集获取：LIBSVM Data: Classification, Regression, and Multi-label (ntu.edu.tw)
platt原论文SMO伪代码：

1. target = desired output vector
2. point = training point matrix
3. procedure takeStep(i1,i2)
4.     if (i1 == i2) return 0
5.     alph1 = Lagrange multiplier for i1
6.     y1 = target[i1]
7.     E1 = SVM output on point[i1] – y1 (check in error cache)
8.     s = y1*y2
9.     Compute L, H via equations (13) and (14)
10.     if (L == H)
11.         return 0
12.     k11 = kernel(point[i1],point[i1])
13.     k12 = kernel(point[i1],point[i2])
14.     k22 = kernel(point[i2],point[i2])
15.     eta = k11+k22-2*k12
16.     if (eta > 0)
17.     {
18.         a2 = alph2 + y2*(E1-E2)/eta
19.         if (a2 < L) a2 = L
20.         else if (a2 > H) a2 = H
21.     }
22.     else
23.     {
24.         Lobj = objective function at a2=L
25.         Hobj = objective function at a2=H
26.         if (Lobj < Hobj-eps)
27.             a2 = L
28.         else if (Lobj > Hobj+eps)
29.             a2 = H
30.         else
31.             a2 = alph2
32.     }
33.     if (|a2-alph2| < eps*(a2+alph2+eps))
34.         return 0
35.     a1 = alph1+s*(alph2-a2)
36.     Update threshold to reflect change in Lagrange multipliers
37.     Update weight vector to reflect change in a1 & a2, if SVM is linear
38.     Update error cache using new Lagrange multipliers
39.     Store a1 in the alpha array
40.     Store a2 in the alpha array
41.     return 1
42. endprocedure
44. procedure examineExample(i2)
45.     y2 = target[i2]
46.     alph2 = Lagrange multiplier for i2
47.     E2 = SVM output on point[i2] – y2 (check in error cache)
48.     r2 = E2*y2
49.     if ((r2 < -tol && alph2 < C) || (r2 > tol && alph2 > 0))
50.     {
51.         if (number of non-zero & non-C alpha > 1)
52.         {
53.             i1 = result of second choice heuristic (section 2.2)
54.             if takeStep(i1,i2)
55.                 return 1
56.         }
57.         loop over all non-zero and non-C alpha, starting at a random point
58.         {
59.             i1 = identity of current alpha
60.             if takeStep(i1,i2)
61.                 return 1
62.         }
63.         loop over all possible i1, starting at a random point
64.         {
65.             i1 = loop variable
66.             if (takeStep(i1,i2)
67.                 return 1
68.         }
69.     }
70.     return 0
71. endprocedure
73. main routine:
74.     numChanged = 0;
75.     examineAll = 1;
76.     while (numChanged > 0 | examineAll)
77.     {
78.         numChanged = 0;
79.         if (examineAll)
80.             loop I over all training examples
81.                 numChanged += examineExample(I)
82.         else
83.             loop I over examples where alpha is not 0 & not C
84.                 numChanged += examineExample(I)
85.         if (examineAll == 1)
86.             examineAll = 0
87.         else if (numChanged == 0)
88.             examineAll = 1
89. }

复制代码

以上算法根本原理雷同，效果雷同，优化是时间复杂度上的优化 ，libsvm时间复杂度最优