使用LBP改进精确算法以检测面部表情

Question

我正在开发一种简单的算法来检测几种面部表情（快乐，悲伤，愤怒......）。我是以this paper为基础的。我正在预处理之前应用LBP统一运算符将标准化图像划分为6x6区域，如下例所示：

通过应用统一的LBP，为每个区域提取59个专长，所以最终我有2124个图像专长（6x6x59）。当我有大约700张图像训练模型时，我认为这是一个太多的壮举。我已经读过，要获得良好的预测并不好。我的问题是如何减少专长的维度或其他技术，以提高算法的精度。

Answer 1

一种简单的方法来减少特征维度 - 同时增加鲁棒性 - 将使用rotation-invariant uniform patterns。对于半径为且由像素形成的圆形邻域，纹理描述符通过10个特征表示每个区域。因此，维数从2124减少到6×6×10 = 360。

Answer 2

PCA可以帮助减少描述符的大小而不会丢失重要信息。只是谷歌“opencv pca示例”。

另一个有用的事情是为您的统一lbp功能添加旋转不变性。这将提高精度，并将描述符的大小从59减小到10。

static cv::Mat rotate_table = (cv::Mat_<uchar>(1, 256) <<
                               0, 1, 1, 3, 1, 5, 3, 7, 1, 9, 5, 11, 3, 13, 7, 15, 1, 17, 9, 19, 5, 21, 11, 23,
                               3, 25, 13, 27, 7, 29, 15, 31, 1, 33, 17, 35, 9, 37, 19, 39, 5, 41, 21, 43, 11,
                               45, 23, 47, 3, 49, 25, 51, 13, 53, 27, 55, 7, 57, 29, 59, 15, 61, 31, 63, 1,
                               65, 33, 67, 17, 69, 35, 71, 9, 73, 37, 75, 19, 77, 39, 79, 5, 81, 41, 83, 21,
                               85, 43, 87, 11, 89, 45, 91, 23, 93, 47, 95, 3, 97, 49, 99, 25, 101, 51, 103,
                               13, 105, 53, 107, 27, 109, 55, 111, 7, 113, 57, 115, 29, 117, 59, 119, 15, 121,
                               61, 123, 31, 125, 63, 127, 1, 3,  65, 7, 33, 97, 67, 15, 17, 49, 69, 113, 35,
                               99, 71, 31, 9, 25, 73, 57, 37, 101, 75, 121, 19, 51, 77, 115,  39, 103, 79, 63,
                               5, 13, 81, 29, 41, 105, 83, 61, 21, 53, 85, 117, 43, 107, 87, 125, 11, 27, 89,
                               59, 45, 109, 91, 123, 23, 55, 93, 119, 47, 111, 95, 127, 3, 7, 97, 15, 49, 113,
                               99, 31, 25, 57, 101, 121, 51, 115, 103, 63, 13, 29, 105, 61, 53, 117, 107, 125,
                               27,  59, 109, 123, 55, 119, 111, 127, 7, 15, 113, 31, 57, 121, 115, 63, 29, 61,
                               117, 125, 59, 123, 119, 127, 15, 31, 121, 63, 61, 125, 123, 127, 31, 63, 125,
                               127, 63, 127, 127, 255
                               );

// the well known original uniform2 pattern
static cv::Mat uniform_table = (cv::Mat_<uchar>(1, 256) <<
                          0,1,2,3,4,58,5,6,7,58,58,58,8,58,9,10,11,58,58,58,58,58,58,58,12,58,58,58,13,58,
                          14,15,16,58,58,58,58,58,58,58,58,58,58,58,58,58,58,58,17,58,58,58,58,58,58,58,18,
                          58,58,58,19,58,20,21,22,58,58,58,58,58,58,58,58,58,58,58,58,58,58,58,58,58,58,58,
                          58,58,58,58,58,58,58,58,58,58,58,58,23,58,58,58,58,58,58,58,58,58,58,58,58,58,
                          58,58,24,58,58,58,58,58,58,58,25,58,58,58,26,58,27,28,29,30,58,31,58,58,58,32,58,
                          58,58,58,58,58,58,33,58,58,58,58,58,58,58,58,58,58,58,58,58,58,58,34,58,58,58,58,
                          58,58,58,58,58,58,58,58,58,58,58,58,58,58,58,58,58,58,58,58,58,58,58,58,58,58,
                          58,35,36,37,58,38,58,58,58,39,58,58,58,58,58,58,58,40,58,58,58,58,58,58,58,58,58,
                          58,58,58,58,58,58,41,42,43,58,44,58,58,58,45,58,58,58,58,58,58,58,46,47,48,58,49,
                          58,58,58,50,51,52,58,53,54,55,56,57
                          );

static cv::Mat rotuni_table = (cv::Mat_<uchar>(1, 256) <<
                         0, 1, 1, 2, 1, 9, 2, 3, 1, 9, 9, 9, 2, 9, 3, 4, 1, 9, 9, 9, 9, 9, 9, 9, 2, 9, 9, 9,
                         3, 9, 4, 5, 1, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 2, 9, 9, 9, 9, 9, 9, 9,
                         3, 9, 9, 9, 4, 9, 5, 6, 1, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9,
                         9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 2, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9,
                         3, 9, 9, 9, 9, 9, 9, 9, 4, 9, 9, 9, 5, 9, 6, 7, 1, 2, 9, 3, 9, 9, 9, 4, 9, 9, 9, 9,
                         9, 9, 9, 5, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 6, 9, 9, 9, 9, 9, 9, 9, 9,
                         9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 7, 2, 3, 9, 4,
                         9, 9, 9, 5, 9, 9, 9, 9, 9, 9, 9, 6, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 7,
                         3, 4, 9, 5, 9, 9, 9, 6, 9, 9, 9, 9, 9, 9, 9, 7, 4, 5, 9, 6, 9, 9, 9, 7, 5, 6, 9, 7,
                         6, 7, 7, 8
                         );

static void hist_patch_uniform(const Mat_<uchar> &fI, Mat &histo,
                               int histSize, bool norm, bool rotinv)
{
        cv::Mat ufI, h, n;
        if (rotinv) {
                cv::Mat r8;
                // rotation invariant transform
                cv::LUT(fI, rotate_table, r8);
                // uniformity for rotation invariant
                cv::LUT(r8, rotuni_table, ufI);
                // histSize is max 10 bins
        } else {
                cv::LUT(fI, uniform_table, ufI);
        }
        // the upper boundary is exclusive
        float range[] = {0, (float)histSize};
        const float *histRange = {range};
        cv::calcHist(&ufI, 1, 0, Mat(), h, 1, &histSize, &histRange, true, false);

        if (norm)
                normalize(h, n);
        else
                n = h;
        histo.push_back(n.reshape(1, 1));
}

输入是您的CV_8U灰度补丁（其中一个补丁）。 out是旋转不变的，均匀的，归一化的重构直方图（1行）。然后将补丁直方图连接到面部描述符。您将拥有6 * 6 * 10 = 360.这本身就很好但是使用pca可以使其达到300或更低而不会丢失重要信息甚至提高检测质量，因为移除了尺寸（假设差异小于5％）不仅占用空间而且主要包含噪声（例如来自传感器的高斯噪声）。

然后你可以将这个concat直方图与面部库或使用svm（rbf内核更适合）进行比较。如果你正确地做了，那么预测一张脸不应该超过1-15ms（我的iphone7上5毫秒）。

希望这可以帮助。