抢占yocto linux中与docker的RT冲突

Question

我的项目遇到了一个奇怪的问题，我将描述该问题： 我在 yocto linux 上修补了 Preempt RT，它实时证明它可以运行 FIFO 和 RR 等实时调度任务，然后我在上面配置了 docker，奇怪的事情发生了，当我启动 docker 时，我的实时调度任务如下FIFO 和 RR 无法运行。有谁有任何想法或者您能指出错误可能发生在哪里吗？ （FIFO和RR测试是在主机中，而不是在任何容器中）

安装docker之前

root@intel-corei7-64:~# uname -ar
Linux intel-corei7-64 5.15.137-rt71-intel-pk-preempt-rt #1 SMP PREEMPT_RT Fri Feb 2 16:01:33 UTC 2024 x86_64 GNU/Linux
root@intel-corei7-64:~# sudo /userdata/test_thread_rr_fifo fifo
My thread FIFO is running 1 seconds...
My thread FIFO is running 2 seconds...
My thread FIFO is running 3 seconds...
My thread FIFO is running 4 seconds...

安装docker后

root@intel-corei7-64:~# tar -xvf docker-24.0.7.tgz
root@intel-corei7-64:~# cp docker/* /usr/bin/
root@intel-corei7-64:~# systemctl enable docker
create symlink /etc/systemd/multi-user.target.wants/docker.service -> /etc/systemd/system/docker.service.
root@intel-corei7-64:~# systemctl start docker
root@intel-corei7-64:~# sudo /userdata/test_thread_rr_fifo fifo
Please use sudo to test. Or pthread_created Failed: This kernel not support FIFO
root@intel-corei7-64:~# docker ps
CONTAINER ID    IMGAE    COMMAND    CREATED    STATUS    PORTS    NAMES
root@intel-corei7-64:~# docker version
Client:
  Version:       24.0.7
  API version:   1.43
  Go version:    go1.20.10
-----------------------------------------------------------

root@intel-corei7-64:~# systemctl disable docker
root@intel-corei7-64:~# systemctl stop docker
root@intel-corei7-64:~# sudo /userdata/test_thread_rr_fifo fifo
My thread FIFO is running 1 seconds...
My thread FIFO is running 2 seconds...
My thread FIFO is running 3 seconds...
My thread FIFO is running 4 seconds...

FIFO和RR测试代码

#include <pthread.h>
#include <unistd.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <errno.h>
#include <iostream>

void* my_thread_body(void* arg)
{
    long cnt = 0;
    while (1) {
        std::cout << "My thread FIFO is running " << ++cnt << " seconds..." << std::endl;
        sleep(1);
    }
}

void* my_thread_body2(void* arg)
{
    long cnt = 0;
    while (1) {
        std::cout << "My second thread RR is running " << ++cnt << " seconds..." << std::endl;
        sleep(1);
    }
}

int main(int argc, char* argv[])
{
    pthread_t my_thread, my_thread2;
    struct sched_param param;
    param.sched_priority = 50;
    pthread_attr_t attr;
    int ret = 0;

    if (argc < 2) {
        std::cout << "Usage: sudo " << argv[0] << " [scheduling policy]\n\nUsable policy: fifo, rr \n\neg.  sudo " <<argv[0]<< " fifo\n" << std::endl;
        return -1;
    }

    if ((strcmp(argv[1], "fifo") == 0) || (strcmp(argv[1], "rr") == 0)) {
        if ((ret = pthread_attr_init(&attr)) != 0) {
            perror("pthread_attr_init()");
            std::cout << "ret code: " << ret << std::endl;
            return -1;
        }
        if (strcmp(argv[1], "rr") == 0) {
            if ((ret = pthread_attr_setschedpolicy(&attr, SCHED_RR)) != 0) {
                std::cout << "pthread_attr_setschedpolicy() Failed" << std::endl;
                return -1;
            }
        }
        if (strcmp(argv[1], "fifo") == 0) {
            if ((ret = pthread_attr_setschedpolicy(&attr, SCHED_FIFO)) != 0) {
                std::cout << "pthread_attr_setschedpolicy() Failed" << std::endl;
                return -1;
            }
        }

        if ((ret = pthread_attr_setschedparam(&attr, &param)) != 0) {
            std::cout << "pthread_attr_setschedparam Failed" << std::endl;
            return -1;
        }
        if ((ret = pthread_attr_setinheritsched(&attr, PTHREAD_EXPLICIT_SCHED)) != 0) {
            std::cout << "pthread_attr_setinheritsched() Failed" << ret << std::endl;
            return -1;
        }

        if (strcmp(argv[1], "fifo") == 0) {
            if ((ret = pthread_create(&my_thread, &attr, my_thread_body, NULL)) != 0) {
                std::cout << "Please use sudo to test. Or pthread_create Failed: This kernel not support FIFO" << std::endl;
                return -1;
            }
            pthread_join(my_thread, NULL);
        }
        else if (strcmp(argv[1], "rr") == 0) {
            if ((ret = pthread_create(&my_thread2, &attr, my_thread_body2, NULL)) != 0) {
                std::cout << "Please use sudo to test. Or pthread_create Failed: " << strerror(ret)  << std::endl;
                return -1;
            }
            pthread_join(my_thread2, NULL);
        }
    }
    else {
        std::cout << "error param" << std::endl;
    }
    std::cout << "Bye!" << std::endl;
    return 0;
}

尝试过：

1. 我将docker版本从1.18更改为2.24，问题仍然发生
2. 我使用官方的 Check-config.sh 从内核检查 docker 配置，一切都很好
3. CONFIG_RT_GOURP_SCHED 在内核中启用
4. 我比较了 x86 yocto linux 和 arm64 yocto linux 的内核设置（这个没有 isusse），没有明显的差异。
5. 当我使用

   systemctl disable docker

   和

   systemctl stop docker

   时，FIFO和RR任务可以工作 再次。

期待：

找出出现此问题的原因？

Answer 1

几周过去了，我发现 Docker 与 RT（intel RT-patch 内核）冲突的原因是内核配置中启用了 CONFIG_RT_GROUP_SCHED。 我禁用之后，docker就不会和RT-Patch冲突了，然后docker-compose.yml中的标签需要这样注释：

#mem_limit : 300m
#memswap_limit : 0 
#mem_reservation : 250m