12个线程后FUSE死锁

问题描述 投票：0回答：1

基本描述

我偶然发现了 FUSE 非常奇怪的行为。

  • 我尝试运行 100 个线程，其中
  • 每个线程都会触及自己的 mmaped FUSE 页面，这会触发 fusion 上的 
    FUSE_read()
  • 一个熔丝页对应一个文件
  • 在
    FUSE_read()
    中，我做了一些日志并阻止：
    sleep(500000)
  • 原来12个线程被阻塞在
    FUSE_read
    之后，系统就陷入了死锁。

来源

这里是保险丝代码：(

fuse_minimal.c
)

#define FUSE_USE_VERSION 30

#include <fuse3/fuse.h>

//...
static int FUSE_read(const char *path, char *dst, size_t size, off_t offset, struct fuse_file_info *fi){
    char log[50];
    snprintf(log, sizeof(log), "[fuse] Reading file %s, size: %lu", path, size);
    printf("%s\n", log);
    
    // thread 'i' opens file '/tmp/fuse/i'.
    // but here @path="/i"
    // so add +1
    int i = atoi(path + 1); // without leading '/'

    printf("[fuse] Thread %d is blocked\n", i);
    sleep(1000000);
    printf("[fuse] Unblocking thread %d\n", i);
    
    return size;
}

// ...
static struct fuse_operations FUSE_ops = {
    // ...
    .read       = FUSE_read,
    // ...
};

int main(int argc, char** argv) {
    return fuse_main(argc, argv, &FUSE_ops, NULL);
}

这是启动 100 个循环的代码：(

test.c
)

void* func(void* arg) {
    size_t i = (size_t)arg; 
    void* fuse_page_i = (void*)0x40000 + 2 * i * 0x1000; // the mmaped page
    printf("[*] starting thread %lu, touching %p\n", i, fuse_page_i);
    uint64_t val = *(uint64_t*)fuse_page_i; // read happens => FUSE_read
    printf("val: %lu", val);
}   
    
int main() {
    setbuf(stdout, 0);
    puts("[*] ./fuse");
    int pid = fork();
    if (pid == 0) {
        system("mkdir -p /tmp/fuse && ./fuse_minimal -f /tmp/fuse"); // -f for printf output
        exit(0);
    } 
    sleep(5); // wait until fuse_main is set up
    for (size_t  i = 0; i < 100; ++i) {
        char file_target[20];
        snprintf(file_target, sizeof(file_target), "/tmp/fuse/%lu", i);
        int fd = open(file_target, O_CREAT | O_RDWR, 0666);
        if (mmap((void*)0x40000 + 2 * i * 0x1000, 0x1000,
                PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_FIXED, fd, 0)
            == (void*)-1) {
            perror("mmap");
            exit(1);
        }
    }

    pthread_t thread[100];
    for (size_t i = 0; i < 100; ++i) {
        pthread_create(&thread[i], NULL, func, (void*)i); // see the `func` above
    }

    for (size_t i = 0; i < 100; ++i) { 
        pthread_join(thread[i], NULL); // will not join for lloooong time
    }
}

编译

gcc  fuse_minimal.c -lfuse3 -o fuse_minimal -D_FILE_OFFSET_BITS=64
gcc  test.c -o test

最有趣的部分。日志

[*] ./fuse
Ignoring invalid max threads value 4294967295 > max (100000).
[fuse] Opened file /0
[fuse] Opened file /1
[fuse] Opened file /2
[fuse] Opened file /3
[fuse] Opened file /4
[fuse] Opened file /5
[fuse] Opened file /6
[fuse] Opened file /7
[fuse] Opened file /8
[fuse] Opened file /9
[fuse] Opened file /10
[fuse] Opened file /11
[fuse] Opened file /12
[fuse] Opened file /13
[fuse] Opened file /14
[fuse] Opened file /15
[fuse] Opened file /16
[fuse] Opened file /17
[fuse] Opened file /18
[fuse] Opened file /19
[fuse] Opened file /20
[fuse] Opened file /21
[fuse] Opened file /22
[fuse] Opened file /23
[fuse] Opened file /24
[fuse] Opened file /25
[fuse] Opened file /26
[fuse] Opened file /27
[fuse] Opened file /28
[fuse] Opened file /29
[fuse] Opened file /30
[fuse] Opened file /31
[fuse] Opened file /32
[fuse] Opened file /33
[fuse] Opened file /34
[fuse] Opened file /35
[fuse] Opened file /36
[fuse] Opened file /37
[fuse] Opened file /38
[fuse] Opened file /39
[fuse] Opened file /40
[fuse] Opened file /41
[fuse] Opened file /42
[fuse] Opened file /43
[fuse] Opened file /44
[fuse] Opened file /45
[fuse] Opened file /46
[fuse] Opened file /47
[fuse] Opened file /48
[fuse] Opened file /49
[fuse] Opened file /50
[fuse] Opened file /51
[fuse] Opened file /52
[fuse] Opened file /53
[fuse] Opened file /54
[fuse] Opened file /55
[fuse] Opened file /56
[fuse] Opened file /57
[fuse] Opened file /58
[fuse] Opened file /59
[fuse] Opened file /60
[fuse] Opened file /61
[fuse] Opened file /62
[fuse] Opened file /63
[fuse] Opened file /64
[fuse] Opened file /65
[fuse] Opened file /66
[fuse] Opened file /67
[fuse] Opened file /68
[fuse] Opened file /69
[fuse] Opened file /70
[fuse] Opened file /71
[fuse] Opened file /72
[fuse] Opened file /73
[fuse] Opened file /74
[fuse] Opened file /75
[fuse] Opened file /76
[fuse] Opened file /77
[fuse] Opened file /78
[fuse] Opened file /79
[fuse] Opened file /80
[fuse] Opened file /81
[fuse] Opened file /82
[fuse] Opened file /83
[fuse] Opened file /84
[fuse] Opened file /85
[fuse] Opened file /86
[fuse] Opened file /87
[fuse] Opened file /88
[fuse] Opened file /89
[fuse] Opened file /90
[fuse] Opened file /91
[fuse] Opened file /92
[fuse] Opened file /93
[fuse] Opened file /94
[fuse] Opened file /95
[fuse] Opened file /96
[fuse] Opened file /97
[fuse] Opened file /98
[fuse] Opened file /99
[*] starting thread 0, touching 0x40000
[*] starting thread 1, touching 0x42000
[*] starting thread 2, touching 0x44000
[fuse] Reading file /0, size: 4096
[fuse] Thread 0 is blocked
[*] starting thread 3, touching 0x46000
[fuse] Reading file /1, size: 4096
[fuse] Thread 1 is blocked
[*] starting thread 4, touching 0x48000
[*] starting thread 5, touching 0x4a000
[*] starting thread 6, touching 0x4c000
[fuse] Reading file /2, size: 4096
[fuse] Thread 2 is blocked
[*] starting thread 7, touching 0x4e000
[*] starting thread 8, touching 0x50000
[fuse] Reading file /3, size: 4096
[fuse] Thread 3 is blocked
[*] starting thread 9, touching 0x52000
[*] starting thread 10, touching 0x54000
[*] starting thread 11, touching 0x56000
[*] starting thread 12, touching 0x58000
[*] starting thread 13, touching 0x5a000
[*] starting thread 14, touching 0x5c000
[*] starting thread 15, touching 0x5e000
[*] starting thread 16, touching 0x60000
[fuse] Reading file /4, size: 4096
[fuse] Thread 4 is blocked
[*] starting thread 17, touching 0x62000
[*] starting thread 18, touching 0x64000
[*] starting thread 19, touching 0x66000
[fuse] Reading file /5, size: 4096
[fuse] Thread 5 is blocked
[*] starting thread 20, touching 0x68000
[fuse] Reading file /6, size: 4096
[fuse] Thread 6 is blocked
[*] starting thread 21, touching 0x6a000
[*] starting thread 22, touching 0x6c000
[fuse] Reading file /7, size: 4096
[fuse] Thread 7 is blocked
[*] starting thread 23, touching 0x6e000
[*] starting thread 24, touching 0x70000
[fuse] Reading file /8, size: 4096
[fuse] Thread 8 is blocked
[*] starting thread 25, touching 0x72000
[fuse] Reading file /9, size: 4096
[*] starting thread 26, touching 0x74000
[fuse] Thread 9 is blocked
[*] starting thread 27, touching 0x76000
[*] starting thread 28, touching 0x78000
[*] starting thread 29, touching 0x7a000
[*] starting thread 30, touching 0x7c000
[*] starting thread 31, touching 0x7e000
[*] starting thread 32, touching 0x80000
[*] starting thread 33, touching 0x82000
[*] starting thread 34, touching 0x84000
[*] starting thread 35, touching 0x86000
[*] starting thread 36, touching 0x88000
[*] starting thread 37, touching 0x8a000
[*] starting thread 38, touching 0x8c000
[*] starting thread 39, touching 0x8e000
[*] starting thread 40, touching 0x90000
[*] starting thread 41, touching 0x92000
[*] starting thread 42, touching 0x94000
[*] starting thread 43, touching 0x96000
[*] starting thread 44, touching 0x98000
[*] starting thread 45, touching 0x9a000
[*] starting thread 46, touching 0x9c000
[*] starting thread 47, touching 0x9e000
[*] starting thread 48, touching 0xa0000
[*] starting thread 49, touching 0xa2000
[*] starting thread 50, touching 0xa4000
[*] starting thread 51, touching 0xa6000
[*] starting thread 52, touching 0xa8000
[*] starting thread 53, touching 0xaa000
[*] starting thread 54, touching 0xac000
[*] starting thread 55, touching 0xae000
[*] starting thread 56, touching 0xb0000
[*] starting thread 57, touching 0xb2000
[*] starting thread 58, touching 0xb4000
[*] starting thread 59, touching 0xb6000
[*] starting thread 60, touching 0xb8000
[*] starting thread 61, touching 0xba000
[*] starting thread 62, touching 0xbc000
[*] starting thread 63, touching 0xbe000
[*] starting thread 64, touching 0xc0000
[*] starting thread 65, touching 0xc2000
[*] starting thread 66, touching 0xc4000
[*] starting thread 67, touching 0xc6000
[*] starting thread 68, touching 0xc8000
[*] starting thread 69, touching 0xca000
[*] starting thread 70, touching 0xcc000
[*] starting thread 71, touching 0xce000
[*] starting thread 72, touching 0xd0000
[*] starting thread 73, touching 0xd2000
[*] starting thread 74, touching 0xd4000
[*] starting thread 75, touching 0xd6000
[*] starting thread 76, touching 0xd8000
[*] starting thread 77, touching 0xda000
[*] starting thread 78, touching 0xdc000
[*] starting thread 79, touching 0xde000
[*] starting thread 80, touching 0xe0000
[*] starting thread 81, touching 0xe2000
[*] starting thread 82, touching 0xe4000
[*] starting thread 83, touching 0xe6000
[*] starting thread 84, touching 0xe8000
[*] starting thread 85, touching 0xea000
[*] starting thread 86, touching 0xec000
[*] starting thread 87, touching 0xee000
[*] starting thread 88, touching 0xf0000
[*] starting thread 89, touching 0xf2000
[*] starting thread 90, touching 0xf4000
[*] starting thread 91, touching 0xf6000
[*] starting thread 92, touching 0xf8000
[*] starting thread 93, touching 0xfa000
[*] starting thread 94, touching 0xfc000
[*] starting thread 95, touching 0xfe000
[*] starting thread 96, touching 0x100000
[*] starting thread 97, touching 0x102000
[*] starting thread 98, touching 0x104000
[*] starting thread 99, touching 0x106000

正如您在日志中看到的，我们首先打开 100 个文件。 然后我们启动 100 个线程，每个线程都尝试触摸自己的 

mmap
ed 文件。 但！其中只有 12 个会触发 
FUSE_read
。 所有其他触摸该文件，但无法进入 FUSE_read。 此时程序陷入僵局。 :(

如果我们像

20s
那样睡眠更短的时间，线程将在
FUSE_read
之后继续
20s
。测试了一下。

所以，根据我对上述内容的理解，它看起来像

  • libfuse
    有自己的用户空间线程（纤程？），预计不会被阻塞？ （如果我错了请纠正我）（编辑：这是真的，请参阅答案）
  • 或者 Linux 内核在 12 个读取请求融合后以某种方式阻止我（编辑：这是错误的）

我真的很想在熔断操作上阻止这 100 个线程。我有什么选择？

c linux fuse libfuse
1个回答
0
投票

好的，所以这里它是：

struct fuse_loop_config *fuse_loop_cfg_create(void)
{
    struct fuse_loop_config *config = calloc(1, sizeof(*config));
    if (config == NULL)
        return NULL;

    config->version_id       = FUSE_LOOP_MT_V2_IDENTIFIER;
    config->max_idle_threads = FUSE_LOOP_MT_DEF_IDLE_THREADS;
    config->max_threads      = FUSE_LOOP_MT_DEF_MAX_THREADS; // spot the fed
    config->clone_fd         = FUSE_LOOP_MT_DEF_CLONE_FD;

    return config;
}

默认

FUSE_LOOP_MT_DEF_MAX_THREADS=10
。我会尝试设置为
100
并重新编译。

在重新编译成功或发布任何更好的答案之前，不会将问题标记为已接受。

最新问题
© www.soinside.com 2019 - 2024. All rights reserved.