在 FreeBSD 13.2 中启用 OpenMP 编译 Imagick 是否会导致 sched_yield() 问题?如果是这样,该如何解决?

问题描述 投票:0回答:1

从我记事起(几年),我们就一直在 Imagick 中使用 OpenMP,但是在我们的系统最近更新到 FreeBSD 13.2 (RELEASE-p9) 后,我们开始遇到进程被锁定在 100% CPU 的随机实例与 truss 的使用揭示了 sched_yield();在锁定的进程上被无限调用。锁定的进程永远不会完成,唯一的解决方案是手动终止它们。

在谷歌上搜索这个问题时,我只找到了两处提及,而且两者似乎都指出这是在 FreeBSD 13.2 上启用 OpenMP 的情况下编译 Imagick 的潜在问题。

还有其他人经历过这种情况吗?如果是这种情况,是否有解决方案?禁用线程对我们来说不是一个可行的解决方案,因为这样做会显着增加处理每个图像所需的时间(5 倍或更多),并且我们每天处理相当大量的图像。

供参考:FreeBSD 13.2-RELEASE-p9、PHP 8.1.27 (cli)、ImageMagick 7.1.1-26。

我见过的几个类似的提及可以在这里找到:

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=274135

https://forums.freebsd.org/threads/issue-with-ule-scheduler.85307/


这是一个示例调试日志,显示使用 sched_yield() 锁定进程,然后永不释放。

    (gdb) thread apply all bt

Thread 1 (LWP 101575 of process 33247):
#0  sched_yield () at sched_yield.S:4
#1  0x00000008456d0491 in __kmp_wait_4_ptr (spinner=0x84573151c <__kmp_initz_lock+28>, checker=1, pred=0x8456eab50 <__kmp_bakery_check(void*, unsigned int)>, obj=0x845731500 <__kmp_initz_lock>) at /usr/src/contrib/llvm-project/openmp/runtime/src/kmp_dispatch.cpp:2695
#2  0x00000008456e97c8 in __kmp_acquire_ticket_lock_timed_template (lck=0x0, gtid=<optimized out>) at /usr/src/contrib/llvm-project/openmp/runtime/src/kmp_lock.cpp:640
#3  __kmp_acquire_ticket_lock (lck=0x0, gtid=<optimized out>) at /usr/src/contrib/llvm-project/openmp/runtime/src/kmp_lock.cpp:645
#4  0x00000008456ef564 in __kmp_acquire_bootstrap_lock (lck=0x84573151c <__kmp_initz_lock+28>) at /usr/src/contrib/llvm-project/openmp/runtime/src/kmp_lock.h:524
#5  __kmp_get_global_thread_id_reg () at /usr/src/contrib/llvm-project/openmp/runtime/src/kmp_runtime.cpp:253
#6  0x00000008456af91e in __kmp_entry_thread () at /usr/src/contrib/llvm-project/openmp/runtime/src/kmp.h:3401
#7  kmpc_malloc (size=35524908316) at /usr/src/contrib/llvm-project/openmp/runtime/src/kmp_alloc.cpp:1098
#8  0x000000084572a1b7 in __kmp_is_address_mapped (addr=0x847bad5c8) at /usr/src/contrib/llvm-project/openmp/runtime/src/z_Linux_util.cpp:2067
#9  0x00000008456f9925 in __kmp_register_library_startup () at /usr/src/contrib/llvm-project/openmp/runtime/src/kmp_runtime.cpp:6774
#10 0x00000008456ef5c4 in __kmp_do_serial_initialize () at /usr/src/contrib/llvm-project/openmp/runtime/src/kmp_runtime.cpp:6966
#11 0x00000008456ef578 in __kmp_get_global_thread_id_reg () at /usr/src/contrib/llvm-project/openmp/runtime/src/kmp_runtime.cpp:255
#12 0x00000008456da0b2 in __kmp_api_omp_init_lock (user_lock=0x84573151c <__kmp_initz_lock+28>) at /usr/src/contrib/llvm-project/openmp/runtime/src/kmp_ftn_entry.h:1083
#13 0x0000000845bb773c in MagickCoreGenesis () from /usr/local/lib/libMagickCore-7.so.10
#14 0x00000008451f4d8e in zm_startup_imagick () from /usr/local/lib/php/20230831-debug/imagick.so
#15 0x000000000083b010 in zend_startup_module_ex ()
#16 0x000000000083bb10 in ?? ()
#17 0x0000000000853131 in zend_hash_apply ()
#18 0x000000000083b90c in zend_startup_modules ()
#19 0x0000000000766b0f in php_module_startup ()
#20 0x00000000009f51e9 in ?? ()
#21 0x00000000009f37a4 in ?? ()
#22 0x0000000000423720 in _start ()
(gdb) 

这是同时锁定的第二个进程(pid 33248 与 33247):

(gdb) thread apply all bt

Thread 1 (LWP 100499 of process 33248):
#0  sched_yield () at sched_yield.S:4
#1  0x00000008495aa491 in __kmp_wait_4_ptr (spinner=0x84960b51c <__kmp_initz_lock+28>, checker=1, pred=0x8495c4b50 <__kmp_bakery_check(void*, unsigned int)>, obj=0x84960b500 <__kmp_initz_lock>) at /usr/src/contrib/llvm-project/openmp/runtime/src/kmp_dispatch.cpp:2695
#2  0x00000008495c37c8 in __kmp_acquire_ticket_lock_timed_template (lck=0x0, gtid=<optimized out>) at /usr/src/contrib/llvm-project/openmp/runtime/src/kmp_lock.cpp:640
#3  __kmp_acquire_ticket_lock (lck=0x0, gtid=<optimized out>) at /usr/src/contrib/llvm-project/openmp/runtime/src/kmp_lock.cpp:645
#4  0x00000008495c9564 in __kmp_acquire_bootstrap_lock (lck=0x84960b51c <__kmp_initz_lock+28>) at /usr/src/contrib/llvm-project/openmp/runtime/src/kmp_lock.h:524
#5  __kmp_get_global_thread_id_reg () at /usr/src/contrib/llvm-project/openmp/runtime/src/kmp_runtime.cpp:253
#6  0x000000084958991e in __kmp_entry_thread () at /usr/src/contrib/llvm-project/openmp/runtime/src/kmp.h:3401
#7  kmpc_malloc (size=35590812956) at /usr/src/contrib/llvm-project/openmp/runtime/src/kmp_alloc.cpp:1098
#8  0x00000008496041b7 in __kmp_is_address_mapped (addr=0x847bad5c8) at /usr/src/contrib/llvm-project/openmp/runtime/src/z_Linux_util.cpp:2067
#9  0x00000008495d3925 in __kmp_register_library_startup () at /usr/src/contrib/llvm-project/openmp/runtime/src/kmp_runtime.cpp:6774
#10 0x00000008495c95c4 in __kmp_do_serial_initialize () at /usr/src/contrib/llvm-project/openmp/runtime/src/kmp_runtime.cpp:6966
#11 0x00000008495c9578 in __kmp_get_global_thread_id_reg () at /usr/src/contrib/llvm-project/openmp/runtime/src/kmp_runtime.cpp:255
#12 0x00000008495b40b2 in __kmp_api_omp_init_lock (user_lock=0x84960b51c <__kmp_initz_lock+28>) at /usr/src/contrib/llvm-project/openmp/runtime/src/kmp_ftn_entry.h:1083
#13 0x000000084b24773c in MagickCoreGenesis () from /usr/local/lib/libMagickCore-7.so.10
#14 0x000000084a26bd8e in zm_startup_imagick () from /usr/local/lib/php/20230831-debug/imagick.so
#15 0x000000000083b010 in zend_startup_module_ex ()
#16 0x000000000083bb10 in ?? ()
#17 0x0000000000853131 in zend_hash_apply ()
#18 0x000000000083b90c in zend_startup_modules ()
#19 0x0000000000766b0f in php_module_startup ()
#20 0x00000000009f51e9 in ?? ()
#21 0x00000000009f37a4 in ?? ()
#22 0x0000000000423720 in _start ()
(gdb) 

我们也尝试更新到PHP 8.3.4,问题依然存在。


更新:我已将 Imagick 排除在外,并使用命令行

convert
操作仅使用 ImageMagick 重新创建了相同的错误。为了在 PHP 之外重现该错误,我创建了两个进程,每个进程都重复向另一个图像添加水印,然后我只是等待某些内容锁定。

我使用以下命令将水印添加到另一张图像的左上角: 转换 'originalfile.jpg' 'watermarkfile.png' -设置色彩空间 sRGB -gravity NorthWest -geometry +3+3 -define compose:args=20,100 -composesolve -composite 'resultingfile.jpg'

仅使用

convert
产生的锁定进程的跟踪信息如下:

(gdb) thread apply all bt

Thread 1 (LWP 101740 of process 79048):
#0  sched_yield () at sched_yield.S:4
#1  0x0000000824f60491 in __kmp_wait_4_ptr (spinner=0x824fc151c <__kmp_initz_lock+28>, checker=1, pred=0x824f7ab50 <__kmp_bakery_check(void*, unsigned int)>, obj=0x824fc1500 <__kmp_initz_lock>) at /usr/src/contrib/llvm-project/openmp/runtime/src/kmp_dispatch.cpp:2695
#2  0x0000000824f797c8 in __kmp_acquire_ticket_lock_timed_template (lck=0x0, gtid=<optimized out>) at /usr/src/contrib/llvm-project/openmp/runtime/src/kmp_lock.cpp:640
#3  __kmp_acquire_ticket_lock (lck=0x0, gtid=<optimized out>) at /usr/src/contrib/llvm-project/openmp/runtime/src/kmp_lock.cpp:645
#4  0x0000000824f7f564 in __kmp_acquire_bootstrap_lock (lck=0x824fc151c <__kmp_initz_lock+28>) at /usr/src/contrib/llvm-project/openmp/runtime/src/kmp_lock.h:524
#5  __kmp_get_global_thread_id_reg () at /usr/src/contrib/llvm-project/openmp/runtime/src/kmp_runtime.cpp:253
#6  0x0000000824f3f91e in __kmp_entry_thread () at /usr/src/contrib/llvm-project/openmp/runtime/src/kmp.h:3401
#7  kmpc_malloc (size=34980238620) at /usr/src/contrib/llvm-project/openmp/runtime/src/kmp_alloc.cpp:1098
#8  0x0000000824fba1b7 in __kmp_is_address_mapped (addr=0x8242955c8) at /usr/src/contrib/llvm-project/openmp/runtime/src/z_Linux_util.cpp:2067
#9  0x0000000824f89925 in __kmp_register_library_startup () at /usr/src/contrib/llvm-project/openmp/runtime/src/kmp_runtime.cpp:6774
#10 0x0000000824f7f5c4 in __kmp_do_serial_initialize () at /usr/src/contrib/llvm-project/openmp/runtime/src/kmp_runtime.cpp:6966
#11 0x0000000824f7f578 in __kmp_get_global_thread_id_reg () at /usr/src/contrib/llvm-project/openmp/runtime/src/kmp_runtime.cpp:255
#12 0x0000000824f6a0b2 in __kmp_api_omp_init_lock (user_lock=0x824fc151c <__kmp_initz_lock+28>) at /usr/src/contrib/llvm-project/openmp/runtime/src/kmp_ftn_entry.h:1083
#13 0x0000000821df77ac in MagickCoreGenesis () from /usr/local/lib/libMagickCore-7.so.10
#14 0x0000000000202512 in ?? ()
#15 0x00000000002022f0 in ?? ()
#16 0x00000000002021f0 in ?? ()
#17 0x0000000000000000 in ?? ()

还有

(gdb) thread apply all bt

Thread 1 (LWP 153345 of process 79049):
#0  sched_yield () at sched_yield.S:4
#1  0x0000000825da5491 in __kmp_wait_4_ptr (spinner=0x825e0651c <__kmp_initz_lock+28>, checker=1, pred=0x825dbfb50 <__kmp_bakery_check(void*, unsigned int)>, obj=0x825e06500 <__kmp_initz_lock>) at /usr/src/contrib/llvm-project/openmp/runtime/src/kmp_dispatch.cpp:2695
#2  0x0000000825dbe7c8 in __kmp_acquire_ticket_lock_timed_template (lck=0x0, gtid=<optimized out>) at /usr/src/contrib/llvm-project/openmp/runtime/src/kmp_lock.cpp:640
#3  __kmp_acquire_ticket_lock (lck=0x0, gtid=<optimized out>) at /usr/src/contrib/llvm-project/openmp/runtime/src/kmp_lock.cpp:645
#4  0x0000000825dc4564 in __kmp_acquire_bootstrap_lock (lck=0x825e0651c <__kmp_initz_lock+28>) at /usr/src/contrib/llvm-project/openmp/runtime/src/kmp_lock.h:524
#5  __kmp_get_global_thread_id_reg () at /usr/src/contrib/llvm-project/openmp/runtime/src/kmp_runtime.cpp:253
#6  0x0000000825d8491e in __kmp_entry_thread () at /usr/src/contrib/llvm-project/openmp/runtime/src/kmp.h:3401
#7  kmpc_malloc (size=34995201308) at /usr/src/contrib/llvm-project/openmp/runtime/src/kmp_alloc.cpp:1098
#8  0x0000000825dff1b7 in __kmp_is_address_mapped (addr=0x8246965c8) at /usr/src/contrib/llvm-project/openmp/runtime/src/z_Linux_util.cpp:2067
#9  0x0000000825dce925 in __kmp_register_library_startup () at /usr/src/contrib/llvm-project/openmp/runtime/src/kmp_runtime.cpp:6774
#10 0x0000000825dc45c4 in __kmp_do_serial_initialize () at /usr/src/contrib/llvm-project/openmp/runtime/src/kmp_runtime.cpp:6966
#11 0x0000000825dc4578 in __kmp_get_global_thread_id_reg () at /usr/src/contrib/llvm-project/openmp/runtime/src/kmp_runtime.cpp:255
#12 0x0000000825daf0b2 in __kmp_api_omp_init_lock (user_lock=0x825e0651c <__kmp_initz_lock+28>) at /usr/src/contrib/llvm-project/openmp/runtime/src/kmp_ftn_entry.h:1083
#13 0x00000008226c37ac in MagickCoreGenesis () from /usr/local/lib/libMagickCore-7.so.10
#14 0x0000000000202512 in ?? ()
#15 0x00000000002022f0 in ?? ()
#16 0x00000000002021f0 in ?? ()
#17 0x0000000000000000 in ?? ()
openmp imagick
1个回答
0
投票

这确实是 LLVM OpenMP 运行时中的一个问题(我为此创建了一个问题:https://github.com/llvm/llvm-project/issues/86684)。

我不确定您为什么之前没有看到这个问题。可能是 FreeBSD 从 gcc 编译的 ImageMagick 迁移到 clang 编译的 ImageMagick。

不幸的是,在问题得到解决之前,我不知道如何缓解该问题。

最新问题
© www.soinside.com 2019 - 2024. All rights reserved.