为什么ReentrantLock不能在Jenkins管道中锁定？

问题描述 投票：0回答：1

我似乎无法使此并发模式在Jenkins Pipeline脚本中按预期工作。我已尽可能简化了方案，结果仍然没有意义。这是整个Jenkinsfile：

import java.util.concurrent.locks.ReentrantLock

// create lock and index vars to make sure concurrent threads write different output files
shellLock = new ReentrantLock()
shellIndex = 0

def doIt() {
    shellLock.lock()
    def threadShellIndex = ++shellIndex
    if (threadShellIndex == 1) {
        sh("rm -rf shell")
        sh("mkdir shell")
    }
    shellLock.unlock()

    sh "touch shell/${threadShellIndex}"
}

node {
    checkout scm
    runs = [
        "1": { doIt() },
        "2": { doIt() },
    ]
    parallel runs
}

正如我所料，这不会在两次运行都继续在其中创建文件之前删除并替换“ shell”目录。相反，控制台输出为：

[Pipeline] parallel
[Pipeline] { (Branch: 1)
[Pipeline] { (Branch: 2)
[Pipeline] sh
[Pipeline] sh
[1] + rm -rf shell
[Pipeline] sh
[2] + touch shell/2
touch: cannot touch ‘shell/2’: No such file or directory
[Pipeline] }
Failed in branch 2
[1] + mkdir shell
[Pipeline] }
Failed in branch 1
[Pipeline] // parallel
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Also:   hudson.AbortException: script returned exit code 1
        at org.jenkinsci.plugins.workflow.steps.durable_task.DurableTaskStep$Execution.handleExit(DurableTaskStep.java:658)
        at org.jenkinsci.plugins.workflow.steps.durable_task.DurableTaskStep$Execution.check(DurableTaskStep.java:604)
        at org.jenkinsci.plugins.workflow.steps.durable_task.DurableTaskStep$Execution.run(DurableTaskStep.java:548)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
java.lang.IllegalMonitorStateException
    at java.util.concurrent.locks.ReentrantLock$Sync.tryRelease(ReentrantLock.java:151)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer.release(AbstractQueuedSynchronizer.java:1261)
    at java.util.concurrent.locks.ReentrantLock.unlock(ReentrantLock.java:457)
    at sun.reflect.GeneratedMethodAccessor1111.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
    at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
    at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1213)
    at groovy.lang.ExpandoMetaClass.invokeMethod(ExpandoMetaClass.java:1125)
    at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022)
    at org.codehaus.groovy.runtime.callsite.PojoMetaClassSite.call(PojoMetaClassSite.java:47)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
    at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:163)
    at org.kohsuke.groovy.sandbox.GroovyInterceptor.onMethodCall(GroovyInterceptor.java:23)
    at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onMethodCall(SandboxInterceptor.java:157)
    at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:161)
    at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:165)
    at com.cloudbees.groovy.cps.sandbox.SandboxInvoker.methodCall(SandboxInvoker.java:17)
    at WorkflowScript.doIt(WorkflowScript:14)
    at WorkflowScript.run(WorkflowScript:22)
    at ___cps.transform___(Native Method)
    at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:86)
    at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:113)
    at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixName(FunctionCallBlock.java:78)
    at sun.reflect.GeneratedMethodAccessor109.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)
    at com.cloudbees.groovy.cps.impl.ConstantBlock.eval(ConstantBlock.java:21)
    at com.cloudbees.groovy.cps.Next.step(Next.java:83)
    at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:174)
    at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:163)
    at org.codehaus.groovy.runtime.GroovyCategorySupport$ThreadCategoryInfo.use(GroovyCategorySupport.java:129)
    at org.codehaus.groovy.runtime.GroovyCategorySupport.use(GroovyCategorySupport.java:268)
    at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:163)
    at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.access$001(SandboxContinuable.java:18)
    at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:51)
    at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:185)
    at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:400)
    at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$400(CpsThreadGroup.java:96)
    at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:312)
    at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:276)
    at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:67)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:131)
    at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
    at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
Finished: FAILURE

如何让运行2等待运行1重新创建目录？

jenkins concurrency jenkins-pipeline java.util.concurrent
1个回答
0
投票

似乎ReentrantLock无效的原因是，管道常规实际上不是多线程的。它使用一个线程来承载多个“绿色”线程。

来自https://github.com/jenkinsci/workflow-cps-plugin：

所有程序逻辑都在“ CPS VM线程”内部运行，这只是一个Java线程池，可以运行二进制方法并找出下一步要执行的延续。并行步骤使用“绿色线程”（也称为合作多任务）：它记录了各种操作的逻辑线程（〜分支）名称，但实际上并未同时运行它们。

由于所有调用都在同一个线程中，因此ReentrantLock重新输入。

就我而言，我可以通过使用AtomicInteger来解决锁定的需要，并且在重新创建目录时可以使用一些符号链接的巧妙方法。

import java.util.concurrent.atomic.AtomicInteger
shellCounter = new AtomicInteger()

@NonCPS
def getNextShellCounter() {
    return shellCounter.incrementAndGet()
}

...

    def threadShellIndex = getNextShellCounter()

    // concurrency-safe creation of output dir and removal of previous
    def shellDir = "shell.${currentBuild.number}"
    sh """#!/bin/bash +x
        mkdir -p ${shellDir}
        ln -sfn ${shellDir} shell
        rm -rf shell.foo `ls -d shell.* | sed -e '/^${shellDir}\$/d' `
    """

[通过此处的线程工作方式，简单的++可能具有相同的功能，但比后悔更安全。

最新问题
© www.soinside.com 2019 - 2024. All rights reserved.