为什么 ReentrantLock 不锁定 Jenkins 管道？
Why doesn't ReentrantLock lock in a Jenkins Pipeline?
我似乎无法让这种并发模式按预期在我的 Jenkins 管道脚本中工作。我已经尽可能地简化了场景，结果仍然没有意义。这是整个 Jenkins 文件：
import java.util.concurrent.locks.ReentrantLock
// create lock and index vars to make sure concurrent threads write different output files
shellLock = new ReentrantLock()
shellIndex = 0
def doIt() {
shellLock.lock()
def threadShellIndex = ++shellIndex
if (threadShellIndex == 1) {
sh("rm -rf shell")
sh("mkdir shell")
}
shellLock.unlock()
sh "touch shell/${threadShellIndex}"
}
node {
checkout scm
runs = [
"1": { doIt() },
"2": { doIt() },
]
parallel runs
}
这并没有像我预期的那样，在两个 运行 继续在其中创建文件之前删除并替换 "shell" 目录。相反，控制台输出是：
[Pipeline] parallel
[Pipeline] { (Branch: 1)
[Pipeline] { (Branch: 2)
[Pipeline] sh
[Pipeline] sh
[1] + rm -rf shell
[Pipeline] sh
[2] + touch shell/2
touch: cannot touch ‘shell/2’: No such file or directory
[Pipeline] }
Failed in branch 2
[1] + mkdir shell
[Pipeline] }
Failed in branch 1
[Pipeline] // parallel
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Also: hudson.AbortException: script returned exit code 1
at org.jenkinsci.plugins.workflow.steps.durable_task.DurableTaskStep$Execution.handleExit(DurableTaskStep.java:658)
at org.jenkinsci.plugins.workflow.steps.durable_task.DurableTaskStep$Execution.check(DurableTaskStep.java:604)
at org.jenkinsci.plugins.workflow.steps.durable_task.DurableTaskStep$Execution.run(DurableTaskStep.java:548)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access1(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
java.lang.IllegalMonitorStateException
at java.util.concurrent.locks.ReentrantLock$Sync.tryRelease(ReentrantLock.java:151)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.release(AbstractQueuedSynchronizer.java:1261)
at java.util.concurrent.locks.ReentrantLock.unlock(ReentrantLock.java:457)
at sun.reflect.GeneratedMethodAccessor1111.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1213)
at groovy.lang.ExpandoMetaClass.invokeMethod(ExpandoMetaClass.java:1125)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022)
at org.codehaus.groovy.runtime.callsite.PojoMetaClassSite.call(PojoMetaClassSite.java:47)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
at org.kohsuke.groovy.sandbox.impl.Checker.call(Checker.java:163)
at org.kohsuke.groovy.sandbox.GroovyInterceptor.onMethodCall(GroovyInterceptor.java:23)
at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onMethodCall(SandboxInterceptor.java:157)
at org.kohsuke.groovy.sandbox.impl.Checker.call(Checker.java:161)
at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:165)
at com.cloudbees.groovy.cps.sandbox.SandboxInvoker.methodCall(SandboxInvoker.java:17)
at WorkflowScript.doIt(WorkflowScript:14)
at WorkflowScript.run(WorkflowScript:22)
at ___cps.transform___(Native Method)
at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:86)
at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:113)
at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixName(FunctionCallBlock.java:78)
at sun.reflect.GeneratedMethodAccessor109.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)
at com.cloudbees.groovy.cps.impl.ConstantBlock.eval(ConstantBlock.java:21)
at com.cloudbees.groovy.cps.Next.step(Next.java:83)
at com.cloudbees.groovy.cps.Continuable.call(Continuable.java:174)
at com.cloudbees.groovy.cps.Continuable.call(Continuable.java:163)
at org.codehaus.groovy.runtime.GroovyCategorySupport$ThreadCategoryInfo.use(GroovyCategorySupport.java:129)
at org.codehaus.groovy.runtime.GroovyCategorySupport.use(GroovyCategorySupport.java:268)
at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:163)
at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.access[=11=]1(SandboxContinuable.java:18)
at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:51)
at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:185)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:400)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access0(CpsThreadGroup.java:96)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.call(CpsThreadGroup.java:312)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.call(CpsThreadGroup.java:276)
at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService.call(CpsVmExecutorService.java:67)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at hudson.remoting.SingleLaneExecutorService.run(SingleLaneExecutorService.java:131)
at jenkins.util.ContextResettingExecutorService.run(ContextResettingExecutorService.java:28)
at jenkins.security.ImpersonatingExecutorService.run(ImpersonatingExecutorService.java:59)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Finished: FAILURE
如何让 运行 2 等到 运行 1 重新创建目录？
似乎ReentrantLock没有效果的原因是管道groovy实际上不是多线程的。它使用单个线程来托管多个 "green" 线程。
来自https://github.com/jenkinsci/workflow-cps-plugin：
All program logic is run inside a “CPS VM thread”, which is just a Java thread pool that can run binary methods and figure out which continuation to do next. The parallel step uses “green threads” (also known as coöperative multitasking): it records logical thread (~ branch) names for various actions, but does not literally run them simultaneously.
由于所有调用都在同一个线程中，ReentrantLock重新进入
在我的例子中，我能够通过使用 AtomicInteger 和重新创建目录的一些符号链接技巧来绕过锁定的需要。
import java.util.concurrent.atomic.AtomicInteger
shellCounter = new AtomicInteger()
@NonCPS
def getNextShellCounter() {
return shellCounter.incrementAndGet()
}
...
def threadShellIndex = getNextShellCounter()
// concurrency-safe creation of output dir and removal of previous
def shellDir = "shell.${currentBuild.number}"
sh """#!/bin/bash +x
mkdir -p ${shellDir}
ln -sfn ${shellDir} shell
rm -rf shell.foo `ls -d shell.* | sed -e '/^${shellDir}$/d' `
"""
这里线程的工作方式，一个简单的 ++ 可能有相同的功能，但安全总比遗憾好。
我似乎无法让这种并发模式按预期在我的 Jenkins 管道脚本中工作。我已经尽可能地简化了场景，结果仍然没有意义。这是整个 Jenkins 文件：
import java.util.concurrent.locks.ReentrantLock
// create lock and index vars to make sure concurrent threads write different output files
shellLock = new ReentrantLock()
shellIndex = 0
def doIt() {
shellLock.lock()
def threadShellIndex = ++shellIndex
if (threadShellIndex == 1) {
sh("rm -rf shell")
sh("mkdir shell")
}
shellLock.unlock()
sh "touch shell/${threadShellIndex}"
}
node {
checkout scm
runs = [
"1": { doIt() },
"2": { doIt() },
]
parallel runs
}
这并没有像我预期的那样，在两个 运行 继续在其中创建文件之前删除并替换 "shell" 目录。相反，控制台输出是：
[Pipeline] parallel
[Pipeline] { (Branch: 1)
[Pipeline] { (Branch: 2)
[Pipeline] sh
[Pipeline] sh
[1] + rm -rf shell
[Pipeline] sh
[2] + touch shell/2
touch: cannot touch ‘shell/2’: No such file or directory
[Pipeline] }
Failed in branch 2
[1] + mkdir shell
[Pipeline] }
Failed in branch 1
[Pipeline] // parallel
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Also: hudson.AbortException: script returned exit code 1
at org.jenkinsci.plugins.workflow.steps.durable_task.DurableTaskStep$Execution.handleExit(DurableTaskStep.java:658)
at org.jenkinsci.plugins.workflow.steps.durable_task.DurableTaskStep$Execution.check(DurableTaskStep.java:604)
at org.jenkinsci.plugins.workflow.steps.durable_task.DurableTaskStep$Execution.run(DurableTaskStep.java:548)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access1(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
java.lang.IllegalMonitorStateException
at java.util.concurrent.locks.ReentrantLock$Sync.tryRelease(ReentrantLock.java:151)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.release(AbstractQueuedSynchronizer.java:1261)
at java.util.concurrent.locks.ReentrantLock.unlock(ReentrantLock.java:457)
at sun.reflect.GeneratedMethodAccessor1111.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1213)
at groovy.lang.ExpandoMetaClass.invokeMethod(ExpandoMetaClass.java:1125)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022)
at org.codehaus.groovy.runtime.callsite.PojoMetaClassSite.call(PojoMetaClassSite.java:47)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
at org.kohsuke.groovy.sandbox.impl.Checker.call(Checker.java:163)
at org.kohsuke.groovy.sandbox.GroovyInterceptor.onMethodCall(GroovyInterceptor.java:23)
at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onMethodCall(SandboxInterceptor.java:157)
at org.kohsuke.groovy.sandbox.impl.Checker.call(Checker.java:161)
at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:165)
at com.cloudbees.groovy.cps.sandbox.SandboxInvoker.methodCall(SandboxInvoker.java:17)
at WorkflowScript.doIt(WorkflowScript:14)
at WorkflowScript.run(WorkflowScript:22)
at ___cps.transform___(Native Method)
at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:86)
at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:113)
at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixName(FunctionCallBlock.java:78)
at sun.reflect.GeneratedMethodAccessor109.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)
at com.cloudbees.groovy.cps.impl.ConstantBlock.eval(ConstantBlock.java:21)
at com.cloudbees.groovy.cps.Next.step(Next.java:83)
at com.cloudbees.groovy.cps.Continuable.call(Continuable.java:174)
at com.cloudbees.groovy.cps.Continuable.call(Continuable.java:163)
at org.codehaus.groovy.runtime.GroovyCategorySupport$ThreadCategoryInfo.use(GroovyCategorySupport.java:129)
at org.codehaus.groovy.runtime.GroovyCategorySupport.use(GroovyCategorySupport.java:268)
at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:163)
at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.access[=11=]1(SandboxContinuable.java:18)
at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:51)
at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:185)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:400)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access0(CpsThreadGroup.java:96)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.call(CpsThreadGroup.java:312)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.call(CpsThreadGroup.java:276)
at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService.call(CpsVmExecutorService.java:67)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at hudson.remoting.SingleLaneExecutorService.run(SingleLaneExecutorService.java:131)
at jenkins.util.ContextResettingExecutorService.run(ContextResettingExecutorService.java:28)
at jenkins.security.ImpersonatingExecutorService.run(ImpersonatingExecutorService.java:59)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Finished: FAILURE
如何让 运行 2 等到 运行 1 重新创建目录？
似乎ReentrantLock没有效果的原因是管道groovy实际上不是多线程的。它使用单个线程来托管多个 "green" 线程。
来自https://github.com/jenkinsci/workflow-cps-plugin：
All program logic is run inside a “CPS VM thread”, which is just a Java thread pool that can run binary methods and figure out which continuation to do next. The parallel step uses “green threads” (also known as coöperative multitasking): it records logical thread (~ branch) names for various actions, but does not literally run them simultaneously.
由于所有调用都在同一个线程中，ReentrantLock重新进入
在我的例子中，我能够通过使用 AtomicInteger 和重新创建目录的一些符号链接技巧来绕过锁定的需要。
import java.util.concurrent.atomic.AtomicInteger
shellCounter = new AtomicInteger()
@NonCPS
def getNextShellCounter() {
return shellCounter.incrementAndGet()
}
...
def threadShellIndex = getNextShellCounter()
// concurrency-safe creation of output dir and removal of previous
def shellDir = "shell.${currentBuild.number}"
sh """#!/bin/bash +x
mkdir -p ${shellDir}
ln -sfn ${shellDir} shell
rm -rf shell.foo `ls -d shell.* | sed -e '/^${shellDir}$/d' `
"""
这里线程的工作方式，一个简单的 ++ 可能有相同的功能，但安全总比遗憾好。