多线程应用程序中的 NTLM 身份验证失败
NTLM Authentication failing in MultiThreaded application
我一直在尝试将一些代码放在一起，这些代码可以将文件上传到使用 NTLM 身份验证的 Sharepoint 站点。该代码的早期版本是单线程的，并且运行良好。他们完全按照预期上传了文件，没有丝毫问题。然而，我最终尝试对这个应用程序进行多线程处理，以便它可以一次上传许多文件，同时继续处理其余的业务。
然而，当我尝试对代码进行多线程处理时，它每次都失败，抛出 IndexOutOfBoundsException。这对我诊断问题的实际原因毫无帮助。
如果您想知道，如果我将 CachedThreadExecutor 更改为 SingleThreadExecutor - 强制代码进入单线程状态 - 它再次正常工作。
正在创建执行器和连接管理器，并构造线程：
class OrderProcessor implements Runnable {
//Other variables for object
private final ExecutorService executorService = Executors
.newCachedThreadPool();
// .newSingleThreadExecutor();
private HttpClientConnectionManager conManager;
private void setup() {
//always called before execution of anything else in object
conManager = new PoolingHttpClientConnectionManager();
}
//lots of other code
}
提交线程的实际代码很复杂，所以这个版本有些简化，但抓住了要点。
for(Request request : requests){
//Do other stuff
simpleSubmitFile(request);
//Do other stuff
}
这里是简化的文件提交方法
public Future<Boolean> simpleSubmitFile(Request request){
transferer = new SharePointTransferer(extractionRequest, conManager);
Future<Boolean> future = executorService.submit(transferer);
return future;
}
SharePointTransferer 代码
//actual values scrubbed
private final String USERNAME = "";
private final String PASSWORD = "";
private final String DOMAIN = "";
private final File sourceFile;
private final String destinationAddress;
private final CloseableHttpClient client;
public SharePointTransferer(final Request extractionRequest, HttpClientConnectionManager conManager) {
super(extractionRequest);
this.sourceFile = this.extractionRequest.getFile();
this.destinationAddress = this.extractionRequest.getDestinationAddress();
this.client = HttpClients.custom()
.setConnectionManager(conManager).build();
}
public Boolean call() throws Exception {
String httpAddress = correctSharePointAddress(destinationAddress);
HttpPut put = new HttpPut(httpAddress + sourceFile.getName());
// construct basic request
put.setEntity(new FileEntity(sourceFile));
HttpClientContext context = HttpClientContext.create();
// set credentials for the SharePoint login
CredentialsProvider credProvider = new BasicCredentialsProvider();
credProvider.setCredentials(AuthScope.ANY, new NTCredentials(USERNAME,
PASSWORD, "", DOMAIN));
context.setCredentialsProvider(credProvider);
// execute request
try {
HttpResponse response = client.execute(put, context);
logger.info("response code was: "
+ response.getStatusLine().getStatusCode());
if (response.getStatusLine().getStatusCode() != 201) {
throw new FileTransferException(
"Could not upload file. Http response code 201 expected."
+ "\nActual status code: "
+ response.getStatusLine().getStatusCode());
}
} catch (ClientProtocolException e) {
throw new FileTransferException(
"Exception Occurred while Transferring file "
+ sourceFile.getName(), e);
} catch (IOException e) {
throw new FileTransferException(
"Exception Occurred while Transferring file "
+ sourceFile.getName(), e);
}finally{
logger.info("deleting source file: " + sourceFile.getName());
sourceFile.delete();
client.close();
}
logger.info("successfully transfered file: "+sourceFile.getName());
return true;
}
如果我提交多个文件，它会为所有文件抛出基本上完全相同的异常。痕迹在下面
异常堆栈跟踪
2015-04-16 11:49:26 ERROR OrderProcessor:224 - error processing file: FILE_NAME_SCRUBBED
PACKAGE_SCRUBBED.FileProcessingException: Could not process file: FILE_NAME_SCRUBBED
at PACKAGE_SCRUBBED.OrderProcessor.finishProcessingOrder(OrderProcessor.java:223)
at PACKAGE_SCRUBBED.OrderProcessor.run(OrderProcessor.java:124)
at PACKAGE_SCRUBBED.FileTransferDaemon.process(FileTransferDaemon.java:48)
at PACKAGE_SCRUBBED.FileTransferDaemon.start(FileTransferDaemon.java:83)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.commons.daemon.support.DaemonLoader.start(DaemonLoader.java:243)
Caused by: java.util.concurrent.ExecutionException: java.lang.ArrayIndexOutOfBoundsException: 41
at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
at java.util.concurrent.FutureTask.get(FutureTask.java:83)
at PACKAGE_SCRUBBED.OrderProcessor.finishProcessingOrder(OrderProcessor.java:208)
... 8 more
Caused by: java.lang.ArrayIndexOutOfBoundsException: 41
at org.apache.http.impl.auth.NTLMEngineImpl$NTLMMessage.addByte(NTLMEngineImpl.java:924)
at org.apache.http.impl.auth.NTLMEngineImpl$NTLMMessage.addUShort(NTLMEngineImpl.java:946)
at org.apache.http.impl.auth.NTLMEngineImpl$Type1Message.getResponse(NTLMEngineImpl.java:1052)
at org.apache.http.impl.auth.NTLMEngineImpl.getType1Message(NTLMEngineImpl.java:148)
at org.apache.http.impl.auth.NTLMEngineImpl.generateType1Msg(NTLMEngineImpl.java:1641)
at org.apache.http.impl.auth.NTLMScheme.authenticate(NTLMScheme.java:139)
at org.apache.http.impl.auth.AuthSchemeBase.authenticate(AuthSchemeBase.java:138)
at org.apache.http.impl.auth.HttpAuthenticator.doAuth(HttpAuthenticator.java:239)
at org.apache.http.impl.auth.HttpAuthenticator.generateAuthResponse(HttpAuthenticator.java:202)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:262)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at PACKAGE_SCRUBBED.SharePointTransferer.call(SharePointTransferer.java:74)
at PACKAGE_SCRUBBED.SharePointTransferer.call(SharePointTransferer.java:1)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
如果有人能找出导致此问题的原因，我将不胜感激。
编辑：我设法找到了解决问题的解决方法，但仍然希望能解释到底发生了什么。
我最终通过将每个路由的连接数设置为 1 来解决这个问题，如下所示。
conManager.setDefaultMaxPerRoute(1);
我仍然不确定为什么会出现问题，或者解决这个问题的正确方法是什么，但这个解决方案对我有用。
您不能在并发环境中重用 NTLMScheme 和 HttpClientContext，因为它们都被标记为 @NotThreadSafe（请参阅 javadoc）。
在我的环境中，我遇到了同样的错误，解决方法如下：
synchronized(context) {
HttpResponse response = client.execute(put, context);
}
经过身份验证的上下文被重用，但一次只有一个线程。
这是一个错误，已在 httpclient 版本 4.5.2 中解决
http://www.apache.org/dist/httpcomponents/httpclient/RELEASE_NOTES-4.5.x.txt
4.5.2 版
变更日志：
- [HTTPCLIENT-1715] NTLMEngineImpl#Type1Message 不是线程安全的，但声明为常量。由 Olivier Lafontaine 和 Gary Gregory 提供
我一直在尝试将一些代码放在一起，这些代码可以将文件上传到使用 NTLM 身份验证的 Sharepoint 站点。该代码的早期版本是单线程的，并且运行良好。他们完全按照预期上传了文件，没有丝毫问题。然而，我最终尝试对这个应用程序进行多线程处理，以便它可以一次上传许多文件，同时继续处理其余的业务。
然而，当我尝试对代码进行多线程处理时，它每次都失败，抛出 IndexOutOfBoundsException。这对我诊断问题的实际原因毫无帮助。
如果您想知道，如果我将 CachedThreadExecutor 更改为 SingleThreadExecutor - 强制代码进入单线程状态 - 它再次正常工作。
正在创建执行器和连接管理器，并构造线程：
class OrderProcessor implements Runnable {
//Other variables for object
private final ExecutorService executorService = Executors
.newCachedThreadPool();
// .newSingleThreadExecutor();
private HttpClientConnectionManager conManager;
private void setup() {
//always called before execution of anything else in object
conManager = new PoolingHttpClientConnectionManager();
}
//lots of other code
}
提交线程的实际代码很复杂，所以这个版本有些简化，但抓住了要点。
for(Request request : requests){
//Do other stuff
simpleSubmitFile(request);
//Do other stuff
}
这里是简化的文件提交方法
public Future<Boolean> simpleSubmitFile(Request request){
transferer = new SharePointTransferer(extractionRequest, conManager);
Future<Boolean> future = executorService.submit(transferer);
return future;
}
SharePointTransferer 代码
//actual values scrubbed
private final String USERNAME = "";
private final String PASSWORD = "";
private final String DOMAIN = "";
private final File sourceFile;
private final String destinationAddress;
private final CloseableHttpClient client;
public SharePointTransferer(final Request extractionRequest, HttpClientConnectionManager conManager) {
super(extractionRequest);
this.sourceFile = this.extractionRequest.getFile();
this.destinationAddress = this.extractionRequest.getDestinationAddress();
this.client = HttpClients.custom()
.setConnectionManager(conManager).build();
}
public Boolean call() throws Exception {
String httpAddress = correctSharePointAddress(destinationAddress);
HttpPut put = new HttpPut(httpAddress + sourceFile.getName());
// construct basic request
put.setEntity(new FileEntity(sourceFile));
HttpClientContext context = HttpClientContext.create();
// set credentials for the SharePoint login
CredentialsProvider credProvider = new BasicCredentialsProvider();
credProvider.setCredentials(AuthScope.ANY, new NTCredentials(USERNAME,
PASSWORD, "", DOMAIN));
context.setCredentialsProvider(credProvider);
// execute request
try {
HttpResponse response = client.execute(put, context);
logger.info("response code was: "
+ response.getStatusLine().getStatusCode());
if (response.getStatusLine().getStatusCode() != 201) {
throw new FileTransferException(
"Could not upload file. Http response code 201 expected."
+ "\nActual status code: "
+ response.getStatusLine().getStatusCode());
}
} catch (ClientProtocolException e) {
throw new FileTransferException(
"Exception Occurred while Transferring file "
+ sourceFile.getName(), e);
} catch (IOException e) {
throw new FileTransferException(
"Exception Occurred while Transferring file "
+ sourceFile.getName(), e);
}finally{
logger.info("deleting source file: " + sourceFile.getName());
sourceFile.delete();
client.close();
}
logger.info("successfully transfered file: "+sourceFile.getName());
return true;
}
如果我提交多个文件，它会为所有文件抛出基本上完全相同的异常。痕迹在下面 异常堆栈跟踪
2015-04-16 11:49:26 ERROR OrderProcessor:224 - error processing file: FILE_NAME_SCRUBBED
PACKAGE_SCRUBBED.FileProcessingException: Could not process file: FILE_NAME_SCRUBBED
at PACKAGE_SCRUBBED.OrderProcessor.finishProcessingOrder(OrderProcessor.java:223)
at PACKAGE_SCRUBBED.OrderProcessor.run(OrderProcessor.java:124)
at PACKAGE_SCRUBBED.FileTransferDaemon.process(FileTransferDaemon.java:48)
at PACKAGE_SCRUBBED.FileTransferDaemon.start(FileTransferDaemon.java:83)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.commons.daemon.support.DaemonLoader.start(DaemonLoader.java:243)
Caused by: java.util.concurrent.ExecutionException: java.lang.ArrayIndexOutOfBoundsException: 41
at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
at java.util.concurrent.FutureTask.get(FutureTask.java:83)
at PACKAGE_SCRUBBED.OrderProcessor.finishProcessingOrder(OrderProcessor.java:208)
... 8 more
Caused by: java.lang.ArrayIndexOutOfBoundsException: 41
at org.apache.http.impl.auth.NTLMEngineImpl$NTLMMessage.addByte(NTLMEngineImpl.java:924)
at org.apache.http.impl.auth.NTLMEngineImpl$NTLMMessage.addUShort(NTLMEngineImpl.java:946)
at org.apache.http.impl.auth.NTLMEngineImpl$Type1Message.getResponse(NTLMEngineImpl.java:1052)
at org.apache.http.impl.auth.NTLMEngineImpl.getType1Message(NTLMEngineImpl.java:148)
at org.apache.http.impl.auth.NTLMEngineImpl.generateType1Msg(NTLMEngineImpl.java:1641)
at org.apache.http.impl.auth.NTLMScheme.authenticate(NTLMScheme.java:139)
at org.apache.http.impl.auth.AuthSchemeBase.authenticate(AuthSchemeBase.java:138)
at org.apache.http.impl.auth.HttpAuthenticator.doAuth(HttpAuthenticator.java:239)
at org.apache.http.impl.auth.HttpAuthenticator.generateAuthResponse(HttpAuthenticator.java:202)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:262)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at PACKAGE_SCRUBBED.SharePointTransferer.call(SharePointTransferer.java:74)
at PACKAGE_SCRUBBED.SharePointTransferer.call(SharePointTransferer.java:1)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
如果有人能找出导致此问题的原因，我将不胜感激。
编辑：我设法找到了解决问题的解决方法，但仍然希望能解释到底发生了什么。
我最终通过将每个路由的连接数设置为 1 来解决这个问题，如下所示。
conManager.setDefaultMaxPerRoute(1);
我仍然不确定为什么会出现问题，或者解决这个问题的正确方法是什么，但这个解决方案对我有用。
您不能在并发环境中重用 NTLMScheme 和 HttpClientContext，因为它们都被标记为 @NotThreadSafe（请参阅 javadoc）。 在我的环境中，我遇到了同样的错误，解决方法如下：
synchronized(context) {
HttpResponse response = client.execute(put, context);
}
经过身份验证的上下文被重用，但一次只有一个线程。
这是一个错误，已在 httpclient 版本 4.5.2 中解决
http://www.apache.org/dist/httpcomponents/httpclient/RELEASE_NOTES-4.5.x.txt
4.5.2 版
变更日志：
- [HTTPCLIENT-1715] NTLMEngineImpl#Type1Message 不是线程安全的，但声明为常量。由 Olivier Lafontaine 和 Gary Gregory 提供