Solr 导入命令不起作用
Solr import command not working
我需要你的帮助来解决一个大问题。我必须用 solr 索引一些来自 EMC xDB 源的数据。
我将 xDB 配置为像 RDBMS 一样被请求。然后我将 Squirrel 配置为请求 xDB。它有效，例如我尝试时：
Select * from books
我使用与 squirrel 相同的驱动程序和相同的连接字符串配置 solr。
这是我的数据-config.xml:
<dataConfig>
<dataSource type="JdbcDataSource" driver="com.emc.ia.xdbjdbc.Driver" url="jdbc:xdbeas://localhost:1235/MaDatabase" user="*****" password="*****"/>
<document name="books">
<entity name="book" query="select * from books">
<field column="author" name="author" />
<field column="price" name="price" />
<field column="name" name="name" />
<field column="description" name="description" />
<field column="title" name="title" />
<field column="publish_date" name="publish_date" />
</entity>
</document>
</dataConfig>
solrconfig.xml
<requestHandler name="/dataimport" class="org.apache.solr.handler.dataimport.DataImportHandler">
<lst name="defaults">
<str name="config">C:\solr\solr-5.2.1\server\solr\jcg\conf\data-config.xml</str>
</lst>
</requestHandler>
我的 xDB 书籍-MetaData.xml:
<?xml version="1.0" encoding="UTF-16"?>
<TABLEMETADATA TABLENAME="books">
<RECORD-COUNT>2</RECORD-COUNT>
<ROOT-ELEMENT>books</ROOT-ELEMENT>
<ROW-ELEMENT>book</ROW-ELEMENT>
<DATATYPES>
<BOOKS-TYPE>
<NAME>author</NAME>
<COLNO>1</COLNO>
<DATATYPE>VARCHAR2(75)</DATATYPE>
</BOOKS-TYPE>
<BOOKS-TYPE>
<NAME>title</NAME>
<COLNO>2</COLNO>
<DATATYPE>VARCHAR2(75)</DATATYPE>
</BOOKS-TYPE>
<BOOKS-TYPE>
<NAME>genre</NAME>
<COLNO>3</COLNO>
<DATATYPE>VARCHAR2(25)</DATATYPE>
</BOOKS-TYPE>
<BOOKS-TYPE>
<NAME>price</NAME>
<COLNO>4</COLNO>
<DATATYPE>VARCHAR2(25)</DATATYPE>
</BOOKS-TYPE>
<BOOKS-TYPE>
<NAME>publish_date</NAME>
<COLNO>5</COLNO>
<DATATYPE>DATETIME</DATATYPE>
</BOOKS-TYPE>
<BOOKS-TYPE>
<NAME>description</NAME>
<COLNO>6</COLNO>
<DATATYPE>VARCHAR2(350)</DATATYPE>
</BOOKS-TYPE>
</DATATYPES>
</TABLEMETADATA>
部分 solr 响应：
"verbose-output": [
"entity:book",
[
"document#1",
[
"query",
"select * from books",
"time-taken",
"0:0:0.114",
"EXCEPTION",
"org.apache.solr.handler.dataimport.DataImportHandlerException: Error reading data from database Processing Document # 1
当我检查 solr 日志时，我有 3 个异常。
数字 1：
getNext() failed for query 'select * from books':org.apache.solr.handler.dataimport.DataImportHandlerException: Error reading data from database Processing Document # 1
at org.apache.solr.handler.dataimport.DataImportHandlerException.wrapAndThrow(DataImportHandlerException.java:70)
at org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.getARow(JdbcDataSource.java:398)
at org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.access0(JdbcDataSource.java:296)
at org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.next(JdbcDataSource.java:336)
at org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.next(JdbcDataSource.java:328)
at org.apache.solr.handler.dataimport.EntityProcessorBase.getNext(EntityProcessorBase.java:133)
at org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEntityProcessor.java:74)
at org.apache.solr.handler.dataimport.EntityProcessorWrapper.nextRow(EntityProcessorWrapper.java:243)
at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:475)
at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:414)
at org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:329)
at org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:232)
at org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:416)
at org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:480)
at org.apache.solr.handler.dataimport.DataImportHandler.handleRequestBody(DataImportHandler.java:185)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:2064)
at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:654)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:450)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:227)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:196)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
at org.eclipse.jetty.server.Server.handle(Server.java:497)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
at org.eclipse.jetty.io.AbstractConnection.run(AbstractConnection.java:540)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
at org.eclipse.jetty.util.thread.QueuedThreadPool.run(QueuedThreadPool.java:555)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.sql.SQLFeatureNotSupportedException: Method com.emc.ia.xdbjdbc.xdb.XdbResultSet.getObject-columnLabel is not yet implemented.
at com.emc.ia.xdbjdbc.Driver.notImplemented(Driver.java:745)
at com.emc.ia.xdbjdbc.xdb.XdbResultSet.getObject(XdbResultSet.java:659)
at org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.getARow(JdbcDataSource.java:358)
... 39 more
2号：
Exception while processing: book document : SolrInputDocument(fields: []):org.apache.solr.handler.dataimport.DataImportHandlerException: Error reading data from database Processing Document # 1
at org.apache.solr.handler.dataimport.DataImportHandlerException.wrapAndThrow(DataImportHandlerException.java:70)
at org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.getARow(JdbcDataSource.java:398)
at org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.access0(JdbcDataSource.java:296)
at org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.next(JdbcDataSource.java:336)
at org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.next(JdbcDataSource.java:328)
at org.apache.solr.handler.dataimport.EntityProcessorBase.getNext(EntityProcessorBase.java:133)
at org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEntityProcessor.java:74)
at org.apache.solr.handler.dataimport.EntityProcessorWrapper.nextRow(EntityProcessorWrapper.java:243)
at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:475)
at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:414)
at org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:329)
at org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:232)
at org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:416)
at org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:480)
at org.apache.solr.handler.dataimport.DataImportHandler.handleRequestBody(DataImportHandler.java:185)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:2064)
at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:654)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:450)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:227)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:196)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
at org.eclipse.jetty.server.Server.handle(Server.java:497)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
at org.eclipse.jetty.io.AbstractConnection.run(AbstractConnection.java:540)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
at org.eclipse.jetty.util.thread.QueuedThreadPool.run(QueuedThreadPool.java:555)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.sql.SQLFeatureNotSupportedException: Method com.emc.ia.xdbjdbc.xdb.XdbResultSet.getObject-columnLabel is not yet implemented.
at com.emc.ia.xdbjdbc.Driver.notImplemented(Driver.java:745)
at com.emc.ia.xdbjdbc.xdb.XdbResultSet.getObject(XdbResultSet.java:659)
at org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.getARow(JdbcDataSource.java:358)
... 39 more
最后一个：
Full Import failed:java.lang.RuntimeException: java.lang.RuntimeException: org.apache.solr.handler.dataimport.DataImportHandlerException: Error reading data from database Processing Document # 1
at org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:270)
at org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:416)
at org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:480)
at org.apache.solr.handler.dataimport.DataImportHandler.handleRequestBody(DataImportHandler.java:185)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:2064)
at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:654)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:450)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:227)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:196)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
at org.eclipse.jetty.server.Server.handle(Server.java:497)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
at org.eclipse.jetty.io.AbstractConnection.run(AbstractConnection.java:540)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
at org.eclipse.jetty.util.thread.QueuedThreadPool.run(QueuedThreadPool.java:555)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: org.apache.solr.handler.dataimport.DataImportHandlerException: Error reading data from database Processing Document # 1
at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:416)
at org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:329)
at org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:232)
... 29 more
Caused by: org.apache.solr.handler.dataimport.DataImportHandlerException: Error reading data from database Processing Document # 1
at org.apache.solr.handler.dataimport.DataImportHandlerException.wrapAndThrow(DataImportHandlerException.java:70)
at org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.getARow(JdbcDataSource.java:398)
at org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.access0(JdbcDataSource.java:296)
at org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.next(JdbcDataSource.java:336)
at org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.next(JdbcDataSource.java:328)
at org.apache.solr.handler.dataimport.EntityProcessorBase.getNext(EntityProcessorBase.java:133)
at org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEntityProcessor.java:74)
at org.apache.solr.handler.dataimport.EntityProcessorWrapper.nextRow(EntityProcessorWrapper.java:243)
at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:475)
at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:414)
... 31 more
Caused by: java.sql.SQLFeatureNotSupportedException: Method com.emc.ia.xdbjdbc.xdb.XdbResultSet.getObject-columnLabel is not yet implemented.
at com.emc.ia.xdbjdbc.Driver.notImplemented(Driver.java:745)
at com.emc.ia.xdbjdbc.xdb.XdbResultSet.getObject(XdbResultSet.java:659)
at org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.getARow(JdbcDataSource.java:358)
... 39 more
提前感谢您的帮助，因为我真的不明白为什么它适用于松鼠而不适用于 solr。
编辑问题以回复@Cheffe
确实是xDB驱动问题，是getObject()方法引起的。我查看了 squirrel 代码，它可以工作，因为它使用列的索引而不是它的名称。
int idx = _columnIndices != null ? _columnIndices[i] : i + 1;
try
{
final int columnType = _rsmd.getColumnType(idx);
//final String columnClassName = _rsmd.getColumnClassName(idx);
switch (columnType)
{
case Types.NULL:
row[i] = null;
break;
// TODO: When JDK1.4 is the earliest JDK supported
// by Squirrel then remove the hardcoding of the
// boolean data type.
case Types.BIT:
case 16:
//案例Types.BOOLEAN：
行[i] = _rs.getObject(idx);
.
.
.
无论如何它都适用于
convertType="true"
检查 Solr 源后：似乎该数据库的驱动程序可能不适合 运行 与 Solr 的 DIH。不幸的是，该数据库是专有的，因此我无法尝试。
异常是从您的供应商的驱动程序中抛出的。所有三个根本原因都存在于此。当您向下滚动到堆栈跟踪的最底部时，您会看到这一点。所有这些堆栈跟踪都告诉你
Method xxx.getObject-columnLabel is not yet implemented.
这意味着：您的数据库供应商 EMC 没有在他们的 JDBC 驱动程序中实现此方法。如果您不想弄脏您的手，您应该向他们提交错误报告或功能请求，说明您需要实现此功能。
您可能有一个镜头是堆栈跟踪中记录 Solr 来源的最后一行导致 the internal getARow method
private Map<String, Object> getARow() {
if (resultSet == null)
return null;
Map<String, Object> result = new HashMap<>();
for (String colName : colNames) {
try {
if (!convertType) {
Object value = resultSet.getObject(colName); // <-- here the exception takes of
if (value instanceof BigDecimal || value instanceof BigInteger) {
result.put(colName, value.toString());
} else {
result.put(colName, value);
}
continue;
}
// more code there, but left out
您可以通过设置 dataimporthandler 的配置参数来绕过 resultSet.getObject 的调用。即这是
convertType (default: false) – Applies an additional conversion from the field type returned by the database to the field type defined in the schema.xml. The default value seems to be safer, because it does not cause extra, magical conversion. However, in special cases (eg BLOB fields), that conversion is one of the ways of solving the problem. [Reference]
您需要在 data-config.xml
中执行此操作
<dataConfig>
<dataSource convertType="true" type="JdbcDataSource" driver="com.emc.ia.xdbjdbc.Driver" url="jdbc:xdbeas://localhost:1235/MaDatabase" user="*****" password="*****"/>
但是我无法告诉您这对您的数据库及其驱动程序是否成功。可能这只会导致另一个异常。在这种情况下，您将需要回退到解决方案
- 让你的供应商去做
- 编写自定义数据导入例程
我需要你的帮助来解决一个大问题。我必须用 solr 索引一些来自 EMC xDB 源的数据。 我将 xDB 配置为像 RDBMS 一样被请求。然后我将 Squirrel 配置为请求 xDB。它有效，例如我尝试时：
Select * from books
我使用与 squirrel 相同的驱动程序和相同的连接字符串配置 solr。
这是我的数据-config.xml:
<dataConfig>
<dataSource type="JdbcDataSource" driver="com.emc.ia.xdbjdbc.Driver" url="jdbc:xdbeas://localhost:1235/MaDatabase" user="*****" password="*****"/>
<document name="books">
<entity name="book" query="select * from books">
<field column="author" name="author" />
<field column="price" name="price" />
<field column="name" name="name" />
<field column="description" name="description" />
<field column="title" name="title" />
<field column="publish_date" name="publish_date" />
</entity>
</document>
</dataConfig>
solrconfig.xml
<requestHandler name="/dataimport" class="org.apache.solr.handler.dataimport.DataImportHandler">
<lst name="defaults">
<str name="config">C:\solr\solr-5.2.1\server\solr\jcg\conf\data-config.xml</str>
</lst>
</requestHandler>
我的 xDB 书籍-MetaData.xml:
<?xml version="1.0" encoding="UTF-16"?>
<TABLEMETADATA TABLENAME="books">
<RECORD-COUNT>2</RECORD-COUNT>
<ROOT-ELEMENT>books</ROOT-ELEMENT>
<ROW-ELEMENT>book</ROW-ELEMENT>
<DATATYPES>
<BOOKS-TYPE>
<NAME>author</NAME>
<COLNO>1</COLNO>
<DATATYPE>VARCHAR2(75)</DATATYPE>
</BOOKS-TYPE>
<BOOKS-TYPE>
<NAME>title</NAME>
<COLNO>2</COLNO>
<DATATYPE>VARCHAR2(75)</DATATYPE>
</BOOKS-TYPE>
<BOOKS-TYPE>
<NAME>genre</NAME>
<COLNO>3</COLNO>
<DATATYPE>VARCHAR2(25)</DATATYPE>
</BOOKS-TYPE>
<BOOKS-TYPE>
<NAME>price</NAME>
<COLNO>4</COLNO>
<DATATYPE>VARCHAR2(25)</DATATYPE>
</BOOKS-TYPE>
<BOOKS-TYPE>
<NAME>publish_date</NAME>
<COLNO>5</COLNO>
<DATATYPE>DATETIME</DATATYPE>
</BOOKS-TYPE>
<BOOKS-TYPE>
<NAME>description</NAME>
<COLNO>6</COLNO>
<DATATYPE>VARCHAR2(350)</DATATYPE>
</BOOKS-TYPE>
</DATATYPES>
</TABLEMETADATA>
部分 solr 响应：
"verbose-output": [
"entity:book",
[
"document#1",
[
"query",
"select * from books",
"time-taken",
"0:0:0.114",
"EXCEPTION",
"org.apache.solr.handler.dataimport.DataImportHandlerException: Error reading data from database Processing Document # 1
当我检查 solr 日志时，我有 3 个异常。 数字 1：
getNext() failed for query 'select * from books':org.apache.solr.handler.dataimport.DataImportHandlerException: Error reading data from database Processing Document # 1
at org.apache.solr.handler.dataimport.DataImportHandlerException.wrapAndThrow(DataImportHandlerException.java:70)
at org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.getARow(JdbcDataSource.java:398)
at org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.access0(JdbcDataSource.java:296)
at org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.next(JdbcDataSource.java:336)
at org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.next(JdbcDataSource.java:328)
at org.apache.solr.handler.dataimport.EntityProcessorBase.getNext(EntityProcessorBase.java:133)
at org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEntityProcessor.java:74)
at org.apache.solr.handler.dataimport.EntityProcessorWrapper.nextRow(EntityProcessorWrapper.java:243)
at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:475)
at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:414)
at org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:329)
at org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:232)
at org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:416)
at org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:480)
at org.apache.solr.handler.dataimport.DataImportHandler.handleRequestBody(DataImportHandler.java:185)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:2064)
at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:654)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:450)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:227)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:196)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
at org.eclipse.jetty.server.Server.handle(Server.java:497)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
at org.eclipse.jetty.io.AbstractConnection.run(AbstractConnection.java:540)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
at org.eclipse.jetty.util.thread.QueuedThreadPool.run(QueuedThreadPool.java:555)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.sql.SQLFeatureNotSupportedException: Method com.emc.ia.xdbjdbc.xdb.XdbResultSet.getObject-columnLabel is not yet implemented.
at com.emc.ia.xdbjdbc.Driver.notImplemented(Driver.java:745)
at com.emc.ia.xdbjdbc.xdb.XdbResultSet.getObject(XdbResultSet.java:659)
at org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.getARow(JdbcDataSource.java:358)
... 39 more
2号：
Exception while processing: book document : SolrInputDocument(fields: []):org.apache.solr.handler.dataimport.DataImportHandlerException: Error reading data from database Processing Document # 1
at org.apache.solr.handler.dataimport.DataImportHandlerException.wrapAndThrow(DataImportHandlerException.java:70)
at org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.getARow(JdbcDataSource.java:398)
at org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.access0(JdbcDataSource.java:296)
at org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.next(JdbcDataSource.java:336)
at org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.next(JdbcDataSource.java:328)
at org.apache.solr.handler.dataimport.EntityProcessorBase.getNext(EntityProcessorBase.java:133)
at org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEntityProcessor.java:74)
at org.apache.solr.handler.dataimport.EntityProcessorWrapper.nextRow(EntityProcessorWrapper.java:243)
at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:475)
at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:414)
at org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:329)
at org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:232)
at org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:416)
at org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:480)
at org.apache.solr.handler.dataimport.DataImportHandler.handleRequestBody(DataImportHandler.java:185)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:2064)
at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:654)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:450)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:227)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:196)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
at org.eclipse.jetty.server.Server.handle(Server.java:497)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
at org.eclipse.jetty.io.AbstractConnection.run(AbstractConnection.java:540)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
at org.eclipse.jetty.util.thread.QueuedThreadPool.run(QueuedThreadPool.java:555)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.sql.SQLFeatureNotSupportedException: Method com.emc.ia.xdbjdbc.xdb.XdbResultSet.getObject-columnLabel is not yet implemented.
at com.emc.ia.xdbjdbc.Driver.notImplemented(Driver.java:745)
at com.emc.ia.xdbjdbc.xdb.XdbResultSet.getObject(XdbResultSet.java:659)
at org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.getARow(JdbcDataSource.java:358)
... 39 more
最后一个：
Full Import failed:java.lang.RuntimeException: java.lang.RuntimeException: org.apache.solr.handler.dataimport.DataImportHandlerException: Error reading data from database Processing Document # 1
at org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:270)
at org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:416)
at org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:480)
at org.apache.solr.handler.dataimport.DataImportHandler.handleRequestBody(DataImportHandler.java:185)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:2064)
at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:654)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:450)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:227)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:196)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
at org.eclipse.jetty.server.Server.handle(Server.java:497)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
at org.eclipse.jetty.io.AbstractConnection.run(AbstractConnection.java:540)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
at org.eclipse.jetty.util.thread.QueuedThreadPool.run(QueuedThreadPool.java:555)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: org.apache.solr.handler.dataimport.DataImportHandlerException: Error reading data from database Processing Document # 1
at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:416)
at org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:329)
at org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:232)
... 29 more
Caused by: org.apache.solr.handler.dataimport.DataImportHandlerException: Error reading data from database Processing Document # 1
at org.apache.solr.handler.dataimport.DataImportHandlerException.wrapAndThrow(DataImportHandlerException.java:70)
at org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.getARow(JdbcDataSource.java:398)
at org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.access0(JdbcDataSource.java:296)
at org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.next(JdbcDataSource.java:336)
at org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.next(JdbcDataSource.java:328)
at org.apache.solr.handler.dataimport.EntityProcessorBase.getNext(EntityProcessorBase.java:133)
at org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEntityProcessor.java:74)
at org.apache.solr.handler.dataimport.EntityProcessorWrapper.nextRow(EntityProcessorWrapper.java:243)
at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:475)
at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:414)
... 31 more
Caused by: java.sql.SQLFeatureNotSupportedException: Method com.emc.ia.xdbjdbc.xdb.XdbResultSet.getObject-columnLabel is not yet implemented.
at com.emc.ia.xdbjdbc.Driver.notImplemented(Driver.java:745)
at com.emc.ia.xdbjdbc.xdb.XdbResultSet.getObject(XdbResultSet.java:659)
at org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.getARow(JdbcDataSource.java:358)
... 39 more
提前感谢您的帮助，因为我真的不明白为什么它适用于松鼠而不适用于 solr。
编辑问题以回复@Cheffe
确实是xDB驱动问题，是getObject()方法引起的。我查看了 squirrel 代码，它可以工作，因为它使用列的索引而不是它的名称。
int idx = _columnIndices != null ? _columnIndices[i] : i + 1;
try
{
final int columnType = _rsmd.getColumnType(idx);
//final String columnClassName = _rsmd.getColumnClassName(idx);
switch (columnType)
{
case Types.NULL:
row[i] = null;
break;
// TODO: When JDK1.4 is the earliest JDK supported
// by Squirrel then remove the hardcoding of the
// boolean data type.
case Types.BIT:
case 16:
//案例Types.BOOLEAN： 行[i] = _rs.getObject(idx); . . .
无论如何它都适用于
convertType="true"
检查 Solr 源后：似乎该数据库的驱动程序可能不适合 运行 与 Solr 的 DIH。不幸的是，该数据库是专有的，因此我无法尝试。
异常是从您的供应商的驱动程序中抛出的。所有三个根本原因都存在于此。当您向下滚动到堆栈跟踪的最底部时，您会看到这一点。所有这些堆栈跟踪都告诉你
Method xxx.getObject-columnLabel is not yet implemented.
这意味着：您的数据库供应商 EMC 没有在他们的 JDBC 驱动程序中实现此方法。如果您不想弄脏您的手，您应该向他们提交错误报告或功能请求，说明您需要实现此功能。
您可能有一个镜头是堆栈跟踪中记录 Solr 来源的最后一行导致 the internal getARow method
private Map<String, Object> getARow() {
if (resultSet == null)
return null;
Map<String, Object> result = new HashMap<>();
for (String colName : colNames) {
try {
if (!convertType) {
Object value = resultSet.getObject(colName); // <-- here the exception takes of
if (value instanceof BigDecimal || value instanceof BigInteger) {
result.put(colName, value.toString());
} else {
result.put(colName, value);
}
continue;
}
// more code there, but left out
您可以通过设置 dataimporthandler 的配置参数来绕过 resultSet.getObject 的调用。即这是
convertType (default: false) – Applies an additional conversion from the field type returned by the database to the field type defined in the schema.xml. The default value seems to be safer, because it does not cause extra, magical conversion. However, in special cases (eg BLOB fields), that conversion is one of the ways of solving the problem. [Reference]
您需要在 data-config.xml
中执行此操作<dataConfig>
<dataSource convertType="true" type="JdbcDataSource" driver="com.emc.ia.xdbjdbc.Driver" url="jdbc:xdbeas://localhost:1235/MaDatabase" user="*****" password="*****"/>
但是我无法告诉您这对您的数据库及其驱动程序是否成功。可能这只会导致另一个异常。在这种情况下，您将需要回退到解决方案
- 让你的供应商去做
- 编写自定义数据导入例程