为 DS-2542(XOAI 日期粒度)修补 Dspace 不会导致行为发生变化

Patching Dspace for DS-2542 (XOAI date granularity) results in no change of behaviour

我正在尝试将补丁应用到我们的测试环境 运行ning Dspace 5.1(带有一些附加模块)以修复由于日期粒度不正确而导致的 OAI 收集问题。问题是 DS-2542 XOAI does not support non granular YYYY-MM-DD harvesting properly

环境

我是如何应用补丁的

首先我需要从 https://github.com/DSpace/DSpace/pull/912. That gives me the following patch (as a diff 中获取 Dspace (dspace-oai) 的补丁,我将其放入 /tmp/

diff --git a/dspace-oai/src/main/java/org/dspace/xoai/filter/DateUntilFilter.java b/dspace-oai/src/main/java/org/dspace/xoai/filter/DateUntilFilter.java
index 1995fc0..cdb17d9 100644
--- a/dspace-oai/src/main/java/org/dspace/xoai/filter/DateUntilFilter.java
+++ b/dspace-oai/src/main/java/org/dspace/xoai/filter/DateUntilFilter.java
@@ -50,6 +50,11 @@ public boolean isShown(DSpaceItem item)
     public SolrFilterResult buildSolrQuery()
     {
         String format = dateProvider.format(date).replace("Z", ".999Z"); // Tweak to set the millisecon
+        if (format.substring(11, 19).equals("00:00:00"))
+        {
+            format = format.substring(0, 11) + "23:59:59" + format.substring(19);
+        }
+
         return new SolrFilterResult("item.lastmodified:[* TO "
                 + ClientUtils.escapeQueryChars(format) + "]");
     }
diff --git a/dspace-oai/src/main/java/org/dspace/xoai/util/DateUtils.java b/dspace-oai/src/main/java/org/dspace/xoai/util/DateUtils.java
index e968414..b73955d 100644
--- a/dspace-oai/src/main/java/org/dspace/xoai/util/DateUtils.java
+++ b/dspace-oai/src/main/java/org/dspace/xoai/util/DateUtils.java
@@ -31,8 +31,7 @@ public static String format(Date date)
     }
     public static String format(Date date, boolean init)
     {
-       SimpleDateFormat sdf = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss.'000Z'");
-       if (!init) sdf = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss.'999Z'");
+        SimpleDateFormat sdf = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss'Z'");
         // We indicate that the returned date is in Zulu time (UTC) so we have
         // to set the time zone of sdf correct.
         sdf.setTimeZone(TimeZone.getTimeZone("ZULU"));

我已使用以下命令序列应用补丁:

sudo su - builder
  cd /usr/local/src/
    git clone 'https://github.com/lyncode/xoai.git' xoai
    cd xoai/
      git branch -a
      git checkout 3.x
      less pom.xml    <-- verify that com.lyncode version is 3.2.10-SNAPSHOT
      mvn package install
      cd ..
    cd dspace-src/
      patch -p1 < /tmp/473f2faaba99671b55372bcca1604aea2acf9601.diff    # (needed tweaking)
      vim dspace-oai/pom.xml   # change <xoai.version> from 3.2.9 to 3.2.10-SNAPSHOT
      mvn package -U -P \!dspace-jspui,\!dspace-lni,\!dspace-rest,\!dspace-swordv2,\!dspace-rdf,\!dspace-xmlui-mirage2
      exit
cd /usr/local/src/dspace-src/dspace/target/dspace-installer/
  ant update
  chown -Rc tomcat:tomcat /usr/local/dspace/
  service tomcat6 restart

测试方法

通过post-打补丁结果和我打补丁前一样。使用 2015-04-07 仍然失败,但打补丁后应该可以通过。 2015-04-07T00:00:00Z 工作相同并通过。

$ curl 'https://test-dspace.example.com/oai/request?verb=ListRecords&metadataPrefix=oai_dc&until=2015-04-07'
...<error code="badArgument">Invalid date given in until parameter</error></OAI-PMH>

$ curl 'https://test-dspace.example.com/oai/request?verb=ListRecords&metadataPrefix=oai_dc&until=2015-04-07T00:00:00Z'
... completeListSize="3731" cursor="0">oai_dc//2015-04-07T00:00:00.000Z//100</resumptionToken></ListRecords></OAI-PMH>

我已经看到并多次阅读 DSpace - Tech email entitled 'Building DSpace after bug fix (DS-2542)',尽管我已经根据该电子邮件记录做了我应该做的一切(包括 "proper" -- 编辑您的 POM -- 和 "quick and dirty" -- 更换 jar -- 其中列出的方法),问题仍然存在。

我尝试过的事情(都无济于事)

我现在在想/质疑什么

我确信 jar 正在按预期创建,并且安装良好,并且没有旧的 jar 在玩。

确信我认为应该 运行 的代码实际上得到了 运行,或者如果其他东西掩盖了代码。 .. 我不知道如何在 Tomcat.

中进行诊断

是否有一个命令(大概是dspace dsrun ...)我可以用来在Tomcat之外复制这个测试?

我应该指出我是系统工程师,而不是 Java 开发人员。

附录

日志设置

添加到 dspace.log

的 appender 部分末尾
log4j.logger.com.lyncode.xoai=DEBUG, A1
log4j.logger.org.dspace.xoai=DEBUG, A1

这让我看到如下日志:

2015-07-28 02:34:07,222 DEBUG org.dspace.xoai.services.impl.solr.DSpaceSolrServerResolver @ Solr Server Initialized

2015-07-28 02:34:07,233 DEBUG com.lyncode.xoai.dataprovider.OAIRequestParameters @ RootParameterMap 'until' = '2015-04-07'

如果我编辑 /usr/local/src/xoai/src/main/java/com/lyncode/xoai/dataprovider/OAIDataProvider.java,我可以把它变成

2015-07-28 02:50:15,374 DEBUG com.lyncode.xoai.dataprovider.OAIRequestParameters @ CAMERON-RootParameterMap 'until' = '2015-04-07'

但是将 CAMERON 添加到似乎唯一生成错误文本的地方的开头并没有显示测试输出的变化(这在同一目录的 OAIDataProvider.java 中):

throw new BadArgumentException("CAMERON Invalid date given in until parameter");

git dspace-src 目录差异(相对于预修补)

# git diff
diff --git a/build.properties b/build.properties
index 844cb93..09cb5c3 100644
--- a/build.properties
+++ b/build.properties
@@ -166,6 +166,7 @@ http.proxy.port = 3128
 loglevel.other = WARN
 # loglevel.other: Log level for other third-party tools/APIs used by DSpace
 # Possible values (from most to least info): DEBUG, INFO, WARN, ERROR, FATAL
-loglevel.dspace = INFO
+#loglevel.dspace = INFO
+loglevel.dspace = DEBUG
 # loglevel.dspace: Log level for all DSpace-specific code (org.dspace.*)
 # Possible values (from most to least info): DEBUG, INFO, WARN, ERROR, FATAL
diff --git a/dspace-oai/pom.xml b/dspace-oai/pom.xml
index 6e56a52..136a10f 100644
--- a/dspace-oai/pom.xml
+++ b/dspace-oai/pom.xml
@@ -16,7 +16,7 @@
         <!-- This is the path to the root [dspace-src] directory. -->
         <root.basedir>${basedir}/..</root.basedir>
         <spring.version>3.2.5.RELEASE</spring.version>
-        <xoai.version>3.2.9</xoai.version>
+        <xoai.version>3.2.10-SNAPSHOT</xoai.version>
         <jtwig.version>2.0.1</jtwig.version>
     </properties>

diff --git a/dspace-oai/src/main/java/org/dspace/xoai/filter/DateUntilFilter.java b/dspace-oai/src/main/java/org/dspace/xoai/filter/DateUntilFilter.java
index 1995fc0..cdb17d9 100644
--- a/dspace-oai/src/main/java/org/dspace/xoai/filter/DateUntilFilter.java
+++ b/dspace-oai/src/main/java/org/dspace/xoai/filter/DateUntilFilter.java
@@ -50,6 +50,11 @@ public class DateUntilFilter extends DSpaceFilter
     public SolrFilterResult buildSolrQuery()
     {
         String format = dateProvider.format(date).replace("Z", ".999Z"); // Tweak to set the millisecon
+        if (format.substring(11, 19).equals("00:00:00"))
+        {
+            format = format.substring(0, 11) + "23:59:59" + format.substring(19);
+        }
+
         return new SolrFilterResult("item.lastmodified:[* TO "
                 + ClientUtils.escapeQueryChars(format) + "]");
     }
diff --git a/dspace-oai/src/main/java/org/dspace/xoai/util/DateUtils.java b/dspace-oai/src/main/java/org/dspace/xoai/util/DateUtils.java
index e968414..b73955d 100644
--- a/dspace-oai/src/main/java/org/dspace/xoai/util/DateUtils.java
+++ b/dspace-oai/src/main/java/org/dspace/xoai/util/DateUtils.java
@@ -31,8 +31,7 @@ public class DateUtils
     }
     public static String format(Date date, boolean init)
     {
-       SimpleDateFormat sdf = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss.'000Z'");
-       if (!init) sdf = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss.'999Z'");
+        SimpleDateFormat sdf = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss'Z'");
         // We indicate that the returned date is in Zulu time (UTC) so we have
         // to set the time zone of sdf correct.
         sdf.setTimeZone(TimeZone.getTimeZone("ZULU"));

Git 来自 /usr/local/src/xoai 的日志(github.com/lyncode/xoai 分支 3.x 的签出)

commit f6721be7ef5bf75d790e220ee81821e8eb70986e
Merge: dd9ef83 71ae14e
Author: João Melo <jmelo@lyncode.com>
Date:   Thu Oct 9 18:33:57 2014 +0100

    Merge pull request #34 from kosarko/dates_fix

    from and until granularity

commit dd9ef830a16209e624bd03c3e91e7c5d7bdbd449
Merge: 8e778c2 c04b0bc
Author: João Melo <jmelo@lyncode.com>
Date:   Thu Oct 9 18:33:29 2014 +0100

    Merge pull request #33 from kosarko/filter_fix

    fixed typo

commit 71ae14ed5ee9e6a5a6e3d6253424c212ac98081a
Author: Ondřej Košarko <kosarko@ufal.mff.cuni.cz>
Date:   Thu Oct 9 19:05:03 2014 +0200

    from and until granularity

    Regardles of the repository setting the from and until parameters must
    handle day granularity. Error message fixed

commit c04b0bcd177007f618f18008e53a61a81aabdb06
Author: Ondřej Košarko <kosarko@ufal.mff.cuni.cz>
Date:   Thu Oct 9 19:01:08 2014 +0200

    fixed typo

xoai-*.jar 实例

$ sudo updatedb
$ locate xoai- | grep '\.jar$' | grep -v bak | xargs ls -l
-rwxr-xr-x 1 tomcat  tomcat   321775 Jul 28 03:04 /usr/local/dspace/lib/xoai-3.2.10-SNAPSHOT.jar
-rwxr-xr-x 1 tomcat  tomcat   321775 Jul 28 03:04 /usr/local/dspace/webapps/oai/WEB-INF/lib/xoai-3.2.10-SNAPSHOT.jar
-rw-rw-r-- 1 builder builder  321832 Jul 28 01:34 /usr/local/src/dspace-src/dspace/modules/oai/target/oai-5.1/WEB-INF/lib/xoai-3.2.10-SNAPSHOT.jar
-rw-rw-r-- 1 builder builder  321832 Jul 28 01:34 /usr/local/src/dspace-src/dspace-oai/target/dspace-oai-5.1/WEB-INF/lib/xoai-3.2.10-SNAPSHOT.jar
-rw-rw-r-- 1 builder builder  321832 Jul 28 01:34 /usr/local/src/dspace-src/dspace/target/dspace-installer/lib/xoai-3.2.10-SNAPSHOT.jar
-rw-rw-r-- 1 builder builder  321832 Jul 28 01:34 /usr/local/src/dspace-src/dspace/target/dspace-installer/webapps/oai/WEB-INF/lib/xoai-3.2.10-SNAPSHOT.jar
-rw-rw-r-- 1 builder builder  321775 Jul 28 03:04 /usr/local/src/xoai/target/xoai-3.2.10-SNAPSHOT.jar
-rw-rw-r-- 1 builder builder 1571092 Jul 28 03:04 /usr/local/src/xoai/target/xoai-3.2.10-SNAPSHOT-javadoc.jar
-rw-rw-r-- 1 builder builder  222305 Jul 28 03:04 /usr/local/src/xoai/target/xoai-3.2.10-SNAPSHOT-sources.jar

验证 tomcat 应该看到相同的内容(在断开链接的情况下,其中似乎有 none)

$ find -L /usr/share/tomcat6 -type f -name 'xoai-*.jar' -print | xargs ls -l
-rwxr-xr-x 1 tomcat tomcat 321775 Jul 28 03:04 /usr/share/tomcat6/webapps/oai/WEB-INF/lib/xoai-3.2.10-SNAPSHOT.jar

Tomcat -详细:class 调查结果

全部com.lyncode.xoai。 classes 来自文件:/usr/local/dspace/webapps/oai/WEB-INF/lib/xoai-3.2.10-SNAPSHOT.jar

非常感谢阅读,

卡梅隆

您是否尝试过清除 oai 响应缓存?我不知道这种类型的东西是否被缓存,但这是我能想到的唯一你还没有提到的东西。

[dspace]/bin/dspace oai clean-cache

https://wiki.duraspace.org/display/DSDOC5x/OAI+2.0+Server#OAI2.0Server-OAIManager%28SolrDataSource%29