尝试通过 MockMVC 和 Junit5 对 Apache Jena 执行测试时出现内存不足异常
Out of Memory Exception when trying to execute test for Apache Jena via MockMVC and Junit5
我正在 运行 连接 Apache Jena Fuseki 服务器作为 SPARQL 端点,我可以在正常使用应用程序时连接到它。一切正常,我得到了结果查询的输出。
但是当我尝试 运行 使用 Springboot、Junit5(我假设)和 MockMVC 进行测试时,它总是卡在以下部分:
ResultSet setNew = exec.execSelect();
然后测试挂起,直到出现内存不足异常,使用 -Xmx 将其设置为 12G 也无法解决问题。有一些根本的事情我就是不明白。每个人都经历过这种情况吗?难道我做错了什么?是不是这个代码也有问题?
如果我需要 post 更多代码,请随时发表评论。我改变了很多东西试图让它工作,但无济于事。我还必须从 gradle 中取消注释 Jacoco 测试套件,因为它也挂在 Jacoco 测试上。所以我认为它与无限循环有关,但我没有看到它。
我确实是有生以来第一次开始使用流,也许这就是问题所在? (主要是我没有在此处 post 编写的代码)。
代码片段:
Sparql 查询
SELECT ?object1 ?predicate2 ?object2
WHERE
{ ?id1 ?predicate1 ?object1
FILTER ( regex(str(?predicate1), "Amsterdam", "i") || regex(str(?object1), "Amsterdam", "i") )
?id2 ?predicate2 ?object2
FILTER ( regex(str(?predicate2), "city", "i") || regex(str(?object2), "city", "i") )
}
查询码
public ResultSet query(Query query, String endpoint) {
ResultSet resultSet = null;
log.info(String.format("Query being executed: %s on %s", query.toString(), endpoint));
try (QueryExecution exec = QueryExecutionFactory.sparqlService(endpoint, query)) {
//((QueryEngineHTTP) exec).addParam("timeout", "30000");
ResultSet setNew = exec.execSelect();
resultSet = ResultSetFactory.copyResults(setNew);
log.info("Query has been executed");
} catch (Exception e) {
log.error(e.getLocalizedMessage());
}
return resultSet;
}
我的测试
@SpringBootTest
@AutoConfigureMockMvc
class FactCheckControllerTest {
@Autowired
private MockMvc mockMvc;
@Test
void testCheckFactSuccess() throws Exception {
mockMvc.perform(post("/factcheck")
.contentType(MediaType.APPLICATION_JSON_VALUE)
.content("{\"text\": \"Amsterdam is a city.\"}"))
.andExpect(status().isOk())
.andExpect(content().contentType(MediaType.APPLICATION_JSON_VALUE));
// .andExpect(jsonPath("$.triplets[0].subject").value("Amsterdam"))
// .andExpect(jsonPath("$.triplets[0].predicate").value("is"))
// .andExpect(jsonPath("$.triplets[0].object").value("small"));
}
}
启用了 --stacktrace 选项的错误日志(gradle 工作人员)
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':test'.
> Process 'Gradle Test Executor 1' finished with non-zero exit value 1
This problem might be caused by incorrect test process configuration.
Please refer to the test execution section in the User Manual at https://docs.gradle.org/6.8.3/userguide/java_testing.html#sec:test_execution
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
BUILD FAILED in 43s
在 运行 正常应用程序时记录:
2021-05-11 13:10:57.425 INFO 2396 --- [nio-8080-exec-2] c.s.factcheck.data.JenaRDFConnector : Query has been executed
2021-05-11 13:10:57.425 INFO 2396 --- [nio-8080-exec-2] c.s.factcheck.domain.FactDeterminer : Setting ResultSet!
2021-05-11 13:10:57.425 INFO 2396 --- [nio-8080-exec-2] c.s.factcheck.domain.FactDeterminer : Checking Fact!
---------------------------------------------------------------------------------------------------------
| object1 | predicate2 | object2 |
=========================================================================================================
| "Amsterdam" | <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> | <https://dbpedia.org/ontology/City> |
---------------------------------------------------------------------------------------------------------
编辑:添加了 turtle 文件,其中包含 Fuseki 中使用的数据。
@prefix schema: <http://schema.org/> .
@prefix pq: <http://www.wikidata.org/prop/qualifier/> .
@prefix bd: <http://www.bigdata.com/rdf#> .
@prefix pr: <http://www.wikidata.org/prop/reference/> .
@prefix ps: <http://www.wikidata.org/prop/statement/> .
@prefix owl: <http://www.w3.org/2002/07/owl#> .
@prefix wdt: <http://www.wikidata.org/prop/direct/> .
@prefix wds: <http://www.wikidata.org/entity/statement/> .
@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .
@prefix wdv: <http://www.wikidata.org/value/> .
@prefix skos: <http://www.w3.org/2004/02/skos/core#> .
@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
@prefix psn: <http://www.wikidata.org/prop/statement/value-normalized/> .
@prefix pqn: <http://www.wikidata.org/prop/qualifier/value-normalized/> .
@prefix wd: <http://www.wikidata.org/entity/> .
@prefix geo: <http://www.opengis.net/ont/geosparql#> .
@prefix psv: <http://www.wikidata.org/prop/statement/value/> .
@prefix dct: <http://purl.org/dc/terms/> .
@prefix pqv: <http://www.wikidata.org/prop/qualifier/value/> .
@prefix ontolex: <http://www.w3.org/ns/lemon/ontolex#> .
@prefix wdata: <http://www.wikidata.org/wiki/Special:EntityData/> .
@prefix wdref: <http://www.wikidata.org/reference/> .
@prefix prov: <http://www.w3.org/ns/prov#> .
@prefix cc: <http://creativecommons.org/ns#> .
@prefix wikibase: <http://wikiba.se/ontology#> .
@prefix prn: <http://www.wikidata.org/prop/reference/value-normalized/> .
@prefix dbo: <https://dbpedia.org/ontology/> .
@prefix wdtn: <http://www.wikidata.org/prop/direct-normalized/> .
@prefix p: <http://www.wikidata.org/prop/> .
@prefix rdf: <https://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix dbr: <https://dbpedia.org/resource/> .
@prefix prv: <http://www.wikidata.org/prop/reference/value/> .
@prefix hint: <http://www.bigdata.com/queryHints#> .
@prefix wdno: <http://www.wikidata.org/prop/novalue/> .
<http://www.dataset.org/turtle/Amsterdam>
a dbo:City ;
rdfs:label "Amsterdam" ;
dbo:areaCode "020" ;
dbo:country dbr:Netherlands .
<http://www.dataset.org/turtle/ArnoldSchwarzenegger>
a wdata:Q5 ;
rdfs:label "Arnold Schwarzenegger" .
我找到的答案是堆大小不断溢出。添加行:
maxHeapSize '2g'
到我 build.gradle 的测试区解决了问题(虽然我仍然不知道为什么会这样)。
完成build.gradle:
plugins {
id 'org.springframework.boot' version '2.4.5'
id 'io.spring.dependency-management' version '1.0.11.RELEASE'
id 'java'
id 'jacoco'
id 'info.solidsoft.pitest' version '1.6.0'
id "org.sonarqube" version "3.1.1"
}
group = 'com.nlxdodge.factcheck'
version = '0.0.1-SNAPSHOT'
sourceCompatibility = '11'
configurations {
compileOnly {
extendsFrom annotationProcessor
}
}
repositories {
mavenCentral()
}
dependencies {
// Spring Boot Web
implementation 'org.springframework.boot:spring-boot-starter-actuator:2.4.5'
implementation 'org.springframework.boot:spring-boot-starter-web:2.4.5'
testImplementation 'org.springframework.boot:spring-boot-starter-test:2.4.5'
// Lombok Annotation Processor
compileOnly 'org.projectlombok:lombok:1.18.20'
annotationProcessor 'org.projectlombok:lombok:1.18.20'
testCompileOnly 'org.projectlombok:lombok:1.18.20'
testAnnotationProcessor 'org.projectlombok:lombok:1.18.20'
// Stanford NLP
implementation 'edu.stanford.nlp:stanford-corenlp:4.2.0'
implementation 'edu.stanford.nlp:stanford-corenlp:4.2.0:models'
implementation 'edu.stanford.nlp:stanford-corenlp:4.2.0:models-english'
// Jena
implementation 'org.apache.jena:apache-jena-libs:4.0.0'
implementation 'org.apache.jena:jena-querybuilder:4.0.0'
}
test {
useJUnitPlatform()
maxHeapSize '2g'
finalizedBy jacocoTestReport
finalizedBy 'pitest'
}
jacocoTestReport {
dependsOn test
reports {
xml.enabled = true
}
afterEvaluate {
afterEvaluate {
classDirectories.setFrom(files(classDirectories.files.collect {
fileTree(dir: it, exclude: [
'com/nlxdodge/factcheck/service/config/**',
'com/nlxdodge/factcheck/FactCheckApplication**'
])
}))
}
}
}
pitest {
junit5PluginVersion = '0.12'
targetClasses = ['com.nlxdodge.factcheck.*']
outputFormats = ['XML', 'HTML']
timestampedReports = false
}
sonarqube {
properties {
property 'sonar.exclusions',
[
"**/com/nlxdodge/factcheck/FactCheckApplication**"
]
}
}
我正在 运行 连接 Apache Jena Fuseki 服务器作为 SPARQL 端点,我可以在正常使用应用程序时连接到它。一切正常,我得到了结果查询的输出。
但是当我尝试 运行 使用 Springboot、Junit5(我假设)和 MockMVC 进行测试时,它总是卡在以下部分:
ResultSet setNew = exec.execSelect();
然后测试挂起,直到出现内存不足异常,使用 -Xmx 将其设置为 12G 也无法解决问题。有一些根本的事情我就是不明白。每个人都经历过这种情况吗?难道我做错了什么?是不是这个代码也有问题?
如果我需要 post 更多代码,请随时发表评论。我改变了很多东西试图让它工作,但无济于事。我还必须从 gradle 中取消注释 Jacoco 测试套件,因为它也挂在 Jacoco 测试上。所以我认为它与无限循环有关,但我没有看到它。
我确实是有生以来第一次开始使用流,也许这就是问题所在? (主要是我没有在此处 post 编写的代码)。
代码片段:
Sparql 查询
SELECT ?object1 ?predicate2 ?object2
WHERE
{ ?id1 ?predicate1 ?object1
FILTER ( regex(str(?predicate1), "Amsterdam", "i") || regex(str(?object1), "Amsterdam", "i") )
?id2 ?predicate2 ?object2
FILTER ( regex(str(?predicate2), "city", "i") || regex(str(?object2), "city", "i") )
}
查询码
public ResultSet query(Query query, String endpoint) {
ResultSet resultSet = null;
log.info(String.format("Query being executed: %s on %s", query.toString(), endpoint));
try (QueryExecution exec = QueryExecutionFactory.sparqlService(endpoint, query)) {
//((QueryEngineHTTP) exec).addParam("timeout", "30000");
ResultSet setNew = exec.execSelect();
resultSet = ResultSetFactory.copyResults(setNew);
log.info("Query has been executed");
} catch (Exception e) {
log.error(e.getLocalizedMessage());
}
return resultSet;
}
我的测试
@SpringBootTest
@AutoConfigureMockMvc
class FactCheckControllerTest {
@Autowired
private MockMvc mockMvc;
@Test
void testCheckFactSuccess() throws Exception {
mockMvc.perform(post("/factcheck")
.contentType(MediaType.APPLICATION_JSON_VALUE)
.content("{\"text\": \"Amsterdam is a city.\"}"))
.andExpect(status().isOk())
.andExpect(content().contentType(MediaType.APPLICATION_JSON_VALUE));
// .andExpect(jsonPath("$.triplets[0].subject").value("Amsterdam"))
// .andExpect(jsonPath("$.triplets[0].predicate").value("is"))
// .andExpect(jsonPath("$.triplets[0].object").value("small"));
}
}
启用了 --stacktrace 选项的错误日志(gradle 工作人员)
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':test'.
> Process 'Gradle Test Executor 1' finished with non-zero exit value 1
This problem might be caused by incorrect test process configuration.
Please refer to the test execution section in the User Manual at https://docs.gradle.org/6.8.3/userguide/java_testing.html#sec:test_execution
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
BUILD FAILED in 43s
在 运行 正常应用程序时记录:
2021-05-11 13:10:57.425 INFO 2396 --- [nio-8080-exec-2] c.s.factcheck.data.JenaRDFConnector : Query has been executed
2021-05-11 13:10:57.425 INFO 2396 --- [nio-8080-exec-2] c.s.factcheck.domain.FactDeterminer : Setting ResultSet!
2021-05-11 13:10:57.425 INFO 2396 --- [nio-8080-exec-2] c.s.factcheck.domain.FactDeterminer : Checking Fact!
---------------------------------------------------------------------------------------------------------
| object1 | predicate2 | object2 |
=========================================================================================================
| "Amsterdam" | <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> | <https://dbpedia.org/ontology/City> |
---------------------------------------------------------------------------------------------------------
编辑:添加了 turtle 文件,其中包含 Fuseki 中使用的数据。
@prefix schema: <http://schema.org/> .
@prefix pq: <http://www.wikidata.org/prop/qualifier/> .
@prefix bd: <http://www.bigdata.com/rdf#> .
@prefix pr: <http://www.wikidata.org/prop/reference/> .
@prefix ps: <http://www.wikidata.org/prop/statement/> .
@prefix owl: <http://www.w3.org/2002/07/owl#> .
@prefix wdt: <http://www.wikidata.org/prop/direct/> .
@prefix wds: <http://www.wikidata.org/entity/statement/> .
@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .
@prefix wdv: <http://www.wikidata.org/value/> .
@prefix skos: <http://www.w3.org/2004/02/skos/core#> .
@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
@prefix psn: <http://www.wikidata.org/prop/statement/value-normalized/> .
@prefix pqn: <http://www.wikidata.org/prop/qualifier/value-normalized/> .
@prefix wd: <http://www.wikidata.org/entity/> .
@prefix geo: <http://www.opengis.net/ont/geosparql#> .
@prefix psv: <http://www.wikidata.org/prop/statement/value/> .
@prefix dct: <http://purl.org/dc/terms/> .
@prefix pqv: <http://www.wikidata.org/prop/qualifier/value/> .
@prefix ontolex: <http://www.w3.org/ns/lemon/ontolex#> .
@prefix wdata: <http://www.wikidata.org/wiki/Special:EntityData/> .
@prefix wdref: <http://www.wikidata.org/reference/> .
@prefix prov: <http://www.w3.org/ns/prov#> .
@prefix cc: <http://creativecommons.org/ns#> .
@prefix wikibase: <http://wikiba.se/ontology#> .
@prefix prn: <http://www.wikidata.org/prop/reference/value-normalized/> .
@prefix dbo: <https://dbpedia.org/ontology/> .
@prefix wdtn: <http://www.wikidata.org/prop/direct-normalized/> .
@prefix p: <http://www.wikidata.org/prop/> .
@prefix rdf: <https://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix dbr: <https://dbpedia.org/resource/> .
@prefix prv: <http://www.wikidata.org/prop/reference/value/> .
@prefix hint: <http://www.bigdata.com/queryHints#> .
@prefix wdno: <http://www.wikidata.org/prop/novalue/> .
<http://www.dataset.org/turtle/Amsterdam>
a dbo:City ;
rdfs:label "Amsterdam" ;
dbo:areaCode "020" ;
dbo:country dbr:Netherlands .
<http://www.dataset.org/turtle/ArnoldSchwarzenegger>
a wdata:Q5 ;
rdfs:label "Arnold Schwarzenegger" .
我找到的答案是堆大小不断溢出。添加行:
maxHeapSize '2g'
到我 build.gradle 的测试区解决了问题(虽然我仍然不知道为什么会这样)。
完成build.gradle:
plugins {
id 'org.springframework.boot' version '2.4.5'
id 'io.spring.dependency-management' version '1.0.11.RELEASE'
id 'java'
id 'jacoco'
id 'info.solidsoft.pitest' version '1.6.0'
id "org.sonarqube" version "3.1.1"
}
group = 'com.nlxdodge.factcheck'
version = '0.0.1-SNAPSHOT'
sourceCompatibility = '11'
configurations {
compileOnly {
extendsFrom annotationProcessor
}
}
repositories {
mavenCentral()
}
dependencies {
// Spring Boot Web
implementation 'org.springframework.boot:spring-boot-starter-actuator:2.4.5'
implementation 'org.springframework.boot:spring-boot-starter-web:2.4.5'
testImplementation 'org.springframework.boot:spring-boot-starter-test:2.4.5'
// Lombok Annotation Processor
compileOnly 'org.projectlombok:lombok:1.18.20'
annotationProcessor 'org.projectlombok:lombok:1.18.20'
testCompileOnly 'org.projectlombok:lombok:1.18.20'
testAnnotationProcessor 'org.projectlombok:lombok:1.18.20'
// Stanford NLP
implementation 'edu.stanford.nlp:stanford-corenlp:4.2.0'
implementation 'edu.stanford.nlp:stanford-corenlp:4.2.0:models'
implementation 'edu.stanford.nlp:stanford-corenlp:4.2.0:models-english'
// Jena
implementation 'org.apache.jena:apache-jena-libs:4.0.0'
implementation 'org.apache.jena:jena-querybuilder:4.0.0'
}
test {
useJUnitPlatform()
maxHeapSize '2g'
finalizedBy jacocoTestReport
finalizedBy 'pitest'
}
jacocoTestReport {
dependsOn test
reports {
xml.enabled = true
}
afterEvaluate {
afterEvaluate {
classDirectories.setFrom(files(classDirectories.files.collect {
fileTree(dir: it, exclude: [
'com/nlxdodge/factcheck/service/config/**',
'com/nlxdodge/factcheck/FactCheckApplication**'
])
}))
}
}
}
pitest {
junit5PluginVersion = '0.12'
targetClasses = ['com.nlxdodge.factcheck.*']
outputFormats = ['XML', 'HTML']
timestampedReports = false
}
sonarqube {
properties {
property 'sonar.exclusions',
[
"**/com/nlxdodge/factcheck/FactCheckApplication**"
]
}
}