当测试失败运行时确保 Maven 构建失败

Question

我注意到，有时运行ning maven 在 Jenkins 上构建时，运行的 Jbehave 测试的数量与另一个运行不同。在分析日志时，我看到以下片段：

    Failed to run story stories/cancel.story
java.lang.InterruptedException: stories/cancel.story
    at org.jbehave.core.embedder.StoryRunner$RunContext.interruptIfCancelled(StoryRunner.java:616)
    at org.jbehave.core.embedder.StoryRunner.runStepsWhileKeepingState(StoryRunner.java:514)
    at org.jbehave.core.embedder.StoryRunner.runScenarioSteps(StoryRunner.java:479)
    at org.jbehave.core.embedder.StoryRunner.runStepsWithLifecycle(StoryRunner.java:445)
    at org.jbehave.core.embedder.StoryRunner.runCancellable(StoryRunner.java:305)
    at org.jbehave.core.embedder.StoryRunner.run(StoryRunner.java:220)
    at org.jbehave.core.embedder.StoryRunner.run(StoryRunner.java:181)
    at org.jbehave.core.embedder.StoryManager$EnqueuedStory.call(StoryManager.java:235)
    at org.jbehave.core.embedder.StoryManager$EnqueuedStory.call(StoryManager.java:207)
    at java.util.concurrent.FutureTask.run(FutureTask.java:262)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)

问题是，当测试被跳过或以这种方式失败时运行，构建仍然被认为是成功的。

是否有 maven surefire 插件配置可确保每当测试失败时运行构建都会失败？这是 Maven surefire 构建配置

<build>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>2.11</version>
            <configuration>
                <skip>true</skip>
            </configuration>
        </plugin>
        <plugin>
            <artifactId>maven-failsafe-plugin</artifactId>
            <version>2.11</version>
            <configuration>
                <includes>
                    <include>**/*TestSuite.java</include>
                </includes>
            </configuration>
            <executions>
                <execution>
                    <goals>
                        <goal>integration-test</goal>
                        <goal>verify</goal>
                    </goals>
                </execution>
            </executions>
        </plugin>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-compiler-plugin</artifactId>
            <version>3.1</version>
        </plugin>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-javadoc-plugin</artifactId>
            <version>2.9</version>
        </plugin>
        <plugin>
            <groupId>net.thucydides.maven.plugins</groupId>
            <artifactId>maven-thucydides-plugin</artifactId>
            <version>${thucydides.version}</version>
            <executions>
                <execution>
                    <id>thucydides-reports</id>
                    <phase>post-integration-test</phase>
                    <goals>
                        <goal>aggregate</goal>
                    </goals>
                </execution>
            </executions>
            <dependencies>
                <dependency>
                    <groupId>log4j</groupId>
                    <artifactId>log4j</artifactId>
                    <version>1.2.17</version>
                </dependency>
            </dependencies>
        </plugin>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-site-plugin</artifactId>
            <version>3.2</version>
            <configuration>
                <reportPlugins>
                    <plugin>
                        <groupId>net.thucydides.maven.plugins</groupId>
                        <artifactId>maven-thucydides-plugin</artifactId>
                        <version>${thucydides.version}</version>
                    </plugin>
                </reportPlugins>
            </configuration>
        </plugin>
    </plugins>
</build>

Answer 1

您的 maven-surefire-plugin 设置为完全跳过测试（使用 <skip>true</skip>），因此测试是运行 maven-failsafe-plugin。该插件应该不会在 integration-test 期间因失败而停止，然后仅在 verify 阶段失败。

所以如果你真的想回答这个问题：

Is there a maven surefire plugin configuration that will ensure that whenever tests fail to run the build results in a failure?

即：您想要 maven-surefire-plugin 到运行测试，而不是 maven-failsafe-plugin，那么答案是：删除

        <configuration>
            <skip>true</skip>
        </configuration>

来自您的 POM。在这种情况下，您也不需要 maven-failsafe-plugin 配置，因为它只会使您的测试运行两次。

但如果您的目标是让 maven-failsafe-plugin 正常工作，那么我认为您可能遇到以下问题之一：

没有运行正确的目标。作为插件 help states，您应该将其调用为
```
mvn verify
```
旧插件，与您使用的测试框架不兼容（当前版本为2.19.1）

或者这个帮助推荐：

For very complex builds, it may be better to separate the executions for the integration-test and verify goals:

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-failsafe-plugin</artifactId>
  <version>2.19.1</version>
  <executions>
    <execution>
      <id>integration-test</id>
      <goals>
        <goal>integration-test</goal>
      </goals>
    </execution>
    <execution>
      <id>verify</id>
      <goals>
        <goal>verify</goal>
      </goals>
    </execution>
  </executions>
</plugin>

当测试失败运行时确保 Maven 构建失败

Ensure Maven build fails when tests fail to run

selenium

jbehave

maven

jenkins

maven-surefire-plugin

当测试失败 运行 时确保 Maven 构建失败

Ensure Maven build fails when tests fail to run

selenium

jbehave

maven

jenkins

maven-surefire-plugin

当测试失败运行时确保 Maven 构建失败