如何并行运行 surefire 测试，同时强制某些顺序运行？

Question

我正在开发具有数千个测试的 Maven 项目，这些测试运行非常缓慢（运行所有测试需要 2 小时）。因此，我尝试通过如下配置 surefire 插件来并行运行测试：

<configuration>
  <failIfNoTests>false</failIfNoTests>
  <reuseForks>false</reuseForks>
  <reuseForks>true</reuseForks>
  <forkCount>2C</forkCount>
  <systemPropertyVariables>
    <jacoco-agent.destfile>target/jacoco.exec</jacoco-agent.destfile>
  </systemPropertyVariables>
  <parallel>suites</parallel>
  <threadCount>12</threadCount>
</configuration>

但是后来一些测试在我们初始化和清理一些资源的@Before和@After方法中失败了（这似乎与端口冲突有关）。

我尝试将此注释 @net.jcip.annotations.NotThreadSafe 添加到失败的测试中，如 surefire documentation 中所述，以运行按顺序添加它们并避免冲突。但是这没有用，这些测试仍然失败！！！

关于如何强制某些 surefire 测试在同一个 JVM 进程和同一个线程上按顺序运行而其余的将运行并行（可能在不同的 JVM 进程上）的任何指示？

编辑 1 现在，我尝试将 surefire 的配置拆分为两个：一个用于顺序测试，另一个用于并行测试。然而，这种方法似乎并没有增加执行时间，因为我仍然有失败的测试，并且运行所有测试仍然需要 2 小时。

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-surefire-plugin</artifactId>
  <version>${surefire.version}</version>
  <!-- Sequential tests -->
  <configuration>
    <includes>**/*Sequential.java</includes>
    <failIfNoTests>false</failIfNoTests>
    <reuseForks>false</reuseForks>
    <systemPropertyVariables>
      <jacoco-agent.destfile>target/jacoco.exec</jacoco-agent.destfile>
    </systemPropertyVariables>
  </configuration>
  <executions>
    <!-- Parallel tests -->
    <execution>
      <id>parallel-tests</id>
      <phase>test</phase>
      <goals>
        <goal>test</goal>
      </goals>
      <configuration>
        <includes>**/*.java</includes>
        <excludes>
          <exclude>**/*Sequential.java</exclude>
        </excludes>
        <failIfNoTests>false</failIfNoTests>
        <reuseForks>true</reuseForks>
        <forkCount>2C</forkCount>
        <parallel>suites</parallel>
        <threadCount>12</threadCount>
      </configuration>
    </execution>
  </executions>
</plugin>

Answer 1

我同意你的 "split the configuration of surefire into two" 方法。

但是你的配置似乎有问题，因为你只定义了一个执行。

可能想尝试这样的事情（用自己的配置定义 2 个执行）：

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-surefire-plugin</artifactId>
  <version>${surefire.version}</version>

  <executions>
     <!-- Sequential tests -->
    <execution>
      <id>sequential-tests</id>
      <phase>test</phase>
      <goals>
        <goal>test</goal>
      </goals>
      <configuration>
        <includes>**/*Sequential.java</includes>
        <failIfNoTests>false</failIfNoTests>
        <reuseForks>false</reuseForks>
        <systemPropertyVariables>
            <jacoco-agent.destfile>target/jacoco.exec</jacoco-agent.destfile>
        </systemPropertyVariables>
      </configuration>
    </execution>      
    <!-- Parallel tests -->
    <execution>
      <id>parallel-tests</id>
      <phase>test</phase>
      <goals>
        <goal>test</goal>
      </goals>
      <configuration>
        <includes>**/*.java</includes>
        <excludes>
          <exclude>**/*Sequential.java</exclude>
        </excludes>
        <failIfNoTests>false</failIfNoTests>
        <reuseForks>true</reuseForks>
        <forkCount>2C</forkCount>
        <parallel>suites</parallel>
        <threadCount>12</threadCount>
      </configuration>
    </execution>
  </executions>
</plugin>

Answer 2

这是我根据需要进行的运行测试的最终 Maven 配置：

    <plugin>
      <groupId>org.apache.maven.plugins</groupId>
      <artifactId>maven-surefire-plugin</artifactId>
      <version>${surefire.version}</version>
      <!-- Sequential tests -->
      <configuration>
        <failIfNoTests>false</failIfNoTests>
        <reuseForks>false</reuseForks>
        <systemPropertyVariables>
          <jacoco-agent.destfile>target/jacoco.exec</jacoco-agent.destfile>
        </systemPropertyVariables>
      </configuration>
      <executions>
        <!-- Default tests -->
        <execution>
          <id>default-test</id>
          <configuration>
            <skip>true</skip>
          </configuration>
        </execution>
        <!-- Parallel tests -->
        <execution>
          <id>parallel-tests</id>
          <phase>test</phase>
          <goals>
            <goal>test</goal>
          </goals>
          <configuration>
            <includes>**/*.java</includes>
            <excludes>
              <exclude>**/*Sequential.java</exclude>
            </excludes>
            <failIfNoTests>false</failIfNoTests>
            <reuseForks>true</reuseForks>
            <forkCount>2C</forkCount>
            <parallel>suites</parallel>
            <threadCount>12</threadCount>
          </configuration>
        </execution>
        <!-- Sequential tests -->
        <execution>
          <id>sequential-tests</id>
          <phase>test</phase>
          <goals>
            <goal>test</goal>
          </goals>
          <configuration>
            <skip>${skip.sequential.tests}</skip>
            <includes>**/*Sequential.java</includes>
            <reuseForks>false</reuseForks>
          </configuration>
        </execution>
      </executions>
    </plugin>

如何并行运行 surefire 测试，同时强制某些顺序运行？

How to run surefire tests in parallel while forcing some to run sequentially?

java

junit

multithreading

maven

maven-surefire-plugin

如何并行 运行 surefire 测试，同时强制某些顺序 运行？

How to run surefire tests in parallel while forcing some to run sequentially?

java

junit

multithreading

maven

maven-surefire-plugin

如何并行运行 surefire 测试，同时强制某些顺序运行？