使用 Maven/Cucumber/Serenity 重新运行失败的场景

Question

是否有人使用过 maven surefire 插件或任何其他机制来重新运行失败的场景。我将 Cucumber 与 Serenity 和 Maven 一起使用。我尝试了以下不同的方法，以便在没有任何手动干预的情况下重新运行失败的场景
例如：如果 5 个测试用例中有 2 个测试用例出现错误，那么我的脚本应该在生成最终平静报告之前自动执行这 2 个失败的场景
1。 Maven surefire
我在 pom.xml 文件

中添加了以下行

<properties>
    <failsafe.rerunFailingTestsCount>2</failsafe.rerunFailingTestsCount>
    <surefire.rerunFailingTestsCount>2</surefire.rerunFailingTestsCount>
</properties>

或

        <plugin>
            <artifactId>maven-failsafe-plugin</artifactId>
            <version>2.19.1</version>
            <configuration>
                <includes>
                    <include>**/*.java</include>
                </includes>
                <argLine>-Xmx512m</argLine>
                <systemPropertyVariables>
                    <webdriver.driver>${webdriver.driver}</webdriver.driver>                        
                </systemPropertyVariables>
                <systemProperties>
                    <webdriver.driver>${webdriver.driver}</webdriver.driver> 
                    <surefire.rerunFailingTestsCount>${surefire.rerunFailingTestsCount}</surefire.rerunFailingTestsCount>
                    <surefire.rerunFailingTestsCount>${surefire.rerunFailingTestsCount}</surefire.rerunFailingTestsCount>
                </systemProperties>
            </configuration>
            <executions>
                <execution>
                    <goals>
                        <goal>integration-test</goal>
                        <goal>verify</goal>
                    </goals>
                </execution>
            </executions>
        </plugin>

然后我尝试执行以下命令（我的场景由@RunThis 标记注释）

mvn -Dcucumber.options="--tags @RunThis" -Dfailsafe.rerunFailingTestsCount=2 测试

执行完上面的命令后，我希望失败的场景应该再次执行，但它不会重新执行

2。使用 Cucumber 格式化程序生成 rerun.txt 并执行它
根据另一个解决方案，我使用黄瓜格式化程序生成 rerun.txt 。出现错误的场景将在 rerun.txt 中列出，然后应该由第二个跑步者 class 拾取并执行那些失败的场景。
我能够成功生成 rerun.txt 但是第二个亚军 class 既没有被执行也没有得到任何类型的错误

亚军class 1

 @RunWith(CucumberWithSerenity.class)
    @CucumberOptions(
            features = "src/test/resources/features",
            tags = "@RunThis",
            monochrome = true
            ,plugin = { "json:target/cucumber-report-composite.json", "pretty",
                "html:target/cucumber/","rerun:target/rerun.txt" })
    public class AcceptanceTest {   

    }

2 级跑步者

@RunWith(CucumberWithSerenity.class)
    @CucumberOptions(strict = true, 
      glue = { "src/test/resources/features" }, 
      features = { "@target/rerun.txt" }, 
      plugin = { "json:target/cucumber-report-composite.json", "pretty",
        "html:target/cucumber/"})
public class AcceptanceTest2 {

}

3。使用@ExtendCucumberOptions
根据另一种解决方案，我们可以添加以下对 pom.xml 文件

的依赖

<dependency>
    <groupId>com.github.mkolisnyk</groupId>
    <artifactId>cucumber-runner</artifactId>
    <version>1.3.1</version>
</dependency>

那我们的跑步者class应该是这样的

@RunWith(CucumberWithSerenity.class)
@ExtendedCucumberOptions(jsonReport = "target/cucumber.json",
retryCount = 3,
jsonUsageReport = "target/cucumber-usage.json",
outputFolder = "target")
@CucumberOptions(
        features = "src/test/resources/features",
        tags = "@RunThis",
        monochrome = true
        ,plugin = { "json:target/cucumber-report-composite.json", "pretty",
                "html:target/cucumber/","rerun:target/rerun.txt" })
public class AcceptanceTest {


}

然后我执行下面的命令
mvn -Dcucumber.options="--tags @RunThis"
我再次看不到失败的场景正在重新执行

完成POM.xml文件

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>GROUPNAME</groupId>
    <artifactId>ARTIFACTNAME</artifactId>
    <version>1.0-SNAPSHOT</version>
    <packaging>jar</packaging>

    <name>Serenity project using Cucumber and WebDriver</name>

    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
        <failsafe.rerunFailingTestsCount>2</failsafe.rerunFailingTestsCount>
        <surefire.rerunFailingTestsCount>2</surefire.rerunFailingTestsCount>
        <serenity.version>1.6.3</serenity.version>
        <serenity.cucumber.version>1.5.9</serenity.cucumber.version>
        <serenity.maven.version>1.2.1</serenity.maven.version>
        <webdriver.driver>chrome</webdriver.driver>
    </properties>

    <repositories>
        <repository>
            <snapshots>
                <enabled>false</enabled>
            </snapshots>
            <id>central</id>
            <name>bintray</name>
            <url>http://jcenter.bintray.com</url>
        </repository>
    </repositories>
    <pluginRepositories>
        <pluginRepository>
            <snapshots>
                <enabled>false</enabled>
            </snapshots>
            <id>central</id>
            <name>bintray-plugins</name>
            <url>http://jcenter.bintray.com</url>
        </pluginRepository>
    </pluginRepositories>

    <dependencies>
        <!-- <dependency> <groupId>net.serenity-bdd.maven.plugins</groupId> <artifactId>serenity-maven-plugin</artifactId> 
            <version>1.5.0-rc.1</version> </dependency> -->

        <dependency>
            <groupId>net.serenity-bdd</groupId>
            <artifactId>serenity-core</artifactId>
            <version>${serenity.version}</version>
        </dependency>
        <dependency>
            <groupId>net.serenity-bdd</groupId>
            <artifactId>serenity-cucumber</artifactId>
            <version>${serenity.cucumber.version}</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>slf4j-log4j12</artifactId>
            <version>1.7.25</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>4.12</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.assertj</groupId>
            <artifactId>assertj-core</artifactId>
            <version>1.7.0</version>
        </dependency>
        <dependency>
            <groupId>com.googlecode.lambdaj</groupId>
            <artifactId>lambdaj</artifactId>
            <version>2.3.3</version>
        </dependency>
        <dependency>
            <groupId>org.json</groupId>
            <artifactId>json</artifactId>
            <version>20171018</version>
        </dependency>
        <dependency>
            <groupId>com.qpid.automation</groupId>
            <artifactId>AutomationToolKit</artifactId>
            <version>1.0.0-SNAPSHOT</version>
        </dependency>
        <dependency>
            <groupId>com.github.mkolisnyk</groupId>
            <artifactId>cucumber-runner</artifactId>
            <version>1.3.1</version>
        </dependency>

    </dependencies>
    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>2.19.1</version>
                <configuration>
                    <!-- <skip>true</skip> -->
                    <argLine>-Dfile.encoding="UTF-8"</argLine>
                </configuration>
            </plugin>
            <plugin>
                <artifactId>maven-failsafe-plugin</artifactId>
                <version>2.19.1</version>
                <configuration>
                    <includes>
                        <include>**/*.java</include>
                    </includes>
                    <argLine>-Xmx512m</argLine>
                    <systemPropertyVariables>
                        <webdriver.driver>${webdriver.driver}</webdriver.driver>                        
                    </systemPropertyVariables>
                    <systemProperties>
                        <webdriver.driver>${webdriver.driver}</webdriver.driver> 
                        <surefire.rerunFailingTestsCount>${surefire.rerunFailingTestsCount}</surefire.rerunFailingTestsCount>
                        <surefire.rerunFailingTestsCount>${surefire.rerunFailingTestsCount}</surefire.rerunFailingTestsCount>
                    </systemProperties>
                </configuration>
                <executions>
                    <execution>
                        <goals>
                            <goal>integration-test</goal>
                            <goal>verify</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>3.2</version>
                <configuration>
                    <source>1.8</source>
                    <target>1.8</target>
                </configuration>
            </plugin>
            <!-- Generate the test reports after the integration tests -->
            <plugin>
                <groupId>net.serenity-bdd.maven.plugins</groupId>
                <artifactId>serenity-maven-plugin</artifactId>
                <version>${serenity.version}</version>
                <executions>
                    <execution>
                        <id>serenity-reports</id>
                        <phase>post-integration-test</phase>
                        <goals>
                            <goal>aggregate</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>
</project>

Answer 1

我可以使用下面的代码实现重新运行。

@RunWith(CucumberWithSerenity.class)
@CucumberOptions(
        features = "src/test/resources/features", tags = "@Smoke1",
        monochrome = true,
        plugin = {"pretty", "html:target/cucumber-reports", "json:target/cucumber.json","rerun:rerun.txt"})
public class AcceptanceTest {

}

我只是执行下面的命令，在上面的插件中添加re运行之后(@CucumberOptions)

mvn clean verify -Dcucumber.options="--tags @Smoke1"

它会自动 re-execute 失败场景。我不需要手动编写另一个运行ner class 和运行它。

我的 pom.xml 是：

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>GROUPID</groupId>
    <artifactId>ARTIFACTID</artifactId>
    <version>1.0-SNAPSHOT</version>
    <packaging>jar</packaging>

    <name>Serenity project using Cucumber and WebDriver</name>

    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
        <serenity.version>1.9.2</serenity.version>
        <serenity.cucumber.version>1.9.3</serenity.cucumber.version>
        <serenity.maven.version>1.8.21</serenity.maven.version>
        <webdriver.driver>chrome</webdriver.driver>
    </properties>

    <repositories>
        <repository>
            <snapshots>
                <enabled>false</enabled>
            </snapshots>
            <id>central</id>
            <name>bintray</name>
            <url>http://jcenter.bintray.com</url>
        </repository>
    </repositories>
    <pluginRepositories>
        <pluginRepository>
            <snapshots>
                <enabled>false</enabled>
            </snapshots>
            <id>central</id>
            <name>bintray-plugins</name>
            <url>http://jcenter.bintray.com</url>
        </pluginRepository>
    </pluginRepositories>

    <dependencies>
        <dependency>
            <groupId>net.serenity-bdd</groupId>
            <artifactId>serenity-core</artifactId>
            <version>${serenity.version}</version>
        </dependency>
        <dependency>
            <groupId>net.serenity-bdd</groupId>
            <artifactId>serenity-cucumber</artifactId>
            <version>${serenity.cucumber.version}</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>slf4j-log4j12</artifactId>
            <version>1.7.25</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.junit.jupiter</groupId>
            <artifactId>junit-jupiter-api</artifactId>
            <version>5.1.0</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.assertj</groupId>
            <artifactId>assertj-core</artifactId>
            <version>1.7.0</version>
        </dependency>
        <dependency>
            <groupId>com.googlecode.lambdaj</groupId>
            <artifactId>lambdaj</artifactId>
            <version>2.3.3</version>
        </dependency>
        <dependency>
            <groupId>org.json</groupId>
            <artifactId>json</artifactId>
            <version>20171018</version>
        </dependency>
        <dependency>
            <groupId>com.qpid.automation</groupId>
            <artifactId>AutomationToolKit</artifactId>
            <version>1.0.0</version>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>2.18.1</version>
                <configuration>
                    <!-- <skip>true</skip> -->
                    <argLine>-Dfile.encoding="UTF-8"</argLine>
                </configuration>
            </plugin>
            <plugin>
                <artifactId>maven-failsafe-plugin</artifactId>
                <version>2.18.1</version>
                <configuration>
                    <includes>
                        <include>**/*.java</include>
                    </includes>
                    <argLine>-Xmx512m</argLine>
                    <systemPropertyVariables>
                        <webdriver.driver>${webdriver.driver}</webdriver.driver>
                    </systemPropertyVariables>
                </configuration>
                <executions>
                    <execution>
                        <goals>
                            <goal>integration-test</goal>
                            <goal>verify</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>3.2</version>
                <configuration>
                    <source>1.8</source>
                    <target>1.8</target>
                </configuration>
            </plugin>
            <!-- Generate the test reports after the integration tests -->
            <plugin>
                <groupId>net.serenity-bdd.maven.plugins</groupId>
                <artifactId>serenity-maven-plugin</artifactId>
                <version>${serenity.version}</version>
                <executions>
                    <execution>
                        <id>serenity-reports</id>
                        <phase>post-integration-test</phase>
                        <goals>
                            <goal>aggregate</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>
</project>

使用 Maven/Cucumber/Serenity 重新运行失败的场景

Rerunning the failed scenario using Maven/Cucumber/Serenity

cucumber

maven

selenium-webdriver

serenity-bdd

cucumber-serenity