selenium testNG 重试结果帐户不正确

Question

我正在使用安装在 Eclipse 中的 testNG 6.9.10。我试图使用重试来确保失败的测试可以运行定义的 maxcount 次。请参阅以下代码。

public class TestRetry implements IRetryAnalyzer {
    private int retryCount = 0;
    private int maxRetryCount = 1;

    public boolean retry(ITestResult result) {

        if (retryCount < maxRetryCount) {
            retryCount++;
            return true;
        }
        return false;
    }

    @Test(retryAnalyzer = TestRetry.class)
    public void testGenX() {
        Assert.assertEquals("google", "google");
    }

    @Test(retryAnalyzer = TestRetry.class)
    public void testGenY() {
        Assert.assertEquals("hello", "hallo");

    }
}

我得到以下结果：

===============================================
    Default test
    Tests run: 3, Failures: 1, Skips: 1
===============================================


===============================================
Default suite
Total tests run: 3, Failures: 1, Skips: 1
===============================================

不过好像结果计数有点问题。我要以下：

===============================================
    Default test
    Tests run: 2, Failures: 1, Skips: 0
===============================================


===============================================
Default suite
Total tests run: 2, Failures: 1, Skips: 0
===============================================

我试图定义监听器来实现它，比如覆盖 onFinish 函数。您可能会在 http://www.seleniumeasy.com/testng-tutorials/retry-listener-failed-tests-count-update 中找到它但最终还是不行。

遇到过这种情况的人可以帮忙吗？

Answer 1

它工作正常，我希望监听器使用有问题。我和你一样创建了 TestRetry，但是没有 @Test 方法。

 public class TestRetry implements IRetryAnalyzer{

private int retryCount = 0;
private int maxRetryCount = 1;

@Override
public boolean retry(ITestResult arg0) {
    // TODO Auto-generated method stub
    if (retryCount < maxRetryCount) {
        retryCount++;
        return true;
    }

    return false;
}
}

已创建监听器class

 public class TestListener implements ITestListener{

@Override
public void onFinish(ITestContext context) {
    // TODO Auto-generated method stub
    Set<ITestResult> failedTests = context.getFailedTests().getAllResults();
    for (ITestResult temp : failedTests) {
        ITestNGMethod method = temp.getMethod();
        if (context.getFailedTests().getResults(method).size() > 1) {
            failedTests.remove(temp);
        } else {
            if (context.getPassedTests().getResults(method).size() > 0) {
                failedTests.remove(temp);
            }
        }
    }
}

@Override
public void onStart(ITestContext arg0) {
    // TODO Auto-generated method stub

}

@Override
public void onTestFailedButWithinSuccessPercentage(ITestResult arg0) {
    // TODO Auto-generated method stub

}

@Override
public void onTestFailure(ITestResult arg0) {
    // TODO Auto-generated method stub

}

@Override
public void onTestSkipped(ITestResult arg0) {
    // TODO Auto-generated method stub

}

@Override
public void onTestStart(ITestResult arg0) {
    // TODO Auto-generated method stub

}

@Override
public void onTestSuccess(ITestResult arg0) {
    // TODO Auto-generated method stub

}

}

我终于用这些方法class 进行了测试

  public class RunTest {


@Test(retryAnalyzer = TestRetry.class)
public void testGenX() {
    Assert.assertEquals("google", "google");
}

@Test(retryAnalyzer = TestRetry.class)
public void testGenY() {
    Assert.assertEquals("hello", "hallo");

}

}

通过指定我的自定义侦听器从 testng.xml 文件执行此 RunTest

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Suite1" parallel="false" preserve-order="true">
<listeners>
   <listener class-name="com.test.TestListener"/>
</listeners>

<test name="TestA">
<classes>
  <class name="com.test.RunTest"/>
</classes>
 </test> <!-- Test -->
 </suite> <!-- Suite -->

请试一试..

谢谢，穆拉利

Answer 2

@murali 可以看看我下面的代码吗？我真的看不出有什么区别。 CustomLinstener.java

package cases;
import java.util.Set;
import org.testng.ITestContext;
import org.testng.ITestListener;
import org.testng.ITestNGMethod;
import org.testng.ITestResult;

public class CustomLinstener implements ITestListener{

    @Override
    public void onFinish(ITestContext context) {
        Set<ITestResult> failedTests = context.getFailedTests().getAllResults();
        for (ITestResult temp : failedTests) {
            ITestNGMethod method = temp.getMethod();
            if (context.getFailedTests().getResults(method).size() > 1) {
                failedTests.remove(temp);
            } else {
                if (context.getPassedTests().getResults(method).size() > 0) {
                    failedTests.remove(temp);
                }
            }
        }
    }

    @Override
    public void onStart(ITestContext arg0) {
        // TODO Auto-generated method stub

    }

    @Override
    public void onTestFailedButWithinSuccessPercentage(ITestResult arg0) {
        // TODO Auto-generated method stub

    }

    @Override
    public void onTestFailure(ITestResult arg0) {
        // TODO Auto-generated method stub

    }

    @Override
    public void onTestSkipped(ITestResult arg0) {
        // TODO Auto-generated method stub

    }

    @Override
    public void onTestStart(ITestResult arg0) {
        // TODO Auto-generated method stub

    }

    @Override
    public void onTestSuccess(ITestResult arg0) {
        // TODO Auto-generated method stub

    }
}

RunTest.java

package cases;

import org.testng.Assert;
import org.testng.annotations.Test;

public class RunTest {


@Test(retryAnalyzer = TestRetry.class)
public void testGenX() {
    Assert.assertEquals("google", "google");
}

@Test(retryAnalyzer = TestRetry.class)
public void testGenY() {
    Assert.assertEquals("hello", "hallo");

}

}

TestRetry.java

package cases;

import org.testng.IRetryAnalyzer;
import org.testng.ITestResult;

public class TestRetry implements IRetryAnalyzer{

private int retryCount = 0;
private int maxRetryCount = 1;

@Override
public boolean retry(ITestResult arg0) {
    // TODO Auto-generated method stub
    if (retryCount < maxRetryCount) {
        retryCount++;
        return true;
    }

    return false;
}
}

终于XML了。我右键单击它并运行作为 testNG 套件。

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Suite1" parallel="false" preserve-order="true">

    <test name="TestA">
        <classes>
            <class name="cases.RunTest" />
        </classes>
    </test> <!-- Test -->
        <listeners>
        <listener class-name="cases.CustomLinstener" />
    </listeners>
</suite> <!-- Suite -->

Answer 3

TestNG IRetryAnalyzer 的文档没有指定测试报告行为：

Interface to implement to be able to have a chance to retry a failed test.

http://testng.org/doc/documentation-main.html and searching across the entire testng.org site only returns links to the documentation of and references to IRetryAnalyzer (see site:testng.org retry - Google Search 上没有提到 "retries"。

由于没有关于如何报告重试测试的文档，我们无法做出很多合理的期望。每次尝试都应该出现在测试结果中吗？如果是这样，除了最后一次尝试之外，每次尝试是否都标记为跳过，最后一次尝试是成功还是失败？它没有记录。该行为是未定义的，它可能会随着任何 TestNG 版本以微妙或突然的方式发生变化。

因此，我建议使用 TestNG 以外的工具来重试逻辑。

例如您可以 Spring Retry (which can be used independently of other Spring 个项目）：

TestRetry.java

public class TestRetry {
    private static RetryOperations retryOperations = createRetryOperations();

    private static RetryOperations createRetryOperations() {
        RetryTemplate retryTemplate = new RetryTemplate();
        retryTemplate.setRetryPolicy(createRetryPolicy());
        return retryTemplate;
    }

    private static RetryPolicy createRetryPolicy() {
        int maxAttempts = 2;
        Map<Class<? extends Throwable>, Boolean> retryableExceptions =
                Collections.singletonMap(AssertionError.class, true);
        return new SimpleRetryPolicy(maxAttempts, retryableExceptions);
    }

    @Test
    public void testGenX() {
        runWithRetries(context -> {
            Assert.assertEquals("google", "google");
        });
    }

    @Test
    public void testGenY() {
        runWithRetries(context -> {
            Assert.assertEquals("hello", "hallo");
        });
    }

    private void runWithRetries(RetryRunner<RuntimeException> runner) {
        retryOperations.execute(runner);
    }
}

RetryRunner.java

/**
 * Runner interface for an operation that can be retried using a
 * {@link RetryOperations}.
 * <p>
 * This is simply a convenience interface that extends
 * {@link RetryCallback} but assumes a {@code void} return type.
 */
interface RetryRunner<E extends Throwable> extends RetryCallback<Void, E> {
    @Override
    default Void doWithRetry(RetryContext context) throws E {
        runWithRetry(context);
        return null;
    }

    void runWithRetry(RetryContext context) throws E;
}

控制台输出

===============================================
Default Suite
Total tests run: 2, Failures: 1, Skips: 0
===============================================

Spring 重试一开始可能看起来稍微复杂一些，但它提供了非常灵活的 features and API and enables separation of concerns 测试重试逻辑和测试报告。

selenium testNG 重试结果帐户不正确

selenium testNG retry with incorrect result account

testng

selenium

testng-dataprovider

testng-eclipse