使用 resource-lock 进行并行测试？

Question

背景: 现在使用java JUnit4，愿意迁移到JUnit5或TestNG。

当前状态：有 100 多个 Selenium 测试。在 Junit4 中，它们中的大多数通过 @RunWith(Parameterized.class) 重复。（即根据提供的参数集创建多个测试实例 class，通常是浏览器类型 + 用户身份的组合。）共享大约 12 个用户的有限集合。

限制：被测试的应用程序阻止同一用户同时在多个地方登录。因此，如果用户在一个线程中的某个测试运行中登录应用程序，则会导致同一用户在同一时刻在另一个线程中的另一个测试运行中立即注销。

问题：当并行执行的测试无法共享某些资源时，是否有任何推荐的方法来管理线程安全？或者如何强制那些使用相同资源的测试在同一个线程中执行？

谢谢你的想法。

这是我目前使用 TestNG 找到的一些解决方案的简化示例...：

public abstract class BaseTestCase {
    protected static ThreadLocal<WebDriver> threadLocalDriver = new ThreadLocal<>();
    protected String testUserName;

    private static final Set<String> inUse = new HashSet<>();

    public BaseTestCase(WebDriver driver, String testUserName) {
        threadLocalDriver.set(driver);
        this.testUserName = testUserName;
    }

    private boolean syncedAddUse(@NotNull String key){
        synchronized (inUse){
            return inUse.add(key);
        }
    }

    private boolean syncedRemoveUse(@NotNull String key){
        synchronized (inUse) {
            return inUse.remove(key);
        }
    }

    @DataProvider(parallel = true)
    public static Object[][] provideTestData() {
        //load pairs WebDriver+user from config file. E.g.:
        //Chrome + chromeUser
        //Chrome + chromeAdmin
        //Firefox + firefoxUser
        //etc...
    }

    @BeforeMethod
    public void syncPoint() throws InterruptedException {
        while( !syncedAddUse(testUserName) ){
            //Waiting due the testUserName is already in use at the moment.
            Thread.sleep(1000);
        }
    }

    @AfterMethod
    public void leaveSyncPoint(){
        syncedRemoveUse(testUserName);
    }
}

所以我可以进行很多测试 class，例如：

public class TestA extends BaseTestCase {

    @Factory(dataProvider = "provideTestData")
    public TestA(WebDriver webDriver, String testUserName) {
        super(webDriver, testUserName);
    }

    public void someTest() {
        WebDriver driver = threadLocalDriver.get();
        threadLocalDriver.get().navigate().to("http://myPage.example.com");
        logintoMyPageWithUser(driver, testUserName);
        doSomeStuffOnPage(driver);
        logoutUserFromPage(driver);
    }
    ...
}

并且所有测试都是通过 testNG.xml 启动的，如下所示：

<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="sample suite" verbose="1" parallel="instances" thread-count="20" data-provider-thread-count="10">
    <test name="sample test" >
        <packages>
            <package name="com.path_to_package_with_example" />
        </packages>
    </test>
</suite>

这个解决方案的孩子。但是，我讨厌那里的 Thread.sleep() 。它创建了许多线程，其中大多数线程一直在等待彼此。我更愿意将使用同一用户的所有测试排列到同一线程并尽量减少等待时间。

Answer 1

我不知道有什么方法可以在每个组都在一个线程中运行的组中组织测试。但是您可以将 "while user busy sleep" 替换为 "try lock on user"。后者在对用户完成另一项测试（即解锁锁）后立即继续执行。

下面的可运行示例应该让您开始了解 "try lock on user" 想法。请记住，如果您获得锁（在您的情况下为 "beforeTest"），您必须确保在 "finally" 块中释放锁（在您的情况下为 "afterTest"）。否则执行会挂起并且永远不会完成。

import java.util.Map;
import java.util.Random;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.locks.ReentrantLock;
import java.util.concurrent.TimeUnit;
import java.util.stream.IntStream;

// 
public class NamedResourceLocks {

    public static void main(String[] args) {

        System.out.println("Starting");
        ExecutorService executor = Executors.newCachedThreadPool();
        try {
            new NamedResourceLocks().run(executor);
        } catch (Exception e) {
            e.printStackTrace();
        } finally {
            executor.shutdownNow();
        }
        System.out.println("Done");
    }

    final static String userPrefix = "user";
    final static int maxUsers = 3;
    final static long maxWait = 10_000; // 10 seconds
    final static long startTime = System.currentTimeMillis();

    final Map<String, ReentrantLock> userLocks = new ConcurrentHashMap<>();
    final int maxTests = maxUsers * 10;
    final CountDownLatch allTestsDone = new CountDownLatch(maxTests);

    void run(ExecutorService executor) throws Exception {

        IntStream.range(0,  maxUsers).forEach(u -> 
            userLocks.put(userPrefix + u, new ReentrantLock(true)));
        IntStream.range(0,  maxTests).forEach(t -> 
            executor.execute(new Test(this, random.nextInt(maxUsers), t)));
        if (allTestsDone.await(maxWait, TimeUnit.MILLISECONDS)) {
            System.out.println("All tests finished");
        }
    }


    void lock(String user) throws Exception {

        ReentrantLock lock = userLocks.get(user);
        if (!lock.tryLock(maxWait, TimeUnit.MILLISECONDS)) {
            throw new RuntimeException("Waited too long.");
        }
    }

    void unlock(String user) {

        userLocks.get(user).unlock();
    }

    void oneTestDone() {

        allTestsDone.countDown();
    }

    final static Random random = new Random();

    static class Test implements Runnable {

        final NamedResourceLocks locks;
        final String user;
        final int testNumber;

        public Test(NamedResourceLocks locks, int userNumber, int testNumber) {
            this.locks = locks;
            this.user = userPrefix + userNumber;
            this.testNumber = testNumber;
        }

        @Override
        public void run() {

            boolean haveLock = false;
            try {
                log(this, "acquiring lock");
                locks.lock(user);
                haveLock = true;
                int sleepTime = random.nextInt(maxUsers) + 1; 
                log(this, "sleeping for " + sleepTime + " ms.");
                Thread.sleep(sleepTime);
            } catch (Exception e) {
                e.printStackTrace();
            } finally {
                if (haveLock) {
                    log(this, "releasing lock");
                    locks.unlock(user);
                }
                locks.oneTestDone();
            }
        }

    }

    static void log(Test test, String msg) {
        System.out.println((System.currentTimeMillis() - startTime) + " - " +
                test.testNumber + " / " + test.user + " - " + msg);
    }
}

使用 resource-lock 进行并行测试？

Parallel tests with resource-lock?

java

testng

multithreading

parallel-testing

junit5