Caffeine LoadingCache 实现的单元测试单独通过，但在运行一起时失败

Question

我为我的 Caffeine CacheLoader 实现编写的单元测试（JUnit、Mockito）在我单独运行时全部成功，但是当我运行它们一起时其中一个失败.我相信我在使用 @Before 进行所有测试对象设置时遵循了最佳实践。

当运行与其他人一起时，测试 testGet_WhenCalledASecondAndThirdTimeBeyondCacheDuration_LoadingMethodCalledASecondTime 每次都失败并出现以下错误：

org.mockito.exceptions.verification.TooLittleActualInvocations: 
testDataSource.getObjectWithKey(
    "mountain-bikes"
);
Wanted 2 times:
-> at ErrorHandlingLoadingCacheFactoryTest.testGet_WhenCalledASecondAndThirdTimeBeyondCacheDuration_LoadingMethodCalledASecondTime(ErrorHandlingLoadingCacheFactoryTest.java:67)
But was 1 time:
-> at ErrorHandlingCacheLoader.load(ErrorHandlingCacheLoader.java:41)

有些东西似乎会在测试之间延续，但考虑到我在我的 @Before 方法中所做的事情，我不确定那是怎么回事。我尝试在 @After 方法中调用以下内容：

invalidateAll()
cleanUp()
Mockito.reset(testDataSource)

我还尝试手动将 singleThreadExecutor 传递给缓存构建器，并等待它完成它在 @After 中所做的任何事情，以防与它有关。

我的 Caffeine CacheLoader 实现只是将 reload 方法覆盖为 return 当前缓存的值，如果尝试刷新它失败（抛出异常）。除此之外，它非常香草。

@Component
public class ErrorHandlingLoadingCacheFactory {

    private final Ticker ticker;

    @Autowired
    public ErrorHandlingLoadingCacheFactory(Ticker ticker) {
        this.ticker = ticker;
    }

    public <T> LoadingCache<String, T> buildCache(String cacheName,
                                                        long duration,
                                                        TimeUnit timeUnit,
                                                        Function<String, T> valueResolver) {
        return Caffeine.newBuilder()
                .refreshAfterWrite(duration, timeUnit)
                .ticker(ticker)
                .build(new ErrorHandlingCacheLoader<>(cacheName, valueResolver));
    }
}

/**
 *  a LoadingCache that retains stale cache values if
 *  an attempt to retrieve a fresh value for a given key fails.
 *
 * @param <K> the cache key type
 * @param <V> the cache value type
 */
class ErrorHandlingCacheLoader<K, V> implements CacheLoader<K, V> {
    private final static Logger logger = LoggerFactory.getLogger(ErrorHandlingCacheLoader.class);

    private final String cacheName;
    private final Function<K, V> valueResolver;

    /**
     * Create a cache.
     *
     * @param cacheName the cache name
     * @param valueResolver the method used to get a value for a key
     */
    public ErrorHandlingCacheLoader(String cacheName, Function<K, V> valueResolver) {
        this.cacheName = cacheName;
        this.valueResolver = valueResolver;
    }

    /**
     * Load the initial cache value for a given key.
     * @param key the cache key
     * @return the initial value to cache
     */
    @Override
    public V load(@NonNull K key) {
        return valueResolver.apply(key);
    }

    /**
     * Attempt to reload a value for a given key.
     * @param key the cache key
     * @param oldValue the currently cached value for the given key
     * @return
     */
    @Override
    public V reload(@NonNull K key, V oldValue) {
        V value = oldValue;
        try {
            value = valueResolver.apply(key);
        } catch (RuntimeException e) {
            logger.warn("Failed to retrieve value for key '{}' in cache '{}'. Returning currently cached value '{}'.", key, cacheName, oldValue);
        }
        return value;
    }
}

public class ErrorHandlingLoadingCacheFactoryTest {

    private ErrorHandlingLoadingCacheFactory errorHandlingLoadingCacheFactory;

    private FakeTicker fakeTicker;
    private TestDataSource testDataSource;

    private LoadingCache<String, TestObject> loadingCache;

    @Before
    public void setUp() {
        fakeTicker = new FakeTicker();
        testDataSource = mock(TestDataSource.class);
        errorHandlingLoadingCacheFactory = new ErrorHandlingLoadingCacheFactory(fakeTicker::read);
        loadingCache = errorHandlingLoadingCacheFactory.buildCache("testCache", 1, TimeUnit.HOURS, testDataSource::getObjectWithKey);
    }

    @After
    public void tearDown() {
        validateMockitoUsage();
    }

    @Test
    public void testGet_WhenCalledTwiceWithinCachePeriod_LoadingMethodCalledOnce() {
        // Arrange
        TestObject testObject = new TestObject("Mountain Bikes");
        when(testDataSource.getObjectWithKey("mountain-bikes")).thenReturn(testObject);

        // Act
        TestObject result1 = loadingCache.get("mountain-bikes");
        TestObject result2 = loadingCache.get("mountain-bikes");

        // Assert
        verify(testDataSource, times(1)).getObjectWithKey("mountain-bikes");
        assertThat(result1).isEqualTo(testObject);
        assertThat(result2).isEqualTo(testObject);
    }

    @Test
    public void testGet_WhenCalledASecondAndThirdTimeBeyondCacheDuration_LoadingMethodCalledASecondTime() {
        // Arrange
        TestObject testObject1 = new TestObject("Mountain Bikes 1");
        TestObject testObject2 = new TestObject("Mountain Bikes 2");
        when(testDataSource.getObjectWithKey("mountain-bikes")).thenReturn(testObject1, testObject2);

        // Act
        TestObject result1 = loadingCache.get("mountain-bikes");
        fakeTicker.advance(2, TimeUnit.HOURS);
        TestObject result2 = loadingCache.get("mountain-bikes");
        TestObject result3 = loadingCache.get("mountain-bikes");

        // Assert
        verify(testDataSource, times(2)).getObjectWithKey("mountain-bikes");
        assertThat(result1).isEqualTo(testObject1);
        assertThat(result2).isEqualTo(testObject1);
        assertThat(result3).isEqualTo(testObject2);
    }

    @Test(expected = RuntimeException.class)
    public void testGet_WhenFirstLoadCallThrowsRuntimeException_ThrowsRuntimeException() {
        // Arrange
        when(testDataSource.getObjectWithKey("mountain-bikes")).thenThrow(new RuntimeException());

        // Act
        loadingCache.get("mountain-bikes");
    }

    @Test
    public void testGet_WhenFirstLoadCallSuccessfulButSecondThrowsRuntimeException_ReturnsCachedValueFromFirstCall() {
        // Arrange
        TestObject testObject1 = new TestObject("Mountain Bikes 1");
        when(testDataSource.getObjectWithKey("mountain-bikes")).thenReturn(testObject1).thenThrow(new RuntimeException());

        // Act
        TestObject result1 = loadingCache.get("mountain-bikes");
        fakeTicker.advance(2, TimeUnit.HOURS);
        TestObject result2 = loadingCache.get("mountain-bikes");

        // Assert
        verify(testDataSource, times(2)).getObjectWithKey("mountain-bikes");
        assertThat(result1).isEqualTo(testObject1);
        assertThat(result2).isEqualTo(testObject1);
    }

    @Test
    public void testGet_WhenFirstLoadCallSuccessfulButSecondThrowsRuntimeException_SubsequentGetsReturnCachedValueFromFirstCall() {
        // Arrange
        TestObject testObject1 = new TestObject("Mountain Bikes 1");
        when(testDataSource.getObjectWithKey("mountain-bikes")).thenReturn(testObject1).thenThrow(new RuntimeException());

        // Act
        TestObject result1 = loadingCache.get("mountain-bikes");
        fakeTicker.advance(2, TimeUnit.HOURS);
        TestObject result2 = loadingCache.get("mountain-bikes");
        TestObject result3 = loadingCache.get("mountain-bikes");

        // Assert
        verify(testDataSource, times(2)).getObjectWithKey("mountain-bikes");
        assertThat(result1).isEqualTo(testObject1);
        assertThat(result2).isEqualTo(testObject1);
        assertThat(result3).isEqualTo(testObject1);
    }

    @Test(expected = NullPointerException.class)
    public void testGet_WhenKeyIsNull_ThrowsNullPointerException() {
        // Arrange
        String key = null;

        // Act
        loadingCache.get(key);
    }

    @Test
    public void testGet_WhenFirstLoadCallReturnsNull_DoesNotCacheResult() {
        // Arrange
        TestObject testObject1 = new TestObject("Mountain Bikes 1");
        when(testDataSource.getObjectWithKey("mountain-bikes")).thenReturn(null).thenReturn(testObject1);

        // Act
        TestObject result1 = loadingCache.get("mountain-bikes");
        TestObject result2 = loadingCache.get("mountain-bikes");

        // Assert
        verify(testDataSource, times(2)).getObjectWithKey("mountain-bikes");
        assertThat(result1).isEqualTo(null);
        assertThat(result2).isEqualTo(testObject1);
    }

    @Data
    class TestObject {
        private String id;
        public TestObject(String id) {
            this.id = id;
        }
    }

    interface TestDataSource {
        TestObject getObjectWithKey(String key);
    }
}

Answer 1

Ben Manes 在他的评论中建议我在运行单元测试时使用 Runnable::run 作为 LoadingCache 的执行器，这成功了！

我在我的工厂中实施了第二个受保护的 buildCache 方法，该方法另外采用 Executor 参数，我的测试 class 使用该参数传递 Runnable::run.

更新后的ErrorHandlingLoadingCacheFactory:

public class ErrorHandlingLoadingCacheFactory {

    private final Ticker ticker;

    @Autowired
    public ErrorHandlingLoadingCacheFactory(Ticker ticker) {
        this.ticker = ticker;
    }

    /**
     * Create an in-memory LoadingCache
     *
     * @param cacheName the name of the cache
     * @param duration how long to keep values in the cache before attempting to refresh them
     * @param timeUnit the unit of time of the given duration
     * @param valueResolver the method to call to get a value to load into the cache for a given key
     * @param <T> the type of object to store into the cache
     * @return the newly created cache
     */
    public <T> LoadingCache<String, T> buildCache(String cacheName,
                                                        long duration,
                                                        TimeUnit timeUnit,
                                                        Function<String, T> valueResolver) {
        return buildCache(cacheName, duration, timeUnit, valueResolver, ForkJoinPool.commonPool());
    }

    /**
     * Create an in-memory LoadingCache
     *
     * @param cacheName the name of the cache
     * @param duration how long to keep values in the cache before attempting to refresh them
     * @param timeUnit the unit of time of the given duration
     * @param valueResolver the method to call to get a value to load into the cache for a given key
     * @param executor the executor for the cache to use
     * @param <T> the type of object to store into the cache
     * @return the newly created cache
     */
    protected <T> LoadingCache<String, T> buildCache(String cacheName,
                                                     long duration,
                                                     TimeUnit timeUnit,
                                                     Function<String, T> valueResolver,
                                                     Executor executor) {
        return Caffeine.newBuilder()
                .refreshAfterWrite(duration, timeUnit)
                .ticker(ticker)
                .executor(executor)
                .build(new ErrorHandlingCacheLoader<>(cacheName, valueResolver));
    }
}

ErrorHandlingLoadingCacheFactoryTest中更新的setUp()方法：

...
@Before
    public void setUp() {
        fakeTicker = new FakeTicker();
        testDataSource = mock(TestDataSource.class);
        errorHandlingLoadingCacheFactory = new ErrorHandlingLoadingCacheFactory(fakeTicker::read);
        loadingCache = errorHandlingLoadingCacheFactory.buildCache("testCache", 1, TimeUnit.HOURS, testDataSource::getObjectWithKey, Runnable::run);
    }
...

我的单线程执行程序没有捕捉到我的测试之间一定存在竞争，可能是因为我没有正确地等待它在我的 @After 方法中终止。 Ben 建议如果我在单线程执行器上使用 awaitTermination，那也可能有效。

Caffeine LoadingCache 实现的单元测试单独通过，但在运行一起时失败

Unit tests on Caffeine LoadingCache implementation pass individually but one fails when run together

java

junit

caching

mockito

caffeine

Caffeine LoadingCache 实现的单元测试单独通过，但在 运行 一起时失败

Unit tests on Caffeine LoadingCache implementation pass individually but one fails when run together

java

junit

caching

mockito

caffeine

Caffeine LoadingCache 实现的单元测试单独通过，但在运行一起时失败