`@MockitoBean` incorrectly injects supertype into subtype field

[nstdio]

Interesting problem when using @MockitoBean with Spring Boot 3.4.0. Here's the test case to reproduce it. Please run it couple of times because it sometimes does work.

package com.example;

import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.springframework.test.context.bean.override.mockito.MockitoBean;
import org.springframework.test.context.junit.jupiter.SpringExtension;

@ExtendWith(SpringExtension.class)
class MockitoBeanTest {

  @MockitoBean
  private Foo foo;

  @MockitoBean
  private Bar bar;

  @Test
  void test() {
  }

  interface Foo {

  }

  interface Bar extends Foo {

  }
}

Sometimes test case fails with following exception:

[main] WARN org.springframework.test.context.TestContextManager -- Caught exception while allowing TestExecutionListener [org.springframework.test.context.bean.override.BeanOverrideTestExecutionListener] to prepare test instance [com.example.MockitoBeanTest@27ace0b1]
org.springframework.beans.factory.BeanCreationException: Could not inject field 'private com.example.MockitoBeanTest$Bar com.example.MockitoBeanTest.bar'
	at org.springframework.test.context.bean.override.BeanOverrideRegistry.inject(BeanOverrideRegistry.java:98)
	at org.springframework.test.context.bean.override.BeanOverrideRegistry.inject(BeanOverrideRegistry.java:88)
	at org.springframework.test.context.bean.override.BeanOverrideTestExecutionListener.injectFields(BeanOverrideTestExecutionListener.java:91)
	at org.springframework.test.context.bean.override.BeanOverrideTestExecutionListener.prepareTestInstance(BeanOverrideTestExecutionListener.java:58)
	at org.springframework.test.context.TestContextManager.prepareTestInstance(TestContextManager.java:260)
	at org.springframework.test.context.junit.jupiter.SpringExtension.postProcessTestInstance(SpringExtension.java:160)
	at org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.lambda$invokeTestInstancePostProcessors$11(ClassBasedTestDescriptor.java:378)
	at org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.executeAndMaskThrowable(ClassBasedTestDescriptor.java:383)
	at org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.lambda$invokeTestInstancePostProcessors$12(ClassBasedTestDescriptor.java:378)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
	at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:179)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
	at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1708)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
	at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596)
	at org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.invokeTestInstancePostProcessors(ClassBasedTestDescriptor.java:377)
	at org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.lambda$instantiateAndPostProcessTestInstance$7(ClassBasedTestDescriptor.java:290)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.instantiateAndPostProcessTestInstance(ClassBasedTestDescriptor.java:289)
	at org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.lambda$testInstancesProvider$5(ClassBasedTestDescriptor.java:279)
	at java.base/java.util.Optional.orElseGet(Optional.java:364)
	at org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.lambda$testInstancesProvider$6(ClassBasedTestDescriptor.java:278)
	at org.junit.jupiter.engine.execution.TestInstancesProvider.getTestInstances(TestInstancesProvider.java:31)
	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$prepare$1(TestMethodTestDescriptor.java:105)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.prepare(TestMethodTestDescriptor.java:104)
	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.prepare(TestMethodTestDescriptor.java:68)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$prepare$2(NodeTestTask.java:128)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.prepare(NodeTestTask.java:128)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:160)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:146)
	at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:144)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:143)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:100)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:160)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:146)
	at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:144)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:143)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:100)
	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:35)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:54)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85)
	at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47)
	at org.junit.platform.launcher.core.SessionPerRequestLauncher.execute(SessionPerRequestLauncher.java:63)
	at com.intellij.junit5.JUnit5IdeaTestRunner.startRunnerWithArgs(JUnit5IdeaTestRunner.java:57)
	at com.intellij.rt.junit.IdeaTestRunner$Repeater$1.execute(IdeaTestRunner.java:38)
	at com.intellij.rt.execution.junit.TestsRepeater.repeat(TestsRepeater.java:11)
	at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:35)
	at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:232)
	at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:55)
Caused by: org.springframework.beans.factory.BeanNotOfRequiredTypeException: Bean named 'com.example.MockitoBeanTest$Bar#0' is expected to be of type 'com.example.MockitoBeanTest$Bar' but was actually of type 'com.example.MockitoBeanTest$Foo$MockitoMock$epkkeH3d'
	at org.springframework.beans.factory.support.AbstractBeanFactory.adaptBeanInstance(AbstractBeanFactory.java:421)
	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:402)
	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:204)
	at org.springframework.test.context.bean.override.BeanOverrideRegistry.inject(BeanOverrideRegistry.java:93)
	... 74 common frames omitted

[nstdio]

Sorry it's bit synthetic but here is consistently failing version

package com.example;

import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.springframework.test.context.bean.override.mockito.MockitoBean;
import org.springframework.test.context.junit.jupiter.SpringExtension;

@ExtendWith(SpringExtension.class)
class MockitoBeanTest {

  @MockitoBean
  private Foo1 foo1;
  @MockitoBean
  private Bar1 bar1;

  @MockitoBean
  private Foo2 foo2;
  @MockitoBean
  private Bar2 bar2;

  @MockitoBean
  private Foo3 foo3;
  @MockitoBean
  private Bar3 bar3;

  @Test
  void test() {
  }

  interface Foo1 {

  }

  interface Bar1 extends Foo1 {

  }

  interface Foo2 {

  }

  interface Bar2 extends Foo2 {

  }

  interface Foo3 {

  }

  interface Bar3 extends Foo3 {

  }
}

For what it's worth I think it depends on beanOverrideHandlers iteration order here

spring-framework/spring-test/src/main/java/org/springframework/test/context/bean/override/BeanOverrideBeanFactoryPostProcessor.java

Lines 96 to 100 in da75840

	public void postProcessBeanFactory(ConfigurableListableBeanFactory beanFactory) throws BeansException {
	for (BeanOverrideHandler handler : this.beanOverrideHandlers) {
	registerBeanOverride(beanFactory, handler);
	}
	}

The beanOverrideHandlers is HashSet acually created here and passed down to BeanOverrideBeanFactoryPostProcessor.

If it happens so that handler for Foo (supertype) comes before Bar (subtype) handler then Bar handler will pickup bean definition for Foo and obviously bean definition for Foo will produce Foo instance. This might explain why my original test class fails only sometimes and this test class with 6 interfaces fails more consistently.

Additionally, if we set unique MockitoBean#name for each field test does not fail.

[sbrannen]

Thanks for bringing this to our attention, @nstdio. 👍

This indeed turns out to be an interesting one, and I've had some "debugging fun" in order to get to the bottom of it.

For what it's worth I think it depends on beanOverrideHandlers iteration order

Yes, it depends on the order in which the handlers/fields are processed.

It turns out this this bug exists with @MockBean in Spring Boot as well.

For example, the following @MockBean test class consistently fails with the Subtype field declared first.

@ExtendWith(SpringExtension.class)
class MockBeanTest {

	@MockBean
	Subtype subtype;

	@MockBean
	Supertype supertype;

	@Test
	void test() {
	}

	interface Supertype {}

	interface Subtype extends Supertype {}
}

Additionally, if we set unique MockitoBean#name for each field test does not fail.

Yep, and it also succeeds if you apply @Qualifier to all or the Bar* fields or to all of the Foo* fields.

Thus, when it fails it's because mocks are created "by type" without an explicit name or qualifier, and if a mock for a subtype is created first (e.g., Bar3), then the subsequent lookup for a bean of the supertype (e.g., Foo3) finds the newly created mock bean for the subtype and replaces that newly created bean with a mock of the supertype, effectively removing the mock for the subtype.

[sbrannen]

This has been fixed in 6.2.x and main and will available in upcoming 6.2.1-SNAPSHOT builds in case you want to test it before the 6.2.1 release.

Thus, when it fails it's because mocks are created "by type" without an explicit name or qualifier, and if a mock for a subtype is created first (e.g., Bar3), then the subsequent lookup for a bean of the supertype (e.g., Foo3) finds the newly created mock bean for the subtype and replaces that newly created bean with a mock of the supertype, effectively removing the mock for the subtype.

And I decided to call that the "Phantom Read problem for Bean Overrides in the TestContext framework". 🤣

See aa7b459 for details.