Skip to content

[Mule 3.7.0 M1] Registry Consolidation, Lifecycle fix, and Dependency Injection

Jump to bottom Edit New page
Mariano Gonzalez edited this page Mar 12, 2015 · 5 revisions

Jira: https://www.mulesoft.org/jira/browse/MULE-7588 Other issues fixed by this:

Motivation / Context

Registry Split

As we all know, the MuleContext contains the concept of an object registry. The mule registry is basically a set of key-value pairs used to gain access to relevant mule components, such as connectors, factories, object stores, managers (such as the QueueManager), etc.

Registries in Mule have a number of attributes: They are composable via a RegistryBroker which allows n registries to be added and used. (The original reason for creating this composite registry model was to support OSGi with each application bundle contributing a configured registry to a shared muleContext with multiple registries(1 per app), this was also then used with tomcat-based hot-deployment.) The registry implementation is decoupled via a Registry interface. This is so to achieve two goals: To allow the core mule to not depend on a third party container such as Spring or Guice. Currently however, the SpringRegistry is the only implementation used. To allow users running in embedded mode to plug their own object container.

CONTEXT NOTE: We checked with support, we currently know of no customer which is actually adding his own registry.

As a result, when a MuleContext is created, it creates a registry broker with two registries; i) an instance of TransientRegistry, which is a very basic implementation of the Registry contract ii) On top of that, the spring-config module also adds a SpringRegistry, which is another implementation of Registry which relies on Spring.

The registered mule components are then split as follows:

The TransientRegistry contains:

  • All the objects declared in a registry-bootstrap.properties file
  • Any objects created and registry manually via muleContext.getRegistry().registerXX(), which usually happens in CoreExtension and Agent instances

The Spring Registry contains:

  • Every component that was created through an XML config (connectors, configs, flows, message processors, etc).

Spring is then used to host most of the registry objects, including every component which is created from reading an application’s XML configuration (since Spring is used to parse the XML). The TransientRegistry is given priority over every other registry. When an object is requested from the main registry, it searches all registries until a match is found. Because the TransientRegistry goes first, its objects will always have priority.

Interoperability and dependency injection

Because the objects live in two separate registries, then some issues arise:

  • Dependency Injection between registries is out of the question. If an object in registry A depends on an object X from registry B, there’s no way to perform that injection (registries start in order)
  • Objects in the TransientRegistry cannot access the ones in other registries. For example, the QueueManager lives in the TransientRegistry. That means that when it’s initialised it cannot access the main mule configuration, because such configuration is set through XML and thus lives in the Spring registry, which is started later.
  • Overrides are inconsistent. For example, the Cluster extension replaces the default QueueManager and ObjectStore instances by cluster aware versions. Because the TransientRegistry has priority, those are only replaced there. That works fine when doing a manual lookup, but if a component uses Spring dependency injection to get a QueueManager, it will get the local version instead.

The bottom line is that there’s a functional dependency between the TransientRegistry and the SpringRegistry, due to the fact that:

  • The Spring registry contains the SimpleRegistryBootstrap
  • The Spring registry contains the MuleContext
  • Because the Spring registry starts flows, connectors, batch jobs and other components that depend on objects started through the registry bootstrap, we can't start the spring registry before the TransientRegistry.

These issues prevents us from:

  • Have a solid IoC mechanism
  • Have a consistent lifecycle across registries

Lifecycle breaks its own contract

Finally but definitely not least, the probably most serious issue with the current registry implementation is how it manages lifecycle. Although this is the most severe problem, I saved it for last because it’s greatly influenced by all the other problems.

The registry has an additional responsibility aside from the ones already described, which Ross Mason described as follows when he first came out with the concept: “The registry is a magic bucket which ensures that all objects are in the same lifecycle state”.

The lifecycle is comprised by 4 phases:

  • Initialise
  • Start
  • Stop
  • Dispose

This is conceptually fine, but the implementation has a lot of problems. The lifecycle is not implemented as something that leverages the registry to make sure that all the important components are transitioned harmoniously. Instead, each registry is in charge of portions of that lifecycle, while some others are handled separately. This leads to inconsistent behavior:

The transient registry performs the initialise() operation whenever an object is registered. This means that if the object depends on some other which hasn’t still been registered, it won’t find it (I’m not talking about dependency injection here, this doesn’t work even with a 90’s like lookup) The SpringRegistry on the other hand, automatically executes the initialise phase when the bean is instantiated, which means that the initialising object might depend on other objects which are not yet instantiated. This only happens with objects that mule registers on its own, if the user registers a spring bean, lifecycle is not applied to it. The SpringRegistry also shows a similar behavior towards the dispose phase. Because the functionality is part of the registry instead of leveraging it, each new registry can potentially have a different behavior.

Uses cases

  • As a developer, I want to be able to use dependency injection either through Spring XML definitions or by the use of JSR-330 annotations
  • As a developer, I want to be able to rely on the platform to have fully instantiated and injected all managed objects BEFORE the lifecycle is applied

Other than this particular use cases, the overall goal of fixing this is that as we start to dig deeper into preparing the next major version of our platform, fixing these issues are critical to achieve the such vision.

Non Functional Requirements

As a developer and a user, I don’t want this fix to break backwards compatibility

Registry unification

As stated before, the main reason to have registries split up is to avoid a direct dependency between the mule-core module and Spring (or any other similar framework). Truth of the matter is that although it is desirable to maintain that separation, the only real reason to have something like the TransientRegistry is running tests. On runtime, the SpringRegistry is discovered and created through SPI, keeping it decoupled from the core. Because Spring is a fully fledged DI container which suites all of our needs, there’s no real reason to build our own DI container or to have the TransientRegistry on runtime.

Converging on one and only Spring registry allows us to:

  • All objects are now in a single registry. No interoperability problems
  • Spring already knows how to create all objects in the correct order and inject dependencies into them, even in cases of circular dependencies.
  • Spring provides OOTB support for JSR-330 (DI for the win!)
  • Spring is a proven tool that we already rely on. It just doesn’t make sense to try to build our own

This unification pretty much solves all the problems in the “Registry split” and “Interoperability and dependency injection” sections of the Motivation part of this document.

This is however, easier said than done.

Replacing the TransientRegistry

TransientRegistry will be deprecated as of Mule 3.7.0. For backwards compatibility reasons, it will still be part of the distribution but it will not be used anymore. Along with it, the InjectProcessor and PreInitProcessor interfaces (and all their implementations) are also deprecated and will also not be used anymore. All these deprecated components are replaced by a matching BeanPostProcessor in the SpringRegistry.

SimpleRegistry

All the tests that directly extend the AbstractMuleContextTestCase need the MuleContext to have a registry to function. Because of the decoupling requirements already stated, we don’t want that registry to be a Spring one. However, we need a registry that behaves “similar enough” to it. So we created a new registry called SimpleRegistry. This registry has the following properties:

Lightweight, * simple enough for testing implementation of the Registry interface

  • NOT RECOMMENDED FOR PRODUCTION USE. The actual runtime will never instantiate it
  • Unlike the TransientRegistry, it will not apply any lifecycle operation when registering/unregistering any objects
  • It has really basic support for JSR-330. It will perform dependency injection only on fields annotated with @Inject. Notice that this is not even close to fully supporting that JSR and that the way it’s implemented is really not performant

This registry will be automatically added to the MuleContext on all subclasses of AbstractMuleContextTestCase which don’t extends FunctionalTestCase.

The FunctionalTestCase base class will continue to behave as usual, only that it will only use the SpringRegistry. No SimpleRegistry or TransientRegistry will be used in that case (which mimics how the actual runtime will behave).

Bootstrap

If the TransientRegistry is being replaced by the SpringRegistry, that means that the latter (or some other component) needs to assume responsibility for all the tasks that it was performing. One of those tasks is processing the registry-bootstrap.properties files.

To provide a bootstrapping mechanism that is independent from Spring, Mule searches the classpath for files called registry-bootstrap.properties. That simple properties file contains keys and class names for objects to be automatically added to the registry. A component called SimpleRegistryBootstrap was responsible for creating Instances of those classes and registering them in the TransientRegistry. This was problematic because:

As previously stated, the TransientRegistry wrongfully applied the initialise() phase when those objects were registered The SimpleRegistryBootstrap object was fired as a spring bean, which created a very odd functional dependency between the TransientRegistry and the SpringRegistry.

SimpleRegistryBootstrap will also be deprecated as of 3.7.0, and is replaced with a new one called SpringRegistryBootstrap which is really similar in terms of functionality but with one key difference: it doesn’t instantiate and registers objects, instead it creates and registers BeanDefinition objects which are added to the Spring BeanFactory before it is initialised. In that way, these bootstrap objects go through the same creation-injection-initialisation cycle as the components defined in XML.

Override mechanisms

Another mechanism that we need to keep backwards compatibility with is Agents and CoreExtensions. A great example of that is the ClusterCoreExtension, which replaces the configured QueueManager and ObjectStore instances, while also adding some objects of its own like the HazelcastManager.

Before this refactor, that mechanism worked because the TransientRegistry already existed to receive those objects, and because it had precedence over the SpringRegistry, a call to muleContext.getRegistry().lookupObject(QueueManager.class) will return the version in the TransientRegistry instead of the one in Spring.

However, this is not a consistent behavior, because the user can declare this:

In this case, the myComponent bean would get the local QueueManager instead of the cluster one.

This issue is solved by the unification of the registries, but it introduces another problem: by the time the CoreExtension is executed, the SpringRegistry doesn’t exist yet.

The solution to that problem goes as follows:

When the MuleContext is created, by default it will still have one registry, but it will no longer be a TransientRegistry but a SimpleRegistry The SimpleRegistry will catch any object registered before Spring kicks in When the SpringRegistry is created, its ConfigurationBuilder will iterate through all other registries and will take over its registered objects. Objects and registries that have been taken over, are said to have been monopolized Each monopolized registry is removed from the MuleContext When the SpringRegistry is asked for a monopolized object, it guarantees that the return instance is the exact same that used to live in the SimpleRegistry

In this way we achieve:

Backwards compatibility with agents and core extensions A consistent mechanism for overriding definitions The overridden objects get a fair treatment regarding initialisation and dependency injection

Adding custom registries

We will continue to support adding user registries, but since we know of nobody doing that and because we already identified that as a source of problems, that functionality will be deprecated.

Also, dependency injection will not be supported on custom registries.

Lifecycle

The only missing piece after doing all the changes above, is to no longer rely on the object container to fire the lifecycle phases anymore. Considering the notion of the registry as a “magic bucket” which makes all components transition together, then yes, registries are still responsible for firing those phases on the objects they own, but that’s a concern of each particular implementation of the Registry interface, not something we delegate into a Spring init-method for example.

In that way, the Registry interface continues to extend the Lifecycle one but lifecycle will only be applied once that:

All objects are instantiated All objects have been fully injected All post processors have been executed A LifecycleManager instance is available to make this transitions. No manual invocation of lifecycle methods at a registry level

Initialisation order

As explained before, the initialisation order was not being respected by either the TransientRegistry nor the Spring one. Now that initialisation only fires once all objects are injected, the order being respected becomes more relevant.

The initialisation order is this:

ObjectStoreManager Config Connector Agent Model FlowConstruct Initialisable

Notice that a new interface has been added: Config. That is a new marker interface that is to be implemented by configs of new connector-ish modules such as the new Http connector introduced in 3.6. For now it’s just an empty interface, but we reserve the right to grow it.

GOTCHA: You might be wondering what happens if a given object implements more than one of those interfaces. There’s logic present to ensure that the same object is not initialised twice.

Initialise dependencies first

Another advantage of this change is that now we can fire the phases not only in a type base order, but we can also do it based on dependency order. For example, if object A depends on B, we will now be able to know that B needs to get its lifecycle phases before A. This will of course work only as long as the dependency is expressed through a Spring bean definition or by the use of JSR-330 annotations Risks

The 11K+ tests of the ESB passes with these changes. However, we did found some tests which had bogus lifecycle code designed to work around the issues described in this document. Thus, there’s a risk of cloud connectors and custom components which are also taking these bugs as a feature to fail initialisation.

However we believe however that if we clearly notify these changes and provide a well guided migration guide, those components (if they exists) should be adapted easily (it would not be new behavior, it would be removing a workaround after the original bug was fixed).

Impact

Studio Impact

Not initial impact in Studio is detected before hand, but the Studio team should advice.

DevKit Impact

Devkit should verify that:

The lifecycle changes don’t break any of the supported connectors Devkit provides manual support for @Inject to be applied to a limited set of types. We should verify that the manual support doesn’t break with the new platform provided support, and that JSR-330 works well with all the Devkit adapters and wrappers

MMC Impact

Sanity check

CH Impact

Sanity check. Verify that the CH custom modules keep functioning correctly

API Manager impact

The gateway use its own registries. This spec was discussed with part of the Gateway team and initially they didn’t see this as a major problem, but they will need to test and investigate further. Sanity check. Verify that the custom modules keep functioning correctly

MUNIT

Sanity check Migration Impact

Only custom components which used the lifecycle bug as a feature. We should provide documentation, blog posts and migration guide for such cases.

Documentation Impact

Add migration guide List deprecated components and its advised replacements Add examples of how to use JSR-330 We could create a docs page explaining the inner mechanics of this (such as the ObjectLimbo), but I’m not convinced that is worth it since this is not really a new feature but a long overdue bug fix

Add a custom footer
Add a custom sidebar
Clone this wiki locally