While we are refactoring CloudStack architecture, to lay out a better component oriented foundation, be open to tools that most developers are familiar with in Java community, we started to experience the usage of Apache Spring Framework, switching to Spring touches a lot of existing CloudStack codebase, however, it does not bring any business logic changes, but it does introduce changes that CloudStack developers need to be aware of.
CloudStack management server contains a collection of various components, components usually come with 4 different flavors, common framework components, manager components, adapter components and DAO components. In post-deploy time, CloudStack allows experienced customer to compose a customized setup through editing of configuration files. For CloudStack OSS (OpenSource) distribution, developers provides default configuration in applicationContext.xml.in and componentContext.xml.in, while for non-OSS distribution, the configuration will be provided in applicationContext.xml.in and nonossComponentContext.xml.in.
For all components that are mandatory and shared among both OSS distribution and non-OSS distribution, they should be declared in applicationContext.xml.in. Optional components, depends on whether or not they are specific to OSS or non-OSS distribution, they can appear in componentContext.xml.in and nonossComponentContext.xml.in, either in one of the two files or the both files.
If a certain component has different configuration for OSS and non-OSS distribution, it should go both to applicationContext.xml.in and noncomponentContext.xml.in.
Use following flow to determine where your newly developed component should go
if (component is mandatory to OSS/non-OSS ) {
if( component shares common configuration in both OSS and non-OSS distributions ) {
put it in applicationContext.xml.in
} else {
put it in both componentContext.xml and nonossComponentContext.xml.in
}
} else {
// optional component
if(component is for OSS only)
put it in componentContext.xml
else if(component is for non-OSS only)
put it in nonossComponentContext.xml
else
put in both componentContext.xml.in and nonossComponentContext.xml.in
} |
Spring component is nothing more than a Java POJO(Plain Old Java Object) object, once you've determined which file the component declaration should go using the flow at above. declaring it is fairly simple. Following gives an example
<bean id="configurationDaoImpl" class="com.cloud.configuration.dao.ConfigurationDaoImpl" /> |
ID of the component should be unique, as original configuration files are generated automatically from legacy code, it is usually named after the component class name.
Avoid using @Component annotation to declare your component, it is not JSR compliance, and we no longer support component auto-scanning to load up components annotated by @Component
One of the biggest advantage of switching to Spring is that Auto-wiring in components has now become consistent everywhere. With legacy ComponentLocator, there are a lot of places that have to use run-time wiring, for example, using ComponentLocator.getManager() to wire a reference to a manager component. The reason for developers to do so is that ComponentLocator does not fully resolve dependent-injection for components, so when inter-component relationship becomes complex in a large system like CloudStack, lots of hacking ways rise up inside CloudStack codebase. Although it solves the immediate needs, it also creates a lot of confusion for developers.
Following is an example of such hacking way, inside BaseCmd class, developer tried to resolve all the references for every used components in all sub-classes into static variables at BaseCmd. It basically means that, when you add a new Command class to the system, you will have to remember to do something about your new service at BaseCmd class
public abstract class BaseCmd {
static public ConfigurationService _configService;
static public AccountService _accountService;
static public UserVmService _userVmService;
static public ManagementService _mgr;
// more code ...
// code to resolve above static references using runtime resolution
}
|
With Spring, developer can now always use @Inject annotation to declare such reference, and most importantly, inject these references only when they are needed at individual Command class, so that each Command class can become more independent to each other. This is one of the top two reasons for us to try Spring, it can give us a cleaner component coding practice. (the other top reason is that we have broader integration support from many other third-party vendors).
Although we've switched whole code base to Spring, there are still a lot of legacy coding-practices like above that need to cleanup. Once you've seen one of these, feel free to help cleanup them.
With legacy ComponentLocator, we have many places that use run-time resolution for fields that can't be resolved automatically by ComponentLocator's injection, and in many of these cases, these run-time resolutions happen at object construction time. Under Spring, such way won't work any more. The reason is that when Spring constructs the component object, the added runtime logic running inside the constructor can not feed information back into Spring injection framework, it may cause the process to break. The solution to this issues is to always leave the object construction and auto-wiring work to Spring, and leave the component initialization later.
Protected or private constructors may be a good coding practice to enforce certain usage pattern of the class. However, protected or private constructors are not friendly to Spring injection process. Please always use public constructors, for the same reason, if you POJO class wants to be Spring component, don't seal the class by putting final modifier to your class
It is a good practice to separate component construction and initialization, always try to make your component be self-dependable in initialization, self-dependable initialization means the component is able to initialize itself with the very minimal dependency to other component's initialization status. Use @PostConstruct to mark such initialization method for Spring to automatically call for you. Following is an example.
public class ConfigurationDaoImpl extends GenericDaoBase<ConfigurationVO, String> implements ConfigurationDao {
@PostConstruct void initComponent() { // more code... }
}
|
Although we've tried the best to cut the relationship to a component container inside component code, we still have very few places in framework components that need to be aware of existence of the component container, to avoid any strong binding to a particular container like Spring, we introduced a class called ComponentContext, it is responsible to bridge CloudStack component with a chosen component container (for now, it is Spring).
As a business logic component developer, you should limit direct use of ComponentContext with one exception: using ComponentContext.inject() to auto-wire components. Following example gives a scenario.
Class<?> cmdClass = getCmdClass(command[0]);
if (cmdClass != null) {
BaseCmd cmdObj = (BaseCmd) cmdClass.newInstance();
cmdObj = ComponentContext.inject(cmdObj);
cmdObj.configure();
cmdObj.setFullUrlParams(paramMap);
cmdObj.setResponseType(responseType);
...
}
public class FooCmd {
@Inject FooService _fooService;
@Inject GeniusService _geniusService;
....
}
|
cmdObj = ComponentContext.inject(cmdObj);
Above one-line enabler makes it possible to use @Inject pattern in all CloudStack command classes consistently across the board. Spring will automatically wire the object in reference for the newly constructed object at run-time. If you have to use run-time constructed objects and have cases to pass over a set of service objects, instead of using pattern like
public class FooObject {
FooService _fooService;
public FooObject(FooService service) {
_fooService = service;
}
....
}
|
You may take advantage of auto-wiring as
public class FooObject {
@Inject FooService _fooService;
public FooObject() {
}
...
} |
If you see yourself constantly need to pass a lot of service objects to a object constructor (or setters), use ComponentContext.inject() to help you out.
We found that out-of-box AOP offering from Spring does not work with CloudStack, Spring AOP uses proxy mechanism, it can only intercept method calls that are "calling into" the component through a generated proxy object, unfortunately some of CloudStack codebase relies on the fact that method interception should happen for inner method calls within the component class itself. To solve this problem, we have to develop a customized AOP under Spring. Following is an example of such use case.
package com.cloud.event;
import java.lang.reflect.Method;
import org.apache.log4j.Logger;
import com.cloud.user.UserContext;
import com.cloud.utils.component.ComponentMethodInterceptor;
public class ActionEventInterceptor implements ComponentMethodInterceptor {
private static final Logger s_logger = Logger.getLogger(ActionEventInterceptor.class);
public ActionEventInterceptor() {
}
@Override
public Object interceptStart(Method method, Object target) {
EventVO event = null;
ActionEvent actionEvent = method.getAnnotation(ActionEvent.class);
if (actionEvent != null) {
boolean async = actionEvent.async();
if(async){
UserContext ctx = UserContext.current();
long userId = ctx.getCallerUserId();
long accountId = ctx.getAccountId();
long startEventId = ctx.getStartEventId();
String eventDescription = actionEvent.eventDescription();
if(ctx.getEventDetails() != null){
eventDescription += ". "+ctx.getEventDetails();
}
EventUtils.saveStartedEvent(userId, accountId, actionEvent.eventType(), eventDescription, startEventId);
}
}
return event;
}
@Override
public void interceptComplete(Method method, Object target, Object event) {
ActionEvent actionEvent = method.getAnnotation(ActionEvent.class);
if (actionEvent != null) {
UserContext ctx = UserContext.current();
long userId = ctx.getCallerUserId();
long accountId = ctx.getAccountId();
long startEventId = ctx.getStartEventId();
String eventDescription = actionEvent.eventDescription();
if(ctx.getEventDetails() != null){
eventDescription += ". "+ctx.getEventDetails();
}
if(actionEvent.create()){
//This start event has to be used for subsequent events of this action
startEventId = EventUtils.saveCreatedEvent(userId, accountId, EventVO.LEVEL_INFO, actionEvent.eventType(), "Successfully created entity for "+eventDescription);
ctx.setStartEventId(startEventId);
} else {
EventUtils.saveEvent(userId, accountId, EventVO.LEVEL_INFO, actionEvent.eventType(), "Successfully completed "+eventDescription, startEventId);
}
}
}
@Override
public void interceptException(Method method, Object target, Object event) {
ActionEvent actionEvent = method.getAnnotation(ActionEvent.class);
if (actionEvent != null) {
UserContext ctx = UserContext.current();
long userId = ctx.getCallerUserId();
long accountId = ctx.getAccountId();
long startEventId = ctx.getStartEventId();
String eventDescription = actionEvent.eventDescription();
if(ctx.getEventDetails() != null){
eventDescription += ". "+ctx.getEventDetails();
}
if(actionEvent.create()){
long eventId = EventUtils.saveCreatedEvent(userId, accountId, EventVO.LEVEL_ERROR, actionEvent.eventType(), "Error while creating entity for "+eventDescription);
ctx.setStartEventId(eventId);
} else {
EventUtils.saveEvent(userId, accountId, EventVO.LEVEL_ERROR, actionEvent.eventType(), "Error while "+eventDescription, startEventId);
}
}
}
@Override
public boolean needToIntercept(Method method) {
ActionEvent actionEvent = method.getAnnotation(ActionEvent.class);
if (actionEvent != null) {
return true;
}
return false;
}
}
|
If you ever need to implement a method interceptor, implement interface ComponentMethodInterceptor and declare it in applicationContext.xml.in. As you see in the above example, ActionEventInterceptor implements an aspect to automatically log events at methods in classes that have been marked as @ActionEvent. To install such interceptor, declare it as following in applicationContext.xml.in
<bean id="actionEventInterceptor" class="com.cloud.event.ActionEventInterceptor" />
<bean id="instantiatePostProcessor" class="com.cloud.utils.component.ComponentInstantiationPostProcessor">
<property name="Interceptors">
<list>
<ref bean="transactionContextBuilder" />
<ref bean="actionEventInterceptor" />
</list>
</property>
</bean>
|
Not any class can be intercepted in this way, both due to implementation limitation and for sake of performance, only classes that have marked itself with a marker interface ComponentMethodInterceptable can be intercepted to apply above AOP pattern. All Manager, Adapter and DAO classes are currently marked with ComponentMethodInterceptable.
ComponentInstantiationPostProcessor is a CloudStack implemented Spring post processor that eventually implements the Customized AOP pattern. ComponentInstantiationPostProcessor uses CGLIB to replace loaded component instance with a CGLIB generated wrapper object that is a sub-class instance of the original object, in this way, it can provide hooks to all method calls to the object through ComponentMethodInterceptor.
The sub-class generated by CGLIB uses the naming convention implemented in ComponentNamingPolicy. All these generated classes are named after "EnhancedByCloudStack" pattern, you may notice it in log messages or in debug session.
Adapter components usually works under the management of its manager component, a same set of adapter components may be used by multiple managers, and sometimes, order of the adapters may also be significant. Whether or not an adapter component is in action depends not only the existence of its <bean> declaration, but also the references in manager components.
The reference configuration of adapters in fact represents the dynamic aspect of CloudStack and gives people the flexibility to tune and customize CloudStack, due to its dynamic nature, we put all of these configurations in componentContext.xml.in and/or nonossComponentContext.xml.in. Following example shows a same set of adapter components, but with a different order of these adapter components.
<!-- Security adapters -->
<bean id="userAuthenticators" class="com.cloud.utils.component.AdapterList">
<property name="Adapters">
<list>
<ref bean="SHA256SaltedUserAuthenticator"/>
<ref bean="MD5UserAuthenticator"/>
<ref bean="LDAPUserAuthenticator"/>
<ref bean="PlainTextUserAuthenticator"/>
</list>
</property>
</bean>
<bean id="userPasswordEncoders" class="com.cloud.utils.component.AdapterList">
<property name="Adapters">
<list>
<ref bean="SHA256SaltedUserAuthenticator"/>
<ref bean="MD5UserAuthenticator"/>
<ref bean="LDAPUserAuthenticator"/>
<ref bean="PlainTextUserAuthenticator"/>
</list>
</property>
</bean>
|
For network plugin developers, you probably need to visit following example often to add or remove your network element component. (baremetal support is disabled in following example)
<bean id="networkElements" class="com.cloud.utils.component.AdapterList">
<property name="Adapters">
<list>
<ref bean="VirtualRouter"/>
<ref bean="Ovs"/>
<ref bean="SecurityGroupProvider"/>
<ref bean="VpcVirtualRouter"/>
<ref bean="NiciraNvp" />
<ref bean="MidoNetElement"/>
!--
<ref bean="BareMetalDhcp"/>
<ref bean="BareMetalPxe"/>
<ref bean="BareMetalUserdata"/>
->
</list>
</property>
</bean>
|
CloudStack contains a lot of software modules, for example, module for account management, module for networking etc, each module in turn can contain its own DAO, manager, adapter components, To help make the component configuration maintainable, please follow following two guidelines.
Due to historic reason, we have a number of tiny components, like OVA/ISO format "components", unless you see a need to take advantage of auto-wiring, avoid turning these classes into components
Because of the removal of component auto-scanning, all current component declaration in applicationContext.xml.in are primarily generated with a tool, and also the module boundaries of most of legacy components are blur, therefore organization of these legacy components does not strictly follow this rule. However, it is recommended for all new large independent modules to follow this rule. For example, baremetal feature module is organized in this way in applicationContext.xml.in
<!--=======================================================================================================-->
<!-- -->
<!-- Module-basis OSS/non-OSS Common components -->
<!-- -->
<!--=======================================================================================================-->
<!--
Baremetal components
-->
<!--
<bean id="BareMetalDhcp" class="com.cloud.baremetal.networkservice.BaremetalDhcpElement">
<property name="name" value="BareMetalDhcp"/>
</bean>
<bean id="BareMetalPxe" class="com.cloud.baremetal.networkservice.BaremetalPxeElement">
<property name="name" value="BareMetalPxe"/>
</bean>
<bean id="BareMetalUserdata" class="com.cloud.baremetal.networkservice.BaremetalUserdataElement">
<property name="name" value="BareMetalUserdata"/>
</bean>
<bean id="BareMetalTemplateAdapter" class="com.cloud.baremetal.manager.BareMetalTemplateAdapter" />
<bean id="BareMetalDiscoverer" class="com.cloud.baremetal.manager.BareMetalDiscoverer">
<property name="name" value="Bare Metal Agent"/>
</bean>
<bean id="BareMetalPlanner" class="com.cloud.baremetal.manager.BareMetalPlanner">
<property name="name" value="BareMetal Fit"/>
</bean>
<bean id="BaremetalGuru" class="com.cloud.baremetal.manager.BareMetalGuru">
<property name="name" value="BaremetalGuru"/>
</bean>
<bean id="BaremetalPlannerSelector" class="com.cloud.baremetal.manager.BaremetalPlannerSelector">
<property name="name" value="BaremetalPlannerSelector"/>
</bean>
<bean id="BaremetalManager" class="com.cloud.baremetal.manager.BaremetalManagerImpl"/>
<bean id="BaremetalDhcpManager" class="com.cloud.baremetal.networkservice.BaremetalDhcpManagerImpl"/>
<bean id="BaremetalKickStartPxeService" class="com.cloud.baremetal.networkservice.BaremetalKickStartServiceImpl"/>
<bean id="BaremetalPingPxeService" class="com.cloud.baremetal.networkservice.BareMetalPingServiceImpl" />
<bean id="BaremetalPxeManager" class="com.cloud.baremetal.networkservice.BaremetalPxeManagerImpl" />
<bean id="BAREMETAL" class="org.apache.cloudstack.storage.image.format.BAREMETAL" />
<bean id="baremetalDhcpDaoImpl" class="com.cloud.baremetal.database.BaremetalDhcpDaoImpl" />
<bean id="baremetalPxeDaoImpl" class="com.cloud.baremetal.database.BaremetalPxeDaoImpl" />
-->
|
In CloudStack, some components are lifecyle sensitive, examples are those manager objects, adapter objects. To indicate the desire to be life-cycle management aware, a component needs to implement interface ComponentLifecycle. ComponentLifecycle currently defines 6 run-levels to represent system startup cycles.
public interface ComponentLifecycle {
public static final int RUN_LEVEL_SYSTEM_BOOTSTRAP = 0; // for system level bootstrap components
public static final int RUN_LEVEL_SYSTEM = 1; // for system level service components (i.e., DAOs)
public static final int RUN_LEVEL_FRAMEWORK_BOOTSTRAP = 2; // for framework startup checkers (i.e., DB migration check)
public static final int RUN_LEVEL_FRAMEWORK = 3; // for framework bootstrap components(i.e., clustering management components)
public static final int RUN_LEVEL_COMPONENT_BOOTSTRAP = 4; // general manager components
public static final int RUN_LEVEL_COMPONENT = 5; // regular adapters, plugin components
public static final int RUN_LEVEL_APPLICATION_MAINLOOP = 6;
public static final int MAX_RUN_LEVELS = 7;
// ...
} |
You usually don't need to implement such interface of your own, GenericDaoBase, AdapterBase, ManagerBase give you a start point. It covers major flavors of the components inside CloudStack.