The ultimate goal here is to merge the Storm and JStorm code bases and communities. The current proposal is to do a feature freeze on storm-core while this work happens, but everything under external should still be open for development.
I (Bobby) envision this merger happening in two phases major, but we may blur the lines between these two as development happens.
Phase 1. Migrate clojure code to java using JStorm code as a reference.
| Namespace | Dependency | JIRA | Notes | JStorm Similar Code |
|---|---|---|---|---|
| backtype.storm.LocalCluster | backtype.storm.config backtype.storm.testing | https://github.com/apache/storm/blob/jstorm-import/jstorm-core/src/main/java/backtype/storm/LocalCluster.java | ||
| backtype.storm.LocalDRPC | backtype.storm.config backtype.storm.daemon.drpc backtype.storm.util | https://github.com/apache/storm/blob/jstorm-import/jstorm-core/src/main/java/backtype/storm/LocalDRPC.java | ||
| backtype.storm.MockAutoCred | backtype.storm.config backtype.storm.testing | Mock implementation of INimbusCredentialPlugin,IAutoCredentials and ICredentialsRenewer for testing only. | NONE | |
| backtype.storm.clojure | backtype.storm.thrift backtype.storm.util | NONE | Clojure Storm API (Needs to be kept for backwards compatability, possibly in separate maven package) | NONE |
| backtype.storm.clojure-test | backtype.storm.clojure backtype.storm.config backtype.storm.daemon.common backtype.storm.testing backtype.storm.thrift | NONE | Tests for Storm Clojure API | NONE |
| backtype.storm.cluster | backtype.storm.config backtype.storm.converter backtype.storm.daemon.common backtype.storm.log backtype.storm.util backtype.storm.zookeeper | |||
| backtype.storm.cluster-test | backtype.storm.cluster backtype.storm.config backtype.storm.log backtype.storm.testing backtype.storm.thrift backtype.storm.util backtype.storm.zookeeper | |||
| backtype.storm.command.activate | backtype.storm.log backtype.storm.thrift | |||
| backtype.storm.command.config-value | backtype.storm.config backtype.storm.log | |||
| backtype.storm.command.deactivate | backtype.storm.log backtype.storm.thrift | |||
| backtype.storm.command.dev-zookeeper | backtype.storm.config backtype.storm.util backtype.storm.zookeeper | |||
| backtype.storm.command.get-errors | backtype.storm.daemon.common backtype.storm.daemon.nimbus backtype.storm.log backtype.storm.thrift backtype.storm.util | |||
| backtype.storm.command.healthcheck | backtype.storm.config backtype.storm.log | |||
| backtype.storm.command.kill-topology | backtype.storm.config backtype.storm.log backtype.storm.thrift | |||
| backtype.storm.command.kill-workers | backtype.storm.config backtype.storm.daemon.common backtype.storm.daemon.supervisor backtype.storm.util | |||
| backtype.storm.command.list | backtype.storm.log backtype.storm.thrift | |||
| backtype.storm.command.monitor | backtype.storm.thrift | |||
backtype.storm.command.rebalance | backtype.storm.config backtype.storm.log backtype.storm.thrift | |||
| backtype.storm.command.set-log-level | backtype.storm.log backtype.storm.thrift | |||
| backtype.storm.command.shell-submission | backtype.storm.config backtype.storm.log backtype.storm.thrift backtype.storm.util backtype.storm.zookeeper | |||
| backtype.storm.command.upload-credentials | backtype.storm.log backtype.storm.util | |||
| backtype.storm.config | backtype.storm.log backtype.storm.util | |||
| backtype.storm.converter | backtype.storm.daemon.common backtype.storm.log backtype.storm.stats backtype.storm.util | |||
| backtype.storm.daemon.acker | backtype.storm.config backtype.storm.log backtype.storm.util | |||
| backtype.storm.daemon.builtin-metrics | backtype.storm.stats | |||
| backtype.storm.daemon.common | backtype.storm.config backtype.storm.daemon.acker backtype.storm.log backtype.storm.thrift backtype.storm.util | |||
| backtype.storm.daemon.drpc | backtype.storm.config backtype.storm.daemon.common backtype.storm.log backtype.storm.ui.helpers backtype.storm.util | |||
| backtype.storm.daemon.executor | backtype.storm.cluster backtype.storm.config backtype.storm.daemon.builtin-metrics backtype.storm.daemon.common backtype.storm.daemon.task backtype.storm.disruptor backtype.storm.log backtype.storm.stats backtype.storm.thrift backtype.storm.timer backtype.storm.tuple backtype.storm.util | |||
| backtype.storm.daemon.logviewer | backtype.storm.config backtype.storm.daemon.common backtype.storm.daemon.supervisor backtype.storm.log backtype.storm.timer backtype.storm.ui.helpers backtype.storm.util | |||
| backtype.storm.daemon.nimbus | backtype.storm.cluster backtype.storm.config backtype.storm.converter backtype.storm.daemon.common backtype.storm.local-state backtype.storm.log backtype.storm.scheduler.DefaultScheduler backtype.storm.stats backtype.storm.timer backtype.storm.tuple backtype.storm.util backtype.storm.zookeeper | |||
| backtype.storm.daemon.supervisor | backtype.storm.cluster backtype.storm.command.healthcheck backtype.storm.config backtype.storm.daemon.common backtype.storm.daemon.worker backtype.storm.event backtype.storm.local-state backtype.storm.log backtype.storm.process-simulator backtype.storm.timer backtype.storm.util | |||
| backtype.storm.daemon.task | backtype.storm.config backtype.storm.daemon.builtin-metrics backtype.storm.daemon.common backtype.storm.log backtype.storm.stats backtype.storm.thrift backtype.storm.util | |||
| backtype.storm.daemon.worker | backtype.storm.cluster backtype.storm.config backtype.storm.daemon.common backtype.storm.daemon.executor backtype.storm.disruptor backtype.storm.local-state backtype.storm.log backtype.storm.messaging.loader backtype.storm.timer backtype.storm.util | |||
| backtype.storm.disruptor | backtype.storm.log backtype.storm.util | |||
| backtype.storm.drpc-test | backtype.storm.clojure backtype.storm.config backtype.storm.daemon.common backtype.storm.daemon.drpc backtype.storm.testing | |||
| backtype.storm.event | backtype.storm.log backtype.storm.util | |||
| backtype.storm.fields-test | ||||
| backtype.storm.grouping-test | backtype.storm.clojure backtype.storm.config backtype.storm.daemon.common backtype.storm.daemon.executor backtype.storm.log backtype.storm.testing backtype.storm.thrift | |||
| backtype.storm.integration-test | backtype.storm.clojure backtype.storm.config backtype.storm.daemon.common backtype.storm.testing backtype.storm.thrift backtype.storm.util | |||
backtype.storm.local-state | backtype.storm.log backtype.storm.util | |||
| backtype.storm.local-state-test | backtype.storm.testing | |||
| backtype.storm.log | ||||
| backtype.storm.logviewer-test | backtype.storm.config backtype.storm.daemon.logviewer backtype.storm.daemon.supervisor backtype.storm.ui.helpers backtype.storm.util | |||
| backtype.storm.messaging-test | backtype.storm.config backtype.storm.daemon.common backtype.storm.testing backtype.storm.thrift | |||
| backtype.storm.messaging.loader | backtype.storm.messaging.local | |||
| backtype.storm.messaging.local | ||||
| backtype.storm.messaging.netty-integration-test | backtype.storm.config backtype.storm.testing backtype.storm.thrift backtype.storm.util | |||
| backtype.storm.messaging.netty-unit-test | backtype.storm.config backtype.storm.daemon.worker backtype.storm.log backtype.storm.testing backtype.storm.util | |||
| backtype.storm.metric.testing | ||||
| backtype.storm.metrics-test | backtype.storm.clojure backtype.storm.config backtype.storm.daemon.common backtype.storm.metric.testing backtype.storm.testing backtype.storm.thrift | |||
| backtype.storm.multilang-test | backtype.storm.config backtype.storm.daemon.common backtype.storm.testing backtype.storm.thrift | |||
| backtype.storm.nimbus-test | backtype.storm.MockAutoCred backtype.storm.cluster backtype.storm.config backtype.storm.converter backtype.storm.daemon.common backtype.storm.daemon.nimbus backtype.storm.log backtype.storm.stats backtype.storm.testing backtype.storm.thrift backtype.storm.timer backtype.storm.util backtype.storm.zookeeper | |||
| backtype.storm.process-simulator | backtype.storm.log backtype.storm.util | |||
| backtype.storm.scheduler-test | backtype.storm.config backtype.storm.daemon.nimbus backtype.storm.scheduler.EvenScheduler backtype.storm.testing | |||
| backtype.storm.scheduler.DefaultScheduler | backtype.storm.config backtype.storm.scheduler.EvenScheduler backtype.storm.util | |||
| backtype.storm.scheduler.EvenScheduler | backtype.storm.config backtype.storm.log backtype.storm.util | |||
| backtype.storm.scheduler.IsolationScheduler | backtype.storm.config backtype.storm.log backtype.storm.scheduler.DefaultScheduler backtype.storm.util | |||
| backtype.storm.scheduler.multitenant-scheduler-test | backtype.storm.config backtype.storm.daemon.nimbus backtype.storm.log backtype.storm.testing | |||
| backtype.storm.scheduler.resource-aware-scheduler-test | backtype.storm.config backtype.storm.daemon.nimbus backtype.storm.testing backtype.storm.thrift backtype.storm.util | |||
| backtype.storm.security.auth.AuthUtils-test | ||||
| backtype.storm.security.auth.DefaultHttpCredentialsPlugin-test | ||||
| backtype.storm.security.auth.ReqContext-test | ||||
| backtype.storm.security.auth.SaslTransportPlugin-test | ||||
| backtype.storm.security.auth.ThriftClient-test | backtype.storm.config backtype.storm.util | |||
| backtype.storm.security.auth.ThriftServer-test | backtype.storm.config | |||
| backtype.storm.security.auth.auth-test | backtype.storm.config backtype.storm.daemon.common backtype.storm.daemon.nimbus backtype.storm.testing backtype.storm.util | |||
| backtype.storm.security.auth.authorizer.DRPCSimpleACLAuthorizer-test | backtype.storm.config backtype.storm.util | |||
| backtype.storm.security.auth.auto-login-module-test | backtype.storm.util | |||
| backtype.storm.security.auth.drpc-auth-test | backtype.storm.config backtype.storm.daemon.common backtype.storm.daemon.drpc backtype.storm.log backtype.storm.testing backtype.storm.util | |||
| backtype.storm.security.auth.nimbus-auth-test | backtype.storm.cluster backtype.storm.config backtype.storm.daemon.common backtype.storm.daemon.nimbus backtype.storm.log backtype.storm.testing backtype.storm.util backtype.storm.zookeeper | |||
| backtype.storm.security.serialization.BlowfishTupleSerializer-test | backtype.storm.util | |||
| backtype.storm.serialization-test | backtype.storm.config backtype.storm.util | |||
| backtype.storm.serialization.SerializationFactory-test | backtype.storm.config | |||
| backtype.storm.stats | backtype.storm.log backtype.storm.util | |||
| backtype.storm.submitter-test | backtype.storm.config backtype.storm.testing | |||
| backtype.storm.subtopology-test | backtype.storm.daemon.common backtype.storm.testing | |||
backtype.storm.supervisor-test | backtype.storm.cluster backtype.storm.config backtype.storm.daemon.common backtype.storm.daemon.supervisor backtype.storm.daemon.worker backtype.storm.testing backtype.storm.thrift backtype.storm.timer backtype.storm.util | |||
| backtype.storm.testing | backtype.storm.cluster backtype.storm.config backtype.storm.daemon.acker backtype.storm.daemon.common backtype.storm.daemon.executor backtype.storm.daemon.nimbus backtype.storm.daemon.supervisor backtype.storm.daemon.worker backtype.storm.local-state backtype.storm.log backtype.storm.messaging.loader backtype.storm.process-simulator backtype.storm.thrift backtype.storm.util backtype.storm.zookeeper | |||
| backtype.storm.testing4j | backtype.storm.LocalCluster backtype.storm.log backtype.storm.testing backtype.storm.util | |||
| backtype.storm.testing4j-test | backtype.storm.clojure backtype.storm.config backtype.storm.integration-test backtype.storm.testing backtype.storm.thrift backtype.storm.util | |||
| backtype.storm.thrift | backtype.storm.config backtype.storm.log backtype.storm.util backtype.storm.zookeeper | |||
| backtype.storm.tick-tuple-test | backtype.storm.clojure backtype.storm.config backtype.storm.daemon.common backtype.storm.testing backtype.storm.thrift | |||
| backtype.storm.timer | backtype.storm.log backtype.storm.util | |||
| backtype.storm.transactional-test | backtype.storm.clojure backtype.storm.config backtype.storm.daemon.common backtype.storm.testing backtype.storm.util | |||
| backtype.storm.tuple | backtype.storm.tuple-test backtype.storm.testing | |||
| backtype.storm.ui.core | backtype.storm.config backtype.storm.converter backtype.storm.daemon.common backtype.storm.log backtype.storm.stats backtype.storm.thrift backtype.storm.tuple backtype.storm.ui.helpers backtype.storm.util backtype.storm.zookeeper | |||
| backtype.storm.ui.helpers | backtype.storm.config backtype.storm.log backtype.storm.util | |||
| backtype.storm.util | backtype.storm.log | |||
| backtype.storm.utils-test | backtype.storm.config backtype.storm.util | |||
| backtype.storm.utils.ZookeeperServerCnxnFactory-test | ||||
| backtype.storm.versioned-store-test | backtype.storm.testing | |||
| backtype.storm.worker-test | backtype.storm.daemon.common backtype.storm.daemon.worker backtype.storm.log backtype.storm.testing backtype.storm.util | |||
| backtype.storm.zookeeper | backtype.storm.config backtype.storm.log backtype.storm.util | |||
| storm.trident.integration-test | backtype.storm.testing backtype.storm.util storm.trident.testing | |||
| storm.trident.state-test | backtype.storm.config backtype.storm.testing backtype.storm.util storm.trident.testing | |||
| storm.trident.testing | backtype.storm.LocalDRPC backtype.storm.testing backtype.storm.util | |||
| storm.trident.tuple-test | backtype.storm.testing backtype.storm.util storm.trident.testing |
Phase 2. Port JStorm features to the new java based storm-core.