...
- Motivation:
- Gobblin has about 400k java lines of code spread across ~76 different modules. Its very difficult to navigate the code, specially for new beginners. the proposal is to re-organise the modules into proper parent categories.
- Proposed Change:
- collapse all modules under following parent modules
New Modules New Purpose Notes Gobblin-core
All gobbline core components required for the platform to run end to end Gobblin-API
All REST APIs that operates on platform Gobblin-connectors
All extractor, convertors, writers and publisher classes for different source and target platforms This can be new repo all together for all kinds of gobblin plugins. Gobblin-plugins
All Gobblin orchastrator or supportive components like Oozie, Azkaban, etc... Gobblin-docs
All documents Gobblin-utils
All additional or supportive utilities like crypto, encryptions, etc... Gobblin-deployments
All gobblin deployment modes, aws, yarn, MR, etc...
- collapse all modules under following parent modules
- Proposed execution:
- execute the change in a way that does not change the commit history or does not change the ownership info to make sure the original contributor info remains the same.
- If the above is not possible, I suggest someone from LinkedIn team executes these migration.
- more actions <TBD>
- New or Changed Public Interfaces: there should not be have any impact on compatibility.
- Migration Plan and Compatibility: since the packaging structure ( bin, conf, lib ) is not changing, there should not be requiring migration plan. ( May only the full package.class_name if its used as utility)
- Rejected Alternatives: N/A ( there is no alternative to arranging the modules)
- TODO: add/update modules as components in JIRA once finalized.
...