...

• nifi: will have been broken up into nifi-api, nifi-framework, nifi-extensions, nifi-release
  • Alternatively: this repo could be reduced to a single pom module containing the org.apache.nifi:nifi pom that inherits from org.apache:apache and can be a parent pom for other projects.
    
    
• nifi-minifi: will become a new assembly of nifi-framework and nifi-extensions that lives in the nifi-release repo

Benefits of Multiple Repositories

Multiple git repositories are not required in order to have multiple Maven projects. An alternative is a single git source code repository with multiple projects in sub directories. This was considered. Pros and cons of both approaches were also discussed on the discussion thread related to this proposal. This section of the proposal serves as a summary of that discussion and the thinking behind why this proposal recommends multiple repositories at this time.

Advantages of single repository:

• Easier to make cross-project changes, especially for new comers. Also for some, it will be easier to review changes that impact multiple projects if they are in a single PR.
• Git history for combined/entire project in one place
• Single focal point for entire community to collaborate on and follow
• Less infra to manage (though potentially mitigated now that access to repos is managed by ASF LDAP groups)

Advantages of multiple repositories:

• Easier for Release Mangers as they are performing source releases of an entire repository.
• Quicker to build an entire repository. Easier to setup reliable, fast CI builds. In a mutli-repo setup, builds will never span multiple projects as commits are, by definition, isolated to a single repo. Also avoids having to "build all project" or setting up CI tools such as Travis or Jenkins to scope builds based on contents of commits.
• Separate repos forces pull requests to stay scoped to a given component, making for overall smaller PRs. Easier to look at a repo and see what work/contributions are still open.
• Easier to understand the history for a single project or module in the git history, e.g., "show me all changes to this project since the last tag / release"
• Lower learning curve for making changes to a single project, as the overall code base is smaller.

At this time (though it is still being discussed), the multiple repository approach seems to be merited, especially based on it being the optimized approach for managing source code releases.

To mitigate the downsides, it is recommended that additional documentation and guidance be prepared for developers and contributors, especially getting started guides that target new comers. Especially helpful would be a document that gives an overview of the project and module structure to help newcomers navigate. 

A Note About Jira

This proposal only impacts source code projects and repositories. At this time, no change is being suggested for our Jira project structure or issue tracking system. It is recommended that even under different source code projects and repositories, issues are still tracked in the existing Jira project (i.e., NIFI) and differentiated as to which source code project/repo/artifact they impact via fields on the issue, such as component, label, fix version, etc.

Proposed Approach

Phase 1: nifi

...

Once the initial restructuring and decomposition of the nifi project/repo is complete, there may be opportunities for farther steps such as breaking nifi-extensions into smaller projects, or dropping support for some legacy extensions.

A Note About Jira

This proposal only impacts source code projects and repositories. At this time, no change is being suggested for our Jira project structure or issue tracking system. It is recommended that even under different source code projects and repositories, issues are still tracked in the existing Jira project (i.e., NIFI) and differentiated as to which source code project/repo/artifact they impact via fields on the issue, such as component, label, fix version, etc.