What is BigTop?
Bigtop is a project for the development of packaging and tests of the Apache Hadoop ecosystem.
The primary goal of Bigtop is to build a community around the packaging and interoperability testing of Hadoop-related projects. This includes testing at various levels (packaging, platform, runtime, upgrade, etc...) developed by a community with a focus on the system as a whole, rather than individual projects.
Build, packaging and integration test code that depends upon official releases of the Apache Hadoop-related projects (HDFS, MapReduce, HBase, Hive, Pig, ZooKeeper, etc...) will be developed and released by this project. As bugs and other issues are found we expect these to be fixed upstream.
Does BigTop patches source releases?
BigTop does NOT patch any source release and does NOT have any mechanism to deal with anything else than bare source and pristine releases. NO patches will be applied. NOT even for build or security issues.
How to add a component to BigTop?
- First, you need to add an entry for your project in bigtop.mk similar to what is there for the others
- Put all the files common to RPM or DEB packaging in src/pkg/common/<YOUR_PROJECT_NAME>/. This may include, but not limited to, common build or installation scripts or service scripts for daemons
- Put your spec file in src/pkg/rpm/<YOUR_PROJECT_NAME>/SPECS/
- Put any additional file needed for the creation of your project's RPMs in src/pkg/rpm/<YOUR_PROJECT_NAME>/SOURCES/
- Put all your files needed for the creation of your project's DEBs in src/pkg/deb/<YOUR_PROJECT_NAME>/SPECS/
How to build a component of BigTop?
You must type from a command line:
Where <TARGET> can be:
- sdeb if you wish to build source DEBs
- deb if you wish to build DEBs
- srpm if you wish to build source RPMs
- rpm if you wish to build RPMs
- apt if you wish to build a repository for the already built DEBs
- yum if you wish to build a repository for the already built RPMs. Note this creates a repomd repository which will only work for GNU/Linux distributions of the Fedora/CentOS/RHEL/openSUSE family
How to learn more about packaging?
Here is a list of useful resources:
- How to create an RPM package
- Maximum RPM
- How to create a GNU Hello RPM package
- Packaging for beginners
- Packaging Hints
- Debian New Maintainers' Guide
- Packaging Tutorial (deb) Guide
- Guru's Guide to RPM (PDF)
What is the packaging policy for distribution X?
Each distribution has its own set of policy. Even if we cannot follow all of these policies, BigTop is trying to follow good practices and to be a good citizen. Here is a non-exhaustive list of packaging policy for several GNU/Linux distributions: