This Confluence has been LDAP enabled, if you are an ASF Committer, please use your LDAP Credentials to login. Any problems file an INFRA jira ticket please.

Page tree
Skip to end of metadata
Go to start of metadata

Fineract-CN projects depend on each other. The simplest option is to build all projects locally so artifacts are published to local maven repository and available from there.

Sometomes still there is need to only work with a single project and resolve the other fineract-cn-* dependencies from public repository. Also in order for continous integration (Travis-ci.com) to build a specific project it needs that all the dependencies are available.

JFrog Artifactory has been set up to host these artifacts in https://mifos.jfrog.io/mifos/. Maven Central would not be suitable for the task as it doesn't hold "work-in-progress" snapshots.

In this artifacotry there are two repositories:

  • libs-release to hold releases
  • libs-snapshot to hold snapshot builds. Snapshot builds are updated every time something new gets pushed to git (and the build passes)

In order to use snapshot builds one needs to have following line in repositories list:

repositories {
  jcenter()
  mavenLocal()
  maven { url 'https://mifos.jfrog.io/mifos/libs-snapshot/' }
}

How snapshots are built by Travis

All apache/fineract-cn-* repositories are configured to be built with Travis-CI.com. Whenever something is pushed to a repository travis tries to run gradle build on it. Travis also detects the branch name that the changes were pushed to.

Changes pushed/merged to "develop" branch

If travis detects that the changes were pushed to "develop" branch and build doesn't fail then it uploads the new snapshot artifacts to the public repository. You can pick them up the same way as artifacts deployed to local Maven:

dependencies {
compile(
[group: 'org.apache.fineract.cn', name: 'reponame', version: '0.1.0-BUILD-SNAPSHOT']
)
)

Changes pushed/merged to other branches (not "develop")

Travis builds and publishes the snapshots out of all branches but for branches not named "develop" it overwrites the version of the artifact (defined in build.gradle) with "branchname-SNAPSHOT". If you want to point some other project to branch "myfeaturebranch" then you need to use:

dependencies {
compile(
[group: 'org.apache.fineract.cn', name: 'reponame', version: 'myfeaturebranch-SNAPSHOT']
)
)

Handling pull requests

Travis also builds all pull requests but for security reasons it doesn't upload any artifacts built from pull requests.

How releases work

If a release is tagged in github then Travis-CI builds that as a release and uploads it under different repoKey - release artifacts are available at https://mifos.jfrog.io/mifos/libs-release

Travis and Artifactory setup

Following instructions are useful if you want to set up a new project or change existing setup.

Configure a new fineract-cn-* project to upload artifacts to artifactory

  • in shared.gradle (or if it doesn't exist then in build.gradle):

    1. If not already, add snapshots repository to list of repositories so it looks like this:
      repositories {
      jcenter()
      mavenLocal()
      maven { url 'https://mifos.jfrog.io/mifos/libs-snapshot/' }
      }
    2. Underheath jar or publishing block add instructions for the new plugin:
      artifactory {
      contextUrl = System.getenv("ARTIFACTORY_URL")
      publish {
      repository {
      repoKey = project.findProperty('artifactoryRepoKey')
      username = System.getenv("ARTIFACTORY_USER")
      password = System.getenv("ARTIFACTORY_PASSWORD")
      }

      defaults {
      publications ('api', 'componentTest', 'service', 'bootService') // TODO add projectnamePublification here
      }
      }
      }
      artifactoryPublish.dependsOn('clean','publishToMavenLocal')


  • in every module build.gradle
    1. In list of plugins add plugin:

      id "com.jfrog.artifactory" version "4.9.5"

    2. In publishing → publications → projectnamePublification

      replace (or add if not present)
        version project.version 
      with: 
        version project.findProperty('externalVersion') ?: project.version
    3. If publification name is mavenJava then give it a correct name (i.e. componentTest)


  • If shared.gradle exists then add to build.gradle (after publishToMavenLocal)

    task artifactoryPublish {
    group 'all'
    dependsOn publishToMavenLocal
    dependsOn gradle.includedBuild('api').task(':artifactoryPublish')
    dependsOn gradle.includedBuild('service').task(':artifactoryPublish')
    dependsOn gradle.includedBuild('component-test').task(':artifactoryPublish')
    // TODO add other modules found in the same file
    }
  • Add travis banner to top of README.md (replace reponame with correct value):
    [![Build Status](https://api.travis-ci.com/apache/fineract-cn-reponame.svg?branch=develop)](https://travis-ci.com/apache/fineract-cn-reponame)
  • Add travis.sh (copy-paste from other project that is already included)
  • Add .travis.yml - travis-ci.com looks for this file

    language: java
    sudo: false
    jdk:
    - openjdk8
    install: true
    script: "./travis.sh"
    env:
    global:
        - BUILD_SNAPSHOTS_BRANCH=develop  
    - ARTIFACTORY_URL=https://mifos.jfrog.io/mifos
    - ARTIFACTORY_USER=travis-ci1
    - secure: "see below chapter Encrypt Artifactory password"

Encrypt Artifactory password


You need command line travis client.

travis encrypt --com -r apache/fineract-cn-reponame ARTIFACTORY_PASSWORD=enter-password-here

copy the encrypted secure value to .travis.yml

How to test build locally


export ARTIFACTORY_URL=https://mifos.jfrog.io/mifos ARTIFACTORY_USER=travis-ci ARTIFACTORY_PASSWORD=xxx

./gradlew -PartifactoryRepoKey=libs-snapshot-local artifactoryPublish


Verify

  • If you push to "develop" branch the code is uploaded to https://mifos.jfrog.io/mifos/libs-snapshot-local/org/apache/fineract/cn/<projectname>/0.1.0-BUILD-SNAPSHOT/<projectname>-0.1.0-BUILD-SNAPSHOT.jar
  • If you push to any other branch, for example to branch "myfeature" then the code is uploaded to https://mifos.jfrog.io/mifos/libs-snapshot-local/org/apache/fineract/cn/<projectname>/myfeature-SNAPSHOT/<projectname>-myfeature-SNAPSHOT.pom
  • If you create a realease in Github (out of any branch), and name it let's say "1.2.0.0" then travis builds the code and uploads to https://mifos.jfrog.io/mifos/libs-release-local/org/apache/fineract/cn/<projectname>/1.2.0.0/<projectname>-1.2.0.0.jar

Design principles

  • travis.sh file should not hold any project-specific configuration (should be the same in every repo)

JFrog Artifactory configuration

  • Common Maven repository configuration is used - this means:
    • Snapshots are uploaded to "libs-snaptshots-local" and releases to "libs-release-local"
    • There are two virutal repositories libs-snapshots (includes libs-snapshots-local) and libs-release (includes libs-release-local)
  • Remote repositories are deleted (as we don't want the reposiotry to be a proxy for JCentral)
  • A user "travis-ci" has been configured who has rights to upload artifacts but it has no UI access
    • the password for this user is encrypted with travis public keys (travis creates a public key for each individual reposiotry) and set into .travis.yml
  • Anonymous access has been enabled for anyone to download the artifacts

Troubleshooting

How to verify that the new artifact was uploaded

Let's take this pull request as an example: https://github.com/apache/fineract-cn-lang/pull/10

When this pull request was merged - new jar (and pom) should have been built and uploaded to Mifos Jfrog Artifactory. Let's check if it happened correctly.

We can check the Travis builds of fineract-cn-lang from here: https://travis-ci.com/apache/fineract-cn-lang/builds

Find the correct pull request from the list (pull request #10) and on the same line click on "develop" in the beginning of row.

Build details page (https://travis-ci.com/apache/fineract-cn-lang/builds/114688396) opens up. 

Scroll to the end of the logs - you can see:

Deploying artifact: https://mifos.jfrog.io/mifos/libs-snapshot-local/org/apache/fineract/cn/lang/0.1.0-BUILD-SNAPSHOT/lang-0.1.0-BUILD-SNAPSHOT.jar

Now we can navigate to Jfrog and check if we see it from there:

Go to Artifactory (no need to log in): https://mifos.jfrog.io/mifos/

Choose "Artifact repository browser" icon from the left (icon is like pile of paper).

You will end up on this page: https://mifos.jfrog.io/mifos/webapp/#/artifacts/browse/tree/General/libs-release

Choose "libs-snapshot" from the left. You will end up here: 

https://mifos.jfrog.io/mifos/webapp/#/artifacts/browse/tree/General/libs-snapshot

Click "find" icon on top and search "fineract-cn-lang".

"org" appears in search result. Expand it to org → apache → fineract → cn → lang → 0.1.0-BUILD-SNAPSHOT

In the end of list you should see the uploaded artifact (you can use the upload date to check).

In our example the uploaded artifact is this:

https://mifos.jfrog.io/mifos/webapp/#/artifacts/browse/tree/General/libs-snapshot/org/apache/fineract/cn/lang/0.1.0-BUILD-SNAPSHOT/lang-0.1.0-BUILD-20190607.052703-26.jar

How to get the latest artifact

Latest artifact is available here: http://mifos.jfrog.io/mifos/libs-snapshot/org/apache/fineract/cn/lang/0.1.0-BUILD-SNAPSHOT/lang-0.1.0-BUILD-SNAPSHOT.jar

But it might not work (especially if you are using a network with a proxy server).

Each snapshot artifact in artifactory has its own filename. So in this case you might need the unique filename. 

One way would be to use the Atifactory UI (https://mifos.jfrog.io/mifos ) to find it (see instructions in previous chapter). Another option would be to use Maven.

Using Maven

You have to craft a pom.xml file with the dependencies that you want. Then you run mvn package and you have the dependency updated in your local maven repo.

See Step 2, Option #1 ("You clone demo-server and let it download all dependendent fineract-cn-* libraries and projects from Artifactory to your local Maven repository") from How To Build Apache Fineract CN for an example.

Using artifactory

Get latest version (see JFrog rest api documentation for more info):

https://mifos.jfrog.io/mifos/api/search/latestVersion?g=org.apache.fineract.cn&a=lang&v=0.1.0-BUILD-SNAPSHOT&repos=libs-snapshot 

It returns something like this "0.1.0-BUILD-20190607.052703-26"

And then use it to pull latest from https://mifos.jfrog.io/mifos/libs-snapshot/org/apache/fineract/cn/lang/0.1.0-BUILD-SNAPSHOT/

like this:

https://mifos.jfrog.io/mifos/libs-snapshot/org/apache/fineract/cn/lang/0.1.0-BUILD-SNAPSHOT/lang-0.1.0-BUILD-20190607.052703-26.jar

  • No labels