Status: Completed



Guide: How To migrate Jira issues history to GH issues

Phase 2: Transfer issues from Jira project to GitHub repository

Technically it's possible to transfer issues from Jira to Github as soon as issues are enabled for the repository. However, there need to be some preparations.

A config is needed and is prepared by Sandra Parsick. She also does a test for each repository to a test repository. A Maven committer needs to do the "real" migration, because of the rights needed.

Planed project order

First some smaller projects will be migrated to ensure that the tool works.

After that wishes are that Maven core and the core plugins should be migrated with a higher priority in preparation of Maven 4.


Guillaume Nodet:

ok, i would think, now that the process has been experimented, the most important is the main MNG one, then the core plugins.  Those that users faces the most.

Precondition

There need to be a config class for the JIRA project/component to be migrated.
Those are programmed by Sandra Parsick. She also does a test run for those.

Only projects with a such a config can be migrated. If configuration is missing for the project to be migrated, reach out for Sandra (for example on Slack).

Step 1: Get access-token for service account 

The actual migration execution is done, using a time limited service account named "jira-importer" provided by INFRA.
For this an access-token is needed. Ask PMC Benjamin Marwell to get the token.

Notes:

  • Each token has its own set of permission, granted manually by INFRA. They don't have global access. 
  • The Github API has a limit per user/token. So you can't just spam the API (see below about split migrations)

Step 2: Clone migration tool

Clone the migration tool from "https://github.com/support-and-care/jira-to-gh-issues"

Step 3: Prepare run configuration in application.property

In the application.property (location: src/main/ressources) the following things need to be set and/or matching the project to be migrated

  • github.access-token (line ~12): The token for the service account (Make sure to NOT commit/push this information!)
  • jira.base-url (~17): The URL of the JIRA server, should always be https://issues.apache.org/Jira
  • jira.projectId (~24): The name of the JIRA project, e.g MRAR
  • github.repository-slug (~39): The name, including organization, of the target github repository, e.g. apache/maven-rar-plugin


Warning: Make sure that "github.delete-create-repository-slug" (~45) is ALWAYS set to FALSE. Otherwise the script will try to delete and recreate the target repository (which will fail)

Step 3.1: Optional configuration

  • If JIRA issues for a component of a shared project are to be migrated the name of the jira component name must be set for the "jira.component" property (~28).
  • If the JIRA project has too many issues (more than 700?), they have to be migrated in multiple runs. Each run has to define a JQL query to only select a subset of all issues. This JQL is set in the property "jira.migrate-jql" (~60). In general Sandra will prepare those JQL during her test runs.

Step 4: Build migration tool

Build the migration tool with the given configuration using "mvn clean package -DskipTests".

Notes:

  • Tests are not working as they are designed for the original purpose of Spring (Info: Skipping surefire is set in the plugin's configuration of the pom.xml now)
  • The build needs exactly Java 17 and Maven 3.x (Enforced via enforcer-plugin now)

Step 5: Disable notification by contacting INFRA

During migration a lot of issues and comments are created. To avoid spamming at least the notification ML, those should be disabled for the time of the migration.
To do this comment out the lines "issues: issues@maven.apache.org" and "pullrequests: issues@maven.apache.org" (or whole notifaction block) in the asf.yml (on master branch) configuration of the target repository.

Disabling notifications in asf.yml does not work. Infra needs to be informed at least 30mins before migration which GitHub repository needs to get ignored sending events. 

Step 6: Prepare run

Copy the created jar from step 4 into a folder and make sure that no "github-migration-failures.txt" is there before run
This file gets created during execution if errors occurs.

If you start migration on project without any former migration run there must also be no "github-issue-mappings.properties" (contains mapping of Jira issue key to Github issue number) and "github-issue-pendings.properties" (contains the last migrated JIRA issues and acts as a reentry point when continuing a migration run) file. 

These files ARE needed when continuing a runAlso make sure that your local webservice port 8080 is not used. The tool needs this port during execution.

Step 7: Execute tool

Execute the tool via command line, using "java -jar jira-to-gh-issues-0.0.1-SNAPSHOT.jar" (adjust path to artefact if not executed within same folder)

Step 7.1: Check log

Check log for errors. If there are such, fix them and rebuild the tool and/or rerun the build.

Experiences errors

ProblemWhat happend?Resolution
Not enough permissions (403 errors in logfile)When the executing user has no rights to create milestones in the target repositories

Contact INFRA (David McDonald)

It looks that the user needs "admin" rights. "Maintainer" rights seem to not be enough for the import API

User can't be mappedDuring migration issues the tool tries to map Jira users to Github users of ASF. If either the person as no (yet known) Github accounts and/or is no ASF member the tool can't be mapped and issues can't be assigned. Update "jira_to_github_user.properties". Either add mapping or comment out the name 



Step 7.2: Add Jira comments with links to created GH issue (optional but recommended )

To let users know that an Jira issue has been migrated to GH issues, the class/app "LastJiraCommentApp.java" exists in "src/main/java/io/pivotal/post".

This creates an comment in the Jira issues with an information and a link to the created issue.

The app requires

  • "jira.user" and "jira.password" properties (with your personal values) in "application.property"
  • the "github-issue-mappings.properties" (created during migration) in to the root folder of the Maven project (not in target or where ever you ran the jar file)

Step 8: Enable notification again

If migration is completed, don't forget to enable notifications in asf.yml again.

It looks like if notifications get enabled too early after migration, mails still get send...

Step 9: Remove old issue Jira link from config

Remove the link to old Jira issues from the ".github/ISSUE_TEMPLATE/config.yml" (Example PR: https://github.com/apache/maven-apache-parent/pull/524/files)

Step 10: Check milestone list

We need exactly one open milestone in order to detect which is current.

If we need more than one open milestone, we have to add a branch to the milestone description


branch: <branch name>

This identifies the default for that branch.

https://github.com/apache/maven-gh-actions-shared?tab=readme-ov-file#milestone-configuration

Repositories / project status (Phase 2 only)


GitHub repoJIra projectissues & milestones migration

config(Planned to be) Done byDone with(out) issues?

Old Jira link

removed

GH Link in Jira issues

maven-siteMNGSITE

Getting issues...

config, test Sandra(tick)#865 (tick)(tick)
core https://github.com/apache/maven-sources/blob/master/default.xml#L30

maven-build-cache-extensionMBUILDCACHE

Getting issues...

test,
config
Olivier(tick) completed without issues(tick)(tick)
mavenMNG

Getting issues...

some components now have own repositories, e.g. wrapper

test,
config

test result:

MNG-925: The description creates a StackOverflowError in the markdown engine. Solution: we delete the problematic character

MNG-5592: Payload is too large for import. Solution: We are snippet problematic comments

Matthias &

Sandra

(tick) completed with duplicated issues. All duplicated issues with status open are closed as duplicated(tick)(tick)
maven-mvndno Jira, so nothing to migrate------

maven-integration-testing

no Jira, so nothing to migrate

-

-----

maven-resolver

MRESOLVER

Getting issues...

configOlivier(tick) completed without issues(tick)(tick)

maven-resolver-ant-tasks

MRESOLVER ant

Getting issues...


Olivier(tick) issues have been migrated twice (once to maven-resolver and then to maven-resolver-ant-tasks)(tick)(tick)

maven-wrapper

MWRAPPER

Getting issues...

configOlivier(tick) completed without issues(tick)(tick)
plugins/core https://github.com/apache/maven-sources/blob/master/default.xml#L39

maven-clean-plugin

Getting issues...

config, testBenjamin(tick) completed without issues
https://github.com/apache/maven-clean-plugin/issues
(tick)(tick)
maven-compiler-plugin

Getting issues...

config, testOlivier(tick)(tick)(tick)
maven-deploy-plugin

Getting issues...

test, configBenjamin(tick) completed without issues(tick)(tick)
maven-install-plugin

Getting issues...

test, configBenjamin(tick) completed without issues(tick)(tick)

maven-resources-plugin

Getting issues...

test, configOlivier(tick)(tick)(tick)
maven-site-plugin

Getting issues...

config
test
Olivier(tick) completed (MSITE-171 not migrated)(tick)(tick)
surefire

Getting issues...

config, test

migration has to be done in five runs.

For every run, the application.properties has to be adjusted:

application.properties
#jira.migrate-jql=project\=${jira.projectId} AND created < 2007-01-01
#jira.migrate-jql=project\=${jira.projectId} AND created >=2007-01-01 AND created < 2010-01-01
#jira.migrate-jql=project\=${jira.projectId} AND created >=2010-01-01 AND created < 2015-01-01
#jira.migrate-jql=project\=${jira.projectId} AND created >=2015-01-01 AND created < 2020-01-01
#jira.migrate-jql=project\=${jira.projectId} AND created >=2020-01-01
Olivier(tick)(tick)(tick)
maven-verifier-plugin

Getting issues...

config,
test
Matthias(tick) no issues#51 (tick)(tick)
plugins/packaging https://github.com/apache/maven-sources/blob/master/default.xml#L48

maven-ear-plugin

Getting issues...

config, testMatthias

(tick)
one issue in first run (no user)

no issues in 2nd run

#459 (tick)(tick)
maven-ejb-plugin

Getting issues...

test,
config
Benjamin(tick)
no issues
one reported, manually checked
(tick)(tick)
maven-jar-plugin

Getting issues...

test, configMatthias

(tick)
three issues in first run (no user)

no issues in 2nd run

#450 (tick)(tick)
maven-rar-plugin

Getting issues...

test,
config
Matthias(tick) no issues#126 (tick)(tick)
maven-war-plugin

Getting issues...

config, testMatthias(tick) no issues#547 (tick)(tick)
maven-acr-plugin

Getting issues...

config,
test
Benjamin(tick)
no issues
(tick)(tick)
maven-shade-plugin

Getting issues...

config, testMatthias(tick)
no issues (in between Jira limit kicked in, but no further problem)
(tick)(tick)
maven-source-plugin

Getting issues...

config, test

MSOURCES-143 was imported, but not with all comments (tool recognized an error during the import)
Matthias(tick) Same error message as during test, but manual compare did not show any difference in comments. They were all there. Unsure what field should cause the "null" value mentioned in the error message.#217 (tick)(tick)
maven-jlink-plugin

Getting issues...

config, test

Benjamin

(tick)

But we lost PRs due to human error

https://github.com/apache/maven-jlink-plugin/issues

(tick)

(tick)

maven-jmod-plugin

Getting issues...

config,
test
Benjamin(tick)
no issues
(tick)(tick)
plugins/reporting https://github.com/apache/maven-sources/blob/master/default.xml#L48

maven-changelog-plugin

Getting issues...

test,

config

Sandra

(tick)

#194 (tick)

(tick)

maven-changes-plugin

Getting issues...

test, config

Sandra

(tick) (first run had some user mapping error)

#596 (tick)

(tick)

maven-checkstyle-plugin

Getting issues...

test, config

Sandra

(tick)

#622 (tick)

(tick)

maven-doap-plugin

Getting issues...

config, test

Sandra

(tick)

#108 (tick)

(tick)

maven-javadoc-plugin

Getting issues...

config

Olivier

(tick)
no issues

(tick)

(tick)

maven-jdeps-plugin

Getting issues...

config, test

Matthias

(tick) no issues

#58 (tick)

(tick)

maven-jxr

Getting issues...

config, test

Olivier

(tick) no issues

(tick)

(tick)

maven-linkcheck-plugin

Getting issues...

config, test

Sandra

(tick) no issues

#56 (tick)

(tick)

maven-pmd-plugin

Getting issues...

config, test

Sandra

(tick) no issues (2 backported issues are created)

#636 (tick)

(tick)

maven-project-info-reports-plugin

Getting issues...

config, test

Sandra

(tick) no issues

#122 (tick)

(tick)

plugins/tools https://github.com/apache/maven-sources/blob/master/default.xml#L70

maven-antrun-pluginMANTRUN

Getting issues...

config, testMatthias(tick) no issues(tick)(tick)
maven-archetypeARCHETYPE

Getting issues...

(question) config, testMatthias(tick) no issues(tick)(tick)

maven-artifact-plugin

MARTIFACT

Getting issues...

config, testMatthias(tick)(tick)(tick)
maven-assembly-pluginMASSEMBLY

Getting issues...

configOlivier(tick) no issues(tick)(tick)
maven-dependency-pluginMDEP

Getting issues...

configOlivier(tick) no issues(tick)(tick) 
maven-enforcerMENFORCER

Getting issues...


Matthias(tick) no issues ( 1 backporting issue created)(tick)(tick)
maven-gpg-pluginMGPG

Getting issues...


Matthias(tick) no issues(tick)(tick)
maven-help-pluginMPH

Getting issues...


Matthias(tick) no issues (one user in mapping updated)(tick)(tick)
maven-invoker-pluginMINVOKER

Getting issues...

config, testOlivier(tick)(tick)(tick)
maven-jarsigner-pluginMJARSIGNER

Getting issues...

config, testOlivier(tick)(tick)(tick)
maven-jdeprscan-pluginMJDEPRSCAN

Getting issues...

config, testSandra(tick)(tick) #52(tick)
maven-plugin-toolsMPLUGIN

Getting issues...

configOlivier(tick)(tick)(tick)
maven-releaseMRELEASE

Getting issues...

configOlivier(tick)(tick)(tick)
maven-remote-resources-pluginMRRESOURCES

Getting issues...

configOlivier(tick)(tick)(tick)
maven-scmSCM

Getting issues...

configOlivier(tick)(tick)(tick)
maven-scm-publish-pluginMSCMPUB

Getting issues...

config, testOlivier(tick)(tick)(tick)
maven-scripting-pluginMSCRIPTING

Getting issues...

config, testSandra(tick)#54 (tick)(tick)
maven-stage-pluginMSTAGE

Getting issues...

confg, testSandra(tick)#49 (tick)(tick)
maven-toolchains-pluginMTOOLCHAINS

Getting issues...

config, testSandra(tick)#84 (tick)(tick)
shared https://github.com/apache/maven-sources/blob/master/default.xml#L70

maven-archiver

MSHARED
maven-archiver

Getting issues...

config

add to application.properties:

application.properties
jira.component=maven-archiver
Matthias(tick) ("only" rate limits)(tick)(tick)
maven-artifact-transfer
MSHARED
maven-artifact-transfer


should get retired - see Phase 3 list





maven-common-artifact-filters
MSHARED
maven-common-artifact-filters

Getting issues...

config

add to application.properties:

application.properties
jira.component=maven-common-artifact-filters
Matthias(tick)  no issues(tick)(tick)
maven-dependency-analyzerMSHARED maven-dependency-analyzer

Getting issues...

test

config

add to application.properties:

application.properties
jira.component=maven-dependency-analyzer
Matthias(tick) no issues(tick)(tick)
maven-dependency-tree

MSHARED

maven-dependency-tree

Getting issues...

config

add to application.properties:

application.properties
jira.component=maven-dependency-tree
Matthias(tick) no issues(tick) (tick)
maven-file-managementMSHARED file-management

Getting issues...

config

add to application.properties:

application.properties
jira.component=file-management
Matthias(tick) no issues(tick)(tick)
maven-filtering

MSHARED

maven-filtering

Getting issues...

config

add to application.properties:

application.properties
jira.component=maven-filtering
Matthias(tick) no issues(tick)(tick)
maven-invoker

MSHARED

maven-invoker

Getting issues...

config

add to application.properties:

application.properties
jira.component=maven-invoker
Matthias(tick) no issues(tick)(tick)
maven-jarsigner

MSHARED

maven-jarsigner

Getting issues...

config

add to application.properties:

application.properties
jira.component=maven-jarsigner
Matthias(tick) no issues(tick)(tick)
maven-mapping

MSHARED

maven-mapping

Getting issues...

add to application.properties:

application.properties
jira.component=maven-mapping
Matthias(tick) no issues(tick)(tick)
maven-project-utils

MSHARED

maven-project-utils


should get retired - see Phase 3 list





maven-reporting-api

MSHARED

maven-reporting-api

Getting issues...


Matthias(tick)(tick)(tick)
maven-reporting-exec

MSHARED

maven-reporting-exec

Getting issues...


Matthias(tick)(tick)(tick)
maven-reporting-impl

MSHARED

maven-reporting-impl

Getting issues...


Matthias(tick)(tick)(tick)
maven-script-interpreter

MSHARED

maven-script-interpreter

Getting issues...


Matthias(tick)(tick)(tick)
maven-shared-incremental

MSHARED

maven-shared-incremental

Getting issues...


Matthias(tick)(tick)(tick)
maven-shared-io

MSHARED

maven-shared-io

Getting issues...


Matthias(tick)(tick)(tick)
maven-shared-jar

MSHARED

maven-shared-jar

Getting issues...


Matthias(tick)(tick)(tick)
maven-shared-resourcesMSHARED maven-shared-resources

Getting issues...

config,
test

Benjamin(tick)(tick)(tick)

maven-shared-utils

MSHARED

maven-shared-utils

Getting issues...


Matthias(tick)(tick)(tick)
maven-verifier

MSHARED

maven-verifier

Getting issues...


Matthias(tick) no issue180 (tick)(tick)
doxia https://github.com/apache/maven-sources/blob/master/default.xml#L113

maven-doxia

DOXIA

Getting issues...

some components got moved to DOXIASITETOOLS

config, testSandra(tick)#309 (tick)(tick)
maven-doxia-siteno jira, nothing to migrate





maven-doxia-sitetoolsDOXIASITETOOLS

Getting issues...

config, testSandra(tick)#213(tick)(tick)
maven-doxia-book-maven-pluginDOXIATOOLS Doxia Book Maven Plugin

Getting issues...

config, test
add to application.properties:

jira.component=Doxia Book Maven Plugin
Sandra(tick)#8 (tick)(tick)
maven-doxia-book-rendererDOXIATOOLS Doxia Book Renderer

Getting issues...

config, test
add to application.properties:

jira.component=Doxia Book Renderer
Sandra(tick)#24 (tick)(tick)
maven-doxia-converterDOXIATOOLS Doxia Converter

Getting issues...

config, test
add to application.properties:

jira.component=Doxia Converter
Sandra(tick)#119 (tick)(tick)
maven-doxia-linkcheckDOXIATOOLS Doxia Linkcheck

Getting issues...

config, test
add to application.properties:

jira.component=Doxia Linkcheck
Sandra(tick)#25 (tick)(tick)
misc https://github.com/apache/maven-sources/blob/master/default.xml#L121

maven-archetypes

Getting issues...

config, testSandra(tick)#115 (tick) (tick)
maven-parentMPOM maven

Getting issues...

config, testBenjamin(tick)(tick)(tick)
maven-apache-parent

MPOM asf

Getting issues...

(question)
config,
test
Benjamin(tick)(tick)(tick)
maven-apache-resourcesMASFRES

Getting issues...

config, testBenjamin(tick) completed without issues
https://github.com/apache/maven-apache-resources/issues
(tick)(tick)
maven-default-skinMSKINS default
should get retired - see Phase 3 list



maven-fluido-skinMSKINS fluido

Getting issues...


Olivier(tick)(tick)(tick)
maven-dist-toolno jira, nothing to migrate--
-

maven-gh-actions-sharedno jira, nothing to migrate--
-

maven-indexerMINDEXER

Getting issues...

configOlivier(tick)(tick)(tick)
maven-plugin-testingMPLUGINTESTING

Getting issues...


Olivier(tick)(tick)(tick)
maven-wagonWAGON

Getting issues...


Olivier(tick)(tick)(tick)
  • No labels