Migrate groups and projects by using direct transfer

DETAILS: Tier: Free, Premium, Ultimate Offering: GitLab.com, Self-managed, GitLab Dedicated

To migrate GitLab groups and projects by using direct transfer, you:

Fulfill the prerequisites.
Connect the source GitLab instance.
Select groups and projects to migrate and begin the migration.
Review the results of the import.

If there are any problems, you can:

Cancel or retry the migration.
Check the troubleshooting information.

Prerequisites

Requirement for Maintainer role instead of Developer role introduced in GitLab 16.0 and backported to GitLab 15.11.1 and GitLab 15.10.5.

Before migrating by using direct transfer, see the following prerequisites.

Network

The network connection between instances or GitLab.com must support HTTPS.
Firewalls must not block the connection between the source and destination GitLab instances.

Versions

To maximize the chance of a successful and performant migration, you should:

Upgrade both the source and destination instances to GitLab 16.8 or later to use bulk import and export of relations. For more information, see epic 9036.
Migrate between versions that are as new as possible. Update the source and destination instances to as late a version as possible to take advantage of bug fixes and improvements added over time.
Configure Sidekiq properly.

We have successfully tested migrations between a source instance running GitLab 16.2 and a destination instance running GitLab 16.8.

Configuration

Both GitLab instances must have group migration by direct transfer enabled in application settings by an instance administrator.
You must have a personal access token for the source GitLab instance:
- For GitLab 15.1 and later source instances, the personal access token must have the api scope.
- For GitLab 15.0 and earlier source instances, the personal access token must have both the api and read_repository scopes.
You must have the Owner role on the source group to migrate from.
You must have a role in the destination namespace that enables you to create a subgroup in that namespace.
To import items stored in object storage, you must either:
- Configure proxy_download.
- Ensure that the destination GitLab instance has access to the object storage of the source GitLab instance.
You cannot import groups with projects when the source instance or group has Default project creation protection set to No one. If required, this setting can be changed:
- For a whole instance.
- For specific groups.

User contributions and membership mapping

DETAILS: Offering: Self-managed, GitLab Dedicated

Mapping of shared and inherited shared members as direct members was introduced in GitLab 16.3.

Introduced in GitLab 16.11, shared and inherited shared members are no longer mapped as direct members if they are already shared or inherited shared members of the imported group or project.

Full support for mapping inherited membership introduced in GitLab 17.1.

Removed from GitLab.com direct transfer migrations in GitLab 17.5 in favor of the alternative.

This method of user contributions and membership mapping is available for GitLab self-managed without enabled feature flags. For information on the other method available for GitLab self-managed with enabled feature flags and for GitLab.com, see User contribution and membership mapping.

Users are never created during a migration. Instead, contributions and membership of users on the source instance are mapped to users on the destination instance. The type of mapping of a user's membership depends on the membership type on source instance:

Direct memberships are mapped as direct memberships on the destination instance.
Inherited memberships are mapped as inherited memberships on the destination instance.
Shared memberships are mapped as direct memberships on the destination instance unless the user has an existing shared membership. Full support for mapping shared memberships is proposed in issue 458345.

When mapping inherited and shared memberships, if the user has an existing membership in the destination namespace with a higher role than the one being mapped, the membership is mapped as a direct membership instead. This ensures the member does not get elevated permissions.

NOTE: There is a known issue affecting the mapping of shared memberships.

Configure users on destination instance

To ensure GitLab maps users and their contributions correctly between the source and destination instances:

Create the required users on the destination GitLab instance. You can create users with the API only on self-managed instances because it requires administrator access. When migrating to GitLab.com or a self-managed GitLab instance you can:
- Create users manually.
- Set up or use your existing SAML SSO provider and leverage user synchronization of SAML SSO groups supported through SCIM. You can bypass the GitLab user account verification with verified email domains.
Ensure that users have a public email on the source GitLab instance that matches any confirmed email address on the destination GitLab instance. Most users receive an email asking them to confirm their email address.
If users already exist on the destination instance and you use SAML SSO for GitLab.com groups, all users must link their SAML identity to their GitLab.com account.

There is no way in the GitLab UI or API to automatically set public email addresses for users. If you need to set a lot of user accounts to have public email addresses, see issue 284495 for a potential workaround.

Connect the source GitLab instance

On the destination GitLab instance, create the group you want to import to and connect the source GitLab instance:

Create either:
- A new group. On the left sidebar, at the top, select Create new ({plus}) and New group. Then select Import group.
- A new subgroup. On existing group's page, either:
  - Select New subgroup.
  - On the left sidebar, at the top, select Create new ({plus}) and New subgroup. Then select the import an existing group link.
Enter the base URL of a GitLab instance.
Enter the personal access token for your source GitLab instance.
Select Connect instance.

Select the groups and projects to import

Introduced in GitLab 15.8, option to import groups with or without projects.

Import user memberships checkbox introduced in GitLab 17.6.

After you have authorized access to the source GitLab instance, you are redirected to the GitLab group importer page. Here you can see a list of the top-level groups on the connected source instance where you have the Owner role.

If you do not want to import all user memberships from the source instance, ensure the Import user memberships checkbox is cleared. For example, the source instance might have 200 members, but you might want to import 50 members only. After the import completes, you can add more members to groups and projects.

By default, the proposed group namespaces match the names as they exist in source instance, but based on your permissions, you can choose to edit these names before you proceed to import any of them. Group and project paths must conform to naming limitations and are normalized if necessary to avoid import failures.
Next to the groups you want to import, select either:
- Import with projects. If this is not available, see prerequisites.
- Import without projects.
The Status column shows the import status of each group. If you leave the page open, it updates in real-time.
After a group has been imported, select its GitLab path to open its GitLab URL.

WARNING: Importing groups with projects is in beta. This feature is not ready for production use.

Group import history

Partially completed status introduced in GitLab 16.7.

You can view all groups migrated by you by direct transfer listed on the group import history page. This list includes:

Paths of source groups.
Paths of destination groups.
Start date of each import.
Status of each import.
Error details if any errors occurred.

To view group import history:

Sign in to GitLab.
On the left sidebar, at the top, select Create new ({plus}) and New group.
Select Import group.
In the upper-right corner, select History.
If there are any errors for a particular import, select Show errors to see their details.

Review results of the import

Introduced in GitLab 16.6 with a flag named bulk_import_details_page. Enabled by default.

Feature flag bulk_import_details_page removed in GitLab 16.8.

Details for partially completed and completed imports added in GitLab 16.9.

Introduced in GitLab 17.0, an Imported badge to indicate that designs, epics, issues, merge requests, notes (system notes and comments), snippets, and user profile activity were imported.

To review the results of an import:

Go to the Group import history page.
To see the details of a failed import, select the Show errors link on any import with a Failed or Partially completed status.
If the import has a Partially completed or Complete status, to see which items were and were not imported, select View details.

You can also see that an item was imported when you see an Imported badge on some items in the GitLab UI.

Cancel a running migration

If required, you can cancel a running migration by using either the REST API or a Rails console.

Cancel with the REST API

For information on cancelling a running migration with the REST API, see Cancel a migration.

Cancel with a Rails console

To cancel a running migration with a Rails console:

Start a Rails console session on the destination GitLab instance.
Find the last import by running the following command. Replace USER_ID with the user ID of the user that started the import:
```
bulk_import = BulkImport.where(user_id: USER_ID).last
```

Cause the import and all items associated with it to fail by running the following command:

bulk_import.entities.each do |entity|
  entity.trackers.each do |tracker|
    tracker.batches.each(&:fail_op!)
  end
  entity.trackers.each(&:fail_op!)
  entity.fail_op!
end
bulk_import.fail_op!

Cancelling a bulk_import doesn't stop workers that are exporting the project on the source instance, but prevents the destination instance from:

Asking the source instance for more projects to be exported.
Making other API calls to the source instance for various checks and information.

Retry failed or partially successful migrations

If your migrations fail, or partially succeed but are missing items, you can retry the migration. To retry a migration of a:

Top-level group and all of its subgroups and projects, use either the GitLab UI or the GitLab REST API.
Specific subgroups or projects, use the GitLab REST API.