The Ultimate GitLab Import/Export Toolkit for Engineers

---

“I’m a DevOps engineer - if I’m doing it manually twice, I’m writing a script.”

---

A few weeks ago, I was handed a deceptively simple‑sounding task: spin up an entire playground environment, pipelines, history, everything for a complex micro‑services platform, from scratch.
The catch? All source code lived in GitLab, and there were dozens of projects spread across nested groups.
I scoured the web for a clean “export → import” recipe that would:

• move commits, MRs, releases, tags, wikis - the whole lineage

• preserve branch protections & default branches

• run reliably for tens of repos without babysitting

Even GitLab’s docs stopped short of a full answer. Most blog posts boiled down to “click this button N times” or “write a bash loop and pray.”
Manual migration? Over my dead keyboard.
So I cracked open the GitLab API, dusted off my Python muscle memory, and the GitLab Migration & Branch‑Housekeeping Toolkit was born.

---

Why Another Solution?

Existing Options	What This Toolkit Delivers
UI‑only exports (one project at a time)	Bulk, unattended export/import of entire groups
3rd‑party “all‑in‑one” SaaS (pricey, opaque)	100 % open‑source Python - reads every line
Incomplete migrations (no MRs, issues)	Captures full project state, Inc. metadata
Risk of projects landing in root/Administrator	Namespace‑safe import with post‑audit cleanup
Branch sprawl post‑import	Automated branch hygiene—keeps demo → spawns demo2

---

A 50,000-ft View

flowchart TD
    subgraph Source GitLab
        A>All projects<br/>in `group-1`] --> B[export.py]
    end
    B -->|*.tar.gz| C((Secure<br/>storage))
    subgraph Destination GitLab
        D[import.py / selected...py] --> E((Group:<br/>group-1))
        E -->|Fix stray imports| F[cleanup.py]
        E -->|Delete stale branches| G[remove_obsolete_branches.py]
    end

• export.py triggers asynchronous exports and downloads resulting archives

• import.py / selected_import.py stream archives into the correct destination group

• cleanup.py deletes projects accidentally imported under default user namespaces

• Branch scripts enforce a single source of truth demo branch and spawn demo2

---

Under the Hood: Key Design Decisions

1. Official python‑gitlab SDK: No scraping, no unofficial endpoints.

2. Idempotency First: Every script can be re-run safely, as existing projects or branches are detected and skipped unless you opt in to overwrite.

3. Streaming Downloads & Uploads: Exports are streamed in 1 MiB chunks; imports upload file-like objects, keeping memory steady.

4. Namespace Double‑Lock: Imports supply both namespace (string) and namespace_id (int) to the API, all but eliminating the dreaded “imported to root” scenario.

5. Branch Hygiene as First-Class Citizen: Post-import, the toolkit unprotects stale branches, deletes them, re-protects the guardians, and sets defaults because CI breaks are not an option.

---

Running the Toolkit in 6 Commands

git clone https://github.com/SubhanshuMG/gitlab-import-export.git
cd gitlab-import-export && python3 -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt

# 1 · bulk export
SRC_GITLAB=... SRC_TOKEN=... python scripts/export.py

# 2 · bulk import
DST_GITLAB=... DST_TOKEN=... python scripts/import.py  # or selected_import.py

# 3 · fix stray namespaces & branches
python scripts/cleanup.py
python scripts/remove_obsolete_branches.py

python scripts/selected_import.py  # edit SELECTED_PROJECTS = ["payment-service"]
python scripts/specific_project_remove_branches.py  # TARGET_PROJECT_NAME="payment-service"

---

Real‑World Payoff

• 4× Speed‑up vs manual UI export/import (12 repos → 5 min)

• Zero branch‑related pipeline failures post‑migration

• Repeatable same scripts now run nightly to refresh our playground

---

Troubleshooting Cheat‑Sheet

Symptom	Diagnosis & Fix
GitlabGetError: 404 on group	Check PAT scope (read_api) and group path correctness
Import stuck at scheduled	Sidekiq busy; verify destination runners aren’t paused
Popen tmp/no space left	Exports bigger than /tmp; set TMPDIR to a larger mount
Protected branch won’t delete	You’re not a Maintainer; branch script auto‑unprotects but needs rights

---

Extending the Toolkit

• Parallelize exports → Wrap the call loop in concurrent.futures.ThreadPoolExecutor

• CI Variables & Releases → After import, iterate /projects/:id/variables & /releases

• SaaS → Self‑Managed → Add mapping for group paths that changed between instances

• GitLab => GitHub? → Swap endpoints; the logic stays 90 % identical

PRs are welcome, just fork and raise an issue!

---

The Repository

☑️ MIT‑licensed ☑️ Zero external dependencies (beyond python‑gitlab)
Check out the full source code and Python scripts w/ instructions:

GitHub → https://github.com/SubhanshuMG/gitlab-import-export.git

---

Final Words

If you’ve ever copy‑pasted repos one by one or worse, lost commit history during a migration; this toolkit is for you.
Fork it, bend it to your needs, and ship playground environments (or entire prod clones) with a single command.

Happy automating,

Signing off
Subhanshu Mohan Gupta (DevOps & Automation addict)