Many internal platforms grow the same way. One VM gets Docker Compose, then a second stack, then another VM for a new tool. After a few years, an LLM gateway, a search engine, workflow automation and a chat interface are spread across several VMs, and each service has an IP address and port that something, somewhere, has hard-coded.

With a Kubernetes platform in place, including GitOps, a Gateway, MetalLB and backups, the question becomes how to move those workloads so that the VMs can actually be switched off, without editing every client that knows <old-ip>:4000.

This playbook sets out the method for that migration: three principles, a repeatable seven-step phase and a way to decide what moves first.

Principle 1: Move the IP, not the clients

The instinct is to give each application a clean new hostname and update its clients. Avoid that on the first move. Clients are where migrations stall: a webhook registered with a third party, a URL in an automation script, a Secret in another namespace, a saved bookmark.

MetalLB can take over the old VM’s address:

1. Create a second address pool, preserved-service-ips, with autoAssign: false so that nothing lands there by accident. It starts empty.
2. At cut-over, after the VM is shut down, add its address as a /32.
3. Create one LoadBalancer Service per application, with loadBalancerIPs: <old-ip> and the old port.
4. Where several applications lived on one VM, their Services share the address. Give them the same metallb.universe.tf/allow-shared-ip key and distinct ports.

MetalLB announces the address with a gratuitous ARP, and within seconds <old-ip>:4000 is served from Kubernetes. Every reverse proxy host, DNS record and hard-coded client keeps working without an edit. Hostnames can follow later through the Gateway.

Three rules keep this safe:

• Never take an address that is still alive. MetalLB will ARP for an address another machine is using, so shut down the VM before adding the /32.
• Keep the DHCP reservation and rename it. It binds the address to the retired VM’s MAC, so DHCP can never hand the address to another device.
• Keep the load balancer range out of the DHCP pool, so no dynamic lease can land on an address MetalLB announces.

Principle 2: Translate Docker’s semantics, do not copy them

docker ps and docker inspect tell you more than the Compose file, because they show what is actually running.

On the VM	In Kubernetes
restart: always or unless-stopped	Deployment or StatefulSet, with liveness and readiness probes and a startupProbe sized for your disks
Container stopped by hand	replicas: 0 in Git, or do not migrate it. Argo CD’s self-heal keeps it at 0
restart: no one-shot container	initContainer or hook Job
Container in a restart loop	Usually do not migrate. Find out why first, because it is often a sidecar the cluster already provides
Named volume	PVC, with databases on the fast tier and bulk files on the large one
Bind mount of a config file	ConfigMap, with secrets as SealedSecrets
ports: "0.0.0.0:5432:5432"	Probably nothing. Database ports published to the whole network are usually accidents. Use ClusterIP unless a real client appears in a week of connection logs
image: ...:latest	The digest the VM is running now (docker inspect -f '{{.Image}}'). Upgrade after the migration has soaked, not during it
Host systemd unit next to Docker	Containerize it, or the VM cannot be decommissioned yet
Container with /var/run/docker.sock	A red flag. The application drives Docker itself, as workflow workers, CI runners and Portainer do. It needs a redesign, not a lift and shift

That final row can disqualify a whole VM from the first wave. Workloads shaped as one container, one volume and one port migrate cleanly.

Principle 3: One VM per phase, with the same seven steps

Each phase runs gate, back up, build, validate, cut over, soak, decommission.

1. Gate. Check host memory, swap, I/O wait and backup storage usage against written thresholds. A migration that starts on a stressed host fails for reasons unrelated to the migration.
2. Back up. – Take a protected VM backup and keep the VM’s configuration with it. A restore without its definition is not a restore. – Take application-level exports that do not depend on the VM: pg_dump, automation data including its encryption key, and archives of application data directories.
3. Build the new version on a temporary MetalLB address, from Git, with data restored from the exports.
4. Validate with golden checks captured from the old service first, and scripted rather than checked by eye. For example: – the same model list from an LLM gateway, and one completion per model; – the same workflow count in an automation tool; – the same top five results for 20 fixed search queries.
5. Cut over. – Stop the containers and shut down the VM. – Add the /32 and move the Services from the temporary address to the old one. – Run the golden checks again. – Rollback is the reverse and takes minutes: remove the /32 and start the VM.
6. Soak for 7 days. Scheduled jobs must run on time, there must be no duplicate notifications, and backups of the new state must exist.
7. Decommission. The VM stays stopped, not destroyed, for another week, and its backup stays protected for 30 days. Only then is it deleted.

The trap for automation tools: running twice

n8n, changedetection and anything else that acts by sending mail, calling webhooks or posting messages will act twice if both copies run. Import every workflow into the new instance as inactive, validate with manual executions, and activate only after the old instance is stopped. The old instance returns only through an explicit rollback.

What surrounds the application

• Reverse proxies and DNS. With IP takeover they need no change. List them anyway, so you know what to check.
• Tailscale and other overlay names. A VM that is a tailnet node takes its MagicDNS name with it. Remove the node first, then give the new Service the same name with the Tailscale Kubernetes operator, or advertise the /32 from a subnet router.
• Monitoring. The VM’s own agents disappear and the cluster’s take over. Update blackbox and alert targets, or the retired VM will show as down forever.
• Dead proxy hosts. The inventory will also find entries pointing at ports where nothing listens. Delete them now, because they confuse the next migration.

Choosing what goes first

Ask four questions about each VM:

• How much state does it hold, and is there a clean export?
• How many clients hard-code it?
• Does it do anything Kubernetes cannot, such as Docker socket access, host services or inbound SMTP?
• How much memory does it free?

Those questions usually produce an order like this:

1. Low-state, high-value services first. An LLM gateway with one Postgres database and many consumers is a strong candidate: little state, high fan-out, and memory returned to the host.
2. Simple applications next, along with any larger application that must move before the VM can be retired.
3. Already-stopped workloads involve no live cut-over, but may need a new node. A hardened golden-image template can make that a roughly 5-minute task.

A VM with Docker socket workers, inbound mail and long-lived messaging sessions should wait. Moving it is a redesign rather than a migration, and it deserves its own project.

The payoff

A VM migrated this way leaves behind:

• an address that still answers, now served by the cluster;
• a Git history of every configuration change;
• a SealedSecret in place of an .env file that nobody backs up;
• probes that restart the application when it hangs;
• 8 to 16 GB of host memory returned.

Most importantly, it leaves a VM you can finally switch off.

---

Get the next one in your inbox

Enjoyed this? I write about local AI, self-hosting, and cloud cost ninjutsu. Join the Ninja Dispatch for occasional, high-signal notes. No fluff.

No spam. Unsubscribe anytime.