{"id":3523,"date":"2026-06-16T13:51:00","date_gmt":"2026-06-16T19:51:00","guid":{"rendered":"https:\/\/yyc.ninja\/index.php\/2026\/10\/20\/from-vms-and-docker-compose-to-rke2-a-migration-playbook\/"},"modified":"2026-10-20T13:00:22","modified_gmt":"2026-10-20T19:00:22","slug":"from-vms-and-docker-compose-to-rke2-a-migration-playbook","status":"publish","type":"post","link":"https:\/\/yyc.ninja\/index.php\/2026\/06\/16\/from-vms-and-docker-compose-to-rke2-a-migration-playbook\/","title":{"rendered":"From VMs and Docker Compose to RKE2: a migration playbook that keeps every IP working"},"content":{"rendered":"<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1200\" height=\"630\" src=\"https:\/\/yyc.ninja\/wp-content\/uploads\/2026\/10\/cg-from-vms-and-docker-compose-to-rke2-a-migration-playbook-cover.png\" alt=\"From VMs and Docker Compose to RKE2: a migration playbook that keeps every IP working\" class=\"wp-image-3522\" sizes=\"auto, (max-width: 1200px) 100vw, 1200px\" srcset=\"https:\/\/yyc.ninja\/wp-content\/uploads\/2026\/10\/cg-from-vms-and-docker-compose-to-rke2-a-migration-playbook-cover.png 1200w, https:\/\/yyc.ninja\/wp-content\/uploads\/2026\/10\/cg-from-vms-and-docker-compose-to-rke2-a-migration-playbook-cover-300x158.png 300w, https:\/\/yyc.ninja\/wp-content\/uploads\/2026\/10\/cg-from-vms-and-docker-compose-to-rke2-a-migration-playbook-cover-1024x538.png 1024w, https:\/\/yyc.ninja\/wp-content\/uploads\/2026\/10\/cg-from-vms-and-docker-compose-to-rke2-a-migration-playbook-cover-768x403.png 768w, https:\/\/yyc.ninja\/wp-content\/uploads\/2026\/10\/cg-from-vms-and-docker-compose-to-rke2-a-migration-playbook-cover-100x53.png 100w\" \/><\/figure>\n<p class=\"wp-block-paragraph\"><em>Part 5 of the data-e2e series. The concrete plan for this homelab is the project&#8217;s planning docs;\nthe phase method is the project&#8217;s planning docs \u00a718.<\/em><\/p>\n<p class=\"wp-block-paragraph\">A typical homelab grows the same way. A VM gets Docker Compose, then a second stack, then a third VM &#8220;for AI\nthings&#8221;. After a few years:<\/p>\n<ul class=\"wp-block-list\">\n<li>one VM runs an LLM gateway, a search engine and a chat UI;<\/li>\n<li>another runs n8n, an RSS reader and an 11-container Dify;<\/li>\n<li>a third runs Windmill with workers that drive the host&#8217;s Docker socket.<\/li>\n<\/ul>\n<p class=\"wp-block-paragraph\">Every service has a LAN IP and port that something, somewhere, has hard-coded.<\/p>\n<p class=\"wp-block-paragraph\">Meanwhile a Kubernetes cluster now exists for the lakehouse (Parts 1-4), with GitOps, a Gateway, MetalLB and\nbackups. The question is how to move the VM workloads onto it <strong>so the VMs can actually be switched off<\/strong>, without\nediting every client that knows <code>&lt;lan&gt;.82:4000<\/code>.<\/p>\n<h3 class=\"wp-block-heading\">Principle 1: move the IP, not the clients<\/h3>\n<p class=\"wp-block-paragraph\">The instinct is to give each app a nice new hostname and update the clients. Resist it for the first move. Clients\nare where migrations die: a webhook registered with a third party, a URL in a Windmill script, a Secret in another\nnamespace, a bookmark on a phone.<\/p>\n<p class=\"wp-block-paragraph\"><strong>MetalLB can take over the old VM&#8217;s IP.<\/strong><\/p>\n<ol class=\"wp-block-list\">\n<li>Create a second address pool, <code>preserved-service-ips<\/code>, with <code>autoAssign: false<\/code> so nothing lands there by\n accident. It starts empty.<\/li>\n<li>At cut-over, <em>after<\/em> the VM is shut down, add the VM&#8217;s address as a <code>\/32<\/code>.<\/li>\n<li>Create one <code>LoadBalancer<\/code> Service per app with <code>loadBalancerIPs: &lt;old ip&gt;<\/code> and the old port.<\/li>\n<li>When several apps lived on one VM, their Services share the IP. Give them the same\n <code>metallb.universe.tf\/allow-shared-ip<\/code> key and distinct ports.<\/li>\n<\/ol>\n<p class=\"wp-block-paragraph\">MetalLB announces the address with a gratuitous ARP, and within seconds <code>&lt;lan&gt;.82:4000<\/code> is LiteLLM-in-Kubernetes.\nEvery reverse-proxy host, DNS record and hard-coded client keeps working <strong>without an edit<\/strong>. Hostnames can come\nlater, at leisure, through the Gateway.<\/p>\n<p class=\"wp-block-paragraph\">Three rules make this safe:<\/p>\n<ul class=\"wp-block-list\">\n<li><strong>Never take an IP that is still alive.<\/strong> MetalLB will happily ARP for an address another machine is using.\n <code>qm shutdown<\/code> first, then add the <code>\/32<\/code>.<\/li>\n<li><strong>Keep the router&#8217;s DHCP reservation<\/strong>, renamed (<code>metallb-llm-mcp<\/code>). The reservation binds the address to the dead\n VM&#8217;s MAC, so DHCP can never hand it to a laptop. Delete it and another device will get <code>.82<\/code> next week.<\/li>\n<li><strong>Keep the load-balancer range out of the DHCP pool<\/strong>, so no dynamic lease can ever land on an address MetalLB announces.<\/li>\n<\/ul>\n<h3 class=\"wp-block-heading\">Principle 2: translate Docker&#8217;s semantics, don&#8217;t copy them<\/h3>\n<p class=\"wp-block-paragraph\"><code>docker ps<\/code> and <code>docker inspect<\/code> tell you more than the Compose file, because they show what is <em>actually<\/em> running.\nThe mapping:<\/p>\n<table>\n<thead>\n<tr>\n<th>On the VM<\/th>\n<th>In Kubernetes<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td><code>restart: always<\/code> \/ <code>unless-stopped<\/code><\/td>\n<td>Deployment or StatefulSet + liveness\/readiness probes + a <strong>startupProbe sized for your disks<\/strong><\/td>\n<\/tr>\n<tr>\n<td>Container stopped by hand (<code>unless-stopped<\/code>, exited)<\/td>\n<td><code>replicas: 0<\/code> in Git, or don&#8217;t migrate it. Argo CD&#8217;s self-heal keeps it at 0<\/td>\n<\/tr>\n<tr>\n<td><code>restart: no<\/code> one-shot (e.g. <code>init_permissions<\/code>)<\/td>\n<td>initContainer or a hook Job<\/td>\n<\/tr>\n<tr>\n<td>Container in a restart loop<\/td>\n<td>Usually don&#8217;t migrate. Find out why first; often it&#8217;s a sidecar the cluster already provides (cAdvisor vs. the cluster&#8217;s metrics)<\/td>\n<\/tr>\n<tr>\n<td>Named volume<\/td>\n<td>PVC. Databases on the fast tier, bulk files on the big one<\/td>\n<\/tr>\n<tr>\n<td>Bind mount of a config file<\/td>\n<td>ConfigMap (secrets \u2192 SealedSecret)<\/td>\n<\/tr>\n<tr>\n<td><code>ports: \"0.0.0.0:5432:5432\"<\/code><\/td>\n<td><strong>Probably nothing.<\/strong> Database ports published to the whole LAN are usually accidents. ClusterIP only, unless a real client shows up in a week of connection logs<\/td>\n<\/tr>\n<tr>\n<td><code>image: \u2026:latest<\/code><\/td>\n<td>The <strong>digest the VM is running now<\/strong> (<code>docker inspect -f '{{.Image}}'<\/code>). Upgrade after the migration has soaked, never during it<\/td>\n<\/tr>\n<tr>\n<td>Host systemd unit next to Docker<\/td>\n<td>Containerise it, or that VM can&#8217;t be decommissioned yet<\/td>\n<\/tr>\n<tr>\n<td>Container with <code>\/var\/run\/docker.sock<\/code><\/td>\n<td>A red flag. The app drives Docker itself (Windmill workers, CI runners, Portainer). Needs a redesign (native or Kubernetes workers), not a lift-and-shift<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p class=\"wp-block-paragraph\">In this homelab that last row disqualified a whole VM from the first wave. The other two VMs are almost entirely\n&#8220;one container, one volume, one port&#8221;, which is the shape that migrates cleanly.<\/p>\n<h3 class=\"wp-block-heading\">Principle 3: one VM per phase, with the same seven steps<\/h3>\n<p class=\"wp-block-paragraph\">Each phase runs <strong>gate \u2192 back up \u2192 build \u2192 validate \u2192 cut over \u2192 soak \u2192 decommission<\/strong>.<\/p>\n<ol class=\"wp-block-list\">\n<li><strong>Gate.<\/strong> Check host RAM, swap, I\/O wait and backup-datastore usage against written thresholds. A migration that\n starts on a stressed host fails for reasons unrelated to the migration.<\/li>\n<li><strong>Back up.<\/strong>\n &#8211; A protected PBS backup of the VM. Keep the <code>qm config<\/code> dump with it; a VM restore without its definition is\n not a restore.\n &#8211; <strong>Application-level exports<\/strong> that don&#8217;t depend on the VM: <code>pg_dump<\/code>, n8n&#8217;s volume <em>including its encryption\n key<\/em>, a tar of the chat UI&#8217;s data directory.<\/li>\n<li><strong>Build<\/strong> the new version on a temporary MetalLB IP, from Git, with the data restored from the exports.<\/li>\n<li><strong>Validate with golden checks<\/strong> captured from the <em>old<\/em> service first:\n &#8211; the same model list from the LLM gateway, and one completion per model;\n &#8211; the same workflow count in n8n;\n &#8211; the same top-5 results for 20 fixed search queries.<\/li>\n<\/ol>\n<p class=\"wp-block-paragraph\">Scripted, not eyeballed.\n5. <strong>Cut over.<\/strong>\n &#8211; Stop the containers, then <code>qm shutdown<\/code>.\n &#8211; Add the <code>\/32<\/code> and move the Services from the temporary IP to the old one.\n &#8211; Run the golden checks again.\n &#8211; Rollback is the reverse and takes minutes: remove the <code>\/32<\/code>, <code>qm start<\/code>.\n6. <strong>Soak<\/strong> 7 days: scheduled jobs run on time, there are no duplicate notifications, and backups of the <em>new<\/em> state\n exist.\n7. <strong>Decommission.<\/strong> The VM stays <em>stopped<\/em> (not destroyed) for another week, and its backup stays protected for 30\n days. Then <code>qm destroy<\/code>.<\/p>\n<h3 class=\"wp-block-heading\">The trap specific to automation tools: running twice<\/h3>\n<p class=\"wp-block-paragraph\">n8n, changedetection and anything else that <em>acts<\/em> (sends mail, calls webhooks, posts messages) will act <strong>twice<\/strong>\nif both copies run. Import every workflow into the new instance <strong>inactive<\/strong>, validate with manual executions, and\nactivate only after the old instance is stopped. The old instance comes back only through an explicit rollback.<\/p>\n<h3 class=\"wp-block-heading\">Don&#8217;t forget the things around the app<\/h3>\n<ul class=\"wp-block-list\">\n<li><strong>Reverse proxies and DNS.<\/strong> With IP takeover they need no change. List them anyway, so you know who to check.\n Here two NPM instances pointed <code>flow.home.example.net<\/code> and <code>fresh.home.example.net<\/code> at <code>.114<\/code>.<\/li>\n<li><strong>Tailscale and other overlay names.<\/strong> A VM that is a tailnet node takes its MagicDNS name with it when it dies.\n Remove the node first, then give the new Service the same name with the Tailscale Kubernetes operator, or advertise\n the <code>\/32<\/code> from a subnet router.<\/li>\n<li><strong>Monitoring.<\/strong> The VM&#8217;s own agents (Alloy, cAdvisor, node-exporter) disappear and the cluster&#8217;s take over. Update\n the blackbox and alert targets, or the dead VM shows as DOWN forever.<\/li>\n<li><strong>Dead proxy hosts.<\/strong> The inventory also finds entries pointing at ports where nothing listens any more. Delete\n them now; they confuse the next migration.<\/li>\n<\/ul>\n<h3 class=\"wp-block-heading\">Choosing what goes first<\/h3>\n<p class=\"wp-block-paragraph\">For each VM, ask four questions:<\/p>\n<ul class=\"wp-block-list\">\n<li>How much state does it hold, and is there a clean export?<\/li>\n<li>How many clients hard-code it?<\/li>\n<li>Does it do anything Kubernetes can&#8217;t (Docker socket, host services, inbound SMTP)?<\/li>\n<li>How much RAM does it free?<\/li>\n<\/ul>\n<p class=\"wp-block-paragraph\">Here that produced:<\/p>\n<ol class=\"wp-block-list\">\n<li><strong>The LLM gateway VM.<\/strong> Little state (one Postgres plus config), the highest fan-out (the lakehouse itself calls\n it), and it frees 8 GB.<\/li>\n<li><strong>The automation VM.<\/strong> Simple apps plus one big one (Dify) that must move before the VM can go.<\/li>\n<li><strong>The search VM.<\/strong> Already stopped, so there&#8217;s no live cut-over, but it <em>adds<\/em> RAM and needs a new node. That is\n what the golden-image template is for: <code>add-rke2-node.sh<\/code> clones a pre-hardened node and joins it to the cluster\n in about 5 minutes.<\/li>\n<\/ol>\n<p class=\"wp-block-paragraph\">The VM with Docker-socket workers, SMTP and a WhatsApp session waits. Moving it is a redesign, not a migration, and it\nshould get that time on its own.<\/p>\n<h3 class=\"wp-block-heading\">The payoff<\/h3>\n<p class=\"wp-block-paragraph\">A VM you migrate this way leaves behind:<\/p>\n<ul class=\"wp-block-list\">\n<li>an IP that still answers, served by the cluster;<\/li>\n<li>a Git history of every config change;<\/li>\n<li>a Sealed Secret instead of an <code>.env<\/code> file on a disk nobody backs up;<\/li>\n<li>probes that restart it when it hangs;<\/li>\n<li>8-16 GB of host RAM returned.<\/li>\n<\/ul>\n<p class=\"wp-block-paragraph\">And, finally, a VM you can switch off.<\/p>\n<p class=\"wp-block-paragraph\"><em>Previous: Part 4 (Upgrades \u00b7 Start: Part 1), one spinning disk.<\/em><\/p>\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h2 class=\"wp-block-heading\">Get the next one in your inbox<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Enjoyed this? I write about local AI, self-hosting, and cloud cost ninjutsu. Join the Ninja Dispatch for occasional, high-signal notes. No fluff.<\/p>\n\n\n\n\t\t\t<form id=\"sib_signup_form_2\" method=\"post\" class=\"sib_signup_form\">\n\t\t\t\t<div class=\"sib_loader\" style=\"display:none;\"><img\n\t\t\t\t\t\t\tsrc=\"https:\/\/yyc.ninja\/wp-includes\/images\/spinner.gif\" alt=\"loader\"><\/div>\n\t\t\t\t<input type=\"hidden\" name=\"sib_form_action\" value=\"subscribe_form_submit\">\n\t\t\t\t<input type=\"hidden\" name=\"sib_form_id\" value=\"2\">\n                <input type=\"hidden\" name=\"sib_form_alert_notice\" value=\"Please fill out all required fields.\">\n                <input type=\"hidden\" name=\"sib_form_invalid_email_notice\" value=\"Please enter a valid email address.\">\n                <input type=\"hidden\" name=\"sib_security\" value=\"e492cd529a\">\n\t\t\t\t<div class=\"sib_signup_box_inside_2\">\n\t\t\t\t\t<div style=\"\/*display:none*\/\" class=\"sib_msg_disp\">\n\t\t\t\t\t<\/div>\n                    \t\t\t\t\t<p class=\"sib-email-area\">\n    <input type=\"email\" class=\"sib-email-area\" name=\"email\" placeholder=\"you@example.com\" required=\"required\">\n<\/p>\n<p class=\"sib-FIRSTNAME-area\">\n    <input type=\"text\" class=\"sib-FIRSTNAME-area\" name=\"FIRSTNAME\" placeholder=\"First name (optional)\">\n<\/p>\n<input type=\"hidden\" class=\"sib-SOURCE_DOMAIN-area\" name=\"SOURCE_DOMAIN\" value=\"yyc.ninja\">\n<input type=\"hidden\" class=\"sib-SIGNUP_SOURCE-area\" name=\"SIGNUP_SOURCE\" value=\"post\">\n<p>\n    <input type=\"submit\" class=\"sib-default-btn\" value=\"Join the Ninja Dispatch\">\n<\/p>\n<p class=\"ninja-fineprint\">No spam. Unsubscribe anytime.<\/p>\t\t\t\t<\/div>\n\t\t\t<\/form>\n\t\t\t<style>\n\t\t\t\tform#sib_signup_form_2{background:#192a3d;border-radius:10px;padding:16px;font-family:inherit}form#sib_signup_form_2 input[type=text],form#sib_signup_form_2 input[type=email],form#sib_signup_form_2 select{width:100%;box-sizing:border-box;margin:0 0 10px 0;padding:10px 12px;border:1px solid #33465c;border-radius:8px;background:#0f1c29;color:#e8eef5;font-size:14px}form#sib_signup_form_2 input::placeholder{color:#8a9bb0}form#sib_signup_form_2 .sib-default-btn{width:100%;margin:2px 0 0 0;padding:11px 14px;background:#e63946;color:#ffffff;border:0;border-radius:8px;font-size:15px;font-weight:600;cursor:pointer}form#sib_signup_form_2 .sib-default-btn:hover{background:#d02f3c}form#sib_signup_form_2 p{margin:0 0 10px 0}form#sib_signup_form_2 .ninja-fineprint{margin:8px 0 0;font-size:11px;color:#8a9bb0;text-align:center}form#sib_signup_form_2 p.sib-alert-message {\n    padding: 6px 12px;\n    margin-bottom: 20px;\n    border: 1px solid transparent;\n    border-radius: 4px;\n    -webkit-box-sizing: border-box;\n    -moz-box-sizing: border-box;\n    box-sizing: border-box;\n}\nform#sib_signup_form_2 p.sib-alert-message-error {\n    background-color: #f2dede;\n    border-color: #ebccd1;\n    color: #a94442;\n}\nform#sib_signup_form_2 p.sib-alert-message-success {\n    background-color: #dff0d8;\n    border-color: #d6e9c6;\n    color: #3c763d;\n}\nform#sib_signup_form_2 p.sib-alert-message-warning {\n    background-color: #fcf8e3;\n    border-color: #faebcc;\n    color: #8a6d3b;\n}\n\t\t\t<\/style>\n\t\t\t\n","protected":false},"excerpt":{"rendered":"<p>the phase method is the project&#8217;s planning docs \u00a718.<\/p>\n","protected":false},"author":1,"featured_media":3522,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[12],"tags":[44,46,45],"class_list":["post-3523","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-how-to","tag-data-engineering","tag-mlops","tag-streaming"],"featured_image_urls_v2":{"full":["https:\/\/yyc.ninja\/wp-content\/uploads\/2026\/10\/cg-from-vms-and-docker-compose-to-rke2-a-migration-playbook-cover.png",1200,630,false],"thumbnail":["https:\/\/yyc.ninja\/wp-content\/uploads\/2026\/10\/cg-from-vms-and-docker-compose-to-rke2-a-migration-playbook-cover-150x150.png",150,150,true],"medium":["https:\/\/yyc.ninja\/wp-content\/uploads\/2026\/10\/cg-from-vms-and-docker-compose-to-rke2-a-migration-playbook-cover-300x158.png",300,158,true],"medium_large":["https:\/\/yyc.ninja\/wp-content\/uploads\/2026\/10\/cg-from-vms-and-docker-compose-to-rke2-a-migration-playbook-cover-768x403.png",768,403,true],"large":["https:\/\/yyc.ninja\/wp-content\/uploads\/2026\/10\/cg-from-vms-and-docker-compose-to-rke2-a-migration-playbook-cover-1024x538.png",1024,538,true],"RoboGalleryMansoryImagesCenter":["https:\/\/yyc.ninja\/wp-content\/uploads\/2026\/10\/cg-from-vms-and-docker-compose-to-rke2-a-migration-playbook-cover-600x630.png",600,630,true],"RoboGalleryPreload":["https:\/\/yyc.ninja\/wp-content\/uploads\/2026\/10\/cg-from-vms-and-docker-compose-to-rke2-a-migration-playbook-cover-100x53.png",100,53,true],"1536x1536":["https:\/\/yyc.ninja\/wp-content\/uploads\/2026\/10\/cg-from-vms-and-docker-compose-to-rke2-a-migration-playbook-cover.png",1200,630,false],"2048x2048":["https:\/\/yyc.ninja\/wp-content\/uploads\/2026\/10\/cg-from-vms-and-docker-compose-to-rke2-a-migration-playbook-cover.png",1200,630,false]},"post_excerpt_stackable_v2":"<p>the phase method is the project&#8217;s planning docs \u00a718.<\/p>\n","category_list_v2":"<a href=\"https:\/\/yyc.ninja\/index.php\/category\/how-to\/\" rel=\"category tag\">How To Ninja<\/a>","author_info_v2":{"name":"YYC Ninja","url":"https:\/\/yyc.ninja\/index.php\/author\/admin\/"},"comments_num_v2":"0 comments","_links":{"self":[{"href":"https:\/\/yyc.ninja\/index.php\/wp-json\/wp\/v2\/posts\/3523","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/yyc.ninja\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/yyc.ninja\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/yyc.ninja\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/yyc.ninja\/index.php\/wp-json\/wp\/v2\/comments?post=3523"}],"version-history":[{"count":1,"href":"https:\/\/yyc.ninja\/index.php\/wp-json\/wp\/v2\/posts\/3523\/revisions"}],"predecessor-version":[{"id":3540,"href":"https:\/\/yyc.ninja\/index.php\/wp-json\/wp\/v2\/posts\/3523\/revisions\/3540"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/yyc.ninja\/index.php\/wp-json\/wp\/v2\/media\/3522"}],"wp:attachment":[{"href":"https:\/\/yyc.ninja\/index.php\/wp-json\/wp\/v2\/media?parent=3523"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/yyc.ninja\/index.php\/wp-json\/wp\/v2\/categories?post=3523"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/yyc.ninja\/index.php\/wp-json\/wp\/v2\/tags?post=3523"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}