Building a reliable home lab with Proxmox, Ceph and VPN
A home lab Proxmox cluster with Ceph storage and VPN gives systems administrators a practical platform for replicated block storage, migration testing and secure administration. It mirrors the virtualisation, networking and failure-handling patterns used in small production environments while remaining affordable enough to run on refurbished mini PCs or desktop hardware.
Australian homes add several constraints. NBN fibre often terminates near the living area while the server lives in a garage or study, so Ethernet, Wi-Fi backhaul and careful cable placement matter. Sydney and Melbourne terraces offer limited rack space, Brisbane summers raise cooling requirements, and Adelaide properties can have long distances between rooms and the equipment cupboard.
The local market has plenty of used Dell OptiPlex, Lenovo ThinkCentre and enterprise SSD deals, generally advertised with 10% GST included. Start with three hosts, a UPS and separate management, cluster and storage networks. That foundation is easier to troubleshoot than a larger lab assembled from mismatched equipment.
Plan hardware, VLANs and resilience
Give each node 32 GB RAM, a reliable SSD for Proxmox and at least two fast data devices for Ceph. One 10 GbE connection per host is valuable for replicated traffic; 1 GbE can run the lab but increases replica latency. Place a UPS on each node so shutdowns do not corrupt monitors or write-ahead logs.
Segment traffic into VLANs for management, Proxmox cluster traffic, Ceph public and private networks, and guest workloads. Use a dedicated NIC or switch trunk for each storage path where possible. Jumbo frames reduce overhead, but set MTU 9000 only across every switch, interface and storage path that supports it.
Plan failure domains before adding capacity. Three full nodes provide the clearest quorum, while a two-node Proxmox cluster needs a qdevice witness and still benefits from a separate test host. In Ceph, spread OSDs across hosts, use the host as the CRUSH failure domain and avoid depending on one large disk.
Assemble the Proxmox and Ceph cluster
Install the same Proxmox VE release on every node, configure consistent time, hostnames and IP addressing, then create the cluster from the first host. Join the others with their management addresses and confirm quorum, task logs and corosync membership. Static DHCP leases are convenient, but documented static addresses avoid surprises after a router replacement.
Add Ceph through the Proxmox interface and deploy three monitors for the three-node design, keeping them distributed across the nodes. Allocate OSDs from SSDs and separate the Ceph public network from the heavier replication path. Keep WAL and database writes on NVMe devices where possible because they are latency-sensitive and should not compete with normal OSD traffic.
Create a replicated RBD pool with three replicas and a minimum of two usable copies, then expose it to Proxmox as shared storage. Treat the replica count as a resilience setting rather than a backup. Monitor Ceph health, scrub errors, recovery activity and capacity trends before production workloads fill the pool.
Choose the right storage and network paths
Several storage models suit different workloads, but only replicated shared storage supports migration and failover between cluster nodes.
| Storage model | Suitable workload | Resilience | Operational caution |
|---|---|---|---|
| Local LVM-thin | Boot disks and disposable test VMs | Local to one node | Cannot provide shared failover |
| Ceph RBD | Clustered VM disks and HA workloads | Replicated across failure domains | Requires healthy monitors and adequate networking |
| CephFS | Shared files and application data | Replicated metadata and data | Requires a Ceph client; unsuitable for most VM images |
Use Ceph RBD for Proxmox virtual machines that may move between nodes. Keep templates and disposable test systems on local storage where appropriate, while CephFS handles shared files or configuration data. Any Proxmox HA resource also needs reliable fencing and tested STONITH behaviour; shared storage alone does not guarantee recovery.
Connect remote sites with an encrypted VPN
WireGuard provides a fast encrypted tunnel for connecting a remote workstation to the lab. Use split tunnelling to route only the management interface and designated lab subnet through the tunnel. Do not send Proxmox cluster or Ceph private traffic through an external VPN, and avoid overlapping the remote and home LAN address ranges.
An NBN FTTO line, and some FTTP plans, may provide a dynamic address or place the router behind carrier-grade NAT. That prevents inbound connections even when port forwarding is configured correctly. Use an outbound-maintained overlay such as Tailscale, or terminate WireGuard on a small VPS and have the home router establish a persistent tunnel to it.
A commercial VPN is an internet exit rather than a private site-to-site link. A field guide to VPN choices for Australians illustrates the checks worth making: logging policy, jurisdiction, breach history and account security. Restrict the Proxmox web interface and SSH to trusted source ranges, enable multi-factor authentication and never expose them directly to the public internet.
Test, maintain and extend the platform
Exercise failures deliberately. Shut down one node, disconnect its storage cable and interrupt a VPN route while monitoring quorum, Ceph health and service recovery. Keep Proxmox Backup Server or an equivalent restic repository outside the cluster, and test restoration rather than assuming snapshots are sufficient.
If the lab replaces an older VMware estate, IT Diversified’s VMware migration guidance provides a useful reference for moving workloads onto Proxmox. Check guest drivers, boot firmware and licence requirements after conversion, then measure storage latency under realistic virtual disk activity.
Build acceptance checks
- Confirm quorum and Ceph health after a host shutdown.
- Restore one VM to an isolated test network.
- Verify VPN routes reach only approved lab subnets.
Routine maintenance
- Review Ceph capacity, scrub results and SSD health monthly.
- Patch Proxmox, guest firmware and firewall rules quarterly.
- Rotate credentials and test backups twice yearly.
Regular failure drills and current documentation turn the platform into a dependable infrastructure testbed rather than a fragile collection of appliances.