The Lab

Infrastructure I build.
Systems I operate.

A working environment for services, software development, and infrastructure experiments. Linux hosts, a Proxmox cluster, network security, and recovery tooling all connect here.

Proxmox Nodes
9+
VMs & Containers
120+
Storage (TB)
100+

01 / Topology

See how the pieces connect.

An overview of the physical and virtual environment. Expand the diagram for a closer look, or visit Architecture for the detailed walkthrough.

Infrastructure overview

Expanded lab network and infrastructure overview

02 / Linux & Compute

Understand the host. Build repeatable systems.

Arch Linux is my daily development environment, with Debian and Ubuntu supporting server workloads. Proxmox brings together virtual machines, containers, storage, and GPU passthrough.

Day-to-day operations span systemd services, journald diagnostics, host firewalling, and storage management. Templates and configuration tooling help reproduce setups, while runbooks and repository documentation capture the decisions needed to maintain them.

Proxmox VE Cluster

Corosync quorum and HA across the cluster, with guest restart after host failure and firewall policies at the datacenter, node, and guest levels.

ZFS Storage

NVMe mirrors and RAIDz2 pools, compression, scheduled scrubs, and snapshots support storage operations and recovery.

Software-Defined Networking

VLAN-aware bridges and virtual network overlays separate workloads across the cluster, with host networking managed alongside the guests.

VFIO GPU Passthrough

Dedicated GPUs passed through to virtual machines, including the RTX 4090 used for inference and other compute workloads.

Cloud-Init & LXC

VM templates, cloud-init configuration, and LXC containers make provisioning repeatable across the lab.

GitOps & IaC

Terraform, Ansible, and version-controlled configuration support provisioning and changes to infrastructure and services.

See the 5090, 4090, and Spark cluster in the AI stack →

03 / Networking & Edge

Connect the workloads. Control the boundaries.

FortiGate handles edge policy and WAN connectivity, UniFi supports switching and wireless, and Tailscale provides remote access to internal services.

SD-WAN: Failover & Load Balancing

FortiGate manages fiber and cable connections with link health checks, failover, and traffic steering based on workload and connection quality.

UTM Inspection

FortiGate inspection, filtering, and threat feeds provide layered controls at the edge, including a CrowdSec blocklist feed.

Tailscale Mesh VPN

Tailscale provides remote access to internal services. A separate Headscale development setup supports control-plane experimentation.

HA & Automation Stitches

Edge high availability and automation stitches support failover, event-driven responses, scheduled configuration backups, and notifications.

04 / Observability & Detection

Follow the signals. Find the cause.

Metrics, logs, and security events support a shared view of the environment, from host health to edge detections.

Sources

Proxmox Cluster

node_exporter + pve-exporter

Hosts & Services

Wazuh agents · syslog-ng

Edge & Firewalls

FortiGate · CrowdSec parsers

↓

Collection & Detection

Prometheus + Loki

Metrics TSDB · log store

Wazuh Manager

SIEM/XDR correlation

CrowdSec LAPI

Decisions → bouncers

↓

Visualization

Grafana

Dashboards · alerts

Wazuh Dashboard

Security analytics

🔒 Reverse proxy (TLS) bound to Tailscale only, no public exposure

Prometheus + Exporters

Host and Proxmox exporters feed metrics into Prometheus. Version-controlled targets make monitoring part of infrastructure configuration.

Wazuh SIEM/XDR

Host agents collect security telemetry and file-integrity events for centralized investigation and correlation.

CrowdSec (Distributed)

Log-based detections feed blocking decisions to configured bouncers, with a mirrored blocklist available to the FortiGate edge.

Centralized Logging

syslog-ng collects infrastructure and firewall logs for Loki. Labels organize the streams, and LogQL supports investigation.

Tailscale-Only Access

Monitoring dashboards are reached over the tailnet, with a TLS reverse proxy providing access to the internal services.

Grafana Dashboards

Dashboards bring together metrics, logs, and security data, with Alertmanager handling notification routing.

05 / Backup & Recovery

Plan for failure. Keep a way back.

Recovery is layered across local snapshots, separate backups, and offsite copies. Each layer covers a different kind of failure.

  1. Layer 01 / Local

    Snapshots & rollback

    Recover from an unwanted system or configuration change.

  2. Layer 02 / Backup

    Separate restore points

    Restore workloads and data using the appropriate backup tool.

  3. Layer 03 / Offsite

    Copies beyond the site

    Retain a recovery option when local infrastructure is unavailable.

BTRFS + Snapper

Workstation snapshots provide a recovery point around system changes and package updates.

Restic

Encrypted, deduplicated backups protect configuration and application data on hosts outside the hypervisor backup workflow.

ZFS + Proxmox Backup Server

ZFS snapshots and replication support local recovery, while Proxmox Backup Server maintains guest backups.

Veeam Backup & Replication

Backup and restore tooling for virtual machines and physical workloads, alongside Microsoft 365 backup workflows.

Synology NAS (DSM)

Active Backup protects Microsoft 365 and Google Workspace data, with Hyper Backup providing an encrypted offsite copy to Wasabi.

Wasabi S3 (Offsite)

S3-compatible storage holds encrypted offsite backup copies, adding a recovery layer outside the local cluster.

Build it with operations in mind.

Infrastructure design, monitoring, and recovery are parts of the same job.