Homelab

A self-hosted, production-oriented infrastructure environment running on Debian 13 with rootless Docker, Tailscale networking, Pi-hole DNS, centralized observability and automated infrastructure and dependency management, hosting 20+ services across staging and production environments.

Each major component is self-contained and includes its own README.md with deployment and configuration details.

This repository serves as a documentation mirror, while the implementation is maintained in a private repository.

Contents

• Purpose
• Design principles
• Technology stack
• Architecture
• Observability
• Infrastructure provisioning
• Repository structure
• Development
• Future work

Purpose

This homelab serves as both functional infrastructure and a personal services hub. It provides practical applications, automation, and quality-of-life enhancements, while also serving as a platform for designing, deploying, and operating production-oriented infrastructure.

Design principles

• Private by default: Services are reachable only through Tailscale, with no port forwarding.
• Reproducible environments: VMs are built from versioned images and configuration, with dependency and tool versions pinned where practical.
• Environment isolation: Staging runs in its own VM so changes can be tested before production deployment.
• Least privilege: application containers run under rootless Docker wherever practical, reducing the privileges exposed to workloads.
• Automated: Provisioning, configuration, dependency updates, checks, and deployments are automated rather than performed manually.
• Single validation pipeline: Local development and CI use the same checks and tooling; the ci profile makes the local check set match CI exactly, while the default local profile provides a faster subset.
• Fast, targeted feedback: Formatters, linters, syntax checks, and file-specific CI jobs run as early as possible and only when relevant files change. Tooling and dependencies are cached to keep CI fast.
• Centralized observability: Host, container, and application metrics and logs feed shared dashboards and alerts.
• No plaintext development secrets: Development secrets come from fnox, backed by encrypted gopass store.
• Centralized access control: Selected internal services are protected by a reverse proxy integrated with Authelia, which provides SSO and centralized authentication.

Technology stack

Area	Technology
Host OS	Debian 13
Virtualization	QEMU/KVM + libvirt
Containers	Rootless Docker
VPN	Tailscale
DNS	Pi-hole + systemd-resolved
Reverse proxy	Caddy
Authentication	Authelia
Telemetry collection	Alloy
Metrics	Prometheus
Logs	Loki
Dashboards	Grafana
Alerting	Alertmanager + alertmanager-ntfy bridge + ntfy
Infrastructure as Code	Terraform + Packer + Ansible
CI/CD	Forgejo Actions + Forgejo Runner
Development secrets	fnox + gopass + GPG
Dependency updates	Renovate

Architecture

%%{init: {
  'layout': 'elk', # Layout 'elk' is not supported on GitHub.
  'theme': 'neutral',
  'themeVariables': {
    'fontSize': '14px'
  },
  'flowchart': {
    'htmlLabels': true,
    'padding': 15,
    'nodeSpacing': 35,
    'rankSpacing': 35
  }
}}%%
flowchart TB
    subgraph External["<b>External</b>"]
        TailscaleClients["Tailscale Clients"]
    end

    subgraph DNS["<b>DNS Layer</b>"]
        direction LR
        PiHoleDNS["Pi-hole<br/>127.0.0.1 · tailscale0 :53"]
        Resolved["systemd-resolved<br/>127.0.0.1:5353"]
        UpstreamDNS["Upstream DNS"]
        DnsmasqConfigs["dnsmasq.d configs<br/>homelab · MagicDNS"]
    end

    subgraph Ingress["<b>Reverse Proxy</b>"]
        Caddy["Caddy"]
    end

    subgraph Auth["<b>Authentication</b>"]
        Authelia["Authelia"]
    end

    subgraph Apps["<b>Application Layer</b>"]
        Forgejo
        Immich
        Syncthing["Syncthing<br/>Web UI"]
        Navidrome

        subgraph Notifications
            ntfy
        end
    end

    subgraph Observability["<b>Observability Stack</b>"]
        direction LR

        subgraph Collectors
        
            NodeExp["node_exporter"]
            CadvisorExp["cAdvisor"]
            BlackboxExp["Blackbox exporter"]
            DockerLogs["Docker container logs"]
            JournalLogs["journald logs"]
            FileLogs["/var/log/* files"]
            SmartctlExp["smartctl_exporter"]
            SystemdExp["systemd_exporter"]
            PiholeExp["pihole-exporter"]
        end

        Alloy["Alloy"]

        subgraph Storage["Storage"]
            direction TB
            Prometheus["Prometheus"]
            Loki["Loki"]
        end

        subgraph Consumers["Consumers"]
            direction TB
            Grafana
            Alertmanager
            AlertBridge["alertmanager-ntfy-bridge"]
        end
    end

    %% DNS flow
    TailscaleClients -->|"*.homelab.internal"| DNS
    PiHoleDNS --> Resolved
    Resolved -->|"per-link"| UpstreamDNS
    PiHoleDNS -->|"reads"| DnsmasqConfigs

    %% Ingress
    TailscaleClients -->|"*.homelab.internal"| Ingress
    Ingress -->|"forward_auth"| Auth
    Ingress -->|"exposes"| Apps
    Ingress -->|"exposes"| Storage

    %% Observability flow
    Collectors -->|"metrics; logs"| Alloy
    Alloy -->|"metrics"| Prometheus
    Alloy -->|"logs"| Loki
    PiHoleDNS -->|"metrics"| PiholeExp

    %% Cannot be rendered properly for 'layout': 'dagre'. Renders okay for layout 'elk' but it's feature incomplete (overall diagram looks awful).
    %% Prometheus -->|"datasource"| Grafana
    %% Loki -->|"datasource"| Grafana 
    Storage -->|"datasource"| Consumers
    Prometheus -->|"alerts"| Alertmanager
    Alertmanager -->|"webhook"| AlertBridge
    AlertBridge -->|"alerts"| Notifications

    %% Styling
    classDef dns fill:#1a3a5c,color:#fff,stroke:#0f2440
    classDef ingress fill:#b5a713,color:#fff,stroke:#402810
    classDef app fill:#558911,color:#fff,stroke:#281040
    classDef obs fill:#7a063c,color:#fff,stroke:#401020

    class PiHoleDNS,Resolved,UpstreamDNS,DnsmasqConfigs dns
    class Caddy ingress
    class ntfy,Forgejo,Immich,Syncthing,Navidrome app
    class Alloy,Prometheus,Loki,Grafana,Alertmanager,AlertBridge,NodeExp,CadvisorExp,BlackboxExp,DockerLogs,JournalLogs,FileLogs,SmartctlExp,SystemdExp,PiholeExp obs

Services are reachable through the private Tailscale network without exposing the host directly to the public internet or requiring port forwarding. Caddy acts as the main ingress point, Authelia handles centralized authentication and Pi-hole resolves internal DNS.

Alloy collects metrics and logs from the host, containers and additional Tailnet devices, forwarding them to centralized Prometheus and Loki instances. Grafana provides visualization, while Alertmanager routes alerts to ntfy.

The homelab uses separate staging and production VMs. Changes can be tested in staging before being deployed to production.

Observability

See observability/ for the observability architecture, configuration, dashboard screenshots, and alerting examples.

Infrastructure provisioning

The pipeline builds a reproducible Debian golden image and uses it to provision fully configured VMs. A complete run typically takes ~6 minutes.

The production VM is managed separately from the staging VM, allowing infrastructure and service changes to be tested before deployment.

See iaac/ for details.

Repository structure

Each top-level directory represents a self-contained component. The most significant components include:

.
├── auth/              # Authelia SSO (depends on Immich's PostgreSQL)
├── autoheal/          # Docker container health 
├── caddy/             # Reverse proxy (Ansible-managed)
├── forgejo/           # Self-hosted Git with CI/CD runner with ntfy integration for notifications
├── iaac/              # VM golden image pipeline (Packer + Ansible + Terraform)
├── immich/            # Photo management (server, ML, PostgreSQL, Valkey)
├── ntfy/              # Push notification server
├── observability/     # Prometheus, Loki, Grafana, Alertmanager, Alloy
├── pihole/            # DNS server and ad blocker (Ansible-managed)
├── scripts/           # Repo-wide helpers (mdsf-check, shellcheck-check)
└── .github/           # CI/CD workflows and ntfy-notify action

Development

Development, including toolchain installation, secrets management, Git hooks, repository checks, and CI/CD details, is documented in DEVELOPMENT.md.

Future work

• Add backup and restore strategy.