version: "1.0.1" name: server-admin description: Server administration knowledge for Evan's personal home server running Arch Linux. Knows all services, their configs, network layout, and operational procedures.

Server Admin Skill

You are running directly on this server as user `evan`. You can run commands directly on the server.

You have deep knowledge of Evan's personal home server. The server configuration is managed via Ansible in the ~/workspace/ansible-personal repository.

Server Overview

• OS: Arch Linux (bare metal)
• Hostname: server (single inventory host)
• Playbook: play-server.yml → roles/machines/server
• Config management: ansible-playbook play-server.yml (or with --tags <tag> for specific services)
• User: primary user is evan (uid 1000)

Network

• LAN interface: lan0 (internal, 10.0.0.0/24)
• WAN interface: wan0 (external)
• Domain: home.evanpurkhiser.com
• DNS server: dnsmasq on lan0, listening only on LAN
• DHCP range: 10.0.0.100–10.0.0.254
• Falls back to systemd-resolved → Cloudflare DNS (1.1.1.1, 1.0.0.1)
• Firewall: ufw — deny all by default, allow all on lan0, allow HTTPS (443) and BitTorrent (51413) on wan0
• Tailscale: enabled (tailscaled), provides VPN access

DHCP Reservations

Storage

• Primary storage: ZFS pool mounted at /mnt/documents
• ZFS replication: zrepl pushes the documents filesystem to offsite TCP receiver (Tailscale IP)
• Snapshots: hourly, prefixed zrepl_
• Retention: 2 weeks daily → 8 weeks weekly → 6 months monthly (receiver)
• NFS export: /mnt/documents exported to 10.0.0.0/24, 100.0.0.0/8 (Tailscale), and fe80::/64 — all with all_squash (maps to evan/users)
• Transmission download dirs: /mnt/documents/downloads/torrents-complete and torrents-incomplete

Nginx / Public Services

All HTTPS traffic proxied through Cloudflare (strict SSL with Cloudflare origin cert). Real IP recovered from CF-Connecting-IP header.

Default server block returns 444 (immediately close) for unknown hosts.

Podman Containers

All containers are managed via systemd units generated by Podman. Auto-update via podman-auto-update.timer (updates containers with io.containers.autoupdate: registry label).

Atuin (Shell History Sync)

• Pod: atuin — ports 7070:8888 (server) and 7071:8000 (abacus)
• Containers:
• atuin-postgres — docker.io/postgres:14, data at /var/lib/atuin/data
• atuin-server — ghcr.io/atuinsh/atuin:18.4.0, config at /var/lib/atuin/config, open registration disabled
• atuin-abacus — ghcr.io/evanpurkhiser/atuin-abacus:latest, stats/analytics API (port 8000), cache TTL 300s
• Systemd units: container-atuin-postgres, container-atuin-server, container-atuin-abacus, pod-atuin

Home Assistant

• Container: home-assistant — ghcr.io/home-assistant/home-assistant:latest
• Network: host networking
• Config: /var/lib/hass/ (configuration.yaml, automations.yaml, scripts.yaml, secrets.yaml, themes/)
• Devices: /dev/zwave (Z-Wave), /dev/zigbee (Zigbee) — GoControl USB adapter, udev rule 99-usb-gocontrol.rules
• Location: NYC, timezone America/New_York
• External URL: https://hass.evanpurkhiser.com
• Integrations: LunchMoney (unreviewed transaction count sensor), YouTube playlist sensors (Morning, Evening Chillout, Nighttime Sleep, Energized), Virtual Doorman REST unlock command, mobile_app, lovelace, history, logbook
• Mosquitto MQTT broker: port 1883, anonymous access, listening 0.0.0.0
• Restart: ansible-playbook play-server.yml --tags hass or sudo systemctl restart container-home-assistant

Waitress (File Share)

• Container: waitress — ghcr.io/evanpurkhiser/waitress:latest
• Port: 9005:80
• Volume: /mnt/documents:/mnt/documents
• Serves: /mnt/documents/public-share → public.evanpurkhiser.com
• Starts after: mnt-documents.mount

Meal Log

• Container: meal-log — ghcr.io/evanpurkhiser/meal-log:latest
• Port: 9006:8888
• Data: /var/lib/meal-log
• Config: /etc/meal-log.conf (OpenAI token, Cloudflare R2 credentials for image storage in meal-log-images bucket)

Instagram Saver

• Container: instagram-saver — ghcr.io/evanpurkhiser/instagram-saver:latest
• Port: 9007:8888
• Config: /etc/instagram-saver.conf (Google Places API key, OpenAI token)

Systemd Timer Services

All timers are monitored via Sentry Cron Monitors.

Transmission (BitTorrent)

• Service: transmission (runs as user evan)
• RPC: port 9091, whitelist 127.0.0.1, 10.0.0.*, 100.*.*.* (Tailscale), no auth required
• RPC URL path: /transmission/
• Peer port: 51413 (open on WAN firewall)
• Download dirs: /mnt/documents/downloads/torrents-complete (done), /mnt/documents/downloads/torrents-incomplete (in progress)
• Download queue: 5 simultaneous
• On torrent complete: triggers transmission-helper-trigger.sh → socket-activated transmission-helper@.service
• Transmission Helper: OpenAI + Telegram integration for torrent completion notifications; config at /etc/transmission-helper.conf; socket: transmission-helper.socket

Purkhiser Bot (Telegram)

• Script: /usr/local/bin/purkhiser-bot.sh
• Config: /etc/purkhiser-bot.conf — Telegram token + channel ID
• Used by: weekly-report, venmo-lunchmoney-ai, transmission-helper

SSH / Access

• SSH: password auth disabled, empty password disabled — key-only
• Tailscale: for remote access outside LAN
• SSH agent proxy: /run/ssh-agent-proxy.sock serves agent sessions and prefers the 1Password SSH agent on an online MacBook reachable over Tailscale
• Agent-witness fallback: when no MacBook is online, the proxy routes SSH authorization requests through /run/agent-witness/agent.sock, which wakes Evan's paired iPhone for passkey authorization
• Sudo: authenticated through pam-ssh-agent using the same proxy; the agent-sudo shim reports whether the request is routed through a MacBook or agent-witness and sends Evan the command and working directory via Telegram
• Authorization failures: Evan may need to approve either the 1Password prompt on a MacBook or the agent-witness request on his iPhone. Avoid parallel sudo requests because prompts can race or cancel each other.

Package Management

• Pacman sync: pacsync role — standard Arch repos
• AUR helper: aur role with aur-builder user — packages installed: zrepl, pam-ssh-agent
• Paccache: automatic cache cleanup
• Time sync: systemd-timesyncd

Common Operations

Apply full playbook

cd ~/workspace/ansible-personal
ansible-playbook play-server.yml

Apply specific service

ansible-playbook play-server.yml --tags <tag>
# Tags: zfs, packages, sshd, network, hardware, pam-ssh-agent, nfs, tailscale, nginx,
#       transmission, hass, waitress, meal-log, instagram-saver, venmo-auto-cashout,
#       venmo-lunchmoney-ai, atuin, podman-auto-update, zrepl, record-file-history,
#       purkhiser-bot, weekly-report, speedtest-monitor, auto-system-update, bambulab-lights-off

Check container status

sudo systemctl status container-home-assistant
sudo systemctl status container-atuin-server
sudo systemctl status container-atuin-postgres
sudo systemctl status container-atuin-abacus
sudo systemctl status container-waitress
sudo systemctl status container-meal-log
sudo systemctl status container-instagram-saver

Check timer status

systemctl list-timers
sudo systemctl status auto-system-update.timer
sudo systemctl status venmo-auto-cashout.timer
sudo systemctl status venmo-lunchmoney-ai.timer

ZFS / zrepl

sudo zfs list
sudo zrepl status
sudo systemctl status zrepl

NFS exports

sudo exportfs -v       # show current exports
sudo exportfs -ra      # reload exports

Nginx

sudo nginx -t                    # test config
sudo systemctl reload nginx      # reload without downtime
sudo systemctl status nginx

Transmission

sudo systemctl status transmission
# Web UI accessible at https://tx.prk.network on Tailscale

Logs

journalctl -u container-home-assistant -f
journalctl -u transmission -f
journalctl -u zrepl -f
journalctl -u dnsmasq -f
journalctl -u nginx -f

Config File Locations