nixnet

How it works

Describe the transports

List every way to reach a peer, or to carry an uplink — LAN, overlay, wired, wireless, cellular — each with a priority and a health check, in one declarative schema.

nixnetd probes, continuously

Every transport gets its own thread and its own ticker, so one slow or hung probe never delays another. A hysteresis state machine tells healthy apart from flapping.

The winner gets published

A peer's address lands in a managed /etc/hosts block; an uplink's interface gets the best route metric. Every tool that already worked keeps working.

Features

One schema, two shapes

peers.<name> and uplinks.<name> share one transport type and one engine. Only the publish backend differs — an /etc/hosts entry for peers, a kernel route metric for uplinks.

Provider-agnostic by contract

A provider contributes ordinary Nix list entries, optionally backed by an exec script — an exit code plus one JSON line on stdout. No plugin loader, and no core changes for a new mesh.

Hysteresis-damped

A minimum hold time stops nixnet switching away from a healthy winner just because a lower-priority option also came up — but a dead winner is never held onto.

No resolver, no listener

Peers publish into /etc/hosts, read by NSS files ahead of dns. Zero query-time protocol, and nixnet never listens on a network port for peer publishing.

Reprioritizes, never replaces

Uplink failover flips a route's metric field only — the gateway and every other attribute DHCP or a static config assigned is left exactly as it was.

Self-watchdogged

nixnetd calls sd_notify(WATCHDOG=1) on its own heartbeat, so systemd force-restarts it if the event loop ever wedges. No bespoke supervision code.

Unprivileged by default

Runs as a dedicated system user under ProtectSystem=strict, never root. A peers-only install using TCP or HTTP probes gets no elevated capabilities at all; CAP_NET_RAW and CAP_NET_ADMIN are granted only when a transport actually needs them.

Nix-unaware daemon

nixnetd only ever reads /etc/nixnet/config.json. The same binary works unmodified from a Nix-rendered config, a hand-written one, or a system-manager render.

Ships real providers

netbird-provider contributes the overlay address and detects local identity going stale or enrolled against the wrong endpoint, re-enrolling headlessly from a setup key. cloudflared-provider restarts a tunnel whose edge connection has wedged without the process dying.

Architecture

nixnet.peers / uplinks
            |
            v
 /etc/nixnet/config.json          (rendered once, at build time)
            |
            v
      +------------+
      |  nixnetd   |   one probe thread + ticker per transport
      +-----+------+
            |
      winner changes?
       /            \
      v              v
+-----------+  +---------------------------+
| /etc/hosts|  | ip route replace default  |
| (managed  |  | dev IFACE metric N        |
|  block)   |  +---------------------------+
+-----------+
      |
      v
ssh, mount, curl, browsers --
every ordinary NSS files-then-dns lookup

Providers, not plugins

A provider registers by contributing ordinary Nix list entries into nixnet.peers.<name>.transports. Core has no registry and no provider-specific code to fork.

One health contract

Built-in TCP, ICMP and HTTP probes cover the common case. A provider needing richer health, or a dynamically discovered address, drops in an exec probe: an exit code plus one optional JSON line.

Reconverges from disk

Hysteresis counters and last-known winners persist to /var/lib/nixnet/state.json, written with the same atomic write-then-rename discipline as every other output. A restarted daemon rebuilds its picture correctly — no special recovery path.

Status

v0.1 — running in production. The winner-selection and hysteresis engine, all four probe methods, both publish backends and the sd_notify watchdog integration are implemented for real, not stubbed; the test suite runs in-derivation on every nix build, and an eval-time check suite guards the module surface. It runs day to day across several hosts, on both the NixOS and the system-manager backend.

That deployment is where the sharpest bugs came from — the ones no amount of eval-time checking finds, because they are facts about a running kernel rather than about what Nix evaluates to: the daemon's process identity versus the hosts file it must rename(2) over, a secret unseal racing the mount holding its key, and a restart publishing nothing because publishing was wired only to winner changes and a settled fleet has none. Each fix landed with a regression check beside it.

Calibrate before adopting: default probe intervals and hold times are reasoned, not measured — the repo's experiments/ notes are the open ledger of which values still need real numbers. The peer//etc/hosts half is considerably better exercised than the uplink/route-metric half. And docs/providers.md lists every place the code fills a gap the option surface didn't pin down, rather than leaving it implicit.

Install

NixOS (flake)

Import the core module and turn it on. Providers are opt-in additional imports.

{
  inputs.nixnet.url =
    "github:julian-corbet/nixnet-corbet-ch";
}

# host configuration.nix
imports = [ inputs.nixnet.nixosModules.default ];
nixnet.enable = true;

Non-NixOS (system-manager)

Same module file, same schema. nixnet only ever touches environment.etc, systemd units and a rendered JSON config — none of the primitives system-manager categorically can't reach.

imports = [
  inputs.nixnet.systemManagerModules.core
];
nixnet.enable = true;

From source

The daemon is a single Rust crate with two binaries; the modules are plain Nix. Cargo.lock is committed, so both build paths work fully offline.

git clone https://github.com/julian-corbet/nixnet-corbet-ch
cd nixnet-corbet-ch
nix build .#nixnet   # or: cargo build --release

Commands

`nixnetctl`	Every peer and uplink group: current winner, per-transport health, and how long the winner has held.
`nixnetctl -json`	The same snapshot as raw JSON, for a script or a status page.
`systemctl status nixnetd`	Whether the daemon itself is up, and since when.
`journalctl -u nixnetd -f`	Follow failover events live — one structured line per state transition.
`systemctl restart nixnetd`	Safe at any time: state persists to disk and every publish is a full atomic recompute, so a restart reconverges cleanly.