Fil-C on Nix

Filnix wiki

mbrock/filnix

tests/nixos/default.nix

NixOS VM tests of Fil-C systems: Nixpkgs' own tests of each service, run with nixosModules.default on every node, plus tests of our own.

nix build .#nixosTests.lighttpd           # one test
nix build .#nixosTests.lighttpd.driver    # just the driver, to run by hand

Every node's pkgs is the Fil-C package set, so the services, their clients, systemd and PAM are Fil-C builds. After a test's own script passes, filc-ps runs on every machine still up and fails the test if a process is native without being an allowed exception; its table ends the log.

The tests want a builder with KVM, and about 2 GiB of memory per node.

λpkgs, pkgsFilc, filcModule, filnix
let
21inherit pkgs
lib
22upstream=
"${pkgs.path}/nixos/tests"

Keyed, so the module is imported once however many times it is listed.

25filc=
26key=
"filnix-tests-filc-module"
27imports=
filcModule
used by filcDefaults native
30filcDefaults=
31imports=
filc
32filc.enable=
true

Fil-C programs need more memory than the default 1 GiB.

34virtualisation.memorySize=
lib.mkOverride900 2048

The test driver talks to the VM over its backdoor, not the guest agent, and the agent comes from a QEMU build Fil-C cannot do yet.

37services.qemuGuest.enable=
lib.mkForcefalse
uses filc lib used by withFilc
40proof=
with subtest("Fil-C: every process is Fil-C or an allowed exception"):
for m in machines:
if not m.is_up():
continue
m.succeed("grep -q libpizlo /proc/1/maps")
print(m.succeed("filc-ps"), flush=True)
used by withFilc

A Fil-C php-fpm worker grows to about 150-250 MB, and these apps run several (Nextcloud starts 12).

52phpApp=
λmemorySize test
let
55memory.defaults.virtualisation=
inherit memorySize;
in
test ? extend
→
test.extendmodules = memory;
else
imports = test memory;
67mediawikiPhp=
pkgs, …: services.mediawiki.phpPackage = pkgs.php84.buildEnvextensions = all, enabled: enabled ++ all.apcu;;
used by t'

Tests whose nodes use Rust or Go programs, or nginx, keep those native. nginx's processes share pointers through shared memory, which Fil-C capabilities do not survive; on a Fil-C system the web server is lighttpd.

78native=
λf test
let
81m.defaults=
82imports=
filc
83filc.native=
f
in
test ? extend
→
test.extendmodules = m;
else
imports = test m;
uses filc

Run test (a test module, or a test Nixpkgs already built with runTest) with the Fil-C module on every node, then check its processes.

98withFilc=
λtest
let
101base=
if test ? extend then test else pkgs.testers.runNixOSTesttest
used by script
102script=
base.config.testScript
uses base
in
base.extend
105modules=

The module chooses the package set.

108node.pkgsReadOnly=
false
110name=
lib.mkForce"filc-${base.config.name}"
111testScript=
lib.mkForcenodes, …@args: if lib.isFunctionscript then scriptargs else script + proof
uses filcDefaults lib proof
in
lib.mapAttrs_: withFilc
120lighttpd=
"${upstream}/lighttpd.nix"

Its hidden service is served by Caddy, which is Go, and its client side runs arti, the Rust Tor.

123tor=
nativen: inherit n caddy arti; "${upstream}/tor.nix"
124bind=
"${upstream}/bind.nix"
125openssh=
126imports=
"${upstream}/openssh.nix"

Its OpenSSH without OpenSSL can't use the Fil-C PAM: pam_unix's crypt_r from libxcrypt doesn't resolve in that sshd (an open issue). The node runs the regular Fil-C OpenSSH instead.

130nodes.server-no-openssl=
pkgs, lib, …: services.openssh.package = lib.mkForcepkgs.openssh;

Its DNS client is knot-dns's kdig; the Fil-C knot-dns stops in its own test suite (open, docs/nixos-userland.md).

138unbound=
nativen: inherit n knot-dns;
139imports=
"${upstream}/unbound.nix"

Without the Python plugin, which the test doesn't use: its configure runs the host's python, which a cross build can't.

142defaults=
pkgs, …: services.unbound.package = lib.mkForcepkgs.unbound-full.overridewithPythonModule = false;;
150mosquitto=
"${upstream}/mosquitto.nix"
153demo-wiki=
import./demo-wiki.nix inherit filnix;
154first-switch=
import./first-switch.nix inherit filnix;

Nixpkgs runs this with PHP and APCu behind nginx, here native.

158php-fpm=
nativen: inherit n nginxStable;
159imports=
"${upstream}/php/fpm.nix"
160_module.args.php=
pkgsFilc.php.buildEnvextensions = enabled, all: enabled ++ all.apcu;

Both also run behind Caddy, which is Go, and nginx; WordPress also behind Fil-C Apache. Each talks to Fil-C PHP-FPM.

167wordpress=
nativen: inherit n caddy nginxStable mariadb; phpApp2048 "${upstream}/wordpress.nix"
174dokuwiki=
nativen: inherit n caddy nginxStable; phpApp2048 "${upstream}/dokuwiki.nix"

mediawiki-mysql, -postgresql, -sqlite, -nohttpd and -nginx, on Fil-C PHP 8.4: the module's default is 8.3, which is not ported. Apache is Fil-C; nginx, in front of PHP-FPM in one of them, is native.

//
lib.mapAttrs'
λn' t
lib.nameValuePair"mediawiki-${n'}"
withFilc
let
188t'=
189imports=
t
uses mediawikiPhp
in
nativen: inherit n mariadb; // lib.optionalAttrsn' ≡ "nginx" inherit n nginxStable; // lib.optionalAttrsn' ≡ "postgresql" inherit n postgresql_17; t'

MariaDB, the default database, is not ported yet; nginx can't be.

import"${upstream}/mediawiki.nix"
207inherit
lib
208runTest=
t: t

Nextcloud's own tests: nextcloud-basic, with Redis and PostgreSQL, and with MySQL and memcached. The Redis one is on 34 because Nixpkgs' notify_push app supports Nextcloud only up to 34, so its 35 test fails on any PHP; its notify_push daemon is Rust, so it stays native, and so does the nginx in front of Nextcloud.

//
lib.mapAttrs'
λtest t
lib.nameValuePair"nextcloud-${lib.removeSuffix"34" lib.removeSuffix"35" test}"
withFilc
native
λn
227inherit n
nginxStable nextcloud-notify_push

The test's WebDAV client: the Fil-C mount.davfs daemon stops after it forks (open, docs/nixos-userland.md).

230inherit n
davfs2

MariaDB is not ported yet.

//
lib.optionalAttrslib.hasInfix"mysql" test inherit n mariadb;

This one also runs notify_push's (Rust) test client.

//
lib.optionalAttrslib.hasInfix"postgresql" test
236inherit n
postgresql_17
237notify_push-test_client=
n.nextcloud-notify_push.passthru.test_client
phpApp4096 t
lib.getAttrs"basic35" "with-postgresql-and-redis34" "with-mysql-and-memcached35"
import"${upstream}/nextcloud"
248inherit
pkgs
249inherit pkgs.stdenv.hostPlatform
system
//

A whole Fil-C NixOS boots and logs a user in; see userland.nix.

255userland=
import./userland.nix inherit pkgs filcModule;