M3: pointer class and Fil-C lowering

This is the first, deliberately naive Fil-C lowering, in src/filc_amd64.pl. Every rule is chosen to be easy to check, not fast. Optimizations come in M7.

cd nanofil/src
../bin/nanofil run ../tests/filc/basic.ir walk 9      # trap out_of_bounds
../bin/nanofil compile ../tests/filc/basic.ir > basic.s
as basic.s -o basic.o && filcc your.c basic.o             # link with Fil-C code

IR

A module that uses class p anywhere is a Fil-C module. It is validated, interpreted and compiled with Fil-C rules. Modules without p keep the plain meaning from M1 and M2 (once checked against real qbe) and compile with the plain backend.

In a Fil-C module:

p is a value class: a flight pointer, as in let(tmp(X), p, ...), parameters param(p, tmp(X)), a p return, call arguments arg(p, V).
let(tmp(Q), p, alloc(Align), [const(N)]) yields p, and let(tmp(Q), p, copy, [global(Sym)]) is the pointer to Sym.
let(tmp(Q), p, add, [tmp(P), X]) and sub (X of class l) move the address and keep the capability. No other arithmetic takes p.
let(tmp(I), l, ptoi, [tmp(P)]) gives the address. let(tmp(P), p, itop, [tmp(I)]) gives a pointer with no capability (Fil-C's inttoptr with no laundering analysis). An integer constant read at p is the same as itop.
Every load and store address is p. let(tmp(V), p, load(p), [A]) and store(p, V, A) move pointers (8 bytes, 8-aligned).
Calls may name functions not defined in the module. Their signature comes from the call site.

Interpreter semantics (the oracle)

A pointer is ptr(Address, Cap), where Cap is none or obj(Id). Objects never die (the naive lowering heap-allocates every alloc, and the GC keeps them while reachable). Memory has bytes plus an aux map from 8-aligned address to the Cap stored there by store(p, ...).

An access of S bytes at ptr(A, obj(I)) traps unless base(I) ≤ A and A + S ≤ base(I) + size(I), and for a write unless I is writable. With none it traps. These are outcomes trap(Reason), which native code must match by panicking.
load(p) and store(p, ...) also trap if A mod 8 ≠ 0.
store(p, ...) writes the address bytes and sets aux(A) = Cap. An integer store writes bytes and leaves aux alone, as Fil-C does, so a later load(p) there returns the new address with the old capability.
load(p) returns the address bytes with aux(A), or none when unset.
Fresh allocations are zero, because Fil-C allocations are.

Lowering

Frame. Each function pushes filc_frame {parent, origin, lowers[N]} in its stack frame and sets thread->top_frame (offset 16) to it. Every p temporary and every p phi shadow slot has a dedicated lowers[i] slot for its whole lifetime, and its address lives in an ordinary slot. All lowers are zeroed at entry. So at every safepoint, every capability the function can still use is in a precisely scanned frame slot, because values are never kept in registers across anything. Frame stores have no barrier: FUGC is grey-stack Dijkstra with a soft-handshake fixpoint, and clang's output stores frame lowers the same way. The frame is popped (top_frame = parent) on every return and on exception propagation. lowers[] is also what callee arguments rely on: callers root what they pass.

Origins. One filc_function_origin with can_catch = 1 for the frame and call sites, and one with can_catch = 0 for pollchecks and check failures. Both use num_lowers_ish = N in staging's 48-byte layout (num_setjmps counts static sites, num_stack_auxes = 0). See setjmp.md for the final per-site lowers. frame.origin is reset to the call-site origin before every call, because filc_pollcheck_slow overwrites it.

Entry. cmpq %rsp, (%rdi); jae filc_stack_overflow_failure after the frame is allocated, then the frame push.

Safepoints. A pollcheck (testb $14, 8(thread), then filc_pollcheck_slow(thread, origin)) on every edge that goes backwards in block layout order. Every cycle has such an edge, so every loop polls.

Access checks, for S bytes at flight pointer (P, L), with failure calling filc_optimized_access_check_fail(P, L, &check_origin):

L ≠ 0
P ≥ L
P ≤ upper − S, with upper = *(L − 16)
writes: *(u8 *)(L − 2) & 6 == 0 (READONLY | FREE)
load(p)/store(p, ...): P & 7 == 0

No separate not-freed check: free sets upper = lower, so check 3 fails for every S ≥ 1.

Pointer load after checks with S = 8: aux = *(L − 8) & (2^48 − 1); if 0 the loaded lower is 0, else w = *(aux + (P − L)), and if w & 1 the lower is *((w & ~1) + 8) (atomic box). The address is *P.

Pointer store of (V, VL) after write checks with S = 8: if aux is 0, aux = filc_object_ensure_aux_ptr_outline(thread, L − 16). If VL ≠ 0 and filc_current_marking_state ≠ 0, call filc_store_barrier_for_lower_slow(thread, VL). Then *(aux + (P − L)) = VL and *P = V.

alloc is filc_allocate(thread, N), or filc_allocate_with_alignment for 16, giving P = L = object + 16.

ptoi/itop. ptoi drops the lower; itop and integer constants used as p have lower 0, so any access through them fails check 1.

Calls use the fast convention: rdi = thread, rsi = 0, then each argument's words in rdx, rcx, r8, r9, then 8-byte stack slots; a p argument is two words (address, lower), w is zero-extended. The result is al = has_exception, rdx and rcx (a p result's lower). If al & 1, the caller pops its frame and returns al = 1. A call goes to pizlonatedFI<sig>_f; for a function outside the module, a signature mismatch is a link error (no fallback thunk yet).

Function symbols for f with signature sig: pizlonatedFIP<sig>_f (implementation), pizlonatedFI<sig>_f (alias, exported functions only), pizlonated_f (getter returning FO + 16 in rax and rdx), and pizlonatedFO_f: {FO+16, (FO+16) + 0x0083 << 48, fast, generic, sig} in .data.rel.ro. The generic entry is per signature and local to the module. It checks args_size, calling filc_cc_args_check_failure if it is too small, unpacks arguments from thread + 128 (data) and thread + 384 (aux), calls the fast entry through *callee_lower, and writes results back to those buffers.

Not yet supported, and rejected: more than 16 arguments. (Data definitions, symbol addresses and indirect calls arrived in M4c; see docs/m4.md.) unreachable is ud2, as in M2.

Testing

All of this runs in checks.x86_64-linux.nanofil.

Interpreter against native. difftest.sh sees a Fil-C module and compiles it with nanofil only, links a clang-built Fil-C harness, and runs each # run: line in its own process. A Fil-C safety panic is the native form of trap. tests/filc/basic.ir has 21 runs, 8 of them traps: out of bounds above and below, a read straddling the end, a forged itop pointer, a pointer constant, a pointer loaded from memory that never held one, a misaligned store(p, ...), and walking off an array through phis, a loop and calls. Each expected outcome was worked out by hand, and the panics name the right check and the nanofil function.
Interop with clang code. tests/filc/interop.c, 16 cases: pointers both ways, a capability stored by nanofil and read back by C, ten argument words (the fourth pointer split between %r9 and the stack), nanofil calling C calling nanofil, a call through a function pointer (getter and function object), a wrong-signature call rejected by the generic entry, a write to a string literal (readonly), use after free, and NULL.
GC. tests/filc/gc.ir builds and walks a 20,000-node list while C forces collections, requests concurrent ones and churns the heap. It runs plain and with FUGC_SCRIBBLE=1 FUGC_VERIFY=1. Three planted rooting bugs were each caught: not linking the frame into top_frame (crash in both modes), declaring zero lowers in the origin (trap, verify mode only), and dropping the store barrier (FUGC's verifier reports unmarked live objects, verify mode only). So the verify-mode run is the one that matters.

Fil-C's malloc rounds sizes up to 16 bytes, which the interop test's overruns account for.