M6: immediate selection and validated scalar stack caching

Archived: a record of finished work, kept as written. nanofil's archive lists the others.

Retired (October 2026). --registers=local, its cache (amd64_cache.pl) and its checker (amd64_cache_check.pl) were removed after the code-quality campaign (perf.md, register modes). Whole-function register homes with register operands and frame cells (--registers=temps, the -O default) do everything the cache did, on all eleven gate workloads, with fewer instructions and in less time (lz4 -1 within noise): geomean 3.32× filcc -O2 for -O --registers=local, 2.18× for -O. The cache's only trusted code of its own was its checker, so keeping it cost about a tenth of the Nix check and ten program builds for a mode nothing used. Immediate selection (below) is not retired: it is the allocating selection, sel(local, …) in filc_amd64.pl, that temps emits. A retired --registers=local is refused with an error. The diagnostic tests/bench-registers.pl with its kernel bench-registers.c, and the checker's attack suite tests/cache_check.pl went with it; they are at 7fc3fa2. The rest of this page is the record of the milestone as it was.

--registers=local selects integer immediates, then caches scalar stack homes. It is not general register allocation or completion of M6. --registers=stack remains the default. Both work independently of -O; build harnesses can set NANOFIL_REGISTERS=local instead. The plain SysV backend is unchanged.

Immediate selection precedes the cache's proof boundary

The opt-in amd64:compute_op//5 selects immediate right operands for w/l add, subtract, multiply, and/or/xor and the three shifts. It leaves the reference compute_op//4 unchanged. Constants normalize modulo the operand width; a qword ALU immediate must equal a sign-extended imm32. Thus 0xffffffff can be immediate at w but not at l, where it means positive 4294967295. All other cases retain register operands. Shifts mask the constant count to 31 or 63 and retain even a zero-count instruction; branches still test their operand explicitly, not the shift's flags.

Selection removes only the constant's scratch move to RCX. It retains the LHS load, result width, operation, defined flags and result store. Variable counts, division/remainder, comparisons, floats, pointer flight operations, helpers and checks are unchanged. The Fil-C function context carries the mode; setjmp forces reference selection before any emission.

The translation checker validates caching of the selected stream, not old selection against new selection. Selection is trusted emitter code, tested through emitted forms and native/interpreter/clang comparisons. An unsupported machine form or original cache-bank use disables caching of the whole implementation, retaining selected-but-uncached code. This differs from setjmp, which bypasses both selection and caching.

The machine contract bounds the transformation

amd64_contract.pl describes known instruction forms, including register families and widths, explicit/implicit reads and writes, flags, memory and control/frame effects. Division names its accumulator and high half, variable shifts read cl, and partial writes retain their widths. Any unknown original form or original use of the r10/r11 bank selects the uncached selected implementation. Calls may clobber both cache registers. Call argument reads are not enumerated: calls are unconditional boundaries, and ordinary registers remain equal. This is not a general call-liveness model for future allocation passes.

filc_amd64 supplies the original temporary classes and placed physical homes before that information disappears. Every alias of an eligible scalar home must be w/l. A home shared with s/d, an unknown class, pointer payload/lower, phi shadow, local payload/aux, or frame bookkeeping is not eligible. All eight bytes stay together: incoming w words can have nonzero upper bits. A cold 32-bit read fills all 64 bits; its register read still zero-extends into the original destination.

Only the implementation body is transformed. Getters, adapters, function objects and origins retain their original lines and independent layouts. Every setjmp function is unchanged, including one with only unreachable setjmp sites. The middle end already preserves those complete functions.

Regions preserve all existing boundaries

A region ends before every branch, call, label, directive, terminal, frame-base/stack-pointer change, and every memory operation other than an exact eligible frame load/store. This includes leave, before it restores the caller's frame. Labels begin empty; no predecessor relies on fallthrough cache contents. Unknown instructions are not merely barriers.

The conservative memory boundary covers partial/overlapping frame accesses, indirect addressing and local payloads without an alias analysis. Helpers inside pointer operations are ordinary call boundaries, regardless of whether they usually exit. Phi reads into shadows, phi writes, polling, rooted helper reloads and exception tests keep their original order.

amd64_cache.pl first removes loads whose complete result is already in the destination register. It remembers one physical-home association per register, invalidating it on register writes and invalidating every association with a home on a store. A dword reload is removable only when the destination's high half is already zero; a dword association alone never justifies removing a qword reload. Boundaries forget all facts, including zero-extension. Copies and partial writes remain opaque.

Then it groups a home's remaining first-to-last references in each region. A deterministic linear scan assigns profitable intervals to r10, then r11; an overlapping interval under pressure stays on stack. There is no eviction or next-use heuristic. Every store updates the resident home version. A first read fills the register, and the last reference writes back a dirty home. All inserted and replacement instructions are flag-preserving moves; no arithmetic instruction moves or changes.

This reduces memory transfers, not necessarily instructions. Two stack moves can become two register moves plus a final spill. The pass retains the frame allocation and every original scalar home.

The checker validates the final instruction list

amd64_cache_check:check_cache/3 derives eligible homes independently and pairs the selected uncached and final cached streams. Only the three permitted move forms may change; a load may also disappear if its complete result equals the current ordinary register. This compares with the reference home, not possibly stale candidate memory. Every other line must be identical and ordered. It symbolically tracks ordinary register families, full-width cache values, and separate reference/actual home contents. Low/high halves distinguish a 32-bit value from its full incoming word. Unchanged arithmetic gives both streams the same fresh result, rather than reimplementing x86.

All homes must agree before every boundary. The checker then forgets cache contents and uses fresh common symbolic entry values. Sparse maps avoid rescanning the whole frame at every boundary. Inserted frame accesses require the original prologue and cannot follow leave. Every fill must immediately replace the next original load of the same home: establishing rbp alone does not permit speculative reads before the stack-overflow check. A failed check is a compiler error, not silent fallback. The checker shares machine facts, not interval choices or claimed eligibility; it derives eligibility from the emitter's trusted class/placement inventory. An omitted load is consumed before trying a later fill's immediate anchor; the fill itself never moves earlier. Matching is deterministic and still checks raw input against final output, not only the intermediate cleanup.

Validation and limits

The native differential, interop, GC, helper-exit, exception and setjmp suites run with both register modes and both middle-end modes. GNU Hello, zlib and Lua also build and run in all four combinations. The unchanged stack mode has 1,030 byte-identical snapshot outputs against the preceding correctness-fix commit.

Hand-authored checker cases reject omitted writebacks, stale values across calls/labels, stale versions after ordinary stores, truncation of high bits, changes to flags/failure registers/exception branches, float-shared homes, unknown forms, and original cache-bank use. In-memory checker mutations removing zero-extension or boundary synchronization are caught. The speculative pre-overflow-check fill was accepted before the anchored- fill rule and is now rejected. Backend tests establish actual integer/float slot sharing and reject a mutation that misclassifies the float aliases. Native mutations also exercise the previously scratch-only pointer-helper and exception regressions; see pizlonation.md.

The trust boundary remains the original emitter, complete class/placement metadata, the audited machine forms, the checker and the assembler. This does not prove the original compiler or Fil-C runtime correct. No pointer allocation, reduced rooting, check/poll removal, cross-region allocation, setjmp allocation or C++ unwinding is introduced.

Static savings cost instructions and compilation time

The initial cache, before redundant-load removal, was measured on 2026-10-25 in a Linux x86-64 orb (Xeon at 2.60 GHz), SWI 9.2.9 and Fil-C staging 0.687. Each CPU/RSS value is the median of three runs, alternating stack/local order. CPU seconds include frontend, validation, middle end and backend, but not module loading or printing. RSS is the whole process. This is a shared VM, not a dedicated benchmark machine. The checker time is included in local's total, not added to it.

All arrows below are stack → local. Frame counts are static operands addressed through rbp, not measured dynamic traffic. Instructions and ELF object text include unchanged adapters. Generated inputs use tests/bigfn.py 1 300 and tests/bigfn.py 2 1000; 00200 is the pinned c-testsuite input used by the corpus gate.

Controls: tests/c/basic.c removes 66/46 frame accesses (off/-O), float.c removes 13/23, and setjmp.c excludes all five setjmp functions in both modes while changing only its ordinary helper functions. There were no unsupported-form fallbacks in these six measurement inputs.

Reproduce each row with preprocessed C (no preprocessing needed for bigfn.py output), then assemble and measure the emitted object:

cd nanofil/src
SWIPL=/tmp/swipl/bin/swipl /usr/bin/time -f '%M KiB' \
  ../bin/nanofil --script ../tests/bench-registers.pl \
  /tmp/input.i o1 local /tmp/output.s </dev/null
as -o /tmp/output.o /tmp/output.s
size /tmp/output.o

Thus the optimized 00200 removes 29.6% of frame accesses but adds 5.7% instructions and 3.2% text. Compilation rises 34.3%. These results justify keeping the cache opt-in; memory-access counts alone do not justify enabling it by default.

Native Lua 5.4.7 measurements do not establish a speedup either. Using the four binaries from the passing nanofil-lua gate, five alternating runs per mode gave these median user CPU seconds, with all outputs checked:

The stack/local ranges overlap in every pair; optimized integer runs span 0.560–0.614 s versus 0.570–0.617 s. These are VM measurements of two small workloads, not evidence of a general runtime improvement. Programs passed as lua -e (no GC-stress environment for timing):

local s=0
for i=1,2000000 do s=(s+i*17)%1000000007 end
print(s)

local t={}
for i=1,1000 do t[i]=i*3 end
local s=0
for r=1,2000 do for i=1,1000 do s=s+t[i] end end
print(s)

Redundant-load removal reduces instructions without widening allocation

This follow-up uses the same toolchain and orb. Its baseline is 76de0216, not the original stack backend. It follows Oracle's smaller proposal: remove proven reloads before interval planning, with no copy propagation or arithmetic rewrite. Instruction reduction is measured on the final stream: replanning intervals can add other fills and writebacks.

Upstream informed the boundary, not an allocation algorithm. At the pinned Fil-C source, Pizlonator leaves ordinary integer operations alone (FilPizlonator.cpp); LLVM runs ordinary cleanup around instrumentation (BackendUtil.cpp). Nanofil likewise keeps safety lowering separate from scalar cleanup. It does not adopt upstream KillUB's division semantics: original native arithmetic and faults remain unchanged.

Thirty GDB samples per Lua workload put 59 of 60 samples in luaV_execute. Its assembly includes result/store/reload sequences, but this coarse sampling does not attribute a percentage of time to them. Thirty zlib text-compression samples landed in longest_match (23) and deflate_slow (7); longest_match also contains the same store/reload pattern, among many pointer operations that remain untouched. lvm.c below is Lua 5.4.7 with the gate's switch-dispatch flags. The kernel is bench-registers.c, two loop-carried unsigned 32-/64-bit chains with runtime count and seed, not a dead-code benchmark.

Arrows are old local → new local, not stack → local. Compiler numbers are medians of three alternating runs of bench-registers.pl; assembly was assembled and measured with size. Frame accesses count operands, not hardware events. Small CPU differences on this VM are not definitive.

The optimized 00200 now has 10.6% fewer instructions than old local, and 5.5% fewer than stack's 23,616, without sacrificing its frame-access savings. Lua's instruction reduction is only 1.0% in its VM translation unit. Stack remains much cheaper to compile: optimized lvm.c takes 2.547 s total and 0.625 s in the backend, versus new local's 4.049/2.019 s.

The new unit cases failed before implementation. Independent checker fixtures cover unknown upper bits, partial and implicit writes, physical home overwrites, stale actual memory, every existing boundary, multiple omissions before a fill, and trailing omissions with dirty writeback. In-memory producer mutations losing zero-extension, home invalidation or division's implicit writes are rejected as compiler errors. Checker mutations losing zero-extension or restoring the premature fill cut are caught by negative and positive fixtures respectively. Fresh-process unit runs pass after these mutations. Stack still has 1,030 byte-identical snapshot outputs; both stack Lua binaries are byte-identical too.

Native benefit is clear in the arithmetic kernel, not generally

Seven alternating trials per executable, after warm-up, pinned to CPU 4. Timings are wall time; timed processes ran serially without GC-stress switches. Trials wait for a quiet VM and discard concurrent external CPU work. Counts were calibrated on the baseline to roughly 0.6 s and then held fixed for each old/new pair. All kernel outputs were checked; zlib binaries and inputs were fingerprinted after correctness validation. This is still a shared VM, not a controlled physical-machine benchmark.

These medians are again old local → new local. Kernel units are nanoseconds per iteration, zlib units milliseconds per uncompressed MiB, and Lua units seconds for the two programs above.

The optimized kernel's 11.5% reduction has nonoverlapping trial ranges: 9.706–10.189 versus 8.550–9.084 ns/iteration. It is the clearest native benefit. The unchanged stack control's .text is byte-identical, yet its median moved from 10.831 to 10.380 ns/iteration (4.2%); small differences must not be overinterpreted. The kernel was validated on 30 count/seed pairs against Fil-C clang and an independent masked-integer Python model, in all modes normally and with GC scribbling/verification.

Zlib's optimized medians improve 0.5–4.6%, but every old/new range overlaps. Unoptimized random compression gets slower. Lua table reads improve 4.7% by the median (ranges 0.471–0.522 versus 0.441–0.486 s), while the integer loop does not improve. These runs do not establish a general application speedup. Keep --registers=local opt-in; instruction selection and wider value retention need their own measured justification.

Zlib is the gate's 1.3.2, all 17 objects compiled through nanofil. Both labels and both register modes were timed with and without -O; each also passed upstream tests and byte-identical compression/cross-inflation against Fil-C clang normally and under GC stress. Timing includes process startup and I/O. Each input is exactly 1 MiB: text concatenates sorted nanofil/src/*.pl, then sorted nanofil/docs/*.md at the baseline commit, repeats and truncates to size; random concatenates SHA256 digests of b"nanofil-integer-bench-v1:" + i.to_bytes(8,"little"), starting at zero. Their SHA256s are f4bf31d7cd2abe0044a5f375b72e6f630d808616f7e49fa7c3ddcf25c7278b60 and c60e5804f983537ce7e87bc82442f4118119d0e90066b3a3ea085ed7e2dc48eb. Compression runs minigzip -1|-6|-9; decompression runs minigzip -d on the level-6 reference output. Repeat whole processes to reach the calibrated duration, dividing elapsed time by repetitions.

For the kernel, compile bench-registers.c in the desired modes, then run with seed 123456789. The optimized local pair used 42,809,000 iterations per trial. The usual driver can reproduce the build (from nanofil/src, with SWIPL and FILCC set):

../bin/nanofil -O --registers=local cc ../tests/bench-registers.c \
  </dev/null > /tmp/kernel.s
as -o /tmp/kernel.o /tmp/kernel.s
"$FILCC" -o /tmp/kernel /tmp/kernel.o
taskset -c 4 /tmp/kernel 42809000 123456789

Immediate selection: fewer instructions with the same frame traffic

This experiment's baseline is a3c1269, including redundant-load removal, not the original stack backend. Oracle reviewed the immediate selector's width rules, scratch-register contract and proof boundary before adoption. Its two-operand immediate imul form was assembled and disassembled: GNU as encodes the accumulator as both source and destination, without an RDX result.

The previously sampled hot functions contain actual adjacent constant-to- RCX moves followed by arithmetic or shifts through CL. In luaV_execute there are 1,421 encodable arithmetic pairs and 707 shifts; longest_match has nine and one; the kernel loop and increment have six and ten. The candidate converts all of them, leaving unencodable qword constants alone. These counts were checked independently in the assembled objects. They describe entire function implementations, not dynamic execution counts.

The reduction is 4.5% of Lua's VM instructions but only 0.5% of the match function. Lua still contains 240 constant shifts by zero and constant-mask expressions; removing/folding those is outside this selection rule. The fixed Fil-C clang reference object instead extracts an opcode with and $0x7f,%eax. It has 18,949 VM instructions, versus Nanofil's 45,326; that remaining gap includes many transformations, not just allocation. Separate clang builds were not instruction-identical, for an uninvestigated reason, so comparisons use one fixed baseline reference executable/object.

Compiler measurements below use the existing bench-registers.pl, three alternating runs per input, the same preprocessed input for both compilers, and the same SWI host. Counts here cover the whole translation unit, including adapters and other functions, unlike the function table above.

The new emission cases failed on the old selector. Native regressions compare arbitrary-high-bit arithmetic with the interpreter and clang; negative and width-sized shift counts use IR, not undefined C shifts. In-memory mutations that truncate every qword constant to imm32 or mask qword counts with 31 fail both emission and native checks. A fresh process then passes all 77 unit tests. The cache checker is unchanged and does not catch those selector mutations: they change its trusted input too.

All four Nix gates pass, including both register modes and middle-end settings, GC-stressed pointer helpers and exceptions, and the Lua/zlib suites. Each actual-helper audit sees 20 ensure-aux exits and 20 store barriers. The default backend has 1,032 byte-identical snapshot outputs against the baseline. Across the benchmark builds, all 108 stack-mode objects have identical disassemblies and relocations at both optimization levels, providing unchanged-code timing controls.

Native timings used seven alternating serial rounds on CPU 4 of the same shared orb, after builds and compiler measurements finished. Every old/new pair used identical calibrated counts and frozen inputs. The runner checked kernel/Lua outputs, input and binary hashes, and the unchanged-stack object controls; zlib's bytes were checked before timing. Two trials were discarded and repeated for competing CPU load, leaving 567 accepted samples plus 81 warmups. These are elapsed times including startup and I/O, not hardware instruction counters. Brackets below are observed min–max, not confidence intervals. Arrows compare old local with immediate-selecting local.

The kernel's ranges do not overlap, while its unchanged optimized stack control moves only 10.2638 → 10.2752 ns/iteration. Without -O, local's kernel also improves: 43.9164 [42.8546, 49.0693] → 41.1927 [40.4790, 42.0015] ns/iteration, or 6.2% less time. This establishes a native benefit for the arithmetic-chain workload, not for arbitrary C programs.

Lua's ranges overlap in both workloads. Its unchanged optimized stack controls move 0.3055 → 0.3168 and 0.2416 → 0.2396 s/million iterations, showing the noise floor. Without -O, local medians improve 4.8% and 2.6%, again with overlapping ranges. These are promising observations, not an established general Lua speedup. Every zlib pair overlaps too, including without -O, where local median changes range from -1.5% to +1.2%. The byte-identical optimized stack controls themselves move by -3.2% to +1.0% across zlib workloads. There is no clear zlib benefit or regression. Local mode remains opt-in; fewer static instructions alone do not warrant changing the default.

To reproduce the strongest pair, build the unchanged integer kernel at the baseline above and at this revision using the earlier build commands, then alternate seven timed runs of each executable with 54882276 123456789, pinned to CPU 4. Expected stdout is 54882276 4bc602bc fa7a402963506b80; divide elapsed seconds by 54,882,276. For Lua, use the earlier scripts with 1,820,540 integer iterations and 2,280 outer table iterations (expected results 125355986 and 3423420000), building all 34 Lua 5.4.7 objects with the gate's switch-dispatch flags. Optimized local zlib repetitions are 6/3/2 for text compression levels 1/6/9, 4 for random compression, and 19/34 for text/random inflation. The text recipe is the same as above but uses this experiment's baseline; its SHA256 is ad941c3c7d5e28031b4a48de7a0c00ccb13185ef6c7b947e7d73bacdf5d1b3c0. The random input and its hash are unchanged. All compared builds use the same inputs and one fixed Fil-C clang reference, with no GC-stress flags during timing; correctness runs separately cover GC stress.