Fil-C on Nix

Filnix wiki

mbrock/filnix/nanofil

main.pl

The nanofil command line: the impure driver that reads a C or IR file, runs it through the compiler's stages and prints the result.

Commands

CommandDoes
check FILEread and validate; prints ok
print FILEread, validate, print the IR (ir_text.pl)
compile FILEread, validate, print x86-64 assembly
cc FILEcompile C (.c, or preprocessed .i) to Fil-C assembly
cc-ir FILEcompile C, print the IR (unchecked; the Fil-C backend's input is pizlonated)
tokens FILEprint C token terms, without locations
run FILE FN ARG...interpret FN on integer arguments
difftest FILE OUTDIRwrite OUTDIR/expected.txt, OUTDIR/harness.c and OUTDIR/mode for the file's runFn, Args facts

A FILE ending in .c or .i goes through the C frontend; any other FILE is IR text. A .c file is preprocessed with the predefined macros and include paths of $FILCC (default filcc).

┌──────┐  ┌─────────┐  ┌─────────┐
│ c_pp │─▶│ c_parse │─▶│ c_lower │──┐
└──────┘  └─────────┘  └─────────┘  │
  ┌─────────────────────────────────┘
  ▼
┌──────────┐  ┌───────────┐  ┌────────────┐
│ opt (-O) │─▶│ pizlonate │─▶│ filc_amd64 │
└──────────┘  └───────────┘  └────────────┘

cc runs the whole pipeline; c_pp lexes with c_lex. IR text enters at the middle end, and a module in plain mode goes to amd64.pl instead of pizlonate and filc_amd64.

-O

-O before the command runs the middle end (opt.pl) after loading. Without it the IR goes to the backend as loaded, which is the reference the optimized path is checked against. With -O, difftest also runs every run line on the unoptimized module and fails unless each optimized outcome refines the original one (translation validation), and prints how many allocs promote removed.

--registers=stack|local independently selects reference emission or validated block-local scalar caching in the Fil-C backend. The default is stack; NANOFIL_REGISTERS supplies the same choice to build harnesses. The plain backend is unchanged. Options precede the command.

difftest

expected.txt holds the interpreter's result for every run line whose outcome a native run can reproduce: a known return value, or for a Fil-C module a trap, written just trap. harness.c prints the same lines from native code; for a Fil-C module it makes one call per process, chosen by argv[1], since a trap ends the process. mode holds plain or filc. Other outcomes are reported on stdout and left out of the comparison.

For a Fil-C module, difftest also runs every run line on the pizlonated module and fails unless each outcome is the same, traps and their reasons included.

usesir

Compile-only commands do not need the interpreter or its CLP(Z) solver.

68:- ifcurrent_prolog_flagdialect, swi
69:- autoloadinterp, run/5, module_prepared/2, run_prepared/5, outcome_text/2
70:- else
usesinterp
72:- endif
usesamd64
usesasm
usesc_lex
usesc_pp
usesopt
useslists
usesformat
usespio
usesdcgs
usespairs
89:- initializationmain
91main ←
Trealla gives argv as character lists, SWI as atoms.
current_prolog_flagargv, RawArgv
maplistarg_atom, RawArgv, Argv0
halt/1 outside catch/3: SWI implements halt as an exception.
catch
optionsArgv0, o0, Default, Opt, Registers, Argv
commandOpt, Registers, Argv
→
Status = 0
else
formatuser_error, "nanofil: failed~n", []
Status = 1
,
Error,
reportError
Status = 1
haltStatus
109register_defaultRegisters ←
getenv'NANOFIL_REGISTERS', R
→
Registers = R
else
Registers = stack
register_modeRegisters
112register_modeR ←
memberchkR, stack, local
→
true
else
thrownanofil_error'registers must be stack or local'
115options'-O'|As, _, R, O, Rs, Args ←
!
optionsAs, o1, R, O, Rs, Args
116optionsA|As, O, _, Opt, Rs, Args ←
atom_concat'--registers=', R, A
!
optionsAs, O, R, Opt, Rs, Args
119optionsArgs, O, R, O, R, Args
arg_atom+Arg, -Atom

Atom is the command-line argument Arg, which is an atom (SWI) or a character list (Trealla).

123arg_atomAtom, Atom ←
atomAtom
!
126arg_atomCs, Atom ←
atom_charsAtom, Cs
report+Error

print Error on stderr, a nanofil_error by its message and anything else as a quoted term. Error is any ball, so the last clause is a default and the cut stays.

132reportnanofil_errorMsg ←
!
formatuser_error, "nanofil: ~w~n", Msg
135reportError ←
formatuser_error, "nanofil: ~q~n", Error

Loading

file_module+File, -Module

Module is the valid IR module in File. A .c or .i file goes through the C frontend, anything else is IR text.

142file_moduleFile, Module ←
atom_concat_, '.c', File
or
atom_concat_, '.i', File
→
c_file_moduleFile, Module
else
read_ir_fileFile, Module, _
c_file_module+File, -Module

Module is the valid IR module of the C file.

153c_file_moduleFile, Module ←
c_file_tokensFile, Tokens
c_parseTokens, Items
→
true
else
thrownanofil_error'C parse failed'
c_moduleItems, Module
c_file_tokens+File, -Tokens

Tokens are the C tokens of File. A .i file is already preprocessed; anything else goes through c_pp with the predefined macros and include paths of $FILCC (default filcc).

164c_file_tokensFile, Tokens ←
atom_concat_, '.i', File
→
phrase_from_fileseqCs, File
c_tokensCs, File, Tokens
else
getenv'FILCC', Clang
→
true
else
Clang = filcc
pp_profileClang, Profile
pp_fileFile, Profile, Tokens

Commands

Only native emission needs Registers; other commands consume Opt.

176commandOpt, Registers, compile, File ←
!
file_moduleFile, Module0
Promote can remove every p from a module; the ABI is the original's.
module_modeModule0, Mode
optimizeOpt, Module0, Module
mode_module_asmMode, Registers, Module, Asm
183commandOpt, Registers, cc, File ←
!
c_file_moduleFile, Module0
optimizeOpt, Module0, Module
mode_module_asmfilc, Registers, Module, Asm
188commandOpt, _, Args ←
commandOpt, Args
command+Opt, +Args

carry out the command line Args (without options), running the middle end when Opt is o1 (-O given) and not when o0. Fails or throws when the command does.

193command_, tokens, File ←
c_file_tokensFile, Tokens
forallmembertokToken, _, Tokens,
write_canonicalToken
write'.'
nl
197command_, check, File ←
format"ok~n", []
200commandOpt, print, File ←
file_moduleFile, Module0
optimizeOpt, Module0, Module
204commandOpt, 'cc-ir', File ←
c_file_moduleFile, Module0
optimizeOpt, Module0, Module
208commandOpt, run, File, Fn|ArgAtoms ←
file_moduleFile, Module0
optimizeOpt, Module0, Module
maplistatom_number, ArgAtoms, Args0
func_namedModule, Fn, func_, _, _, Params, _
maplistcanonical_arg, Params, Args0, Args
oncerunModule, Fn, Args, Fuel, Outcome
outcome_textOutcome, Text
format"~w~n", Text
218commandOpt, difftest, File, Dir ←
file_moduleFile, Module0
module_modeModule0, Mode
optimizeOpt, Module0, Module
file_runsFile, Runs
opt_valid_moduleOpt, Module
module_preparedModule0, Prepared0
module_preparedModule, Prepared
One findall over all runs, so each run's heap is reclaimed on
backtracking before the next starts.
findallResult−Verdict,
memberRun, Runs
validated_runOpt, Module0−Prepared0, Module−Prepared, Run, Result, Verdict
,
ResultVerdicts
pairs_keys_valuesResultVerdicts, Results, Verdicts
opt_report_validationOpt, File, Module0, Module, Verdicts
mode_pizlonation_agreesMode, File, Module−Prepared, Runs
includecomparableMode, Results, Compared
excludecomparableMode, Results, Skipped
atom_concatDir, '/expected.txt', ExpectedFile
atom_concatDir, '/harness.c', HarnessFile
atom_concatDir, '/mode', ModeFile
write_fileExpectedFile, expected_linesCompared
write_fileHarnessFile, harnessMode, Module, Compared
write_fileModeFile, fmt"~w~n", Mode
lengthCompared, NCompared
lengthSkipped, NSkipped
format"~w: ~w runs compared, ~w not comparable~n", File, NCompared, NSkipped
forallmemberrFn, Args, Outcome, Skipped,
outcome_textOutcome, Text
call_textFn, Args, CallText
format" ~w = ~w~n", CallText, Text
mode_module_asm+Mode, +Registers, +Module, -Asm

Asm is the asm.pl lines of Module by the backend for Mode, filc or plain.

257mode_module_asmplain, _, Module, Asm ←
module_asmModule, Asm
258mode_module_asmfilc, Registers, Module, Asm ←
checked_moduleModule, Checked
filc_module_asmChecked, Registers, Asm, _
checked_module+Module, -Checked

Checked is Module pizlonated, which must be valid checked IR (ir.pl): the validator verifies the pass.

264checked_moduleModule, Checked ←
pizlonate_moduleModule, Checked
oncevalid_moduleChecked

mode_pizlonation_agrees(+Mode, +File, +Module-Prepared, +Runs): for a filc module, the pizlonated Module has the same outcome as Module on every run, traps and their reasons included, or the command throws: the explicit checks are exactly the implicit ones. A missing or too weak check shows as the outcome unchecked(Why). Both run with the same fuel, since pizlonate keeps every block.

274mode_pizlonation_agreesplain, _, _, _
275mode_pizlonation_agreesfilc, File, Module−Prepared, Runs ←
checked_moduleModule, Checked
module_preparedChecked, CheckedPrepared
findallBad,
memberRun, Runs
pizlonation_verdictModule−Prepared, Checked−CheckedPrepared, Run, Bad
Bad ≢ ok
,
Bads
forallmemberbadCallText, Text0, Text, Bads, format"PIZLONATION FAILED ~w ~w: ~w unchecked, ~w checked~n", File, CallText, Text0, Text
Bads ≡ []
→
true
else
thrownanofil_error'pizlonation changed an outcome'
292pizlonation_verdictOriginal, Checked, Run, Verdict ←
module_run_resultOriginal, Run, Fuel, rFn, Args, Outcome0
max_fuelMaxFuel
Outcome0 ≢ out_of_fuel
or
Fuel ≡ MaxFuel
!
module_run_resultChecked, Run, Fuel, r_, _, Outcome
groundOutcome0
→
Outcome0 ≡ Outcome
else
¬¬Outcome0 = Outcome
→
Verdict = ok
else
call_textFn, Args, CallText
outcome_textOutcome0, Text0
outcome_textOutcome, Text
Verdict = badCallText, Text0, Text
opt_valid_module+Opt, +Module

with -O (o1), the optimized Module is valid. valid_module's clauses leave choice points that throw on retry.

313opt_valid_moduleo0, _
314opt_valid_moduleo1, Module ←
oncevalid_moduleModule
opt_report_validation+Opt, +File, +Module0, +Module, +Verdicts

with -O (o1), report_validation/4.

318opt_report_validationo0, _, _, _, _
319opt_report_validationo1, File, Module0, Module, Verdicts ←
report_validationFile, Module0, Module, Verdicts
report_validation+File, +Module0, +Module, +Verdicts

print the -O difftest report: promote's effect, then every validation failure, which together make the command throw.

325report_validationFile, Module0, Module, Verdicts ←
alloc_countModule0, Allocs0
alloc_countModule, Allocs
format"~w: promote: ~w allocs -> ~w~n", File, Allocs0, Allocs
forallmemberbadCallText, Text0, Text, Verdicts, format"VALIDATION FAILED ~w ~w: ~w before the middle end, ~w after~n", File, CallText, Text0, Text
memberchkbad_, _, _, Verdicts
→
thrownanofil_error'translation validation failed'
else
true

Running and validating

Fuel (blocks run) for a command-line run and the last validation step.

340max_fuel1000000

validated_run(+Opt, +Original, +Optimized, +Run, -Result, -Verdict): Result is Run's r/3 on the module that difftest compares, and Verdict ok or bad(CallText, OriginalText, OptimizedText).

Translation validation: the unoptimized module's outcome on the same run must allow the optimized one's. Both runs happen in the same findall step, so an unknown result keeps its constraints. Promote keeps every block and branch, so a correct result needs exactly the blocks the original needs: both run with the same fuel, starting small and growing until the original finishes. A broken pass that loops then ends as out_of_fuel, cheaply (the interpreter's memory grows quickly with the number of blocks run, so a looping run at full fuel would exhaust it).

355validated_runo0, _, Optimized, Run, Result, ok ←
module_run_resultOptimized, Run, Fuel, Result
358validated_runo1, Original, Optimized, Run, Result, Verdict ←
module_run_resultOriginal, Run, Fuel, rFn, Args, Outcome0
max_fuelMaxFuel
Outcome0 ≢ out_of_fuel
or
Fuel ≡ MaxFuel
Commit to the first fuel step at which the original finishes, or
to the last one.
!
module_run_resultOptimized, Run, Fuel, Result
Result = r_, _, Outcome
refinesOutcome0, Outcome
→
Verdict = ok
else
call_textFn, Args, CallText
outcome_textOutcome0, Text0
outcome_textOutcome, Text
Verdict = badCallText, Text0, Text
fuel_step-Fuelmulti

Fuel is one of the growing fuel limits, smallest first, ending at max_fuel.

379fuel_stepFuel ←
memberFuel, 1000, 8000, 64000, 512000, 1000000
canonical_arg+Param, +Arg0, -Arg

Arg is Arg0 reduced modulo the class of Param.

384canonical_argparamClass, _, Arg0, Arg ←
modulusClass, Modulus
Arg is Arg0 mod Modulus
module_run_result+Module−Prepared, +Run, +Fuel, -Result

Result is the r(Fn, Args, Outcome) of the "# run:" line Run on a module and the interpreter's preparation of it, with Fuel. Throws if the line does not fit the module or the interpreter finds no outcome.

392module_run_resultModule−Prepared, runFn, Args0, Fuel, rFn, Args, Outcome ←
func_namedModule, Fn, func_, export, _, Params, _
→
true
else
thrownanofil_errorFn−'run line names a function that is not exported'
lengthParams, N
lengthArgs0, N
→
true
else
thrownanofil_errorFn−'run line has the wrong number of arguments'
maplistcanonical_arg, Params, Args0, Args
oncerun_preparedPrepared, Fn, Args, Fuel, Outcome
→
true
else
call_textFn, Args, CallText
thrownanofil_errorCallText−'interpreter found no outcome'
comparable+Mode, +Resultsemidet

a native run in Mode (plain or filc) reproduces Result's outcome, a known return value or, for filc, a trap, which is a Fil-C safety panic natively. Nothing else has a native counterpart.

412comparable_, r_, _, retV ←
integerV
414comparable_, r_, _, ret
415comparablefilc, r_, _, trap_
call_text+Fn, +Args, -Text

Text is the atom 'Fn(A1, A2, ...)'.

418call_textFn, Args, Text ←
atomic_list_concatArgs, ', ', ArgText
formatatomText, '~w(~w)', Fn, ArgText

difftest output files

write_file+File, :Body2

write the characters Body2 (a DCG body) describes to File.

426write_fileFile, Body2 ←
phraseBody2, Cs
openFile, write, Stream
formatStream, "~s", Cs
closeStream
expected_lines+Results//

one "CALL = OUTCOME" line per result, a trap's outcome shown as just "trap" (the native run cannot tell why).

434expected_lines[] ⟶
[]
435expected_linesrFn, Args, Outcome|Results ⟶
call_textFn, Args, CallText
Outcome = trap_
→
Text = trap
else
outcome_textOutcome, Text
fmt"~w = ~w~n", CallText, Text
harness+Mode, +Module, +Results//

a C program that calls Module's functions as Results do and prints what expected_lines//1 prints.

445harnessplain, Module, Results ⟶
"#include <stdio.h>\n#include <stdint.h>\n\n"
results_fnsResults, Fns
prototypesFns, Module
"\nint main(void) {\n"
harness_callsResults, Module, labelled
" return 0;\n}\n"

Fil-C: one run per process, selected by argv[1], since a trap ends the process. The call text is printed first, so a trap leaves it unfinished.

454harnessfilc, Module, Results ⟶
"#include <stdio.h>\n#include <stdint.h>\n#include <stdlib.h>\n\n"
results_fnsResults, Fns
prototypesFns, Module
"\nint main(int argc, char **argv) {\n switch (atoi(argv[1])) {\n"
filc_casesResults, 0, Module
" }\n return 0;\n}\n"
results_fns+Results, -Fns

Fns is the ordset of the functions Results call.

464results_fnsResults, Fns ←
findallFn, memberrFn, _, _, Results, Fns0
sortFns0, Fns
filc_cases+Results, +I, +Module//

the switch cases I, I+1, ... of the Fil-C harness, one per result.

470filc_cases[], _, _ ⟶
[]
471filc_casesResult|Results, I, Module ⟶
Result = rFn, Args, _
call_textFn, Args, CallText
fmt" case ~w: fputs(\"~w = \", stdout); fflush(stdout);~n", I, CallText
harness_callsResult, Module, bare
" break;\n"
I1 is I + 1
filc_casesResults, I1, Module
prototypes+Fns, +Module//

an extern C declaration of each of Fns.

482prototypes[], _ ⟶
[]
483prototypesFn|Fns, Module ⟶
func_namedModule, Fn, func_, _, Ret, Params, _
c_typeRet, RetType
findallType,
memberparamClass, _, Params
c_typeClass, Type
,
ParamTypes
types_textParamTypes, ParamText
fmt"extern ~w ~w(~w);~n", RetType, Fn, ParamText
prototypesFns, Module
types_text+Types, -Text

Text is the C parameter list of Types.

493types_text[], void
494types_textType|Types, Text ←
atomic_list_concatType|Types, ', ', Text
c_type?Class, ?Type

Type is the C type of the IR class.

c_type
498wuint32_t
499luint64_t
500voidvoid
harness_calls+Results, +Module, +Style//

Style labelled prints "CALL = ret N" lines from one main; bare prints "ret N" inside a case.

504harness_calls[], _, _ ⟶
[]
505harness_callsrFn, Args, _|Results, Module, Style ⟶
func_namedModule, Fn, func_, _, Ret, _, _
call_textFn, Args, CallText
findallCArg,
memberX, Args
formatatomCArg, '~wull', X
,
CArgs
atomic_list_concatCArgs, ', ', CArgText
harness_callStyle, Ret, Fn, CArgText, CallText
harness_callsResults, Module, Style
harness_call+Style, +Ret, +Fn, +CArgText, +CallText//

the line of harness_calls//3 for one call of Fn, which returns Ret.

516harness_callStyle, Ret, Fn, CArgText, CallText ⟶
ret_resultRet, Result
style_callStyle, Result, Fn, CArgText, CallText
ret_result?Ret, ?Result

a function that returns Ret has a Result of none or value.

ret_result
522voidnone
523wvalue
524lvalue
525pvalue
526tuple_value
style_call+Style, +Result, +Fn, +CArgText, +CallText//

as harness_call//5, for a call with Result.

530style_callbare, Result, Fn, CArgText, _ ⟶
bare_callResult, Fn, CArgText
532style_calllabelled, Result, Fn, CArgText, CallText ⟶
labelled_callResult, Fn, CArgText, CallText
bare_call+Result, +Fn, +CArgText//

a case's line for a call of Fn.

536bare_callnone, Fn, CArgText ⟶
fmt" ~w(~w); puts(\"ret\");~n", Fn, CArgText
538bare_callvalue, Fn, CArgText ⟶
fmt" printf(\"ret %llu\\n\", (unsigned long long)~w(~w));~n", Fn, CArgText
labelled_call+Result, +Fn, +CArgText, +CallText//

main's line for a call of Fn.

544labelled_callnone, Fn, CArgText, CallText ⟶
fmt" ~w(~w); printf(\"%s = ret\\n\", \"~w\");~n", Fn, CArgText, CallText
547labelled_callvalue, Fn, CArgText, CallText ⟶
fmt" printf(\"%s = ret %llu\\n\", \"~w\", (unsigned long long)~w(~w));~n", CallText, Fn, CArgText

Runs

file_runs+File, -Runs

Runs are the run(Fn, Args) facts of IR File, in order.

555file_runsFile, Runs ←
read_ir_fileFile, _, Runs

library(format)'s format_//2 raises type_error(list, '') for an argument of '' (Trealla 12f4cbd), which a call with no arguments produces. fmt(+Format, +Args)//: the characters format/3 makes of Format and Args.

560fmtFormat, Args ⟶
formatcharsCs, Format, Args
seqCs