luv

Workshop wiki

technical-debt.org

Technical debt that multiplies truths

What this audit counts as debt #J97QHX

This is a source audit of luv and luvcraft at 58b0a3afb75f. It ranks code that makes one change require several coordinated edits, preserves an obsolete runtime path, or puts a contract on the wrong side of a subsystem boundary. Large files and long explanations are not debt by themselves. The problem is when they conceal several independent sources of truth.

The ranking comes from reading the systems, packages, largest implementation files, backend seams, session lifetime, shader lowering, voxel lighting, and their call sites. It is a work map, not a request to rewrite everything.

Ranked cut list #OUKQ2A

RankDebtWhy it compoundsFirst safe cut
1Luvcraft session and render lifetimeOne GPU object appears in several ledgersExtract one renderer that owns its resources
2McCLIM GPU command dispatchA new command edits four type tablesPut command phases behind generic functions
3Shared shader IR lives in SPIR-VBoth backends depend on one backend's nameExtract luv.shader and backend lowerings
4The HAL boundary is only half-ownedBackends disagree on portable inputs; video bypasses HALNormalize once; add an interop protocol
5Three voxel-light runtime solversOracle, experiment, and production coexistClose dispatch; move or delete old paths
6Serial umbrella systems hide islandsOptional features enlarge every core loadExtract only the already-visible islands
7Compatibility entry points without clientsTests keep obsolete surfaces aliveDelete the high-confidence queue

Ranks 1--5 are architectural. Ranks 6--7 are useful supporting cuts, but splitting systems or deleting aliases will not repair duplicated ownership or dispatch on its own.

1. The session has several resource ownership truths #V9VH79

luvcraft-session has about one hundred slots spanning world production, simulation, input, GPU attachments, layouts, pipelines, video, text, overlays, and frame state. luvcraft:start-luvcraft is 623 lines. During startup a GPU object can simultaneously be a lexical binding, a named session slot, an element of resources, or an element of the separate pipelines list:

flet
keep
resource
pushresourceresources
resource
let*
color-texture
keep
getfattachments:color-texture
...
pipeline
let
artifact
make-luvcraft-live-pipeline...
pushartifactpipelines
artifact
session
make-instance'luvcraft-session:color-texturecolor-texture:block-pipelinepipeline:resourcesresources
...

Resize code must replace named slots and update the resource bag. luvcraft:stop-luvcraft separately enumerates ten pipeline accessors, then destroys the resource bag, then releases video, text, canvas, and device. The teardown is careful; the debt is that correctness depends on every future resource author remembering every ledger.

The first extraction should be semantic, not a giant constructor macro:

defclassluvcraft-renderer
frame-attachments:initarg:frame-attachments:accessorframe-attachments
pipelines:initarg:pipelines:readerrenderer-pipelines
layouts:initarg:layouts:readerrenderer-layouts

The renderer should be the sole runtime owner of those objects. A small transactional resource helper may protect construction failure, but the finished owner must not also depend on a catch-all session resource list. Later extractions can give production/streaming and simulation the same shape; the session then coordinates owners instead of being all of them.

defmethod update-instance-for-redefined-class :after app.lisp:379
defmethodupdate-instance-for-redefined-class:after
added-slotsdiscarded-slotsretired-values&restinitargs
declare
ignoreinitargs
when
and
member'rendereradded-slots
intersectiondiscarded-slots

A running game is valuable state. Moving the ownership boundary must not require abandoning it or leave its old GPU objects unreachable. See #V9VH79.

setf
slot-valuesession'renderer
defclass luvcraft-renderer renderer.lisp:26
defclassluvcraft-renderer
device:initarg:device:readerluvcraft-renderer-device
context:initarg:context:readerluvcraft-renderer-context
atlas-texture:initarg:atlas-texture:accessorluvcraft-renderer-atlas-texture
atlas-view:initarg:atlas-view:accessorluvcraft-renderer-atlas-view
atlas-sampler:initarg:atlas-sampler:readerluvcraft-renderer-atlas-sampler
normal-atlas-texture:initarg:normal-atlas-texture:accessorluvcraft-renderer-normal-atlas-texture
normal-atlas-view:initarg:normal-atlas-view:accessorluvcraft-renderer-normal-atlas-view

One slot is the publication point for the complete extent-sized cohort. Readers can therefore never observe a new colour image with an old depth or presentation image while a live resize is being installed.

frame-attachments:initarg:frame-attachments:initformnil:accessorluvcraft-renderer-frame-attachments
shadow-depth-texture:initarg:shadow-depth-texture:readerluvcraft-renderer-shadow-depth-texture
shadow-depth-view:initarg:shadow-depth-view:readerluvcraft-renderer-shadow-depth-view
shadow-depth-sampler:initarg:shadow-depth-sampler:readerluvcraft-renderer-shadow-depth-sampler
shadow-comparison-sampler:initarg:shadow-comparison-sampler:readerluvcraft-renderer-shadow-comparison-sampler
layout:initarg:layout:readerluvcraft-renderer-layout
shadow-layout:initarg:shadow-layout:readerluvcraft-renderer-shadow-layout
post-layout:initarg:post-layout:readerluvcraft-renderer-post-layout
bloom-layout:initarg:bloom-layout:initformnil:readerluvcraft-renderer-bloom-layout
linear-sampler:initarg:linear-sampler:initformnil:readerluvcraft-renderer-linear-sampler
block-pipeline:initarg:block-pipeline:readerluvcraft-renderer-block-pipeline
shadow-pipeline:initarg:shadow-pipeline:readerluvcraft-renderer-shadow-pipeline
sky-vertex-buffer:initarg:sky-vertex-buffer:initformnil:readerluvcraft-renderer-sky-vertex-buffer
sky-pipeline:initarg:sky-pipeline:initformnil:readerluvcraft-renderer-sky-pipeline
crosshair-vertex-buffer:initarg:crosshair-vertex-buffer:readerluvcraft-renderer-crosshair-vertex-buffer
cursor-vertex-buffer:initarg:cursor-vertex-buffer:initformnil:readerluvcraft-renderer-cursor-vertex-buffer
crosshair-pipeline:initarg:crosshair-pipeline:readerluvcraft-renderer-crosshair-pipeline
cursor-pipeline:initarg:cursor-pipeline:readerluvcraft-renderer-cursor-pipeline
post-pipeline:initarg:post-pipeline:readerluvcraft-renderer-post-pipeline
bloom-bright-pipeline:initarg:bloom-bright-pipeline:initformnil:readerluvcraft-renderer-bloom-bright-pipeline
bloom-horizontal-pipeline:initarg:bloom-horizontal-pipeline:initformnil:readerluvcraft-renderer-bloom-horizontal-pipeline
bloom-vertical-pipeline:initarg:bloom-vertical-pipeline:initformnil:readerluvcraft-renderer-bloom-vertical-pipeline
sun-shaft-pipeline:initarg:sun-shaft-pipeline:initformnil:readerluvcraft-renderer-sun-shaft-pipeline

The cache and both inventories are private implementation details of this owner. A construction-time rollback list may precede it, but a completed session has no second resource ledger. See #V9VH79.

frame-states:initarg:frame-states:initform
make-hash-table:test#'eql
:readerluvcraft-renderer-frame-states
resources:initarg:resources:initformnil:accessorluvcraft-renderer-resources
:documentation

The sole owner of a session's frame attachments, layouts, pipelines, and per-drawable GPU state. The session coordinates this owner with simulation, streaming, overlays, and presentation; it does not duplicate its inventory.

DONE Give rendering resources one transactional owner #V5V0JJ

luvcraft-renderer now solely owns named pipelines, layouts, resources, frame state, atlas state, and one atomically published frame-attachment cohort. Startup, resize, atlas replacement, live-session migration, and teardown all use that owner. A failed renderer destruction retains its retry handle; focused tests exercise partial construction, coherent publication, migration, retry, and idempotent release. Backend-native retirement has separate durable HAL custody described in the cut below.

2. McCLIM implements one open command vocabulary four times #A0JOG7

mcclim/gpu.lisp defines solid, analytic, relief, gradient, lattice, image, and text command structures. Their algebra is repeated in four etypecase tables:

PhaseFunctionRepeated decision
Compose pane streamscompose-gpu-mirror-mediaCopy and rebase the command
Prepare frame dataprepare-gpu-frame-commandsPreserve or turn it into a prepared command
Set clippinggpu-frame-command-clipSelect the command's clip accessor
Encode drawsrender-gpu-mirror-frameSelect pipeline, bindings, buffer, and draw fields

For example, the clip operation alone knows every prepared command type:

defungpu-frame-command-clip
command
etypecasecommand
gpu-solid-command
gpu-solid-command-clipcommand
...
gpu-prepared-text-command
gpu-prepared-text-command-clipcommand

Adding one draw family therefore requires shotgun surgery across distant parts of a 2,692-line file. This is an owned, growing vocabulary, so it wants an open protocol:

defgenericrebase-gpu-command
commandoffsets
defgenericprepare-gpu-command
commandmirrorframe-build
defgenericgpu-command-clip
command
defgenericencode-gpu-command
commandpassframe-state

The command structures can remain compact method specializers, and a common included draw structure can hold first-vertex, vertex-count, and clip. The dense vertex arrays should remain arrays. This follows the already useful portable encode=/=enqueue command seam described by #D8F3QM; it does not ask for a retained object graph or a class per vertex.

DONE Make command phases an open protocol #GC3QF4

The four command-family type tables are gone. Internal generic functions own rebase, preparation, clipping, and encoding while the command structures and dense arrays stay compact. Direct luvcraft replay builds one dynamic-extent context, so cached frame state retains only its persistent destination GPU resources. A recording encoder checks the exact pipeline, binding, buffer, offset, and draw mapping of all seven native command families.

3. The shared shader language is trapped inside the SPIR-V backend #TUVTOM

hal/shader/language.lisp is 5,757 lines and begins with (in-package #:luv.spir-v). It contains the backend-neutral type system, source graph, parser, quantity checking, definition revision tracking, lowering protocol, and SPIR-V lowering. Metal's direct MSL lowering consequently specializes a SPIR-V-named protocol over SPIR-V-named portable objects:

defmethodspv:lower-shader-specification

Lower the shared shader graph directly to a structured MSL document.

...

The source has already outgrown that ownership: #LVKEYO describes one shared typed graph with sibling SPIR-V and MSL lowerings. The package graph still says that SPIR-V owns both siblings. The permanent source also carries eval-when migrations that unintern old portable operators and repair old accessors in a live image.

Extract three explicit layers:

luv.shader       types, source graph, parser, revision, lowering protocol
luv.spir-v       literal SPIR-V model and SPIR-V lowering methods
luv.msl          structured MSL model and MSL lowering methods

Do this as a package/file move before redesigning the language. Update the remaining shader-abstraction-revision callers to shader-source-revision, restart the durable image at the package boundary, and then delete the one-time live migration shims. The split makes the architecture in #LVKEYO true in the loader as well as in the prose.

DONE Give the shared shader graph its own home #C18ML5

The neutral type graph, parser, quantity checks, source revisions, live definition tracking, and lowering generics now live in luv/shader and the LUV.SHADER package. SPIR-V's literal model and lowering context live in the sibling luv/spir-v system; luv/msl depends only on the shared language. Repository callers name neutral concepts through shader: and reserve spv: for actual modules, instructions, compilation, and lowering results. Cold loads prove that the shared system loads neither backend and MSL loads without SPIR-V. The one-time package/accessor migration shims are gone.

4. The HAL boundary is only half-owned #YI8ZE7

The portable descriptor types live in hal/gpu.lisp, but each backend independently defines what their input shapes and errors mean. Vulkan accepts a list or vector of two or three texture dimensions and canonicalizes it to (width height 1). Metal accepts only a two-element list and leaves it two-dimensional. Vulkan reports :unsupported-features and :unsupported-limits separately; Metal collapses both into :unsupported-requirements. Similar keyword/list/vector normalization is repeated for buffer and texture usage.

Backend capability differences are legitimate. Portable syntax and error shape changing with the selected backend are not. Move descriptor canonicalization and common structural checks into the portable layer; hand a canonical descriptor to each backend for capability checks and native enum translation. Closed Vulkan and Metal enum tables should remain backend-local.

The same uncertain boundary is more serious in luvcraft/video-screen.lisp. Game code tests private backend classes, reads luv::vulkan-device-* and luv::metal-native-object slots, interprets libav::av-vulkan-frame, and performs CoreVideo FFI. Reader conditionals then choose the platform path. This contradicts the useful HAL seam in #A7N4XP:

when
list:instance
luv::vulkan-device-instancedevice
...
luv.objective-c:objective-c-pointer
luv::metal-native-objectdevice

Introduce a narrow interop extension, for example video-decode-configuration and adopt-decoded-video-frame generic functions. Backend/video bridge files should own native handles and return portable textures/views plus an explicit owner. Luvcraft should own playback policy and the screen; it should not decode Vulkan or CoreVideo ownership records.

DONE Make the portable HAL boundary tell the whole truth #QYZ3XP

Portable create and adopt-native-texture methods now copy and canonicalize buffer/texture descriptors before backend dispatch. Size, dimension, and usage syntax has one error vocabulary; every texture extent is (w h 1); Metal and Vulkan retain only format/capability translation. Backend-free probe tests cover copies, vectors, duplicates, malformed sequences, adoption, and condition provenance, while the Darwin suite checks Metal's canonical owned/imported extents and two-dimensional writes.

Video decoding crosses a separate open bridge. Portable video-decode-configuration and frame-importer generics own the policy seam; Vulkan and Metal bridge files alone inspect AVVkFrame, queues, CoreVideo, and native handles. A screen publishes one complete two-plane picture cohort, rolls back partial import, and retires views before textures.

While a backend queue is live, logical GPU destruction transfers a complete, per-call resumable native teardown into its FIFO retirement ledger before invalidating the wrapper. Nonempty submission and retirement work process-roots the queue; a lazy service advances it even after the application drops its last reference. Device destruction closes admission and refuses to discard unfinished ownership. Without a live queue, teardown must succeed before invalidation. Vulkan leak finalizers transfer into a separate process-global ledger before reporting the leak.

Each adopted video plane retains its frame importer until that native retirement callback succeeds. Partial picture construction, partial screen construction, and failed startup teardown enter small process-rooted release backlogs, so no lexical unwind can discard the final retry handle. Vulkan's two plane wrappers also share one generation-scoped external timeline state; post-submit layout/semaphore publication remains in the rooted submission until every owner callback succeeds. video-screen.lisp contains no backend or native ownership names.

5. Voxel lighting keeps oracle, experiment, and production live #S6421N

The default *voxel-light-solver* is :compiled, but the runtime still offers three complete from-scratch paths: :legacy, handwritten :frontier, and :compiled. The incremental protocol is more dangerous: its unspecialized method ignores the solver and silently invokes the legacy reconciler.

defmethodreconcile-light-region-using
solverstateregion
declare
ignoresolver
reconcile-legacy-light-regionstateregion

A misspelled or future solver therefore succeeds using the wrong algorithm. The handwritten frontier transfer loop also duplicates the relation compiled by the production frontier realization. The differential work in #X7Q90E, #PJY6E1, and #K3WRD3 has made this duplication useful evidence; it need not remain a permanent runtime choice.

Make unknown solver names signal immediately. Delete the handwritten :frontier runtime path now that the compiled path is the production proof. Keep the legacy solver only as a differential test oracle in a test/reference system; if an operational recovery path is genuinely required, name it explicitly and record its use rather than making it the fallback. Rename comparison slots from frontier-* to candidate-* so the diagnostic does not preserve the retired implementation in its interface.

DONE Keep one production light solver #J2UE30

Production from-scratch and incremental dispatch now accept only :compiled; unknown and retired names signal. The handwritten frontier solver and legacy incremental reconciler are gone. A simple from-scratch oracle and the renamed candidate comparator remain available only through the explicit luvcraft/light-reference system, preserving differential evidence without a fallback runtime truth.

6. Serial umbrella systems conceal useful dependency islands #74HG89

luvcraft/core serially loads roughly forty files and unconditionally depends on the GPU stack, Ghostty, libav, terminal/canvas, world, simulation, media, network-facing features, rendering, and capture/tool helpers. Its 686-line package file also defines both luvcraft.frontier and the broad luvcraft package. The luv umbrella similarly loads the shared shader language only after Vulkan's SPIR-V package and model.

The result is unnecessary compile/load fanout and an image that is harder to repair when one optional dependency changes. luvcraft/world proves that a small renderer-independent island can exist without turning the project into dozens of micro-systems.

Extract only boundaries already demanded above: luv/shader, a frontier kernel system, the renderer owner, and platform video bridges. Keep luv and luvcraft/core as convenience umbrellas. ASDF decomposition should express ownership and optionality, not merely make the component list shorter.

DONE Expose the shader, frontier, and reference islands already earned #JV6E2Z

luv/shader, luv/spir-v, and luv/msl are now independent sibling systems, and luvcraft/light-reference keeps the oracle out of production loading. luvcraft/frontier now owns the frontier package and compiler over only luv/arithmetic and luvcraft/world; a cold load does not acquire the renderer, video, GPU, Ghostty, or libav umbrella. Renderer and video ownership boundaries remain separate source components under the convenient luvcraft/core umbrella.

7. A high-confidence deletion queue #H03JVK

These are smaller than the architectural debts, but removing them first will reduce the surface that later refactors must carry:

The persistence migration for old saved gnome edits is different: saved data is an external boundary. Keep versioned persistence migrations until the supported save floor is intentionally raised.

DONE Delete the queue without deleting history #OHTJXI

The gnome aliases, literal crosshair module, shader revision alias, palette constructor/install compatibility, and compatibility meshing entry points are gone; their tests now exercise production specifications, chunks, snapshots, and vocabulary-backed columns. The saved-gnome migration remains because it serves persisted external data rather than repository callers.

A workable order #CWKEY9

  1. Delete the uncalled aliases, obsolete crosshair module, and compatibility constructors; point their tests at production entry points.
  2. Close voxel solver dispatch and retire the handwritten runtime solver.
  3. Replace McCLIM's four command type tables with methods, one phase at a time.
  4. Extract the renderer as the sole owner of frame attachments, layouts, and pipelines; then make session start/stop orchestration use that owner.
  5. Split the shader package and normalize portable GPU descriptors. This is the deliberate durable-image restart boundary.
  6. Move native video import behind backend methods and reflect the resulting ownership islands in ASDF.

Each step has a local stopping condition: one fewer parallel dispatch table, one fewer resource ledger, one fewer selectable obsolete algorithm, or no backend-private name above HAL. That is a more useful measure than smaller files alone.