Status: implementation notes and remaining plan #SK7L2T

This page records the sky and lighting system for luvcraft: an animated sky and primary light, caves and overhangs which become dark for the right reason, light-emitting blocks, relighting after edits and residency changes, and a clean route to HDR bloom and directional shadows.

The important outcome of the design review is that shader-language growth is part of the work, not a cost to evade. Frame-wide facts still belong in Lisp; spatially varying image mathematics belongs in shaders; and the boundary between them should become more capable as the renderer asks more of it.

Phases 0--4 and the first directional-shadow pass now have implementations in the codebase, and the HDR presentation stack the last phase asked for has landed: a linear floating-point scene attachment, a filmic curve, a bloom chain, and light shafts (#IC14P3). The sky it presents became procedural image mathematics rather than a two-colour gradient (#9SSXDJ), and the block surface grew a shape of its own from what the mesher knows about each face's edges (#J19EBO) and what the atlas paints beside each material's colour (#CHUKWD). A later round took the shading itself toward physical argument: no two faces are the same card any more (#3AVEKC), the specular lobe and the ambient are microfacet and hemispheric rather than fitted (#2T8CCH), the sky became an atmosphere (#8X33G2), and the cloud deck became a plane both halves of the renderer can find (#RSGLTL). Each phase below has a visible result and an evidence gate, so it can be revised from measurements without undoing the preceding phases.

What the critique changed #SKCRIT

The first draft had a useful inventory and a good instinct for dense chunk data, immutable mesh snapshots, and last-known-good shader replacement. It also optimized too hard for making no new machinery. These are the changes which make the plan durable.

• Grow the shader language on purpose. Avoiding max, clamp, smoothstep, normalization, and exponentiation is not a design objective. They are ordinary graphics vocabulary needed immediately by fog, a sky gradient and sun disc, tunable light response, and later tone mapping. Add them as one coherent typed-math extension and keep their source-to-SPIR-V provenance inspectable.
• Do not make the present world height a lighting invariant. The current generated world resides only at chunk Y=0, but block-world and voxel-space already admit arbitrary chunk coordinates and shapes. A 3x3-horizontal regional proof becomes false once a sky opening can affect a tall column. The runtime solver will therefore be incremental and height-independent; a from-scratch solver remains its oracle.
• Keep source and derived revisions distinct. Block content revisions say that authored world data changed. Light revisions say that a derived field changed. Relighting must not impersonate a block edit by bumping content revisions; mesh dependency stamps should name the source and derived field revisions explicitly.
• Residency is a lighting input. Arrival and departure alter the boundary conditions of retained chunks. Eviction cannot merely invalidate a mesh and leave its baked light untouched. Lighting receives content and residency dirtiness, and unknown terrain is not silently reclassified as open sky.
• Add light in linear space. Screen blend is bounded and convenient, but it is not a useful foundation for several light sources and it destroys the headroom emissive surfaces need. The material sums sky, direct, and block illumination, adds self-emission separately, then fogs and eventually tone maps.
• Do not spend alpha as an undocumented bloom bus. Propagated blocklight is not the same fact as material emission, and the current sRGB scene target cannot retain HDR energy anyway. Emission gets an explicit material input; bloom follows a deliberate linear-HDR scene path.
• Measure scheduling claims. There is no evidence yet that a relight is sub-millisecond or that a one-mesh-per-frame ripple is pleasant. The system will expose work counters and latency, publish light atomically, and move captured work to the existing producer only if measurements justify it.

What exists today: verified touch points #SK9INV

• hal/shader/language.lisp is already an open shared frontend. Operators are symbols, and parsing, typing, result naming, and documentation are EQL-specialized generic functions. The lower-shader-call protocol is shared while SPIR-V and MSL own sibling methods and result products. A new operator should be a cohesive method cluster, not another central dispatch table.
• The expression language has floats, vec2..4, arithmetic, dot, mix, swizzle, texture sampling, and the first GLSL extended-math family: min, max, clamp, smoothstep, abs, sqrt, expt, and normalize. A small shadow-ready follow-up added step, a typed :depth-texture-2d resource, and depth-image sampling that feeds ordinary scalar math through swizzle. It now has expression conditionals and bounded counted folds, but no general loops, matrices, first-class boolean values, or general uniform packing. Uniform blocks intentionally consist only of aligned vec4 lanes.
• hal/vulkan/spir-v/instructions.lisp now models OpExtInstImport and OpExtInst as structured module sections in logical-layout order. Lowering requests GLSL.std.450 once per module when needed, while modules with no extended math acquire no import.
• The live shader path in luvcraft/live-pipeline.lisp transactionally rebuilds complete vertex/fragment pipelines and retains the last-known-good pipeline after a bad edit. New sky and post materials should use that path rather than a parallel compilation mechanism.
• The block vertex shader and sky shaders share one frame uniform block, and the host derives the uniform byte size from the shader-visible layout. Fog now has explicit near/far semantics with clamped quadratic shaping.
• The visible material reads the shared sky/environment lanes, samples raw vertex light, computes sky, sun, and block contributions in linear space, adds surface emission, and then fogs.
• The mesh vertex is four :float32x3 attributes: position, UV plus AO, normal, and (sky block emission). The 48-byte stride fits the backend's current vertex format support.
• Meshing copies a one-cell block and light halo to an immutable worker snapshot. Corner AO and corner light sampling use separate named rules, and mesh results are rejected when their dependency stamp is stale.
• The current scene target is the canvas's LDR sRGB format. It is rendered offscreen and then copied to the surface, which is a useful post-processing seam but not yet an HDR or multi-pass pipeline.

The rendering model #SKRMOD

There are two related systems, not one.

1. The environment is continuous and changes every frame: sky radiance, primary-light direction and colour, ambient colour, fog, exposure, and time of day. Lisp evaluates its small time/profile model once per frame and writes a uniform block. Sky and material shaders evaluate spatial image functions from those parameters.
2. The voxel light field changes when block content or residency changes: sky access, propagated blocklight, opacity, and emission. It is a dense derived field associated with resident chunks, revised independently from block content, and sampled into immutable mesh snapshots.

The block fragment material starts from a linear-light equation rather than a blend trick. In schematic form:

The exact response curves and balances are art parameters, not baked mesh facts. Vertices carry normalized raw levels; shader edits can change the curve without remeshing the world. AO modulates diffuse sky access, not self-emission. When the HDR phase arrives, radiance above 1.0 survives until tone mapping and can feed bloom naturally.

Propagated skylight is not a directional shadow map. It makes caves dark and canopies reduce ambient sky access. A separate GPU depth pass now shadows the moving sun; vertical canopy shade remains ambient access, not an angled sun shadow.

Phase 0: give the shader language the mathematics this work needs #SKSHDR

Add the SPIR-V and expression-language capability before designing visual effects around its absence.

Mentioned in: One arithmetic medium, many clients

Phase 1: a real sky driven by a small Lisp clock #SKDYCY

A sky-clock belongs to the luvcraft session because it has mutable runtime identity worth inspecting: time, rate, paused state, and an optional pinned time for deterministic capture. A sky profile is ordinary data: keyframes for sky/horizon colour, primary-light colour, ambient colour, fog, and exposure. There is no class hierarchy or generic dispatch until more than one actual sky model exists.

sky-frame-parameters evaluates the clock and profile once per frame. CPU trigonometry is appropriate for a single sun direction; it is not a reason to move per-pixel sky shaping out of the shader. The frame uniform grows by aligned lanes for at least:

• sun direction plus day factor;
• sun colour plus angular size or intensity;
• zenith and horizon colours;
• ambient colour plus exposure;
• fog colour and near/far shaping.

The three hardcoded sky literals have been replaced by this one frame result. The clear value is merely a safe fallback.

A live :sky pipeline draws a fullscreen triangle before block geometry in the existing scene pass, with depth writes disabled. Its fragment shader reconstructs or receives a view ray and uses normalize, smoothstep, max, and expt for a vertical gradient, horizon band, sun disc, and a soft forward glow. A tiny static fullscreen vertex buffer is acceptable; integer vertex-index support is not a prerequisite unless it is independently useful.

The block shader reads the same environment lanes directly. Fog becomes a clamped function with explicit near/far semantics. The sky pass itself is not fogged into its own colour.

The clock advances in render-luvcraft-frame from its bounded delta. SLY can pause it, set a time, change its rate, or replace the profile without restarting. Smoke capture pins it; interactive play animates it.

Mentioned in: Sky, temporal reconstruction, and mesh shaders bound the proof

Phase 2: make voxel light an explicit derived field #SKSTOR

Each resident block-chunk may own a chunk-light-field with:

• sky and block arrays of (unsigned-byte 8) values constrained to 0..15;
• a light revision and six light-boundary revisions;
• a state such as :unlit, :stable, or :provisional.

The semantic light-field object belongs at the chunk boundary; its 4096-site columns remain dense arrays. Two u8 arrays cost 8 KiB for a 16-cubed chunk. Nibble packing is deferred until memory or bandwidth measurements say it matters.

Block light behavior is distinct from collision. Add both protocols now:

Air answers zero for all three. A block-kind method reads configured slots with opaque/non-emissive defaults; particular kind instances carry the data. Propagation strength and visible surface radiance are separate even when one crystal configures both. The solver dispatches once per palette entry to build dense opacity and light-emission lookup vectors, then its hot loops index those vectors. It does not call a generic for every cell.

Light publication advances light revisions only. The mesh dependency stamp contains the target chunk's content and light revisions plus the opposing content and light boundary revisions of sampled neighbors. That preserves the reason a mesh became stale and avoids feedback in which derived light looks like another authored edit.

This field is a sibling materialization over the chunk's domain, not a second spatial domain merely because it has its own revisions. The current light-region nevertheless rebuilds world decomposition, dense offsets, and face enumeration, while meshing rebuilds related traversal for its resolved neighborhood and immutable halo. #WUB581 records that concrete duplication; #K3KZTG makes the shared domain/window vocabulary the next structural proof.

Lighting semantics and boundaries #SKBOUND

• Blocklight begins at block-light-emission and loses at least one level per six-neighbor step, with opacity contributing further loss.
• Direct sky enters from a boundary known to be open to the sky. It travels downward through transparent cells without losing a level; lateral and upward propagation attenuate. This rule works across any number of vertical chunks.
• A missing resident neighbor has one of three meanings: known open boundary, known closed/outside boundary, or unknown terrain. Only the world/source policy may decide which. The lighting solver never equates :absent with open sky merely because the mesher currently chooses :air for missing geometry.
• Provisional results may be rendered at a residency edge, but their dependency includes that boundary state. Arrival or departure dirties the retained face and schedules reconciliation before a new stable light revision is published.
• Residency events accumulate until reconciliation. A same-key remove-and-arrive replacement must run both sides of the transition: remove the old incarnation's influence from retained neighbors, then seed the new resident chunk.

The little generated world can initially declare open sky above its known vertical extent and a closed lower boundary. A future vertically streamed source can provide sky-column knowledge or retain :unknown until the upper stack is known. That source protocol, rather than a hardcoded chunk Y=0 test, is the growth seam.

The solver: explicit reference truth plus one runtime implementation #SKRELT

Maintain two roles with the same semantics but deliberately different loader and dispatch status.

1. A from-scratch reference relight clears a finite captured region, seeds known sky boundaries and emitters, imports explicit boundary values, and propagates to fixation. It favors obvious correctness and is loaded only by the explicit luvcraft/light-reference test/reference system.
2. The runtime relighter maintains removal and addition queues. Content changes first recompute direct-sky seeds for affected columns. Decreased or invalidated values enter the removal queue; surviving sources and alternate paths enter the addition queue. The same mechanism crosses resident chunk faces and works for both edits and residency transitions.

The old 3x3-horizontal recomputation remains useful historical test evidence, not a current-world fallback. Production dispatch accepts only :compiled; unknown, misspelled, and retired solver names signal immediately. The reference system can still compare a complete captured world without placing the oracle on the operational recovery path.

A luvcraft-lighting-state owned by the session records dirty cell/column or chunk coordinates, residency-boundary changes, queue counters, and an in-progress candidate. World content hooks and residency publication feed that state; a dirty bit stored only on a chunk would lose departures and give too little information for future vertical columns.

Reconciliation runs before mesh snapshots are captured, but it does not expose half-solved arrays. Work is accumulated in candidate copies of touched light fields and published as one owner-thread transaction when the queues settle. Until then meshes continue to use the previous stable revision. A new edit either merges into or invalidates the candidate explicitly.

Start with bounded owner-thread work and record cells popped, chunks touched, milliseconds, restarts, and edit-to-publish latency. If real captures show hitches, capture immutable block/light/boundary inputs in a new production request and publish a complete candidate through the existing perform-production-request / publish-production-result protocol. Its dependency stamp rejects stale results. Do not add a second worker system.

Mentioned in: One arithmetic medium, many clients, Discover, relax, and invalidate are three different dynamics

Meshing carries raw light, not an art-directed bake #SKMESH

make-block-mesh-snapshot copies sky and block u8 columns for the same one-cell halo as block samples, after stable lighting publication. A sample-light-at generic is justified at this representation boundary: the live world/neighborhood and immutable snapshot each provide one method, while the dense inner representation remains arrays.

Corner lighting samples the face-adjacent air cells around each vertex. It may share the existing AO traversal after a test establishes the intended sample coordinates, but AO occupancy and light averaging remain separately named results. Occluded samples and absent/unknown samples have explicit rules rather than falling through block-solid-p.

The mesh adds one attribute at location 3:

light-material-input :vec3 = (sky-level / 15,
                              block-level / 15,
                              surface emission)

That widens the current 36-byte vertex to four :float32x3 lanes and a 48-byte stride, which the backend can describe today. It is intentionally plain for the first proof. Record generated vertex bytes and mesh time; if bandwidth becomes material, add normalized packed vertex formats to the GPU API as a measured optimization rather than obscuring the initial format.

Light revision changes schedule remeshing through the existing snapshot and stale-result machinery. Relit visible chunks receive priority; an artificially slow “pleasant ripple” is not an acceptance criterion. Measure solver completion and mesh publication separately so either bottleneck is visible.

Mentioned in: Meshing also carries what a fragment cannot see

Meshing also carries what a fragment cannot see #J19EBO

#SKMESH establishes why light readings belong in the vertex product: they are what the mesher knows and the fragment stage does not. Face edges are the same kind of fact, and they turn out to be the difference between a block world which reads as carved solids and one which reads as a lit grid.

A block face is flat geometry, but a real surface rounds over where it ends and fillets where it meets a wall. Perturbing the shading normal near the face's boundary gives both, and needs no mesh change beyond knowing where the fragment sits inside its cell. Doing it at every boundary, though, draws a seam across the middle of an open plain, where the face merely continues into an identically oriented neighbour and there is no edge at all. A fragment cannot tell those cases apart; the mesher can.

So block-face-edge-shaping-components classifies each face's four in-plane edges once, from the same neighbourhood the corner occlusion already samples:

The four signed values become a fifth vertex lane, :float32x4, and the fragment stage multiplies each of its four boundary ramps by the matching sign. One expression therefore serves all three cases, and the sign is what turns a round-over into a fillet. The two in-plane axes are chosen from the face normal by the same rule on both sides of the ABI, which is what makes U and V mean the same thing to the mesher and to the shader.

This widened the vertex from a 48-byte to a 64-byte stride, and required the portable vertex-attribute vocabulary to grow past :float32x3 alone --- a gap #SKMESH's "four :float32x3 lanes" had left standing in both backends.

Two later corrections, both about making the round-over read as a shape (#YUB8CH).

Mentioned in: Status: implementation notes and remaining plan, A round-over is an arc, not a leaning plane, Decisions now made; questions left to evidence

A round-over is an arc, not a leaning plane #YUB8CH

The first version of #J19EBO's shaping added a scaled tangent to the face normal and normalized the sum. That can only ever reach the arctangent of the scale --- a strength of 0.42 buys 23 degrees --- so a legible edge needs an implausible scale, and even then the shading bunches into the last sliver of the ramp because the tangent grows linearly while the angle saturates.

Rotating the normal through an angle instead sweeps it evenly from the face toward the edge, which is what a fillet of a given radius actually does. The in-plane lean gives the direction and the ramp gives the fraction of a quarter turn, and the normal is the corresponding point on that arc. The guarded denominator matters more than it looks: over the flat middle of a face the lean is exactly zero, so the direction is undefined and only the guard keeps it from being a division by zero --- but the angle is zero there too, so the undefined direction is multiplied away.

The second correction is that a round-over is only legible if it is a few pixels wide. Held to a fixed fraction of a cell it dwindles as blocks recede, and everything in the middle distance flattens back into a card long before it is genuinely unresolvable. The block material already measures its own texel footprint from screen derivatives of the atlas coordinate, for the relief fade; the same measurement grows the bevel width until the edge is worth drawing, bounded above so it never eats the face it is an edge of, and then fades it out. So the two shapings share one measurement and use it in opposite directions: relief shrinks with distance, the round-over widens.

Mentioned in: Meshing also carries what a fragment cannot see, No two faces are the same card

The atlas paints relief as well as colour #CHUKWD

Colour is only half of what a material looks like. The other half is its micro-surface: whether it is granular, grooved, tufted, or faceted. luvcraft's atlas was already generated arithmetic rather than an asset, one paint-block-atlas-tile method per numbered tile, so the honest place for that half is the same arithmetic. paint-block-atlas-relief paints a height field; colour variation reads that field, and make-block-normal-atlas materializes its tile-clamped central difference as a linear tangent-space normal map. The block material reads that map once instead of reconstructing the gradient with four dependent texture taps per fragment.

The colour atlas stays sRGB with ordinary opaque coverage. The normal atlas is linear RGBA8: RGB is the encoded unit normal and alpha retains the source height for inspection. The shader declares those sampled meanings as :surface-normal-sample and :surface-relief rather than treating channels as unlabeled numbers. #SKOPEN's decision that bloom must not commandeer scene alpha remains independent of both material textures.

Relief and bevel are both sub-block detail, so both fade out on a measured texel footprint from the atlas UV derivatives: distant terrain neither sparkles nor shows a lit grid. A material's relief is one EQL method, so a live image can re-sculpt grass tufts or stone cracks and rebuild both dense atlases without touching the rest of the palette.

Mentioned in: Status: implementation notes and remaining plan, No two faces are the same card, A roughness the relief already knew, HDR crystal bloom, Decisions now made; questions left to evidence

No two faces are the same card #3AVEKC

The atlas has one tile per material and the world has thousands of faces per tile, so #CHUKWD's relief only made every grass block the same card seen again in better light. Three fields now make each face its own, and none of them costs a texel.

A hash of the cell a face belongs to is constant across that face and different on the next one. The cell is found by stepping half a cell back along the face's own normal, which lands inside the block whichever of the six faces this is. Value noise over the world point knows nothing about where faces begin, so it drifts across a whole plain the way ground does, at a scale of about a dozen cells. A grain finer than a cell supplies the last octave, and --- the part that matters --- two more taps of that grain, a step along each of the face's own in-plane axes, give its gradient there, which leans the shading normal. The ground is then not merely painted unevenly but lit unevenly, which is what the eye actually reads as texture.

Value and hue move together but not identically: brighter patches are also warmer, and a face's own constant drifts its hue as well as its value, so a plain of grass is many greens rather than one green at many brightnesses. Every scale fades out on the footprint that scale resolves at --- the texel-scale terms on the relief fade, the cell-scale ones on the bevel fade of #YUB8CH --- and all three amplitudes ride one surface-detail knob.

Mentioned in: Status: implementation notes and remaining plan, Materials and sky taken toward physical shading

A roughness the relief already knew #2T8CCH

The block material's specular was one fitted power under a Fresnel weight. It is an ordinary microfacet lobe now --- GGX distribution, Smith's height-correlated visibility in the form that already carries the 1/(4\cos\theta_l\cos\theta_v), Schlick's Fresnel at a dielectric's four per cent --- which would be unremarkable except for where its roughness comes from.

Nothing in the palette declares a roughness. But #CHUKWD's normal atlas is each material's own slope at texel scale, and a steep texel is the best evidence available about what happens below a texel: grass tufts are rough, a glass pane is smooth, and the atlas already knows which is which. So the roughness is read from the tangent normal's tilt. When that relief fades out with distance its variance has to go somewhere, and the honest place is the same roughness --- Toksvig's argument --- so a highlight keeps its size across the draw distance instead of sharpening into a sparkle on every distant face at once.

What is not the sun also stopped being one grey. A face turned up sees the zenith, a face turned sideways the horizon, and a face turned down what the ground bounced; one interpolation over the normal's own vertical component, against the zenith and horizon colours the frame environment already carries for the sky, is the whole of it. The same sky evaluated in the direction the surface actually reflects, weighted by roughness and a grazing Fresnel, gives the sheen that tells a smooth face from a rough one out of the sun. Those two terms are the difference between blocks standing under this weather and blocks standing under a light bulb.

One thing had to move that is not shading at all. The palette was authored display-referred, before the renderer worked in linear radiance, and sand at 0.59 linear reflects more light than fresh snow does. A microfacet lobe over an albedo that bright is a white card with a highlight on it, so the natural materials came down to the range a real one occupies and the exposure came down with them.

Mentioned in: Status: implementation notes and remaining plan, Materials and sky taken toward physical shading