luv

Workshop wiki

streaming.lisp

luvcraft/streaming.lisp

system luvcraft/core · 63 definitions · on GitHub

Asynchronous chunk residency: generation and meshing off the frame thread.

The world/canvas thread is the only writer of residency and the only owner of GPU objects. It ships immutable work descriptions (chunk load requests, mesh snapshots) to the production worker, then validates each returned product against the current desired set and dependency stamps before publishing a bounded number of results per frame. Stale products fail validation harmlessly instead of requiring cancellation of active work.

in-package#:luvcraft
defunwait-for-luvcraft-products
session&keyminimum
timeout10.0

Wait outside frame encoding for a useful initial set of chunk meshes.

minimum defaults to nine or the whole desired set when it is smaller. The visible startup path asks only for the nearest product before its first presentation; deterministic captures wait for the broader default set.

let*
minimum
orminimum
min9
hash-table-count
luvcraft-session-desired-chunkssession
deadline
+
get-internal-real-time
round
*timeoutinternal-time-units-per-second
next-report
+
get-internal-real-time
internal-time-units-per-second
loop
let
productsnil
request-canvas-frame
luvcraft-session-canvassession
lambda
timestamp
declare
ignoretimestamp
setfproducts
hash-table-count
luvcraft-session-chunk-productssession
when
>=productsminimum
returnsession
when
>=
get-internal-real-time
next-report
formatt"~&luvcraft: waiting for chunk meshes (~D/~D)~%"productsminimum
finish-output
incfnext-reportinternal-time-units-per-second
when
>=
get-internal-real-time
deadline
error"Only ~D chunk meshes arrived within ~,2F seconds; expected ~D.~@[ Last worker error: ~A~]"productstimeoutminimum
let
result
first
luvcraft-session-production-errorssession
andresult
production-result-conditionresult
sleep0.005
defclassblock-mesh-production-request
absent-neighbor-policy:initarg:absent-neighbor-policy:readerblock-mesh-production-request-absent-neighbor-policy
snapshot:initarg:snapshot:readerblock-mesh-production-request-snapshot
zdefmethod
perform-production-request:zone:production/mesh-chunk
mesh-block-snapshot
make-instance'exposed-face-mesher:absent-neighbor-policy
block-mesh-production-request-absent-neighbor-policyrequest
block-mesh-production-request-snapshotrequest
defclassblock-light-production-request
region:initarg:region:readerblock-light-production-request-region
solver:initarg:solver:initform*voxel-light-solver*:readerblock-light-production-request-solver
dependency-stamp:initarg:dependency-stamp:readerblock-light-production-request-dependency-stamp
defstructblock-light-production-payloadregioncells-visitedelapsed-seconds
zdefmethod
perform-production-request:zone:production/light-world
let
start
get-internal-real-time
multiple-value-bind
regionvisited
solve-light-region-using
block-light-production-request-solverrequest
block-light-production-request-regionrequest
make-block-light-production-payload:regionregion:cells-visitedvisited:elapsed-seconds
/
-
get-internal-real-time
start
coerceinternal-time-units-per-second'double-float
defclassblock-chunk-load-payload
key:initarg:key:readerblock-chunk-load-payload-key
content:initarg:content:readerblock-chunk-load-payload-content
defclassblock-chunk-load-request
demand-token:initarg:demand-token:readerblock-chunk-load-request-demand-token
:documentation

The source-neutral half of asynchronous chunk loading.

A concrete world source subclasses this with whatever it needs to rebuild one chunk in isolation on the worker, and answers perform-production-request with a block-chunk-load-payload. Publication and demand-token validation live here, on the base class.

defgenericmake-block-chunk-load-request
sourceworldkeydemand-tokenpriority
:documentation

Make the off-thread load request which produces resident chunk key.

Return NIL when source cannot generate chunks away from the owning thread; the session then keeps key desired without scheduling work for it. A returned request must carry the production key (:LOAD key) so it matches the session's outstanding-work and cancellation bookkeeping.

defmethodmake-block-chunk-load-request
sourcet
worldkeydemand-tokenpriority
declare
ignoreworldkeydemand-tokenpriority
nil

The little world's realization of the load protocol. The request captures everything the worker needs as plain values: the seed, the chunk shape, deterministic landmarks owned by the chunk, and the sparse edits inside it.

defclasslittle-world-load-request
seed:initarg:seed:readerlittle-world-load-request-seed
width:initarg:width:readerlittle-world-load-request-width
height:initarg:height:readerlittle-world-load-request-height
depth:initarg:depth:readerlittle-world-load-request-depth
landmarks:initarg:landmarks:initformnil:readerlittle-world-load-request-landmarks
edits:initarg:edits:initformnil:readerlittle-world-load-request-edits
defmethodmake-block-chunk-load-request
worldkeydemand-tokenpriority
let*
shape
voxel-space-chunk-shape
block-world-spaceworld
width
chunk-shape-widthshape
height
chunk-shape-heightshape
depth
chunk-shape-depthshape
captured-editsnil
maphash
lambda
coordinateblock
destructuring-bind
xyz
coordinate
when
and
=
firstkey
=
floorzdepth
thirdkey
push
listblockxyz
captured-edits
block-edit-overlay-entries
little-world-source-editssource
make-instance'little-world-load-request:key
list:loadkey
:prioritypriority:seed
little-world-source-seedsource
:demand-tokendemand-token:widthwidth:heightheight:depthdepth:landmarks:editscaptured-edits
zdefmethod
perform-production-request:zone:production/load-chunk
"Generate one isolated chunk and transfer only its dense content columns."
destructuring-bind
chunk-xchunk-ychunk-z
second
production-request-keyrequest
let*
source
make-instance'little-world-source:seed
little-world-load-request-seedrequest
world
make-block-world:chunk-width
little-world-load-request-widthrequest
:chunk-height
little-world-load-request-heightrequest
:chunk-depth
little-world-load-request-depthrequest
:sourcesource
materialize-block-world-chunksourceworldchunk-xchunk-ychunk-z
dolist
landmark
little-world-load-request-landmarksrequest
destructuring-bind
blockxyz
landmark
setfblock
dolist
edit
little-world-load-request-editsrequest
destructuring-bind
blockxyz
edit
setfblock
let
chunk
world-chunk-atworldchunk-xchunk-ychunk-z
make-instance'block-chunk-load-payload:key
listchunk-xchunk-ychunk-z
:content
block-chunk-contentchunk
defclassluvcraft-chunk-product
coordinate:initarg:coordinate:readerluvcraft-chunk-product-coordinate
dependency-stamp:initarg:dependency-stamp:readerluvcraft-chunk-product-dependency-stamp
mesh:initarg:mesh:readerluvcraft-chunk-product-mesh
vertex-buffer:initarg:vertex-buffer:readerluvcraft-chunk-product-vertex-buffer
defuncancel-luvcraft-chunk-production
sessionkey
dolist
kind'
:load:mesh
let
production-key
listkindkey

Active work cannot be canceled, so retain its ticket until its result returns and fails desired-set/incarnation validation.

when
cancel-production-request
luvcraft-session-production-systemsession
production-key
remhashproduction-key
luvcraft-session-outstanding-productionsession
defunluvcraft-player-chunk-center
worldplayer
ifplayer
let
shape
voxel-space-chunk-shape
block-world-spaceworld
list
floor
player-xplayer
chunk-shape-widthshape
floor
player-zplayer
chunk-shape-depthshape
'
00
defunmaintain-generated-luvcraft-residency
sessionworldplayerradius
let
unless
equalcenter
luvcraft-session-residency-centersession
when
tracy-message
formatnil"residency center ~{~D~^,~}"center
:color#x4EA1FF
let
desired
luvcraft-session-desired-chunkssession
next-desired
make-hash-table:test#'equal
loopforchunk-xfrom
-
firstcenter
radius
to
+
firstcenter
radius
do
loopforchunk-zfrom
-
secondcenter
radius
to
+
secondcenter
radius
forkey=
chunk-keychunk-x0chunk-z
do
setf
gethashkeynext-desired
or
gethashkeydesired
incf
luvcraft-session-next-residency-demandsession
maphash
lambda
old-keytoken
declare
ignoretoken
unless
gethashold-keynext-desired
desired
clrhashdesired
maphash
lambda
keytoken
setf
gethashkeydesired
token
next-desired
setf
luvcraft-session-residency-centersession
center

Eviction is an owner-side publication. Pending work is either canceled before it starts or allowed to finish and fail its desired-set/incarnation validation harmlessly.

dolist
let
unless
gethashkeydesired
destructuring-bind
xyz
key
defunmaintain-static-luvcraft-residency
sessionworldplayer

Treat a caller-owned resident set as desired without loading or eviction.

let
desired
luvcraft-session-desired-chunkssession
resident
make-hash-table:test#'equal
dolist
let
setf
gethashkeyresident
t
unless
gethashkeydesired
setf
gethashkeydesired
incf
luvcraft-session-next-residency-demandsession
let
departednil
maphash
lambda
keytoken
declare
ignoretoken
unless
gethashkeyresident
pushkeydeparted
desired
dolist
keydeparted
remhashkeydesired
setf
luvcraft-session-residency-centersession
defgenericmaintain-block-world-residency
sourcesessionworld
:documentation

Reconcile session's desired chunk set as world's source prescribes.

A source which can rematerialize its terrain keeps a sliding window around the player, letting the session load and evict chunks as the player moves. The default method treats a caller-owned resident set as desired without loading or eviction: those chunks still receive immutable mesh production, but nothing could regenerate them once evicted.

defmethodmaintain-block-world-residency
sourcet
sessionworld
maintain-static-luvcraft-residencysessionworld
luvcraft-session-playersession
defmethodmaintain-block-world-residency
sessionworld
let
player
luvcraft-session-playersession
radius
luvcraft-session-residency-radiussession

Without a player or radius there is no window to slide, so even a generative world falls back to caller-owned residency.

if
andplayerradius
call-next-method
zdefun
maintain-luvcraft-residency:zone:streaming/reconcile-residency:value
hash-table-count
luvcraft-session-desired-chunkssession
session
"Reconcile desired residency without generating chunks on the frame thread."
let
world
luvcraft-session-worldsession
maintain-block-world-residency
block-world-sourceworld
sessionworld
defunchunk-light-stamp-revision
chunk

The chunk's light revision, or -1 while it has no published field.

let
field
block-chunk-light-fieldchunk
iffield
chunk-light-field-revisionfield
-1
defunchunk-light-stamp-boundary-revision
chunkdirection
let
field
block-chunk-light-fieldchunk
defunchunk-mesh-dependency-stamp
worldchunk

Describe exactly which resident data chunk's exposed mesh observes.

Content and light are named as separate revision domains: relighting must invalidate a mesh without impersonating an authored edit, and the stamp preserves which domain made a product stale.

let
coordinate
chunk-domain-coordinate
block-chunk-domainchunk
cons
list
block-chunk-incarnationchunk
block-chunk-revisionchunk
loopfordirectionin*voxel-face-directions*collect
let
neighbor-coordinate
chunk-coordinate-neighborcoordinatedirection
declare
dynamic-extentneighbor-coordinate
multiple-value-bind
neighborpresent-p
world-chunk-at-coordinateworldneighbor-coordinate
ifpresent-p

Only the neighbor boundary facing this chunk contributes.

let
list
block-chunk-incarnationneighbor
'
nil
defunluvcraft-session-products-in-order
session
let
products
luvcraft-session-chunk-productssession
loopforproductbeingthehash-valuesofproductscollectproductintoresultfinally
return
sortresult
lambda
leftright
let
a
luvcraft-chunk-product-coordinateleft
b
luvcraft-chunk-product-coordinateright
or
<
chunk-coordinate-xa
chunk-coordinate-xb
and
=
chunk-coordinate-xa
chunk-coordinate-xb
or
<
chunk-coordinate-ya
chunk-coordinate-yb
and
=
chunk-coordinate-ya
chunk-coordinate-yb
<
chunk-coordinate-za
chunk-coordinate-zb
defunluvcraft-session-mesh
session

Return a combined, inspectable snapshot of session's chunk meshes.

let
vertices
make-array0:element-type'single-float:adjustablet:fill-pointer0
vertex-declarationnil
vertex-count0
face-count0
dolist
let
mesh
luvcraft-chunk-product-meshproduct
setfvertex-declaration
loopforcomponentacross
block-mesh-verticesmesh
do
vector-push-extendcomponentvertices
incfvertex-count
block-mesh-vertex-countmesh
incfface-count
block-mesh-face-countmesh
make-instance'block-mesh:vertex-declaration
orvertex-declaration
:verticesvertices:vertex-countvertex-count:face-countface-count
defundestroy-luvcraft-chunk-products
session
dolist
products
list
luvcraft-session-chunk-productssession
luvcraft-session-staged-chunk-productssession
maphash
lambda
keyproduct
declare
ignorekey
let
buffer
luvcraft-chunk-product-vertex-bufferproduct
whenbuffer
destroybuffer
products
clrhashproducts
values
defuncurrent-luvcraft-chunk-product-p
sessionkeyproduct
let*
world
luvcraft-session-worldsession
chunk
apply#'world-chunk-atworldkey
andchunk
gethashkey
luvcraft-session-desired-chunkssession
equal
luvcraft-chunk-product-dependency-stampproduct
defunstale-luvcraft-visible-product-keys
session

Return the keys of visible products which no longer describe the world.

let
keysnil
maphash
lambda
keyproduct
when
and
gethashkey
luvcraft-session-desired-chunkssession
pushkeykeys
luvcraft-session-chunk-productssession
keys
defunchunk-neighbor-key
keydirection
destructuring-bind
xyz
key
chunk-key
+x
voxel-direction-dxdirection
+y
voxel-direction-dydirection
+z
voxel-direction-dzdirection
defunluvcraft-stale-product-components
session

Group stale visible chunk products connected across block faces.

let
remaining
make-hash-table:test#'equal
componentsnil
dolist
setf
gethashkeyremaining
t
loopwhile
plusp
hash-table-countremaining
forseed=
loopforkeybeingthehash-keysofremainingdo
do
let
frontier
listseed
componentnil
remhashseedremaining
loopwhilefrontierforkey=
popfrontier
do
pushkeycomponent
dolist
let
neighbor
chunk-neighbor-keykeydirection
when
gethashneighborremaining
remhashneighborremaining
pushneighborfrontier
pushcomponentcomponents
components
defunready-luvcraft-mesh-publication-groups
session

Return staged mesh groups which may replace visible products together.

A block edit can invalidate its own chunk and each face-neighbor whose mesh observed the edited boundary. Connected stale visible products therefore form one publication cohort: the old cohort remains drawable until every current replacement is staged. A chunk with no visible predecessor may be published independently.

let*
staged
luvcraft-session-staged-chunk-productssession
stale-components
stale-keys
make-hash-table:test#'equal
groupsnil
dolist
componentstale-components
dolist
keycomponent
setf
gethashkeystale-keys
t
when
every
lambda
key
let
product
gethashkeystaged
andproduct
component
pushcomponentgroups
maphash
lambda
keyproduct
when
and
not
gethashkeystale-keys
push
listkey
groups
staged
groups
zdefun
discard-stale-luvcraft-staged-products:zone:streaming/discard-stale:value
hash-table-count
luvcraft-session-staged-chunk-productssession
session
let
discardednil
staged
luvcraft-session-staged-chunk-productssession
maphash
lambda
keyproduct
unless
let
buffer
luvcraft-chunk-product-vertex-bufferproduct
whenbuffer
destroybuffer
pushkeydiscarded
staged
dolist
keydiscarded
remhashkeystaged
lengthdiscarded
zdefun
publish-ready-luvcraft-meshes:zone:streaming/publish-meshes:value
hash-table-count
luvcraft-session-staged-chunk-productssession
session
"Atomically replace every complete stale mesh cohort at a frame boundary."
let
published0
retirednil
products
luvcraft-session-chunk-productssession
staged
luvcraft-session-staged-chunk-productssession
dolist

No rendering can interleave with these owner-thread hash updates. Install the complete cohort before retiring any of its predecessors.

dolist
keygroup
let
candidate
gethashkeystaged
old
gethashkeyproducts
setf
gethashkeyproducts
candidate
remhashkeystaged
whenold
pusholdretired
incfpublished
dolist
productretired
let
buffer
luvcraft-chunk-product-vertex-bufferproduct
whenbuffer
destroybuffer
published
defunchunk-key-distance-squared
keycenter
+
expt
-
firstkey
firstcenter
2
expt
-
thirdkey
secondcenter
2
defunchunk-key-nearer-p
leftrightcenter
let
left-distance
right-distance
or
<left-distanceright-distance
and
=left-distanceright-distance
loopforainleftforbinrightwhen
/=ab
return
<ab
finally
zdefun
schedule-luvcraft-chunk-loads:zone:streaming/schedule-loads:value
hash-table-count
luvcraft-session-desired-chunkssession
session
let*
world
luvcraft-session-worldsession
source
block-world-sourceworld
center
luvcraft-session-residency-centersession
outstanding
luvcraft-session-outstanding-productionsession
candidatesnil
maphash
lambda
keydemand-token
let
production-key
list:loadkey
unless
or
nth-value1
apply#'world-chunk-atworldkey
gethashproduction-keyoutstanding
push
listkeydemand-tokenproduction-key
candidates
luvcraft-session-desired-chunkssession
setfcandidates
sortcandidates
lambda
leftright
chunk-key-nearer-p
firstleft
firstright
center
looprepeat
luvcraft-session-load-schedule-limitsession
forcandidateincandidatesdo
destructuring-bind
keydemand-tokenproduction-key
candidate
let
request
whenrequest
setf
gethashproduction-keyoutstanding
schedule-production-request
luvcraft-session-production-systemsession
request
zdefun
schedule-luvcraft-meshes:zone:streaming/schedule-meshes:value
hash-table-count
luvcraft-session-desired-chunkssession
session
"Capture at most the configured number of immutable mesh inputs this frame."
let*
world
luvcraft-session-worldsession
products
luvcraft-session-chunk-productssession
staged
luvcraft-session-staged-chunk-productssession
outstanding
luvcraft-session-outstanding-productionsession
center
luvcraft-session-residency-centersession
candidatesnil
dolist
let*
production-key
list:meshkey
old
gethashkeyproducts
candidate
gethashkeystaged
when
and
gethashkey
luvcraft-session-desired-chunkssession
not
gethashproduction-keyoutstanding
not
and
orcandidateold
equalstamp
luvcraft-chunk-product-dependency-stamp
orcandidateold
push
listchunkkeyproduction-keystamp
candidates
setfcandidates
sortcandidates#'<:key#'first
looprepeat
luvcraft-session-mesh-capture-limitsession
forcandidateincandidatesdo
destructuring-bind
prioritychunkkeyproduction-keystamp
candidate
declare
ignorekey
let*
snapshot
request
make-instance'block-mesh-production-request:keyproduction-key:prioritypriority:absent-neighbor-policy
exposed-face-mesher-absent-neighbor-policy
luvcraft-session-meshersession
:snapshotsnapshot
setf
gethashproduction-keyoutstanding
schedule-production-request
luvcraft-session-production-systemsession
request
defunblock-world-light-dependency-stamp
world

Name the exact resident content captured by an asynchronous light solve.

sort
loopforchunkincollect
list
copy-list
block-chunk-incarnationchunk
block-chunk-revisionchunk
lambda
leftright
loopforain
firstleft
forbin
firstright
when
/=ab
return
<ab
finally
defunluvcraft-load-production-pending-p
session
loopforkeybeingthehash-keysof
luvcraft-session-outstanding-productionsession
thereis
eq
firstkey
:load
defunmark-luvcraft-lighting-for-retry
state
dolist
chunk
resident-world-chunks
lighting-state-worldstate
setf
gethash
chunk-domain-coordinate
block-chunk-domainchunk
lighting-state-arrivalsstate
t
zdefun
schedule-luvcraft-lighting:zone:streaming/schedule-lighting:value
let
state
luvcraft-session-lighting-statesession
ifstate
+
hash-table-count
lighting-state-dirty-cellsstate
hash-table-count
lighting-state-arrivalsstate
hash-table-count
lighting-state-departuresstate
0
session

Reconcile cell edits incrementally or schedule a residency-wide relight.

Authored edits normally touch one settled cell and are much cheaper to reconcile against the published field than to solve every resident chunk from scratch. Chunk arrivals and departures still use a coalesced immutable worker request, keeping residency-scale lighting out of the frame callback.

let*
state
luvcraft-session-lighting-statesession
outstanding
luvcraft-session-outstanding-productionsession
production-key'
:light
when
andstate
not
gethashproduction-keyoutstanding
if
let*
world
lighting-state-worldstate
request
make-instance'block-light-production-request:keyproduction-key:priority-1:dependency-stamp:solver*voxel-light-solver*:region
capture-light-regionworld:immutable-pt

New hooks which fire after this capture accumulate for the next request. A stale or failed result explicitly restores dirtiness.

clrhash
lighting-state-dirty-cellsstate
clrhash
lighting-state-arrivalsstate
clrhash
lighting-state-departuresstate
setf
gethashproduction-keyoutstanding
schedule-production-request
luvcraft-session-production-systemsession
request
with-cpu-trace-zone
:streaming/reconcile-cell-lighting
defunluvcraft-lighting-settled-p
session
let
state
luvcraft-session-lighting-statesession
or
nullstate
and
not
gethash'
:light
luvcraft-session-outstanding-productionsession
defgenericpublish-production-result
sessionrequestvalue
:documentation

Validate and accept one worker product on the render/GPU owning thread.

This is the owner-side mirror of perform-production-request: each request class carries its own publication rule, so a new kind of asynchronous product plugs in with one method on each generic rather than an edit to the drain loop. A stale product simply fails its own validation here. Product kinds whose visible dependencies span several chunks may stage a complete candidate here and install its publication cohort later at the frame boundary.

zdefmethod
publish-production-result:zone:production/publish-mesh:value
truncate
length
block-mesh-verticesmesh
+block-mesh-floats-per-vertex+
let*
snapshot
block-mesh-production-request-snapshotrequest
key
block-mesh-snapshot-keysnapshot
world
luvcraft-session-worldsession
chunk
apply#'world-chunk-atworldkey
when
andchunk
gethashkey
luvcraft-session-desired-chunkssession
equal
block-mesh-snapshot-dependency-stampsnapshot
let
buffernil
completed-pnil
old
gethashkey
luvcraft-session-staged-chunk-productssession
unwind-protect
progn
setfbuffer
create
luvcraft-session-devicesession
make-buffer-descriptor:label
formatnil"block chunk ~{~D~^,~} async mesh"key
:size
max4
*4
length
block-mesh-verticesmesh
:usage'
:vertex
write-bufferbuffer
block-mesh-verticesmesh
setf
gethashkey
luvcraft-session-staged-chunk-productssession
make-instance'luvcraft-chunk-product:coordinate
chunk-domain-coordinate
block-chunk-domainchunk
:dependency-stamp
block-mesh-snapshot-dependency-stampsnapshot
:meshmesh:vertex-bufferbuffer
completed-pt
whenold
let
old-buffer
luvcraft-chunk-product-vertex-bufferold
whenold-buffer
destroyold-buffer
unlesscompleted-p
whenbuffer
destroybuffer
zdefmethod
publish-production-result:zone:production/publish-light:value
block-light-production-payload-cells-visitedpayload
let*
state
luvcraft-session-lighting-statesession
world
luvcraft-session-worldsession
current-p
equal
block-light-production-request-dependency-stamprequest
ifcurrent-p
let
changed
publish-light-region
block-light-production-payload-regionpayload
incf
lighting-state-cells-visitedstate
block-light-production-payload-cells-visitedpayload
incf
lighting-state-chunks-touchedstate
hash-table-count
light-region-entries
block-light-production-payload-regionpayload
incf
lighting-state-publicationsstate
setf
lighting-state-last-latency-secondsstate
block-light-production-payload-elapsed-secondspayload
changed
zdefmethod
publish-production-result:zone:production/publish-chunk
let*
key
block-chunk-load-payload-keypayload
world
luvcraft-session-worldsession
when
and
eql
gethashkey
luvcraft-session-desired-chunkssession
block-chunk-load-request-demand-tokenrequest
not
nth-value1
apply#'world-chunk-atworldkey
destructuring-bind
xyz
key
install-world-chunk-storageworldxyz
block-chunk-load-payload-contentpayload
zdefun
drain-luvcraft-production:zone:streaming/drain-results:value
min
luvcraft-session-publication-limitsession
production-system-completed-count
luvcraft-session-production-systemsession
session
"Publish a bounded number of completed CPU products this frame."
looprepeat
luvcraft-session-publication-limitsession
do
multiple-value-bind
resultpresent-p
receive-production-result-no-hang
luvcraft-session-production-systemsession
unlesspresent-p
let*
request
production-result-requestresult
key
production-request-keyrequest
ticket
gethashkey
luvcraft-session-outstanding-productionsession
when
eqlticket
production-request-ticketrequest
remhashkey
luvcraft-session-outstanding-productionsession
if
production-result-conditionresult
progn
when
mark-luvcraft-lighting-for-retry
luvcraft-session-lighting-statesession
pushresult
luvcraft-session-production-errorssession
publish-production-resultsessionrequest
production-result-valueresult
zdefun
evict-luvcraft-products:zone:streaming/evict-products:value
hash-table-count
luvcraft-session-desired-chunkssession
session
let
evictednil
desired
luvcraft-session-desired-chunkssession
product-tables
list
luvcraft-session-chunk-productssession
luvcraft-session-staged-chunk-productssession
dolist
productsproduct-tables
setfevictednil
maphash
lambda
keyproduct
unless
gethashkeydesired
let
buffer
luvcraft-chunk-product-vertex-bufferproduct
whenbuffer
destroybuffer
pushkeyevicted
products
dolist
keyevicted
remhashkeyproducts
zdefun
refresh-luvcraft-mesh:zone:streaming/refresh
session
"Advance asynchronous loading/meshing without doing either computation here."

Batch arrivals, then solve a frozen region off-thread. Publication still precedes mesh capture, so dependency stamps observe complete light fields.

A result captured before this frame's lighting publication may already be stale. Reject it before deciding whether a complete mesh cohort can cross the frame boundary.

setf
luvcraft-session-meshed-world-revisionsession
block-world-revision
luvcraft-session-worldsession
defunluvcraft-streaming-trace-state
session

Snapshot the owner-side streaming state on the canvas thread.

let
statenil
request-canvas-frame
luvcraft-session-canvassession
lambda
timestamp
declare
ignoretimestamp
let
lighting
luvcraft-session-lighting-statesession
setfstate
list:center
copy-list
luvcraft-session-residency-centersession
:desired
hash-table-count
luvcraft-session-desired-chunkssession
:outstanding
hash-table-count
luvcraft-session-outstanding-productionsession
:staged
hash-table-count
luvcraft-session-staged-chunk-productssession
:products
hash-table-count
luvcraft-session-chunk-productssession
:lighting-dirty-p:errors
length
luvcraft-session-production-errorssession
state
defunluvcraft-streaming-trace-state-quiescent-p
state&optionalcenter
and
zerop
getfstate:errors
getfstate:center
or
nullcenter
equalcenter
getfstate:center
plusp
getfstate:desired
=
getfstate:desired
getfstate:products
zerop
getfstate:outstanding
zerop
getfstate:staged
not
getfstate:lighting-dirty-p
defunwait-for-luvcraft-streaming-quiescence
session&keycenter
timeout30d0

Wait until session has no unpublished streaming or lighting work.

When center is supplied, also wait for the residency window to reach it. This is intended for repeatable live profiling orchestration, not frame code.

let
deadline
+
get-internal-real-time
round
*timeoutinternal-time-units-per-second
statenil
loop
when
plusp
getfstate:errors
error"Luvcraft production failed while waiting for quiescence: ~S"state
when
>=
get-internal-real-time
deadline
error"Streaming did not become quiescent within ~,2F seconds: ~S"timeoutstate
sleep0.01
defunwait-for-luvcraft-tracy-connection
&key
timeout10d0
let
deadline
+
get-internal-real-time
round
*timeoutinternal-time-units-per-second
loopuntildo
when
>=
get-internal-real-time
deadline
error"No Tracy capture connected within ~,2F seconds."timeout
sleep0.01
defuntrace-luvcraft-streaming-boundary
session&key
baseline-seconds0.5d0
timeout10d0

Trace quiet play followed by one natural +X residency-window advance.

The Tracy client must be enabled. This waits for a connected capture and a fully quiescent session, marks the baseline, leaves the game untouched for baseline-seconds, then moves the player just across the next chunk boundary. It returns only after the resulting loads, lighting, and meshes are published.

unless*tracy*
error"Start luvcraft with --tracy before tracing streaming."
let*
before
old-center
getfbefore:center
new-center
list
1+
firstold-center
secondold-center
tracy-message"streaming trace: quiescent baseline begins":color#x54C878
sleepbaseline-seconds
request-canvas-frame
luvcraft-session-canvassession
lambda
timestamp
declare
ignoretimestamp
let*
world
luvcraft-session-worldsession
shape
voxel-space-chunk-shape
block-world-spaceworld
player
luvcraft-session-playersession
setf
player-xplayer
+
*
firstnew-center
chunk-shape-widthshape
0.5d0
0d00d00d0
sync-camera-to-player
luvcraft-session-camerasession
player
tracy-message"streaming trace: chunk boundary crossed":color#xFFB347
let
after
wait-for-luvcraft-streaming-quiescencesession:centernew-center:timeouttimeout
tracy-message"streaming trace: publication complete":color#x54C878
after

--------------------------------------------------------------------- The streaming knobs.

The window is only rebuilt when its centre moves, so a knob over its radius must forget the centre: that is the residency-realization. The per-frame budgets are read each frame and need nothing.

defclassresidency-realization
:documentation

The value shapes the streaming window, which is only rebuilt when its centre moves; forget the centre so it rebuilds now.

defmethodrealize-knobprogn
setf
luvcraft-session-residency-centersession
nil
define-knobresidency-radius
:label"view distance":group:streaming:class'residency-knob:quantity
:quantity:chunk-radius:unit:one
:unit-label" chunks":minimum1:maximum12:step1
luvcraft-session-residency-radiussession
define-knobload-schedule-limit
:label"loads per frame":group:streaming:quantity
:quantity:frame-budget:unit:one
:minimum0:maximum32:step1
luvcraft-session-load-schedule-limitsession
define-knobmesh-capture-limit
:label"meshes per frame":group:streaming:quantity
:quantity:frame-budget:unit:one
:minimum0:maximum8:step1
luvcraft-session-mesh-capture-limitsession
define-knobpublication-limit
:label"uploads per frame":group:streaming:quantity
:quantity:frame-budget:unit:one
:minimum0:maximum16:step1
luvcraft-session-publication-limitsession